[PUP-1061] Install multiple packages within a single call to the package manager Created: 2013/12/16  Updated: 2018/06/08

Status: Accepted
Project: Puppet
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Normal
Reporter: redmine.exporter Assignee: Unassigned
Resolution: Unresolved Votes: 30
Labels: customer, redmine
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Blocks
blocks PUP-5539 circular dependent RPM packages can't... Closed
Relates
relates to PUP-146 Install multiple packages with one ca... Open
relates to PUP-4039 package resource doesn't find new pac... Accepted
Template:
Epic Link: Batch Processing of Resources
Team: Coremunity
QA Contact: Erik Dasher

 Description   

During the configuration applying process the package manager is called for each package installation.
It is possible to reduce the number of calls to the package manager by gathering package installation and delayed some package installation.
Naturally, this modification should not break the dependency graph.



 Comments   
Comment by Felix Frank [ 2014/05/11 ]

To save fellow archaeologists some time, the most recent summary of the work in progress is http://projects.puppetlabs.com/issues/2198#note-41 - although we are apparently still in the design phase.

Comment by William Van Hevelingen [ 2014/08/29 ]

+1

Please add it to apt and yum first.

Comment by Mathieu Gagné [ 2014/08/29 ]

This issue is very important for us.

We use Vagrant+Puppet to help our developers get a copy of production infrastructures for test and development purposes.

There is about ~20 boxes in ONE of our stacks and we have about 25+ stacks available. (a stack being one setup/system in production)

Our developers often destroy and provision their stacks several times per day for various reasons and the main complain we get is the time it takes to provision them.

To give you an idea of the impact such feature could have for us: Out of the 33m 57s spent provisioning the boxes in that particular stack, 29m 10s is spent managing (install/remove) packages by Puppet.

Being able to install all packages in a single batch would greatly help us cut down the time spent provisioning our boxes.

Comment by Bill Broadley [ 2014/08/29 ]

Mathieu for comparison purposes it would be useful to know how long an equivalent apt-get or yum command takes if you feed all the same package names to it.

I've been moving the first boot/package management outside puppet into a cobbler managed first boot script for this reason, but haven't made a comparison yet.

Comment by Felix Frank [ 2014/08/29 ]

Kylo Ginsberg has sketched 7 steps towards this goal. As far as I know, not even one has been implemented yet.

I guess the best way forward would be to create a subtask for step 1:

Define report schema changes

The others should follow, but I don't think it would be wise to create them all on the spot - I agree that we will likely learn some things along the way that will shift our perspective on what needs to happen and how.

Needless to say, code contributions would go a long way.

Comment by Grant Street [ 2014/08/31 ]

The package list will have to be able to be batched as there is a command line length limit. This is why you can't assume they can translate it into a single batch. This batching should be transparent.

Comment by Dustin J. Mitchell [ 2014/10/01 ]

It's worth noting that this bug makes updating puppet itself a dangerous operation. In trying to install puppet-3.7.0 (with ensure => '3.7.0-1.el6") on a puppet master, I got

Error: Could not update: Failed to update to version 3.7.0-1.el6, got version 3.7.1-1.el6 instead
Wrapped exception:
Failed to update to version 3.7.0-1.el6, got version 3.7.1-1.el6 instead
Error: /Stage[main]/Packages::Puppet/Package[puppet]/ensure: change from 3.6.1-1.el6 to 3.7.0-1.el6 failed: Could not update: Failed to update to version 3.7.0-1.el6, got version 3.7.1-1.el6 instead
Notice: /Stage[main]/Packages::Puppetserver/Package[puppet-server]: Dependency Package[puppet] has failures: true
Warning: /Stage[main]/Packages::Puppetserver/Package[puppet-server]: Skipping because of failed dependencies

which is to say, because of the bidirectional dependency between puppet and puppet-server, the pinning operation failed and puppet installed an untested version of itself (which version subsequently failed and caused an outage). So, if only for the PR benefit of puppet not destroying itself on upgrade, it'd be good to see motion here

Regarding command-line length, at least with yum it's possible to build up a transaction one package at a time, then trigger the entire transaction.

Comment by Erik Dalén [ 2016/04/22 ]

Here are some details about how mgmt has solved this: https://ttboj.wordpress.com/2016/03/30/automatic-grouping-in-mgmt/

Comment by Eric Sorenson [ 2016/04/26 ]

Branan Riley yeah i'd love to see this in, as you can see it's still an active issue for people and the design feels pretty "shovel ready".

Generated at Wed Nov 13 11:48:07 PST 2019 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.