The puppet plugin download command doesn't re-use connections to the
Puppet Server when downloading plugins from modules. This adds considerable time to
downloads along with extra load on the Puppet Server due to multiple redundant TLS handshakes.
Reproduction Case
- Install puppet-agent 5.5.0 and puppetserver 5.3.0 on a CentOS 7
node:
rpm -Uvh http://yum.puppetlabs.com/puppet5/puppet5-release-el-7.noarch.rpm
|
yum install -y puppet-agent-5.5.0 puppetserver-5.3.0
|
- Configure the puppet agent to connect to the local host and start
puppetserver:
/opt/puppetlabs/bin/puppet config set --section main server $(hostname -f)
|
systemctl start puppetserver
|
- Run puppet agent -t to verify things work.
- Install the puppetlabs-windows module, which brings in a ton of
plugins that will have to be synced:
/opt/puppetlabs/bin/puppet module install puppetlabs-windows
|
- Empty the plugin cache, and time a puppet agent run along with the
number of network connections it opens:
rm -rf /opt/puppetlabs/puppet/cache/lib
|
time /opt/puppetlabs/bin/puppet agent -t --http_debug 2>&1|grep -e Info: -e 'opening connection'
|
- Empty the cache again and time a puppet plugin download run along
with the number of network connections opened:
rm -rf /opt/puppetlabs/puppet/cache/lib
|
time /opt/puppetlabs/bin/puppet plugin download --verbose --http_debug 2>&1|grep -e Info: -e 'opening connection'
|
Outcome
The puppet agent run uses a handful of connections:
# rm -rf /opt/puppetlabs/puppet/cache/lib
|
# time /opt/puppetlabs/bin/puppet agent -t --http_debug 2>&1|grep -e Info: -e 'opening connection'
|
|
opening connection to wzwhytp204ipraf.delivery.puppetlabs.net:8140...
|
Info: Using configured environment 'production'
|
Info: Retrieving pluginfacts
|
Info: Retrieving plugin
|
opening connection to wzwhytp204ipraf.delivery.puppetlabs.net:8140...
|
Info: Retrieving locales
|
Info: Loading facts
|
Info: Caching catalog for wzwhytp204ipraf.delivery.puppetlabs.net
|
Info: Applying configuration version '1523665266'
|
|
real 0m43.990s
|
user 0m16.685s
|
sys 0m1.497s
|
The puppet plugin download run uses hundreds of connections and takes much
longer than the puppet agent run despite only doing a subset of the work:
# rm -rf /opt/puppetlabs/puppet/cache/lib
|
# time /opt/puppetlabs/bin/puppet plugin download --verbose --http_debug 2>&1|grep -e Info: -e 'opening connection'
|
|
Info: Retrieving pluginfacts
|
opening connection to wzwhytp204ipraf.delivery.puppetlabs.net:8140...
|
Info: Retrieving plugin
|
opening connection to wzwhytp204ipraf.delivery.puppetlabs.net:8140...
|
opening connection to wzwhytp204ipraf.delivery.puppetlabs.net:8140...
|
opening connection to wzwhytp204ipraf.delivery.puppetlabs.net:8140...
|
... omitted hundreds of open calls for brevity ...
|
opening connection to wzwhytp204ipraf.delivery.puppetlabs.net:8140...
|
opening connection to wzwhytp204ipraf.delivery.puppetlabs.net:8140...
|
opening connection to wzwhytp204ipraf.delivery.puppetlabs.net:8140...
|
opening connection to wzwhytp204ipraf.delivery.puppetlabs.net:8140...
|
Info: Retrieving locales
|
opening connection to wzwhytp204ipraf.delivery.puppetlabs.net:8140...
|
|
real 3m39.345s
|
user 1m29.576s
|
sys 0m2.405s
|
Expected Outcome
The puppet plugin download action should complete much faster than
puppet agent -t when populating a cold plugin cache.
Bug
Major
