Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-10160

Puppet should not load types and providers during environment convergence

    XMLWordPrintable

Details

    • Coremunity
    • Platform Core KANBAN
    • Needs Assessment
    • Bug Fix
    • Hide
      Previously puppet agents could fail to apply a catalog if the agent switched environments based on node classification and there were different versions of a module in those environments. As a result of this change, an agent should converge to its server-assigned environment more quickly, only load types and providers once, and only update its cached catalog after the environment converges.
      Show
      Previously puppet agents could fail to apply a catalog if the agent switched environments based on node classification and there were different versions of a module in those environments. As a result of this change, an agent should converge to its server-assigned environment more quickly, only load types and providers once, and only update its cached catalog after the environment converges.
    • Needs Assessment

    Description

      During environment convergence, puppet loads types and providers for pluginsynced modules, potentially multiple times. Type/provider loading is triggered when writing the cached catalog and/or when converting the resource catalog to a RAL catalog.

      However, if the environment doesn't converge the first time, then puppet will switch environments, and pluginsync in a different environment, potentially receiving different versions of type/provider code. Puppet can reload new versions of types and providers, but can't reload helper code required by type/providers. This can result in newer providers calling methods in older helper code still in memory.

      This problem may occur when the classifier uses pluginsynced facts to determine environment, a node doesn't have that fact yet (or a wrong version of the fact), puppetdb doesn't have a facts for that node in the "final" environment, the node attempts to get a catalog, and there are different versions of a module in different environments.

      The problem may not occur, or may not be noticed if the different versions of modules don't conflict in a way that causes an exception.

      The problem won't occur if the agent resolves the environment during the node request (before pluginsync). However, the agent doesn't send facts with the node request (a long standing request), so this generally applies to nodes that have already resolved the environment once.

      Here's a stacktrace showing the inifile type/provider being loaded during the environment resolution process (using 5.5.17):

      # puppet --version
      		 
      5.5.17
      		 
      # rm -rf /opt/puppetlabs/puppet/cache/lib/ && puppet config set certname "agent55x$RANDOM" --section main && puppet agent -t --trace --logdest /tmp/out.log
      		 
       
      		 
      --> #0  #<Class:Puppet::Util::Autoload>.mark_loaded(name#String, file#String) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:43
      		 
          #1  #<Class:Puppet::Util::Autoload>.load_file(name#String, env#Puppet::Node::Environment::Remote) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:70
      		 
          #2  Puppet::Util::Autoload.load(name#Symbol, env#Puppet::Node::Environment::Remote) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:198
      		 
          #3  Puppet::MetaType::Manager.type(name#Symbol) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/metatype/manager.rb:174
      		 
          #4  #<Class:Puppet::Resource>.resource_type(type#String, title#String, environment#Puppet::Node::Environment::Remote) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:360
      		 
          #5  Puppet::Resource.resource_type at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:348
      		 
          #6  Puppet::Resource.parse_title at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:656
      		 
          #7  Puppet::Resource.to_hash at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:392
      		 
          #8  Puppet::Resource.to_data_hash at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:93
      		 
          #9  block in Puppet::Resource::Catalog.block in to_data_hash at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:506
      		 
          ͱ-- #10 Array.map at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:506
      		 
          #11 Puppet::Resource::Catalog.to_data_hash at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:506
      		 
          #12 Puppet::Network::FormatSupport.to_json(*args#Array) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/format_support.rb:100
      		 
          #13 Puppet::Network::Format.render(instance#Puppet::Resource::Catalog) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/format.rb:65
      		 
          #14 Puppet::Network::FormatSupport.render(format#String) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/format_support.rb:106
      		 
          #15 Puppet::Resource::Catalog::Json.to_json(object#Puppet::Resource::Catalog) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/json.rb:19
      		 
          #16 block in Puppet::Indirector::JSON.block in save(request#Puppet::Indirector::Request) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/json.rb:17
      		 
          #17 #<Class:Puppet::Util>.replace_file(file#Pathname, default_mode#Integer, &block#Proc) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:603
      		 
          #18 Puppet::Indirector::JSON.save(request#Puppet::Indirector::Request) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/json.rb:17
      		 
          #19 Puppet::Indirector::Indirection.find(key#String, options#Hash) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:204
      		 
          #20 block in Puppet configuration client.block in retrieve_new_catalog(query_options#Hash) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:455

      To fix this problem the conversion of resource catalog to RAL catalog and writing the cached catalog need to occur after the environment converges.

      Attachments

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. PUP-5600 Add "none" parameter to catalog_cache_terminus configuration reference

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Accepted

          Activity

            People

              josh Josh Cooper
              josh Josh Cooper
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Zendesk Support