Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-10160

Puppet should not load types and providers during environment convergence

    XMLWordPrintable

    Details

    • Template:
      PUP Bug Template
    • Team:
      Coremunity
    • Sprint:
      Platform Core KANBAN
    • Method Found:
      Needs Assessment
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Hide
      Previously puppet agents could fail to apply a catalog if the agent switched environments based on node classification and there were different versions of a module in those environments. As a result of this change, an agent should converge to its server-assigned environment more quickly, only load types and providers once, and only update its cached catalog after the environment converges.
      Show
      Previously puppet agents could fail to apply a catalog if the agent switched environments based on node classification and there were different versions of a module in those environments. As a result of this change, an agent should converge to its server-assigned environment more quickly, only load types and providers once, and only update its cached catalog after the environment converges.
    • QA Risk Assessment:
      Needs Assessment

      Description

      During environment convergence, puppet loads types and providers for pluginsynced modules, potentially multiple times. Type/provider loading is triggered when writing the cached catalog and/or when converting the resource catalog to a RAL catalog.

      However, if the environment doesn't converge the first time, then puppet will switch environments, and pluginsync in a different environment, potentially receiving different versions of type/provider code. Puppet can reload new versions of types and providers, but can't reload helper code required by type/providers. This can result in newer providers calling methods in older helper code still in memory.

      This problem may occur when the classifier uses pluginsynced facts to determine environment, a node doesn't have that fact yet (or a wrong version of the fact), puppetdb doesn't have a facts for that node in the "final" environment, the node attempts to get a catalog, and there are different versions of a module in different environments.

      The problem may not occur, or may not be noticed if the different versions of modules don't conflict in a way that causes an exception.

      The problem won't occur if the agent resolves the environment during the node request (before pluginsync). However, the agent doesn't send facts with the node request (a long standing request), so this generally applies to nodes that have already resolved the environment once.

      Here's a stacktrace showing the inifile type/provider being loaded during the environment resolution process (using 5.5.17):

      # puppet --version
      5.5.17
      # rm -rf /opt/puppetlabs/puppet/cache/lib/ && puppet config set certname "agent55x$RANDOM" --section main && puppet agent -t --trace --logdest /tmp/out.log
       
      --> #0  #<Class:Puppet::Util::Autoload>.mark_loaded(name#String, file#String) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:43
          #1  #<Class:Puppet::Util::Autoload>.load_file(name#String, env#Puppet::Node::Environment::Remote) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:70
          #2  Puppet::Util::Autoload.load(name#Symbol, env#Puppet::Node::Environment::Remote) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:198
          #3  Puppet::MetaType::Manager.type(name#Symbol) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/metatype/manager.rb:174
          #4  #<Class:Puppet::Resource>.resource_type(type#String, title#String, environment#Puppet::Node::Environment::Remote) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:360
          #5  Puppet::Resource.resource_type at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:348
          #6  Puppet::Resource.parse_title at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:656
          #7  Puppet::Resource.to_hash at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:392
          #8  Puppet::Resource.to_data_hash at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:93
          #9  block in Puppet::Resource::Catalog.block in to_data_hash at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:506
          ͱ-- #10 Array.map at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:506
          #11 Puppet::Resource::Catalog.to_data_hash at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:506
          #12 Puppet::Network::FormatSupport.to_json(*args#Array) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/format_support.rb:100
          #13 Puppet::Network::Format.render(instance#Puppet::Resource::Catalog) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/format.rb:65
          #14 Puppet::Network::FormatSupport.render(format#String) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/format_support.rb:106
          #15 Puppet::Resource::Catalog::Json.to_json(object#Puppet::Resource::Catalog) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/json.rb:19
          #16 block in Puppet::Indirector::JSON.block in save(request#Puppet::Indirector::Request) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/json.rb:17
          #17 #<Class:Puppet::Util>.replace_file(file#Pathname, default_mode#Integer, &block#Proc) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:603
          #18 Puppet::Indirector::JSON.save(request#Puppet::Indirector::Request) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/json.rb:17
          #19 Puppet::Indirector::Indirection.find(key#String, options#Hash) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:204
          #20 block in Puppet configuration client.block in retrieve_new_catalog(query_options#Hash) at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:455
      

      To fix this problem the conversion of resource catalog to RAL catalog and writing the cached catalog need to occur after the environment converges.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              josh Josh Cooper
              Reporter:
              josh Josh Cooper
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support