Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-10572

Puppet cannot apply a cached catalog containing binary rich data

    XMLWordPrintable

    Details

    • Template:
      PUP Bug Template
    • Team:
      Coremunity
    • Sprint:
      Platform Core KANBAN
    • Method Found:
      Needs Assessment
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Puppet could not apply a cached catalog if it contained Binary or Deferred data types, causing it to always fallback to requesting a new catalog.
    • QA Risk Assessment:
      Needs Assessment

      Description

      If a cached catalog contains rich data (such as Binary) then puppet will fail to load the cached catalog and fallback to loading a new catalog. This doesn't happen with Sensitive because that data type is handled differently:

      Cached catalog for vernal-creator.delivery.puppetlabs.net failed: Could not parse JSON data for catalog vernal-creator.delivery.puppetlabs.net: Could not intern from rich_data_json: Internal Error: Puppet Context ':loaders' missing
      

      To reproduce, create the following resource:

      notify { 'message':
        message => Binary.new('aGk=')
      }
      

      Note aGk= is base64 for hi

      Run the agent once to cache the catalog:

      [root@vernal-creator ~]# puppet agent --no-daemonize --onetime --verbose
      Info: Using configured environment 'production'
      Info: Retrieving pluginfacts
      Info: Retrieving plugin
      Info: Retrieving locales
      Info: Caching catalog for vernal-creator.delivery.puppetlabs.net
      Info: Applying configuration version '1593554072'
      Notice: hi
      Notice: /Stage[main]/Main/Notify[message]/message: defined 'message' as 'hi'
      Notice: Applied catalog in 0.01 seconds
      

      Run it again, but this time force the agent to use a cached catalog:

      [root@vernal-creator ~]# puppet agent --no-daemonize --onetime --use_cached_catalog --verbose --trace
      Error: Cached catalog for vernal-creator.delivery.puppetlabs.net failed: Could not parse JSON data for catalog vernal-creator.delivery.puppetlabs.net: Could not intern from rich_data_json: Internal Error: Puppet Context ':loaders' missing
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loaders.rb:176:in `loaders'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/loaders.rb:89:in `find_loader'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/types/types.rb:319:in `create'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:216:in `pcore_type_hash_to_value'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:139:in `block in initialize'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:155:in `convert'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:157:in `block (3 levels) in convert'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:171:in `with'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:157:in `block (2 levels) in convert'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:157:in `each_pair'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:157:in `block in convert'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:179:in `with_value'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:195:in `build'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:157:in `convert'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/from_data_converter.rb:72:in `convert'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:55:in `initialize_from_hash'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:42:in `from_data_hash'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:437:in `block in from_data_hash'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:436:in `collect'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:436:in `from_data_hash'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/formats.rb:216:in `block in data_to_instance'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:314:in `override'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/formats.rb:214:in `data_to_instance'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/formats.rb:189:in `block in intern'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:314:in `override'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/formats.rb:188:in `intern'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/format_support.rb:12:in `convert_from'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/json.rb:11:in `from_json'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/json.rb:62:in `load_json_from_file'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/json.rb:10:in `find'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:265:in `find_in_cache'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:215:in `find'
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:497:in `block in retrieve_catalog_from_cache'
      

      Since the cached catalog fails, puppet will fallback to requesting a new catalog (though it defeats the purpose of using a cached catalog):

      # continued from above
      Info: Using configured environment 'production'
      Info: Retrieving pluginfacts
      Info: Retrieving plugin
      Info: Retrieving locales
      Info: Caching catalog for vernal-creator.delivery.puppetlabs.net
      Info: Applying configuration version '1593554078'
      Notice: hi
      Notice: /Stage[main]/Main/Notify[message]/message: defined 'message' as 'hi'
      Notice: Applied catalog in 0.01 seconds
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              josh Josh Cooper
              Reporter:
              josh Josh Cooper
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support