Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-10818

Cached catalog contains the result of deferred evaluation instead of the deferred function

    XMLWordPrintable

    Details

    • Template:
      PUP Bug Template
    • Acceptance Criteria:
      Hide

      Running puppet agentĀ -t --use_cached_catalog should reevaluate deferred functions in the cached catalog.

      On the server create a site.pp. Due to PUP-10820, we have to defer casting the TimeStamp to a String:

      # cat /etc/puppetlabs/code/environments/production/manifests/site.pp
      notify { "deferred":
        message => Deferred("new", [String, Deferred("new", [TimeStamp])])
      }
      

      On the agent, run:

      # puppet agent -t
      Info: Using configured environment 'production'
      Info: Retrieving pluginfacts
      Info: Retrieving plugin
      Info: Retrieving locales
      Info: Caching catalog for localhost
      Info: Applying configuration version '1607632459'
      Notice: 2020-12-10T20:34:20.056492000 UTC
      Notice: /Stage[main]/Main/Notify[deferred]/message: defined 'message' as '2020-12-10T20:34:20.056492000 UTC'
      Notice: Applied catalog in 0.01 seconds
      

      Run the agent again with cached catalog, and the deferred message timestamps should be different

      # puppet agent -t --use_cached_catalog
      Info: Using cached catalog from environment 'production'
      Info: Caching catalog for localhost
      Info: Applying configuration version '1607632459'
      Notice: 2020-12-10T20:34:26.732950000 UTC
      Notice: /Stage[main]/Main/Notify[deferred]/message: defined 'message' as '2020-12-10T20:34:26.732950000 UTC'
      Notice: Applied catalog in 0.01 seconds
      

      Show
      Running puppet agentĀ -t --use_cached_catalog should reevaluate deferred functions in the cached catalog. On the server create a site.pp. Due to PUP-10820 , we have to defer casting the TimeStamp to a String: # cat /etc/puppetlabs/code/environments/production/manifests/site.pp notify { "deferred": message => Deferred("new", [String, Deferred("new", [TimeStamp])]) } On the agent, run: # puppet agent -t Info: Using configured environment 'production' Info: Retrieving pluginfacts Info: Retrieving plugin Info: Retrieving locales Info: Caching catalog for localhost Info: Applying configuration version '1607632459' Notice: 2020-12-10T20:34:20.056492000 UTC Notice: /Stage[main]/Main/Notify[deferred]/message: defined 'message' as '2020-12-10T20:34:20.056492000 UTC' Notice: Applied catalog in 0.01 seconds Run the agent again with cached catalog, and the deferred message timestamps should be different # puppet agent -t --use_cached_catalog Info: Using cached catalog from environment 'production' Info: Caching catalog for localhost Info: Applying configuration version '1607632459' Notice: 2020-12-10T20:34:26.732950000 UTC Notice: /Stage[main]/Main/Notify[deferred]/message: defined 'message' as '2020-12-10T20:34:26.732950000 UTC' Notice: Applied catalog in 0.01 seconds
    • Team:
      Coremunity
    • Sprint:
      Platform Core KANBAN
    • Method Found:
      Needs Assessment
    • Zendesk Ticket IDs:
      42173
    • Zendesk Ticket Count:
      1
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Puppet 6.12.0 introduced a regression that caused the result of a deferred function to be stored in the cached catalog. As a result, an agent running with a cached catalog, would not re-evaluate the deferred function. Now it does.
    • QA Risk Assessment:
      Needs Assessment

      Description

      The cached catalog contains the result of evaluating the deferred function, instead of the function. So if the agent applies a cached catalog, it will always apply the result of the deferred function when the cached catalog was saved instead of when the agent runs.

      To reproduce, create:

      cat /etc/puppetlabs/code/environments/production/manifests/site.pp
      $epp_var = {
        'deferred_function' => Deferred('to_yaml', [{'test' => 'foo'}])
      }
      file { '/tmp/deferred_function.txt':
        content => Deferred('inline_epp', ['<%= $deferred_function -%>', $epp_var]),
      }
      

      In 6.11.0, we correctly cache the deferred function:

      $ cat ~/.puppetlabs/opt/puppet/cache/client_data/catalog/$(facter fqdn).json| jq .
      ...
        {
            "type": "File",
            "title": "/tmp/deferred_function.txt",
            "tags": [
              "file",
              "class"
            ],
            "file": "/etc/puppetlabs/code/environments/production/manifests/site.pp",
            "line": 4,
            "exported": false,
            "parameters": {
              "content": {
                "__ptype": "Deferred",
                "name": "inline_epp",
                "arguments": [
                  "<%= $deferred_function -%>",
                  {
                    "deferred_function": {
                      "__ptype": "Deferred",
                      "name": "to_yaml",
                      "arguments": [
                        {
                          "test": "foo"
                        }
                      ]
                    }
                  }
                ]
              }
            }
      

      and the file contains the result of evaluating the deferred function:

      $ cat /tmp/deferred_function.txt
       ---
      test: foo
      

      In 6.12.0 the cached catalog is incorrect:

          {
            "type": "File",
            "title": "/tmp/deferred_function.txt",
            "tags": [
              "file",
              "class"
            ],
            "file": "/etc/puppetlabs/code/environments/production/manifests/site.pp",
            "line": 4,
            "exported": false,
            "parameters": {
              "content": "---\ntest: foo\n"
            }
          }
      

      The bug was introduced in a17761d60b2e5d42cc01a36e1d6ba3d790c9dcf8, as there was a semantic conflict due to the environment convergence changes in 5.5.x and the deferred functionality in 6.x.

        Attachments

          Activity

            People

            Assignee:
            josh Josh Cooper
            Reporter:
            josh Josh Cooper
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Zendesk Support