Uploaded image for project: 'Modules'
  1. Modules
  2. MODULES-10763

apt: apt_update always results in a corrective change report, even when no change is made

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Template:
      MODULES Bug Template
    • Acceptance Criteria:
      Hide

      Idempotent apt resources should not report corrective changes when no changes were made.

      Show
      Idempotent apt resources should not report corrective changes when no changes were made.
    • Team:
      Modules
    • Method Found:
      Customer Feedback
    • Zendesk Ticket IDs:
      40262
    • Zendesk Ticket Count:
      1
    • QA Risk Assessment:
      Needs Assessment

      Description

      Basic Info
      Module Version: all; reproduced on 7.2.0 and 7.4.2
      Puppet Version: reproduced on 6.15 and 6.16
      OS Name/Version: reproduced on Debian 9, Debian 10, and Ubuntu 18.04

      If an apt resource is set to run the apt_update exec, every Puppet run that applies it reports a corrective change, regardless of whether any caches were updated.

      The apt-update command, and the actions it performs, is inherently idempotent. We even use it as an example in the Puppet docs of an inherently idempotent command: https://puppet.com/docs/puppet/6.17/type.html#exec

      Any command in an exec resource must be able to run multiple times without causing harm — that is, it must be idempotent. There are three main ways for an exec to be idempotent:

      • The command itself is already idempotent. (For example, apt-get update.)

      However, the apt module's implementation of apt-get update falsely suggests otherwise.

      The module README also implies that setting the resource's loglevel to debug (or higher than whatever the agent's log_level setting is) will prevent corrective changes from appearing in agent reports: https://forge.puppet.com/puppetlabs/apt#update-the-list-of-packages

      When Exec['apt_update'] is triggered, it generates a Notice message. Because the default logging level for agents is notice, this causes the repository update to appear in logs and agent reports. Some tools, such as The Foreman, report the update notice as a significant change. To eliminate these updates from reports, set the loglevel metaparameter for Exec['apt_update'] above the agent logging level:

      class { 'apt':
        update => {
          frequency => 'daily',
          loglevel  => 'debug',
        },
      }

      However, the PE console still reports a corrective change even when the resource's loglevel is debug or higher, because the corrective change is submitted in the report and the logs don't matter. Aside from the docs not being relevant to the PE console, it's also not true for Foreman as documented: https://github.com/puppetlabs/puppetlabs-apt/pull/690#issuecomment-347472196

      I've defined apt::update with loglevel set to debug and I actually don't see any notes from this resource in the report. But Foreman still shows the status of such Puppet runs as "updated" / "changed", so it's always blue, not green.

      ...

      yeah, I had thought it would properly suppress that in Foreman but it doesn't. I left the MR in place because, well, it's nice not to have it at stdout.

      The apt_update exec resource in question: https://github.com/puppetlabs/puppetlabs-apt/blob/2e794c96ce78485bba77183eb57544a1e7415c6f/manifests/update.pp#L59-L67

        exec { 'apt_update':
          command     => "${::apt::provider} update",
          loglevel    => $::apt::_update['loglevel'],
          logoutput   => 'on_failure',
          refreshonly => $_refresh,
          timeout     => $::apt::_update['timeout'],
          tries       => $::apt::_update['tries'],
          try_sleep   => 1
        }

      Steps to reproduce

      1. Install PE 2019.8.0 on Ubuntu 18.04.
      2. Install the puppetlabs-apt module.
      3. Set the default apt resource behavior to always update with a debug loglevel:

      class { 'apt':
        update => {
          frequency => 'always',
          loglevel => 'debug',
        }
      }

      4. Run the Puppet agent multiple times

      Desired Behavior:

      If there's no corrective change, don't report one.

      Actual Behavior:

      Every run on every node with an apt resource reports a corrective change regardless of the lack of changes on the node, making all reports' statuses effectively meaningless; nodes with changes are indistinguishable from unchanged nodes.

      apt_update,Exec,corrective change,notrun,"[""0""]",/etc/puppetlabs/code/environments/production/modules/apt/manifests/update.pp,59

      Following the docs suggestion of changing the log level removes the result from the notice-level agent output:

      root@scenic-scandal:/etc/puppetlabs# while true; do puppet agent --test; sleep 5; done
      Info: Using configured environment 'production'
      Info: Retrieving pluginfacts
      Info: Retrieving plugin
      Info: Retrieving locales
      Info: Loading facts
      Info: Caching catalog for scenic-scandal.delivery.puppetlabs.net
      Info: Applying configuration version '1596485482'
      Notice: Applied catalog in 20.12 seconds
      

      but not the report, so it will still always be reported by the PE console and Foreman, contradicting the docs:

        Exec[apt_update]:
          title: apt_update
          file: "/etc/puppetlabs/code/environments/production/modules/apt/manifests/update.pp"
          line: 59
          resource: Exec[apt_update]
          resource_type: Exec
          provider_used: posix
          containment_path:
          - Stage[main]
          - Apt::Update
          - Exec[apt_update]
          evaluation_time: 2.001353447
          tags:
          - exec
          - apt_update
          - class
          - apt::update
          - apt
          - update
          - node
          - default
          time: '2020-08-03T19:55:57.523108153+00:00'
          failed: false
          failed_to_restart: false
          changed: true
          out_of_sync: true
          skipped: false
          change_count: 1
          out_of_sync_count: 1
          events:
          - audited: false
            property: returns
            previous_value: notrun
            desired_value:
            - '0'
            historical_value:
            message: executed successfully (corrective)
            name: executed_command
            status: success
            time: '2020-08-03T19:55:57.523287195+00:00'
            redacted:
            corrective_change: true
          corrective_change: true
      

      Suggested fix

      I understand that exec resources always report a corrective change if they successfully run. An exec resource is therefore not appropriate for the apt_update feature because the result (to always report a change) is inherently deceptive (apt-get update can run without performing a corrective change).

      apt_update should report a corrective change only when apt-get update results in an updated cache, and the module documentation should not imply that there is a supported way to silence a corrective change in reports.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              paula Paula Muir
              Reporter:
              garrett.guillotte Garrett Guillotte
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support