apt: apt_update always results in a corrective change report, even when no change is made

Basic Info
Module Version: all; reproduced on 7.2.0 and 7.4.2
Puppet Version: reproduced on 6.15 and 6.16
OS Name/Version: reproduced on Debian 9, Debian 10, and Ubuntu 18.04

If an apt resource is set to run the apt_update exec, every Puppet run that applies it reports a corrective change, regardless of whether any caches were updated.

The apt-update command, and the actions it performs, is inherently idempotent. We even use it as an example in the Puppet docs of an inherently idempotent command: https://puppet.com/docs/puppet/6.17/type.html#exec

Any command in an exec resource must be able to run multiple times without causing harm — that is, it must be idempotent. There are three main ways for an exec to be idempotent:

• The command itself is already idempotent. (For example, apt-get update.)

However, the apt module's implementation of apt-get update falsely suggests otherwise.

The module README also implies that setting the resource's loglevel to debug (or higher than whatever the agent's log_level setting is) will prevent corrective changes from appearing in agent reports: https://forge.puppet.com/puppetlabs/apt#update-the-list-of-packages

When Exec['apt_update'] is triggered, it generates a Notice message. Because the default logging level for agents is notice, this causes the repository update to appear in logs and agent reports. Some tools, such as The Foreman, report the update notice as a significant change. To eliminate these updates from reports, set the loglevel metaparameter for Exec['apt_update'] above the agent logging level:

class { 'apt':

update => {

frequency => 'daily',

loglevel => 'debug',

},

}

However, the PE console still reports a corrective change even when the resource's loglevel is debug or higher, because the corrective change is submitted in the report and the logs don't matter. Aside from the docs not being relevant to the PE console, it's also not true for Foreman as documented: https://github.com/puppetlabs/puppetlabs-apt/pull/690#issuecomment-347472196

I've defined apt::update with loglevel set to debug and I actually don't see any notes from this resource in the report. But Foreman still shows the status of such Puppet runs as "updated" / "changed", so it's always blue, not green.

...

yeah, I had thought it would properly suppress that in Foreman but it doesn't. I left the MR in place because, well, it's nice not to have it at stdout.

The apt_update exec resource in question: https://github.com/puppetlabs/puppetlabs-apt/blob/2e794c96ce78485bba77183eb57544a1e7415c6f/manifests/update.pp#L59-L67

  exec { 'apt_update':

    command     => "${::apt::provider} update",

    loglevel    => $::apt::_update['loglevel'],

    logoutput   => 'on_failure',

    refreshonly => $_refresh,

    timeout     => $::apt::_update['timeout'],

    tries       => $::apt::_update['tries'],

    try_sleep   => 1

  }

Steps to reproduce

1. Install PE 2019.8.0 on Ubuntu 18.04.
2. Install the puppetlabs-apt module.
3. Set the default apt resource behavior to always update with a debug loglevel:

class { 'apt':

  update => {

    frequency => 'always',

    loglevel => 'debug',

  }

}

4. Run the Puppet agent multiple times

Desired Behavior:

If there's no corrective change, don't report one.

Actual Behavior:

Every run on every node with an apt resource reports a corrective change regardless of the lack of changes on the node, making all reports' statuses effectively meaningless; nodes with changes are indistinguishable from unchanged nodes.

apt_update,Exec,corrective change,notrun,"[""0""]",/etc/puppetlabs/code/environments/production/modules/apt/manifests/update.pp,59

Following the docs suggestion of changing the log level removes the result from the notice-level agent output:

root@scenic-scandal:/etc/puppetlabs# while true; do puppet agent --test; sleep 5; done

Info: Using configured environment 'production'

Info: Retrieving pluginfacts

Info: Retrieving plugin

Info: Retrieving locales

Info: Loading facts

Info: Caching catalog for scenic-scandal.delivery.puppetlabs.net

Info: Applying configuration version '1596485482'

Notice: Applied catalog in 20.12 seconds

but not the report, so it will still always be reported by the PE console and Foreman, contradicting the docs:

  Exec[apt_update]:

    title: apt_update

    file: "/etc/puppetlabs/code/environments/production/modules/apt/manifests/update.pp"

    line: 59

    resource: Exec[apt_update]

    resource_type: Exec

    provider_used: posix

    containment_path:

    - Stage[main]

    - Apt::Update

    - Exec[apt_update]

    evaluation_time: 2.001353447

    tags:

    - exec

    - apt_update

    - class

    - apt::update

    - apt

    - update

    - node

    - default

    time: '2020-08-03T19:55:57.523108153+00:00'

    failed: false

    failed_to_restart: false

    changed: true

    out_of_sync: true

    skipped: false

    change_count: 1

    out_of_sync_count: 1

    events:

    - audited: false

      property: returns

      previous_value: notrun

      desired_value:

      - '0'

      historical_value:

      message: executed successfully (corrective)

      name: executed_command

      status: success

      time: '2020-08-03T19:55:57.523287195+00:00'

      redacted:

      corrective_change: true

    corrective_change: true

Suggested fix

I understand that exec resources always report a corrective change if they successfully run. An exec resource is therefore not appropriate for the apt_update feature because the result (to always report a change) is inherently deceptive (apt-get update can run without performing a corrective change).

apt_update should report a corrective change only when apt-get update results in an updated cache, and the module documentation should not imply that there is a supported way to silence a corrective change in reports.