Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-6295

Include corrective_change field in reports

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: PUP 4.6.0
    • Component/s: None
    • Labels:
      None
    • Template:
    • Story Points:
      8
    • Sprint:
      CR 2016-05-18, CR 2016-06-01, CR 2016-06-15, CR 2016-06-29, CR 2016-07-13, CR 2016-07-27, CR 2016-08-10 FC
    • Release Notes:
      New Feature
    • Release Notes Summary:
      Hide
      This release adds a new report event field called ‘corrective_change’ that is designed to detect manual change that has been corrected by a Puppet run.

      This feature should help users be able to detect when an unexpected change occurred outside of Puppet, allowing better auditing and understanding around changes.

      This feature achieves this by storing the last best known value for each property that is applied by Puppet, and comparing that against the values in the next Puppet run.

      As part of the requirement is to store values, this feature also introduces a new local storage mechanism, and introduces a new configuration option ‘transactionstorefile’ which points at the YAML file used. This storage is queried for each run for old values during comparison, and persists the new values for next transaction to do its calculation.

      While we’ve done our best to ensure this feature works well, this entire process is still in development and is quite new, and has some known points of interest:

      * For noop events in particular, these a treated especially. We will continue to return a positive ‘corrective_change’ flag if there will be a corrective_change, if Puppet was to be ran in enforcement mode.
      * Today, idempotency issues are raised as ‘corrective_change’ because we can’t tell the difference. An idempotency issue is when either a provider has a bug applying a change twice consistently, or when Puppet DSL code (or external dependences) is used that has idempotency issues (common in service, and exec resources for example). For properties that have known idempotency issues, we have introduced an ‘idempotent’ flag for declaring that corrective_change calculation can be skipped. An example can be found in the notify provider, as the message property on notify has been a long-standing and well known non-idempotent property: https://github.com/puppetlabs/puppet/blob/master/lib/puppet/type/notify.rb#L9
      * The API for comparison in Puppet for older arbitrary values is brittle, and some custom Puppet types may show the incorrect thing for corrective_change as a consequence. We ask users to raise bugs when these cases are discovered.
      * For now, if there is any exception during value comparison we will still continue to run Puppet, but return an error to the user so it can be debugged. Also, we return a ‘corrective_change’ as nil instead, indicating an unknown state. Any cases where this occurs should be raised as bugs to the appropriate project.
      * Comparison of secret values is currently out of scope. For us to ensure we could compare these values, we would have to store them in doing so leak secret information. We’ve decided we would step back from this problem, and for now secret properties are not supported for being flagged as corrective.

      Along with flagging each event with the ‘corrective_change’ field, we also flag a resource that has such events, and the entire report. Metrics have been included to allow report consumers to see a count of events that are marked as ‘corrective_change’ also.
      Show
      This release adds a new report event field called ‘corrective_change’ that is designed to detect manual change that has been corrected by a Puppet run. This feature should help users be able to detect when an unexpected change occurred outside of Puppet, allowing better auditing and understanding around changes. This feature achieves this by storing the last best known value for each property that is applied by Puppet, and comparing that against the values in the next Puppet run. As part of the requirement is to store values, this feature also introduces a new local storage mechanism, and introduces a new configuration option ‘transactionstorefile’ which points at the YAML file used. This storage is queried for each run for old values during comparison, and persists the new values for next transaction to do its calculation. While we’ve done our best to ensure this feature works well, this entire process is still in development and is quite new, and has some known points of interest: * For noop events in particular, these a treated especially. We will continue to return a positive ‘corrective_change’ flag if there will be a corrective_change, if Puppet was to be ran in enforcement mode. * Today, idempotency issues are raised as ‘corrective_change’ because we can’t tell the difference. An idempotency issue is when either a provider has a bug applying a change twice consistently, or when Puppet DSL code (or external dependences) is used that has idempotency issues (common in service, and exec resources for example). For properties that have known idempotency issues, we have introduced an ‘idempotent’ flag for declaring that corrective_change calculation can be skipped. An example can be found in the notify provider, as the message property on notify has been a long-standing and well known non-idempotent property: https://github.com/puppetlabs/puppet/blob/master/lib/puppet/type/notify.rb#L9 * The API for comparison in Puppet for older arbitrary values is brittle, and some custom Puppet types may show the incorrect thing for corrective_change as a consequence. We ask users to raise bugs when these cases are discovered. * For now, if there is any exception during value comparison we will still continue to run Puppet, but return an error to the user so it can be debugged. Also, we return a ‘corrective_change’ as nil instead, indicating an unknown state. Any cases where this occurs should be raised as bugs to the appropriate project. * Comparison of secret values is currently out of scope. For us to ensure we could compare these values, we would have to store them in doing so leak secret information. We’ve decided we would step back from this problem, and for now secret properties are not supported for being flagged as corrective. Along with flagging each event with the ‘corrective_change’ field, we also flag a resource that has such events, and the entire report. Metrics have been included to allow report consumers to see a count of events that are marked as ‘corrective_change’ also.

      Description

      This is necessary for PuppetDB to synchronously process an agent's report and catalog in order to make inferences on deviation between expected and realized state. The easiest thing to do would probably be to stick the catalog under a :catalog key in the report, but more involved format changes could be explored.

        Attachments

          Issue Links

            Activity

              jsd-sla-details-panel

                People

                • Assignee:
                  james.stocks James Stocks
                  Reporter:
                  wyatt Wyatt Alt
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: