[PUP-1079] Exported resources is exporting to the database with --noop flag Created: 2013/12/16  Updated: 2018/03/14

Status: Needs Information
Project: Puppet
Component/s: Catalog Application
Affects Version/s: PUP 3.3.1
Fix Version/s: None

Type: Bug Priority: Normal
Reporter: redmine.exporter Assignee: Unassigned
Resolution: Unresolved Votes: 9
Labels: customer, needs_decision, redmine
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Blocks
Relates
relates to PUP-7541 Explore removing export / collect / v... Open
relates to DOCUMENT-832 Puppetdb node-ttl and node-purge-ttl ... Closed
Template:
Epic Link: Transaction Events
QA Contact: Narmadha Perumal

 Description   

When I run puppet with the noop flag resources are being exported to the
db.

I would expect it to simulate the transaction with the db...

This is causing things like nagios host resources being created before the plugins get there, so I get invalid alerts.

-Andrew



 Comments   
Comment by Jason Antman [ 2014/01/11 ]

There's a LOT of history in redmine for this. For some of us (myself included, and we're a PE customer) this is soon going to be a really big problem for us (i.e. critical issue).

Comment by Christophe Vanlancker [ 2014/02/26 ]

Quick recap:

There is a valid use-case for both behaviours:

  • --noop exports resources but doesn't affect locally (migration to PuppetDB and have everything pre-populated)
  • --noop (in combination with another option likely) doesn't export resources (for testing changes which could potentially remove necessary exported resources like apache vhosts, nagios_hosts, firewall rules and the likes)

This also affects infrastructures where PuppetDB is in use.

Comment by Rob Nelson [ 2014/12/04 ]

Verified with Jason that this is now a critical issue, --noop is now unusable. This appears to need some engineering decisions on how to go forward or to leave as is.

Comment by Deepak Giridharagopal [ 2014/12/19 ]

This is something we should look at sooner rather than later, though the obvious conundrum is that people are pretty attached to both behaviors Christophe identified.

For the folks that want noop to no longer export resources, a few questions:

  • should facts get exported during a noop run?
  • should this affect all resources marked as noop (say with the noop metaparameter)?
  • if we had, say, an option in the puppetdb terminus config to prevent noop stuff from getting submitted, would that suffice?

Pinging Ryan Senior, Ken Barber, and Wyatt Alt for awareness.

Comment by Jason Antman [ 2014/12/20 ]

Deepak Giridharagopal Yeah, that is a bit of a dilemma.

Speaking for myself:

  • Whether or not facts get exported isn't a big deal to me, as that shouldn't affect my actual catalogs. People who are using something like puppetdbquery might feel otherwise, and want to prevent fact updates as well. I can see arguments in favor of 'yes' or 'no' about this, but don't have a strong opinion for my use case.
  • Resources marked as noop... I'm honestly not sure. I'm leaning towards no (it shouldn't affect them) on the assumption that if a resource is exported with noop => true, it will end up (correctly) being a noop on the collecting node, but I haven't tested this.
  • Yes. I understand that this would be a change to current behavior, and I'd be perfectly happy with making that a configuration option, so those of us who want the behavior to change have to explicitly specify it, so long as it's mentioned clearly in the Exported Resouces docs (one of my big gripes with this ticket is that as far as I know, there's nothing clear in the docs to explain that --noop runs still export resources).
Comment by Rob Nelson [ 2014/12/20 ]

Deepak Giridharagopal For me, I'd say that noop, by definition, should not be expected to effect actual changes, so both facts and resources should not be exported. Since the master or agent in a masterless setup are what send the data to puppetdb, I feel any change in behavior should be made there, including config options. Otherwise, data could be sent over the wire that shouldn't be, even if puppetdb discards it. Even with encryption that's still putting some data at risk and increasing the network load.

As mentioned, the docs don't explain this behavior and I would be surprised if new users expected this behavior, so it may be beneficial to set any new option to default to off in the next major release.

Comment by Henrik Lindberg [ 2017/05/17 ]

This seems to have been critical and urgent in 2014 - what happened since then? Does this work as expected now (export of noop-resources)?
Ping Russell Mull, Josh Cooper

Comment by Rob Nelson [ 2017/05/17 ]

I still observe the behavior that noop runs actually export resources which may not be realized. As an example, if I run with --noop on the first run of an agent, it exports the ssh key, and then all the other nodes in my system start to collect the ssh key. That's not destructive like monitoring resources can be, but it's still not desired behavior.

Comment by Russell Mull [ 2017/05/18 ]

The obvious fix is to just not put the catalog from a --noop run into puppetdb. While I think that would be easy enough to implement, I'm worried that someone's workflow depends on this. Maybe we could just not send --noop catalogs by default but have an option to send them?

Comment by Dimitri Tischenko [ 2018/03/14 ]

(Comment triggered by DOCUMENT-832) FWIW, I have a customer who is happy with the --noop mode preventing exported resources from expiration. From their perspective, if the agent does something useful on the node and a report gets sent, it's OK for exported resources to be updated.

Generated at Fri Aug 07 09:02:10 PDT 2020 using Jira 8.5.2#805002-sha1:a66f9354b9e12ac788984e5d84669c903a370049.