[BOLT-836] Support Hiera lookup() in plans outside of apply blocks Created: 2018/09/13  Updated: 2019/08/26  Resolved: 2019/08/26

Status: Closed
Project: Puppet Task Runner
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Normal
Reporter: Nick Maludy Assignee: Unassigned
Resolution: Won't Do Votes: 3
Labels: closed-github-move, ghm
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Template:
QA Risk Assessment: Needs Assessment

 Description   

Feature Request

It would be handy to support the hiera lookup() function within a plan.

Open Question

What goes into the hierarchy for a plan? Without facter how are those values set? 

Usecases

Lookup different values to use the same plan in different environments or contexts

As a plan author i need to store data about how to execute the plan that may be environment or deployment specific. These settings would be looked up from the "perspective" of the bolt controller node. Example: may need different API keys, credentials, etc to use for API calls. Example 2: may be deploying in a different manner based on a setting in a config that is easy to switch.

Use hiera to lookup task parameters for different targets

As a plan author i need to perform actions differently depending on certain facts and/or attributes of a node. These facts may be gather by the `facts` task. From these facts/attributes i then need to lookup() data to determine how the action should be run.  These facts would be looked up from the "perspective" of specific targets. Example: I'm deploying a software pacakge against a variety of hosts, some may be 32-bit some 64-bit. I need to deploy different versions of software to the 32-bit systems. I would run a 'facts' task to determine architecture, then choose which software package to copy and install. The data hierarchy would be used to uniquely specify which software package to utilize.

Learning from spike

  • currently just exposing hiera allows backends to work without interpolation as long as all interpolation is based on $facts and $trusted. The fact that those are set during plan execution feels wrong and may be a bug.
  • We probably need the ability for users to specify a separate hiearchy for the plan level so we don't limit interpolation options.
  • There is value in exposing backends so that hiera-eyaml can be used but this problem may be solved by supporting eyaml in inventory.

h1 Questions for commentors

  • What levels of heirarchy would you use at the plan level?
  • Would there be any fact like interpolations in the hierarchy? What values would be used.
  • Do you expect values from plan level data to be automatically set anywhere with data bindings like classes in manifests.


 Comments   
Comment by Henrik Lindberg [ 2019/05/30 ]

Some thoughts about implementation...

One thing to consider here is that switching "perpective" (bolt controller, each node) using a single hiera instance will lead to its cache being evicted on each switch. This would potentially be quite bad for performance if many such switches are required as a bolt run progresses. Imagine looping over a set of nodes and in each iteration some plan specific, and some node specific value needs to be obtained.

From a UX perspective everything could be hidden unless it should be possible to lookup node specific values in a plan. For that, it must be possible to use a different perspective. This could be another function for example (lookup_for_node()), or a method on a node/target.

When designing the "perpectives", a big question is if it should be possible to override a node perspective in the plan perspective. If not, it may be possible to have one hiera/lookup-context per node in memory if performance is bad using switching of the set of values used to determine the paths in the hierarchy. Maybe it is possible to make hiera switch between named (i.e. "perspective specific") caches in an effective way.

It seems like a design with one hierarchy for the plan perspective, and one for all node specific perspectives would be best, and if override of node perspective is needed in plan perspective, then that can be done manually in the plan's logic.

Comment by Lucy Wyman [ 2019/08/26 ]

This issue was automatically closed when the Bolt team moved to using Github Issues for ticket management (August 2019). If you'd like to reopen the issue or discuss it further please open a github issue at http://github.com/puppetlabs/bolt/issues.

Generated at Mon Jan 27 11:13:29 PST 2020 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.