[SERVER-2434] Flexible catalog compilation Created: 2019/01/18  Updated: 2019/04/29  Resolved: 2019/04/29

Status: Closed
Project: Puppet Server
Component/s: None
Affects Version/s: None
Fix Version/s: SERVER 6.3.0

Type: Epic Priority: Normal
Reporter: Maggie Dreyer Assignee: Unassigned
Resolution: Done Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
blocks PUP-9055 Compile catalogs on demand with user-... Resolved
Epic Name: Flexible catalog compilation
Acceptance Criteria:

See details in PE-25621 and PE-25714.

Epic Status: In Progress
Release Notes: New Feature
Release Notes Summary: Puppet Server now has a new endpoint for catalog retrieval that allows many more options than the previous endpoint. This endpoint is controlled by tk-auth, and by default is not generally accessible. It is intended for use by other puppet services (like CD4PE). For details on the API, see https://github.com/puppetlabs/puppetserver/blob/master/documentation/puppet-api/v4/catalog.markdown.
QA Risk Assessment: Needs Assessment


Puppet Server currently only has one endpoint for requesting a catalog. This endpoint requires that the requester be requesting their own catalog, i.e. that the name on cert used to authenticate the request match the name of the node for which the catalog is being compiled. It also does not allow the caller to specify what happens with reports or facts.

As we expand our portfolio, the need has arisen for an endpoint that allows requesting catalogs for other nodes. For example, CD4PE needs to request catalogs for arbitrary node names as part of Impact Analysis, and managing network devices requires a "proxy agent" to make catalog requests on the behalf of the devices being managed.

We want to create a new catalog endpoint that allows certain entities to request catalogs for arbitrary nodes. In the case of CD4PE, this would probably be controlled via an RBAC permission (see PE-25566), which for device management it would be locked down to the proxy agent's cert.

In order to service both of these use cases, this endpoint needs to be able to control whether or not reports and facts are saved to PuppetDB (CD4PE does not want this, Network Automation does), and under what name to save them. It also needs the ability to allow environments to be specified as part of the request, as an alternative to going through the classifier.

This endpoint might also enable us to create a CLI tool satisfying PUP-9055, that calls this endpoint on the backend.

Comment by Shane Madden [ 2019/02/07 ]

Hey folks - not sure if this is the right place for this, but it'd be great if https://forge.puppet.com/puppetlabs/catalog_preview could be updated to use these new endpoints, since its purpose is also to generate and inspect catalogs for arbitrary nodes and it's currently broken due to deprecations in 5.5 and 6.0.  We're stretching to find a tool that actually works current versions to replicate functionality that was readily available for our upgrade from 3 to 4. 

Comment by Maggie Dreyer [ 2019/02/07 ]

I think that this will probably give you what you need! We're shipping this endpoint in open source, so it should be available for you. It'd be great if we could solve even more problems with this work

Comment by Shane Madden [ 2019/02/07 ]

Great, thanks Maggie Dreyer! Just to clarify, the catalog_preview module is a project on here, I've opened a couple issues at https://tickets.puppetlabs.com/browse/PRE-144 and https://tickets.puppetlabs.com/browse/PRE-146 with the compatibility issues I've run into on newer versions in the hopes that it can be brought back to functional, but it's beyond my skills to dig quite that deep in the ruby indirectors. In any case, I'm excited for this endpoint to enable some really useful tools - being able to override the trusted hash for compilation testing is going to solve a problem we've run into - thanks a lot!

Comment by Maggie Dreyer [ 2019/04/29 ]

This endpoint worked with minimal difficulty for both consuming teams.

Generated at Mon Feb 24 16:48:09 PST 2020 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.