[PUP-6062] Allow non-root environments Created: 2016/03/16  Updated: 2019/04/04  Resolved: 2016/03/23

Status: Closed
Project: Puppet
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Normal
Reporter: Timothy Nelson Assignee: Unassigned
Resolution: Won't Do Votes: 0
Labels: environments, non-root, user
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Unix; probably others, but not verified



Apache allows access of files on server www.example.com in ~username/public_html/example_dir/ via http://www.example.com/~username/example_dir/

It would be amazing if puppet were to allow environments in eg. ~username/puppet/test_branch/ to be accessed with the environment name ~username/test_branch (or $user_environments::test_branch, or whatever is appropriate). This way, root access on the puppet master could be restricted, but could still allow non-root users to do a full set of testing, but only on machines in their particular test environment. It would even allow for cases like ~systems/production (which would be production for all systems functions; root access would no longer be required for changing the puppet config) and ~deployment/production (which would be the deployment functions of the deployment team, who also don't have root access).

Comment by Timothy Nelson [ 2016/03/16 ]

The idea is that apache/passenger could do some of the work, and puppet's main worry would be with picking up the right files and compiling them. If it only works with apache/passenger, that's fine .

Comment by Timothy Nelson [ 2016/03/16 ]

Also, just to make it trickier, it should allow hiera to specify environments in ~username/hiera/test_branch or something.

Comment by Kylo Ginsberg [ 2016/03/17 ]

Timothy Nelson please note that puppet support for passenger is deprecated. More info at https://docs.puppetlabs.com/puppet/4.3/reference/deprecated_servers.html.

Ping Chris Price for visibility to the enhancement idea in general.

Comment by Chris Price [ 2016/03/17 ]

ping Eric Sorenson for visibility to the enhancement idea as well.

Comment by Timothy Nelson [ 2016/03/20 ]

Kylo/Chris/Eric: Thanks for the consideration given to the idea.

Kylo: Thanks. I only set up Passenger because everyone on the 'Net recommended it. We're not ready to migrate now, but, thanks to your heads-up, we should be ready by the time we're ready to migrate to Puppet 5.0.

Comment by Kylo Ginsberg [ 2016/03/20 ]

Btw, Timothy Nelson, I should have mentioned that the reason passenger support is deprecated is because of: https://docs.puppetlabs.com/puppetserver/2.3/services_master_puppetserver.html.

Comment by Henrik Lindberg [ 2016/03/21 ]

I have given this some thought, and the suggested cannot be supported as it would mean that the system would have to search every user's home directory to be able to enumerate the available environments.

The proposed does not seem different from creating a directory for users (say userenvs), under which each user has one environment (named after the user). The userenvs directory is then added to the environmentpath. With that configuration, only users that need an environment will have one and there is no need to search all users home directories. There is one downside with this in that all environments share one namespace so names of regular environments and users cannot overlap.

If the idea is to have namespaced environments, e.g. helindbe::testenv, we could search the environmentpath for the first helindbe directory, and then search for testenv in that. While the search is simple to handle, I am concerned about the possibly far reaching consequences of changing the naming rule for environments from a simple name to a qualified one.

If the idea is that if an environment is given as say $USER::test and that this would mean helindbe::test if I am running the process. Then, that would work if it is a shell expansion when running things from the command line. I would not work at all if such an notation was in a configuration.

If the idea is that the master process should run under a different userid than root when certain environments are being used then we need a setting that restricts certain environments from running as root. This cannot be done in environment.conf since a user would have control over that. Not sure what the consequences of supporting this would be (it may mean having to spin up a separate master, since what was read and processed by master as user root cannot internally be protected from loaded ruby logic even if the process shifted uid. It does not seem doable in practice by using only a single Ruby instance. Puppet Server could do this though.
More thought has to go into how the process selects the user id the process should run under; if it is the ownership of the environment directory userenvs/helindbe or something else, and if what works in practice given git/r10k workflows etc.

I think that namespaced environments may be of value when having many users and many branches of puppet code being tested as a naming scheme among branches is otherwise needed e.g. bruce_test vs batman_test. Not sure how often branch names clash in practice as they would be named after tickets or similar.

Comment by Timothy Nelson [ 2016/03/22 ]

Kylo: Thanks again.
Henrik: Yes, my idea is to have namespaced environments if possible.

Regarding enumerating all the environments, I don't understand why environments need to be enumerated (if you mention some reasons, I might be able to make suggestions as to other alternatives). However, my assumption is that there would be a 'null' namespace, which would contain environments in their current form (ie. "production" would map to "::production"); would it be possible that enumerating environments only enumerate the ones in the specified namespace (with the default being null)? Also, another idea regarding enumerating users; we could restrict the whole thing to users in the "puppet" group.

Regarding the /userenvs/<username> option (maybe /etc/puppet/userenvs/<username>?), I'd find this less wonderful, but perfectly acceptable if the files in /etc/puppet//userenv/fred were actually owned and writable by the user fred.

I was putting the ~ in to indicate that a user directory had to be searched, however, if this is deemed unsuitable, I can live without it.

When you mention far-reaching consequences of namespaced environments, do you mean a) this could have strange effects at other places in puppet, or b) it would be a pain to rewrite puppet that way?

Regarding the master process and usernames, that wasn't my intent, I think; shouldn't it be sufficient to have users chgrp their files to puppet or something?

I have no experience with r10k at this time, but I think the ideas discussed would work better with my git workflow .

Comment by Henrik Lindberg [ 2016/03/23 ]

Anything that makes "dynamic environments" come back again is a non starter due to all of the problems that were associated with those. The dynamic environments were removed in 4.0.0. Thus any "non-enumerable" scheme is a non-starter because you cannot know if an environment exist or not (or there is a very high price to determine if it exists).

Implementing name-spaced environments could be quite costly as Environment is a central concept and there are many use cases that touch this logic. There are also several 3d party tools that have their own implementation / interpretation of how environments are laid out, presented in user interfaces etc. Hence, "far reaching consequences".

Even without the "far reaching" consequences the implementation will require review and overhaul of quite a lot of code. The current assumption that a directory under environments is a single environment would no longer be true. All code that uses globs from a root would have to be revised, new rules would need to be dictated (Can you have a fully functioning environment inside of another? Can a name space be split on many locations? etc).

Hence, the implementation is both expensive and has far reaching consequences. I agree on the value, it enables users to create branches/environments without having to have a centralized naming policy/authority (which you would have if all branches were in the same git repo, or if branch names were flattened on repo name e.g. hlindberg_test1 vs batman_test1). There are however many things you can do with r10k wrt configuring and mapping repositories to environments. I suggest starting there, and then taking the discussion to the community on the dev mailing list.

As a side note: How about letting users edit their environments in local directories. When they want to deploy the code, they give a command. That maps their username and the environment they created to a flattened name and copies/git-push-pull/whatever the files over to where they are supposed to be, makes sure permissions are right etc.

Comment by Eric Sorenson [ 2016/03/23 ]

tl;dr - Interesting idea, but closed/wontfix.

I'm going to close this as "Won't Do" - the ideas are interesting and I appreciate the thoughtful description of the use-cases. But there is already an effective way to accomplish the same goal (namely: using per-user topic branches + r10k + directory environments) that has widespread adoption and workflows that are built-in to Puppet. You can see a PuppetConf talk describing exactly this workflow at scale (from Phil Zimmerman at Time Warner Cable) here: https://www.youtube.com/watch?v=b1wnXF7z2Iw

Generated at Mon Oct 14 06:21:22 PDT 2019 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.