[PUP-1596] Make modulepath, manifest, and config_version configurable per-environment Created: 2014/02/05  Updated: 2019/04/04  Resolved: 2014/03/26

Status: Closed
Project: Puppet
Component/s: Compiler
Affects Version/s: PUP 3.5.0
Fix Version/s: PUP 3.6.0

Type: Improvement Priority: Normal
Reporter: Andrew Parker Assignee: Unassigned
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by PUP-2618 Unable to define multiple module path... Closed
Relates
relates to PUP-1699 Cache environments Closed
Template:
Epic Link: Directory Environments
Story Points: 2
Sprint: Week 2014-3-05 to 2014-3-12, Week 2014-3-12 to 2014-3-19, Week 2014-3-19 to 2014-3-26

 Description   

In order to make the new environment system fully usable, the default modulepath, manifest, and config_version needs to be overridable on a per-environment basis. This is possible in the current system with the environment stanzas and similar flexibility needs to be retained in the directory environments.

Each environment should optionally have an environment.conf file that has the same format as puppet.conf, but is restricted about what can be in it. The environment.conf can only contain:

  • modulepath
  • manifest
  • config_version

Any other settings or any sections in the file will generate an error. The error will be reported in the logs and the environment will not be usable.

Any relative paths specified in the environment.conf are interpreted relative to the environment's directory.



 Comments   
Comment by Andrew Parker [ 2014/02/05 ]

The problem with this system is that there isn't any way to express the default configuration, which means that variations of that configuration are also off limits. I'm thinking that this problem could be resolved by extending the settings interpolation syntax to allow referring to other sections of the configuration. For instance if ${main::modulepath} referred to the modulepath setting as it exists in the main section, then you could write the default configuration of an environment as

modulepath = modules:${main::modulepath}
manifest = manifests

Comment by Damon Atkins [ 2014/02/21 ]

modulepath also needs OS flavour e.g. solaris, linux, aix, windows etc.
You could do osfamily

manifest = $confdir/manifest/$environment
modulepath = $confdir/modules/$environment:$confdir/modules/$environment/$osflavour:configdir/modules/$osflavour:$confdir/modules/common
Note: $osfamily is to specific....... i.e. RHEL, OEL, Deb etc are 97% the same so "linux" would be best to describe them as a set.

If you have windows systems and linux systems in the same environment unix code will try to run on windows system?

$osfamily=<uname -s>

Comment by Andrew Parker [ 2014/02/21 ]

Damon Atkins, interpolating facts into the settings isn't something that is currently support, nor do I think it is something that will be supported. Deciding what modules to use based on the node is something that should be done in the puppet manifests.

Comment by Nigel Kersten [ 2014/03/12 ]

Andrew Parker, really like the idea of being able to refer to other sections of the configuration, that would have stopped me writing custom tools to interact with the configuration in the past.

I do have some concerns around ${main::modulepath} looking so similar to Puppet DSL syntax for fully qualified variables. Feels a bit uncanny valley.

Comment by Joshua Partlow [ 2014/03/12 ]

Still needs:

  • An investigation of the performance hit, because we do not cache directory environments, so every Puppet::Environments::Directories#get calls list, which creates an env for each directory in environmentpath which loads an environment.conf if present. We seem to do this 79 times when starting up a master... (that's with a single directory environment...so 79 per directory environment). Need to see what we can do with the new benchmark system to test master startup with x directory envs, with and without environment.conf.
  • Predocs
Comment by Andrew Parker [ 2014/03/13 ]

Joshua Partlow, we have a ticket to track getting performance up to snuff (PUP-1699), which we should do soon. Last time I took a stab at that it turned out to be much more difficult than I anticipated

Comment by Andrew Parker [ 2014/03/14 ]

I had previously stated that we needed to extend interpolation in the config file to support a syntax for accessing the variables from another section so that we can express the default value of an environment's modulepath setting in the file. That was all predicated on the idea that the default modulepath was done by using the master's modulepath setting, but we changed it to use basemodulepath.

Now that the default uses basemodulepath the default is easily expressed without having to extend the interpolation syntax. We won't be doing that extension for this feature.

Comment by Andrew Parker [ 2014/03/14 ]

Nigel Kersten, since we won't be doing the extended syntax of config interpolation for this feature you might want to open a feature request outlining what you would want to see.

Comment by Nigel Kersten [ 2014/03/17 ]

I'm more than happy to wait and see what the re-factored world of environment configuration looks like before putting that in, as it's entirely possible that this need may go away with sufficient APIs and dynamic behavior around environments.

I really was just a bit concerned about another way of interpolating values that wasn't quite consistent with the rest of our methods, DSL, Hiera configs, etc etc.

Comment by Andrew Parker [ 2014/03/18 ]

Merged into master in 8e1ebb

Comment by Kurt Wall [ 2014/03/26 ]

Verified in puppet-3.5.0-0.1rc2.121.el6. Using the environment configuration below, modules were installed to and uninstalled from the specified environment:

# /etc/puppet/environment/stage/environment.conf
modulepath = /tmp/modules
manifest = /tmp/manifests
config_version = /tmp/42
 
# puppet module install puppetlabs/stdlib --environment stage
[root@centos6-master stage]# tree -d /tmp/modules
/tmp/modules
└── stdlib
    ├── lib
    │   ├── facter
    │   │   └── util
    │   └── puppet
    │       ├── parser
    │       │   └── functions
    │       ├── provider
    │       │   └── file_line
    │       └── type
    ├── manifests
    ├── spec
    │   ├── classes
    │   ├── fixtures
    │   │   ├── manifests
    │   │   └── modules
    │   ├── functions
    │   ├── monkey_patches
    │   └── unit
    │       ├── facter
    │       │   └── util
    │       └── puppet
    │           ├── parser
    │           │   └── functions
    │           ├── provider
    │           │   └── file_line
    │           └── type
    └── tests
# puppet config print modulepath --environment stage
/tmp/modules
# puppet config print manifest --environment stage
/tmp/manifests
# puppet config print config_version --environment stage
/tmp/42
# puppet module uninstall puppetlabs/stdlib --environment dev
Notice: Preparing to uninstall 'puppetlabs-stdlib' ...
Error: Could not uninstall module 'puppetlabs-stdlib'
  Module 'puppetlabs-stdlib' is not installed
# puppet module uninstall puppetlabs/stdlib --environment stage
Notice: Preparing to uninstall 'puppetlabs-stdlib' ...
Removed 'puppetlabs-stdlib' (v4.1.0) from /tmp/modules

Generated at Sat Aug 08 17:54:21 PDT 2020 using Jira 8.5.2#805002-sha1:a66f9354b9e12ac788984e5d84669c903a370049.