Details

    • Type: New Feature
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: PUP 4.3.0
    • Component/s: Docs, Language
    • Labels:
      None
    • Template:
    • Epic Link:
    • Story Points:
      5
    • Sprint:
      Language 2015-08-19, Language 2015-09-02
    • Release Notes:
      New Feature
    • Release Notes Summary:
      Hide
      This introduces the capability to have hiera data per environment and per module. This also introduces new features in the hiera.yaml configuration file that is used for environments and modules. The new features does not apply to the regular hiera, which remains unaffected.

      In summary:
      * It is now possible to have more precise control over the hierarchy where hiera earlier gave all paths to all backends
      * backends written for the regular hiera cannot be used with the new 'data in modules' based implementation (json and yaml support is included in puppet).
      * Data files in json and hiera as used with regular hiera are compatible with the data in modules implementation
      * The hiera.yaml configuration file is not compatible with regular hiera.
      * Automatic data binding, and the `lookup` function operates on the data in the global regular hiera, environments, and modules. The hiera_* functions only operate on the global regular hiera data.
      Show
      This introduces the capability to have hiera data per environment and per module. This also introduces new features in the hiera.yaml configuration file that is used for environments and modules. The new features does not apply to the regular hiera, which remains unaffected. In summary: * It is now possible to have more precise control over the hierarchy where hiera earlier gave all paths to all backends * backends written for the regular hiera cannot be used with the new 'data in modules' based implementation (json and yaml support is included in puppet). * Data files in json and hiera as used with regular hiera are compatible with the data in modules implementation * The hiera.yaml configuration file is not compatible with regular hiera. * Automatic data binding, and the `lookup` function operates on the data in the global regular hiera, environments, and modules. The hiera_* functions only operate on the global regular hiera data.

      Description

      Add DataProvider subtypes for environment and module that is hiera like. This data type should read a 'hiera.yaml' from <environment-root>/hiera.yaml, or <module-root>/hiera.yaml, respectively.

      • The hiera.yaml is a new version of the configuration file. It should include a version entry.
      • If the version is missing or set to a value < 4, then the current ("old") format is read and the semantics are retained (i.e. search with cartesian product of backends/hierarchy and only one data location per backend)
      • If the version is set, it must be 4 (or an error is raised)
      • The logger entry is not supported in either version - logging is always done using the Puppet logger
      • If an environment is using the hiera data provider and has no hiera.yaml file a default configuration is used comparable to the current default configuration file but with a data dir referencing <environment>/data or <moduleroot>/data respectively for the two kinds of providers.

      New config file format:

      The new config file format basically merges the backend and hierarchy settings into one common structure.

      ---
      :version: 4
      :hierarchy:
        - :name: yaml1
          :backend: yaml
          :datadir: some path
          :paths:
            - path
            - path
       
        - :name: yaml2
          :backend: yaml
          :datadir: some other path
          :path: a single path
       
        - :name: yaml3
          :backend: yaml
          :datadir: some path
          :paths:
               - path
               - path
       
        - :name:  json3
          :backend: json
          :datadir: some path
          :paths: 
               - path
               - path
       
      :datadir: default datadir path if missing in other entries
      

      Also:

      • the default default datadir is <root>/data
      • a relative datadir path is relative to <root>
      • <root> is the root of an environment, or a module respectively
      • relative path are relative to the effective datadir for that entry
      • absolute paths are not allowed in datadir or in paths for an environment or a module (they only make sense for a global-across-all-environments)
      • validation of the above rules is a bit tricky because datadir and paths are really generic arguments to a provider - i.e. they may be opaque URIs - thus each backend must be able to validate its configuration and know if it is used for a module, environment (and later global).
        This example configuration file has three entries in the hierarchy. The entries are searched from top to bottom. An entry may have one or more backends. If multiple backends are specified, they are searched in order using a common set of paths. If they have their own paths, those are searched before the common set of paths. A datadir may be specified per backend, a hierarchy entry, or for all. This acts as a default if a more specific entry does not specify the datadir. The name of an entry is used for human "explain" output as it is not enough to identify a backend by type alone.

      The backends are not hiera backends as specified for hiera <= 3.0, they are data providers. Each instantiated data provider gets the datadir and the paths as arguments. This makes it possible to use other kinds of providers than file based - where datadir and paths may be irrelevant, or bad names. As an example, it is of value to be able to use a function - it could default to the data function, but other function names could be used as well and searched just like a file path is searched. (Some elaboration and decision is needed on the mechanism/naming).

      Cacheing and lifecycle: Since the first implementation is only for modules and environments, the data should be cached for each compilation (i.e. associated with the catalog). Data files are not watched.

      Validation and Schema: There should be a schema for the configuration file, and it should be validated when loaded. This is made difficult if support is retained for the old format. Alternatively we supply a utility that converts hiera.yaml version < 3 to version 3. This should be a very simple transformation that also helps user untangle/select what they really want (i.e. pruning useless combinations that are created by the current cartesian product approach). To be elaborated on and decided

      Data Provider Infrastructure: In order to support data providers that perform hiera-like string interpolation this functionality must be available in some form to those data providers.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                henrik.lindberg Henrik Lindberg
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Zendesk Support