Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-7327

Hiera 5 doesn't call lookup_with_segments method on hiera3_backend

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: PUP 4.9.3
    • Fix Version/s: PUP 4.10.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Puppet 4.9.3 on Ubuntu 16.04

    • Template:
    • Team:
      Puppet Developer Experience
    • Story Points:
      1
    • Sprint:
      PDE 2017-03-08
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      A regression from Puppet 4.8 was caused by the hiera 5 implementation in Puppet 4.9.0 that resulted in the support for hiera3_backends using a feature to take over resolution of "dotted keys" stopped working. This behavior is now restored.
    • QA Risk Assessment:
      No Action
    • QA Risk Assessment Reason:
      covered by unit tests; custom_backend object created. dotted key sending tested. other custom backend functionality covered in existing acceptance.

      Description

      We use a custom Hiera backend we call file_eyaml. It is based on the hiera-file backend from https://github.com/voxpupuli/hiera-file, which allows looking up file contents with Hiera. Our backend goes one step further and allows for the file contents to be Eyaml encrypted. Looking up a file this way decrypts the contents if they are encrypted or just returns them if they are not. The backend code is not really public but I've created a gist here (please don't judge, it is rather hackish and not that pretty): https://gist.github.com/antaflos/f4cb8844efa27d60d5179c77ad5b83ce

      The problem here is that the files in question (and thus the lookup keys) very often have periods in their filenames, so we implemented the lookup_with_segments method in our backend as discussed in https://tickets.puppetlabs.com/browse/HI-496. This works fine in Hiera 3 and Puppet 4.8.x (and Hiera 1 and Puppet 3.8.x)

      In Puppet 4.9.3 and Hiera 5 the lookup_with_segments method doesn't seem to get called at all.

      See below for the hiera.yaml file used and the defined hierarchy.

      In this example I have a file /etc/puppetlabs/code/environments/production/hierafiles/ssldata/common.d/example_ca.crt.pem that contains an unencrypted, regular base64-encoded X509 certificate.

      Running /opt/puppetlabs/bin/puppet lookup --debug --explain example_ca.crt.pem fails to get the contents of this file, saying it cannot find a value for the name, which is understandable since it seems to only look for example_ca (no such file exists), instead of example_ca.crt.pem.

      The full output is here: https://gist.github.com/antaflos/d4553a3a0a46b9a0514266d2f1244090

      I have added Hiera.debug() statements to the lookup and lookup_with_segments methods in our file_eyaml backend and can see from the debug output of the above command that lookup_with_segments is never called.

      hiera.yaml:

      ---
      version: 5
       
      hierarchy:
        - name: Eyaml encrypted data
          hiera3_backend: eyaml
          datadir: "/etc/puppetlabs/code/environments/%{::environment}/hieradata"
          options:
            extension: yaml
            encrypt_method: gpg
            gpg_gnupghome: "/etc/puppetlabs/puppet/keys/gpg"
            gpg_always_trust: true
          paths:
            - "nodes/%{::trusted.certname}"
            - "os/%{::facts.os.name}/%{::facts.os.distro.codename}"
            - common
            - users
            - repos
            - ssldata/common
       
        - name: File_eyaml encrypted data
          hiera3_backend: file_eyaml
          datadir: "/etc/puppetlabs/code/environments/%{::environment}/hierafiles"
          options:
            interpolate: false
          paths:
            - "nodes/%{::trusted.certname}"
            - "os/%{::facts.os.name}/%{::facts.os.distro.codename}"
            - common
            - users
            - repos
            - ssldata/common
      

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            daff Andreas Ntaflos
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Zendesk Support