Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-7327

Hiera 5 doesn't call lookup_with_segments method on hiera3_backend



    • Bug
    • Status: Closed
    • Normal
    • Resolution: Fixed
    • PUP 4.9.3
    • PUP 4.10.0
    • None
    • None
    • Puppet 4.9.3 on Ubuntu 16.04

    • Puppet Developer Experience
    • 1
    • PDE 2017-03-08
    • Bug Fix
    • A regression from Puppet 4.8 was caused by the hiera 5 implementation in Puppet 4.9.0 that resulted in the support for hiera3_backends using a feature to take over resolution of "dotted keys" stopped working. This behavior is now restored.
    • No Action
    • covered by unit tests; custom_backend object created. dotted key sending tested. other custom backend functionality covered in existing acceptance.


      We use a custom Hiera backend we call file_eyaml. It is based on the hiera-file backend from https://github.com/voxpupuli/hiera-file, which allows looking up file contents with Hiera. Our backend goes one step further and allows for the file contents to be Eyaml encrypted. Looking up a file this way decrypts the contents if they are encrypted or just returns them if they are not. The backend code is not really public but I've created a gist here (please don't judge, it is rather hackish and not that pretty): https://gist.github.com/antaflos/f4cb8844efa27d60d5179c77ad5b83ce

      The problem here is that the files in question (and thus the lookup keys) very often have periods in their filenames, so we implemented the lookup_with_segments method in our backend as discussed in https://tickets.puppetlabs.com/browse/HI-496. This works fine in Hiera 3 and Puppet 4.8.x (and Hiera 1 and Puppet 3.8.x)

      In Puppet 4.9.3 and Hiera 5 the lookup_with_segments method doesn't seem to get called at all.

      See below for the hiera.yaml file used and the defined hierarchy.

      In this example I have a file /etc/puppetlabs/code/environments/production/hierafiles/ssldata/common.d/example_ca.crt.pem that contains an unencrypted, regular base64-encoded X509 certificate.

      Running /opt/puppetlabs/bin/puppet lookup --debug --explain example_ca.crt.pem fails to get the contents of this file, saying it cannot find a value for the name, which is understandable since it seems to only look for example_ca (no such file exists), instead of example_ca.crt.pem.

      The full output is here: https://gist.github.com/antaflos/d4553a3a0a46b9a0514266d2f1244090

      I have added Hiera.debug() statements to the lookup and lookup_with_segments methods in our file_eyaml backend and can see from the debug output of the above command that lookup_with_segments is never called.


      version: 5
        - name: Eyaml encrypted data
          hiera3_backend: eyaml
          datadir: "/etc/puppetlabs/code/environments/%{::environment}/hieradata"
            extension: yaml
            encrypt_method: gpg
            gpg_gnupghome: "/etc/puppetlabs/puppet/keys/gpg"
            gpg_always_trust: true
            - "nodes/%{::trusted.certname}"
            - "os/%{::facts.os.name}/%{::facts.os.distro.codename}"
            - common
            - users
            - repos
            - ssldata/common
        - name: File_eyaml encrypted data
          hiera3_backend: file_eyaml
          datadir: "/etc/puppetlabs/code/environments/%{::environment}/hierafiles"
            interpolate: false
            - "nodes/%{::trusted.certname}"
            - "os/%{::facts.os.name}/%{::facts.os.distro.codename}"
            - common
            - users
            - repos
            - ssldata/common




            Unassigned Unassigned
            daff Andreas Ntaflos
            0 Vote for this issue
            5 Start watching this issue



              Zendesk Support