Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-4695

acceptance test: call all hiera functions from ERB templates

    Details

    • Type: Task
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: PUP 4.3.0
    • Component/s: QA
    • Labels:
      None
    • Template:
    • Story Points:
      2
    • Sprint:
      Language 2015-09-02, Language 2015-09-16, Language 2015-09-30
    • Release Notes:
      Not Needed

      Description

      escaped and reported from outside:
      https://groups.google.com/forum/#!msg/puppet-users/Ht_5d2Wb6VA/v1lVtw031DkJ

      regressed with update to puppet 4.0?

      Henrik Lindberg i'd like to get your perspective on what is required here.

        Issue Links

          Activity

          Hide
          henrik.lindberg Henrik Lindberg added a comment -

          The regressions in this area were introduced because hiera functions moved to the 3x API while the old ERB template functions are still 3.x and we have not provided a mechanism to call 4.x functions from a 3.x context. Essentially, the user's ruby logic in a template is running inside of the puppet runtime - and this was something we overlooked in the transition.

          So, the general problem is "calling 3.x and 4.x functions from a 3.x scope"

          Show
          henrik.lindberg Henrik Lindberg added a comment - The regressions in this area were introduced because hiera functions moved to the 3x API while the old ERB template functions are still 3.x and we have not provided a mechanism to call 4.x functions from a 3.x context. Essentially, the user's ruby logic in a template is running inside of the puppet runtime - and this was something we overlooked in the transition. So, the general problem is "calling 3.x and 4.x functions from a 3.x scope"
          Hide
          dalen Erik Dalén added a comment -

          This is a bit of an issue for us with the 4.x migration. I was also going to switch some of my modules to use the 4.x API, but that will mean additional issues for people using them from erb templates as it is now.

          Show
          dalen Erik Dalén added a comment - This is a bit of an issue for us with the 4.x migration. I was also going to switch some of my modules to use the 4.x API, but that will mean additional issues for people using them from erb templates as it is now.
          Hide
          henrik.lindberg Henrik Lindberg added a comment -

          While it may seem crazy, it is possible to get the wanted behavior by evaluating puppet code in the ERB template (or anywhere in a 3.x function by doing he below (e.g. calling the function foo with a string argument):

          scope = self # i.e. in ERB and functions
          arg = 'hello, I am the argument'
          evaluator = Puppet::Pops::Parser::EvaluatingParser.new
          result = evaluator.evaluate_string(
            scope,
            "foo('#{arg}')",
            "nested puppet code in #{__FILE__}"
          )
          

          A function called this way can be implemented in 3.x or 4.x, there is no issue with parameter conversion (since it is expressed using the Puppet Language), and it will get proper error handling / a message that include the location "nested puppet code in..." with a reference to the ruby file that the puppet source is nested in.

          This is a good workaround because it gets all the mechanisms and semantics as if the call had been written in Puppet. The call itself is a high level call that is unlikely to change.

          This work around can be used with both 3.x future parser and 4.0.

          Show
          henrik.lindberg Henrik Lindberg added a comment - While it may seem crazy, it is possible to get the wanted behavior by evaluating puppet code in the ERB template (or anywhere in a 3.x function by doing he below (e.g. calling the function foo with a string argument): scope = self # i.e. in ERB and functions arg = 'hello, I am the argument' evaluator = Puppet::Pops::Parser::EvaluatingParser. new result = evaluator.evaluate_string( scope, "foo('#{arg}')" , "nested puppet code in #{__FILE__}" ) A function called this way can be implemented in 3.x or 4.x, there is no issue with parameter conversion (since it is expressed using the Puppet Language), and it will get proper error handling / a message that include the location "nested puppet code in..." with a reference to the ruby file that the puppet source is nested in. This is a good workaround because it gets all the mechanisms and semantics as if the call had been written in Puppet. The call itself is a high level call that is unlikely to change. This work around can be used with both 3.x future parser and 4.0.
          Hide
          henrik.lindberg Henrik Lindberg added a comment -

          Is there a ticket logged for the underlying problem?

          Show
          henrik.lindberg Henrik Lindberg added a comment - Is there a ticket logged for the underlying problem?
          Hide
          sean Sean Griffin added a comment -

          Thanks for the comments, Eric Thompson. I think I have addressed them all.

          -Sean

          Show
          sean Sean Griffin added a comment - Thanks for the comments, Eric Thompson . I think I have addressed them all. -Sean
          Hide
          erict Eric Thompson added a comment -

          this is failing against windows and ubuntu:
          https://jenkins.puppetlabs.com/view/All%20in%20One%20Agent/view/Master/view/Puppet/job/platform_aio-puppet_intn-sys_master/203/

          in ubuntu here:

            * Verifying hiera_array() call. #1
          Warning:
          hiera_array\('message'\) failed. Expected: '["message from production.yaml", "message1 from {osfamily}.yaml", "message2 from {osfamily}.yaml", "message from default.yaml"]'.
          Expected /hiera_array\('message'\): \[\"message from production.yaml\", \"message1 from {osfamily}.yaml\", \"message2 from {osfamily}.yaml\", \"message from default.yaml\"\]/ to match "\nhiera('message'): message from production.yaml\nhiera('hash_value'): {\"key1\"=>\"hash_value1 from {osfamily}.yaml\", \"key2\"=>\"hash_value2 from {osfamily}.yaml\"}\nhiera('includes'): hieratest::mod_production\nhiera_array('message'): [\"message from production.yaml\", \"message from default.yaml\"]\nhiera_array('includes'): [\"hieratest::mod_production\", \"hieratest::mod_default\"]\nhiera_hash('hash_value'): {\"key1\"=>\"hash_value1 from {osfamily}.yaml\", \"key2\"=>\"hash_value2 from {osfamily}.yaml\"}\n".
           
          /Users/erict/repos/er0ck/puppet/acceptance/tests/parser_functions/hiera_in_templates.rb:274
          /Library/Ruby/Gems/2.0.0/gems/beaker-2.20.0/bin/beaker:6
          tests/parser_functions/hiera_in_templates.rb failed in 64.38 seconds
          

          looks like msg1os is failing to return a value in hiera_array call?

          Show
          erict Eric Thompson added a comment - this is failing against windows and ubuntu: https://jenkins.puppetlabs.com/view/All%20in%20One%20Agent/view/Master/view/Puppet/job/platform_aio-puppet_intn-sys_master/203/ in ubuntu here: * Verifying hiera_array() call. #1 Warning: hiera_array\('message'\) failed. Expected: '["message from production.yaml", "message1 from {osfamily}.yaml", "message2 from {osfamily}.yaml", "message from default.yaml"]'. Expected /hiera_array\('message'\): \[\"message from production.yaml\", \"message1 from {osfamily}.yaml\", \"message2 from {osfamily}.yaml\", \"message from default.yaml\"\]/ to match "\nhiera('message'): message from production.yaml\nhiera('hash_value'): {\"key1\"=>\"hash_value1 from {osfamily}.yaml\", \"key2\"=>\"hash_value2 from {osfamily}.yaml\"}\nhiera('includes'): hieratest::mod_production\nhiera_array('message'): [\"message from production.yaml\", \"message from default.yaml\"]\nhiera_array('includes'): [\"hieratest::mod_production\", \"hieratest::mod_default\"]\nhiera_hash('hash_value'): {\"key1\"=>\"hash_value1 from {osfamily}.yaml\", \"key2\"=>\"hash_value2 from {osfamily}.yaml\"}\n".   /Users/erict/repos/er0ck/puppet/acceptance/tests/parser_functions/hiera_in_templates.rb:274 /Library/Ruby/Gems/2.0.0/gems/beaker-2.20.0/bin/beaker:6 tests/parser_functions/hiera_in_templates.rb failed in 64.38 seconds looks like msg1os is failing to return a value in hiera_array call?
          Hide
          sean Sean Griffin added a comment -

          Eric Thompson
          I checked in a fixed version of the test. Could you please review my PR. Thanks. -Sean

          Show
          sean Sean Griffin added a comment - Eric Thompson I checked in a fixed version of the test. Could you please review my PR. Thanks. -Sean
          Hide
          erict Eric Thompson added a comment -

          looks good. merged to master.
          Sean Griffin if you aren't still working on something for this, this ticket can be moved to ready for CI

          Show
          erict Eric Thompson added a comment - looks good. merged to master. Sean Griffin if you aren't still working on something for this, this ticket can be moved to ready for CI
          Hide
          sean Sean Griffin added a comment -

          This test failed to run when there is more than one agent in the test.
          Working on a fix.

          Show
          sean Sean Griffin added a comment - This test failed to run when there is more than one agent in the test. Working on a fix.
          Hide
          thomas.hallgren Thomas Hallgren added a comment -

          I guess the PR is not ready for this one yet so I'm changing it's status.

          Show
          thomas.hallgren Thomas Hallgren added a comment - I guess the PR is not ready for this one yet so I'm changing it's status.
          Hide
          sean Sean Griffin added a comment -

          Fix merged. Testrail link verified.

          Show
          sean Sean Griffin added a comment - Fix merged. Testrail link verified.

            People

            • Assignee:
              Unassigned
              Reporter:
              erict Eric Thompson
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Agile