Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-4753

cannot call 4.x functions from 3.x function ERB templates

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: PUP 3.8.2, PUP 4.2.0
    • Component/s: DOCS
    • Labels:
      None
    • Template:
    • Story Points:
      2
    • Sprint:
      Language 2015-06-24
    • Release Notes:
      Bug Fix

      Description

      It is not possible to call a 4.x function from a 3.x function or ERB template because that logic does not have ready access to a mechanism to perform such a call correctly.

      We should add a new method of scope that works the same way as how the puppet language evaluator performs the call. i.e.:

      • The call is made with arguments using ruby/4.x semantics (no funny undef stuff)
      • If a 4.x function is found it wins
      • If a 3.x function is found the arguments are transformed to the 3.x convention (i.e. undef funny stuff).
      • in a 3.x version, 4.x. functions are only found if parser==future.

      The solution is to add a method call_function on Scope. It accepts the name of a function as its first argument, followed by all arguments in an Array. It also accepts a Ruby block (as a lambda).

      ORIGINAL REPORT


      escaped and reported from outside:
      https://groups.google.com/forum/#!msg/puppet-users/Ht_5d2Wb6VA/v1lVtw031DkJ

        Issue Links

          Activity

          Hide
          henrik.lindberg Henrik Lindberg added a comment -

          I have a solution to this problem - by adding a method to Scope named call_function, a user can agnostically call a 3.x or 4.x function. Arguments are given in an Array, and it accepts a ruby block (to enable calling 4.x iterative functions).

          It is the caller's responsability to catch an ArgumentError if there is desire to provide better error than that the function does not exist - which ultimately ends up being reported against the original caller - e.g. "Failed to parse inline template" (which is a horrible error message that is shown for all failures - yeah, do use EPP instead), or that the failure relates to the call to the function that in turned called a non existing function.

          The solution to hand this over to the user is that the API would otherwise require to know things like _FILE, and __LINE_ and would look incredibly ugly.

          The new method also works in 3.x without the future parser, but then only calling 3.x functions.

          The arguments should have 4.x semantics, i.e. give nil and not :undef or empty string as undef. Any needed transformations if calling a 3.x function will be done by the implementation.

          Unforrtunately this adds yet one more place where calls are made. When refactoring the call support, this must be taken into account .

          PR coming shortly.

          Show
          henrik.lindberg Henrik Lindberg added a comment - I have a solution to this problem - by adding a method to Scope named call_function , a user can agnostically call a 3.x or 4.x function. Arguments are given in an Array, and it accepts a ruby block (to enable calling 4.x iterative functions). It is the caller's responsability to catch an ArgumentError if there is desire to provide better error than that the function does not exist - which ultimately ends up being reported against the original caller - e.g. "Failed to parse inline template" (which is a horrible error message that is shown for all failures - yeah, do use EPP instead), or that the failure relates to the call to the function that in turned called a non existing function. The solution to hand this over to the user is that the API would otherwise require to know things like _ FILE , and __LINE _ and would look incredibly ugly. The new method also works in 3.x without the future parser, but then only calling 3.x functions. The arguments should have 4.x semantics, i.e. give nil and not :undef or empty string as undef. Any needed transformations if calling a 3.x function will be done by the implementation. Unforrtunately this adds yet one more place where calls are made. When refactoring the call support, this must be taken into account . PR coming shortly.
          Hide
          thomas.hallgren Thomas Hallgren added a comment -

          Merged to 3.x at b2d5834, to stable at 9d31f55, and to master at 4b8c043.

          Show
          thomas.hallgren Thomas Hallgren added a comment - Merged to 3.x at b2d5834 , to stable at 9d31f55 , and to master at 4b8c043 .
          Hide
          henrik.lindberg Henrik Lindberg added a comment -

          For functional review - this is easiest to test with an inline template:

          With and without future parser on 3.x and on 4.x

          notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>"))
          

          Only on 4.x

          notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) |x,y| {x+y}  %>"))
          

          Show
          henrik.lindberg Henrik Lindberg added a comment - For functional review - this is easiest to test with an inline template: With and without future parser on 3.x and on 4.x notice(inline_template( "<%= scope().call_function('fqdn_rand', [30]) %>" )) Only on 4.x notice(inline_template( "<%= scope().call_function('reduce', [[1,2,3]]) |x,y| {x+y} %>" ))
          Hide
          kurt.wall Kurt Wall added a comment -

          Thanks Henrik! I fought this all day before remembering inline_template. SIgh. In master at SHA=6a4ee36c43bd1a3b18516be8ae0a7c1c74616be4:

          $ gl
          6a4ee36 Merge branch 'stable'
          4c1272d Merge branch '3.x' into stable
          5e7fb72 (PUP-3088) Make spec test provider agnostic
          a2d1716 Merge branch 'stable'
          2bc7a9e Merge branch '3.x' into stable
          721d7b0 Merge pull request #4000 from hunner/fix_zone
          f4b50ee (PUP-3088) Fix provider logging calls
          4ed8531 Merge remote-tracking branch 'origin/stable'
          8dd3f64 Merge pull request #4035 from peterhuene/maint
           
          $ cat x.pp
          notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>"))
          notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) |x,y| {x+y}  %>"))
           
          $ be puppet --version
          4.2.0
           
          $ be puppet apply x.pp
          Notice: Scope(Class[main]): 21
          Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')'
          ...nction('reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.fo...
          ...                               ^
          (erb):1: syntax error, unexpected '}', expecting =>
          ...reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.force_enco...
          ...                               ^
          

          Same code on 3.x, SHA=5e7fb72e4ea6ae801a70d7aff582dc9ba4fdfdff:

          $ gl
          5e7fb72 (PUP-3088) Make spec test provider agnostic
          721d7b0 Merge pull request #4000 from hunner/fix_zone
          f4b50ee (PUP-3088) Fix provider logging calls
          b2d5834 Merge pull request #4033 from hlindberg/PUP-4753_add-agnostic-call-function-to-scope
          2ae5598 (PUP-4753) Add test of call_function
          6058f95 (PUP-4753) Fix faulty code - operating on wrong object
          6019339 (PUP-4753) Fix typo in comment
          e738bda (PUP-4753) Add ability to call 3.x and 4.x function from scope
          a9e8b2f Merge pull request #4031 from geoffnichols/RE-4831/3.x/update_pinned_ruby_shas_for_openssl_100s
          $ be puppet --version
          3.8.1
          $ be puppet apply x.pp
          Notice: Scope(Class[main]): 21
          Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')'
          ...nction('reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.fo...
          ...                               ^
          (erb):1: syntax error, unexpected '}', expecting =>
          ...reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.force_enco...
          ...                               ^
           
          $ be puppet apply --parser=future x.pp
          Notice: Scope(Class[main]): 21
          Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')'
          ...nction('reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.fo...
          ...                               ^
          (erb):1: syntax error, unexpected '}', expecting =>
          ...reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.force_enco...
          ...                               ^
          

          Show
          kurt.wall Kurt Wall added a comment - Thanks Henrik! I fought this all day before remembering inline_template . SIgh. In master at SHA=6a4ee36c43bd1a3b18516be8ae0a7c1c74616be4: $ gl 6a4ee36 Merge branch 'stable' 4c1272d Merge branch '3.x' into stable 5e7fb72 (PUP-3088) Make spec test provider agnostic a2d1716 Merge branch 'stable' 2bc7a9e Merge branch '3.x' into stable 721d7b0 Merge pull request #4000 from hunner/fix_zone f4b50ee (PUP-3088) Fix provider logging calls 4ed8531 Merge remote-tracking branch 'origin/stable' 8dd3f64 Merge pull request #4035 from peterhuene/maint   $ cat x.pp notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>")) notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) |x,y| {x+y} %>"))   $ be puppet --version 4.2.0   $ be puppet apply x.pp Notice: Scope(Class[main]): 21 Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')' ...nction('reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.fo... ... ^ (erb):1: syntax error, unexpected '}', expecting => ...reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.force_enco... ... ^ Same code on 3.x, SHA=5e7fb72e4ea6ae801a70d7aff582dc9ba4fdfdff: $ gl 5e7fb72 (PUP-3088) Make spec test provider agnostic 721d7b0 Merge pull request #4000 from hunner/fix_zone f4b50ee (PUP-3088) Fix provider logging calls b2d5834 Merge pull request #4033 from hlindberg/PUP-4753_add-agnostic-call-function-to-scope 2ae5598 (PUP-4753) Add test of call_function 6058f95 (PUP-4753) Fix faulty code - operating on wrong object 6019339 (PUP-4753) Fix typo in comment e738bda (PUP-4753) Add ability to call 3.x and 4.x function from scope a9e8b2f Merge pull request #4031 from geoffnichols/RE-4831/3.x/update_pinned_ruby_shas_for_openssl_100s $ be puppet --version 3.8.1 $ be puppet apply x.pp Notice: Scope(Class[main]): 21 Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')' ...nction('reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.fo... ... ^ (erb):1: syntax error, unexpected '}', expecting => ...reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.force_enco... ... ^   $ be puppet apply --parser=future x.pp Notice: Scope(Class[main]): 21 Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')' ...nction('reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.fo... ... ^ (erb):1: syntax error, unexpected '}', expecting => ...reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.force_enco... ... ^
          Hide
          kylo Kylo Ginsberg added a comment -

          Henrik Lindberg or Thomas Hallgren: any ideas on the error above? It seems to be Henrik's second example, including a block.

          Show
          kylo Kylo Ginsberg added a comment - Henrik Lindberg or Thomas Hallgren : any ideas on the error above? It seems to be Henrik's second example, including a block.
          Hide
          henrik.lindberg Henrik Lindberg added a comment -

          It fails because it should be a Ruby block (args inside) and not a puppet one (args outside)

          • henrik
          Show
          henrik.lindberg Henrik Lindberg added a comment - It fails because it should be a Ruby block (args inside) and not a puppet one (args outside) henrik
          Hide
          erict Eric Thompson added a comment - - edited

          validated on ubuntu14.04 at 3.x SHA: 5e7fb72

          [root@aux1ftmae9spuxm ~]# cat x.pp
          notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>"))
          [root@aux1ftmae9spuxm ~]# puppet --version
          3.8.1
          [root@aux1ftmae9spuxm ~]# puppet apply x.pp
          Notice: Scope(Class[main]): 10
          Notice: Compiled catalog for aux1ftmae9spuxm.delivery.puppetlabs.net in environment production in 0.03 seconds
          Notice: Finished catalog run in 0.01 seconds
           
          [root@aux1ftmae9spuxm ~]# puppet apply x.pp --parser future
          Notice: Scope(Class[main]): 10
          Notice: Compiled catalog for aux1ftmae9spuxm.delivery.puppetlabs.net in environment production in 0.42 seconds
          Notice: Finished catalog run in 0.01 seconds
          

          Show
          erict Eric Thompson added a comment - - edited validated on ubuntu14.04 at 3.x SHA: 5e7fb72 [root@aux1ftmae9spuxm ~]# cat x.pp notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>")) [root@aux1ftmae9spuxm ~]# puppet --version 3.8.1 [root@aux1ftmae9spuxm ~]# puppet apply x.pp Notice: Scope(Class[main]): 10 Notice: Compiled catalog for aux1ftmae9spuxm.delivery.puppetlabs.net in environment production in 0.03 seconds Notice: Finished catalog run in 0.01 seconds   [root@aux1ftmae9spuxm ~]# puppet apply x.pp --parser future Notice: Scope(Class[main]): 10 Notice: Compiled catalog for aux1ftmae9spuxm.delivery.puppetlabs.net in environment production in 0.42 seconds Notice: Finished catalog run in 0.01 seconds
          Hide
          erict Eric Thompson added a comment -

          validated on ubuntu14.04 at stable SHA: d93f591

          [root@joc3xk50poh0nx2 ~]# cat x.pp
          notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>"))
          notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) {|x,y| x+y}  %>"))
          [root@joc3xk50poh0nx2 ~]# puppet --version
          4.2.0
          [root@joc3xk50poh0nx2 ~]# puppet apply x.pp
          Notice: Scope(Class[main]): 20
          Notice: Scope(Class[main]): 6
          Notice: Compiled catalog for joc3xk50poh0nx2.delivery.puppetlabs.net in environment production in 0.42 seconds
          Notice: Applied catalog in 0.01 seconds
          

          Show
          erict Eric Thompson added a comment - validated on ubuntu14.04 at stable SHA: d93f591 [root@joc3xk50poh0nx2 ~]# cat x.pp notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>")) notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) {|x,y| x+y} %>")) [root@joc3xk50poh0nx2 ~]# puppet --version 4.2.0 [root@joc3xk50poh0nx2 ~]# puppet apply x.pp Notice: Scope(Class[main]): 20 Notice: Scope(Class[main]): 6 Notice: Compiled catalog for joc3xk50poh0nx2.delivery.puppetlabs.net in environment production in 0.42 seconds Notice: Applied catalog in 0.01 seconds
          Hide
          erict Eric Thompson added a comment -

          validated on windows2012r2 at stable SHA: d93f591

          Administrator@dqmvaynv21auy6c ~
          $ cmd /c puppet apply x.pp
          Notice: Scope(Class[main]): 26
          Notice: Scope(Class[main]): 6
          Notice: Compiled catalog for dqmvaynv21auy6c.delivery.puppetlabs.net in environment production in 0.44 seconds
          Notice: Applied catalog in 0.14 seconds
           
          Administrator@dqmvaynv21auy6c ~
          $ cat x.pp
          notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>"))
          notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) {|x,y| x+y}  %>"))
          

          Show
          erict Eric Thompson added a comment - validated on windows2012r2 at stable SHA: d93f591 Administrator@dqmvaynv21auy6c ~ $ cmd /c puppet apply x.pp Notice: Scope(Class[main]): 26 Notice: Scope(Class[main]): 6 Notice: Compiled catalog for dqmvaynv21auy6c.delivery.puppetlabs.net in environment production in 0.44 seconds Notice: Applied catalog in 0.14 seconds   Administrator@dqmvaynv21auy6c ~ $ cat x.pp notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>")) notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) {|x,y| x+y} %>"))
          Hide
          rkool Roland Kool added a comment -

          It woud be nice to catch this in the puppet preview tool as well.

          Show
          rkool Roland Kool added a comment - It woud be nice to catch this in the puppet preview tool as well.
          Hide
          henrik.lindberg Henrik Lindberg added a comment -

          Roland Kool that is incredibly difficult as ERB is just Ruby logic. Also, being able to do anything that would report calls that use old APIs would require a new version of 3.x and 3x is now End of Life so don't see that happening.

          What we could possibly do on 4.x is to warn about certain kinds of calls. OTOH that has the potential of creating lots of warnings. If you think that is of value, please open a separate ticket with such a request.

          Show
          henrik.lindberg Henrik Lindberg added a comment - Roland Kool that is incredibly difficult as ERB is just Ruby logic. Also, being able to do anything that would report calls that use old APIs would require a new version of 3.x and 3x is now End of Life so don't see that happening. What we could possibly do on 4.x is to warn about certain kinds of calls. OTOH that has the potential of creating lots of warnings. If you think that is of value, please open a separate ticket with such a request.

            People

            • Assignee:
              Unassigned
              Reporter:
              erict Eric Thompson
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Agile