Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-4753

cannot call 4.x functions from 3.x function ERB templates

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: PUP 3.8.2, PUP 4.2.0
    • Component/s: DOCS
    • Labels:
      None
    • Template:
    • Story Points:
      2
    • Sprint:
      Language 2015-06-24
    • Release Notes:
      Bug Fix

      Description

      It is not possible to call a 4.x function from a 3.x function or ERB template because that logic does not have ready access to a mechanism to perform such a call correctly.

      We should add a new method of scope that works the same way as how the puppet language evaluator performs the call. i.e.:

      • The call is made with arguments using ruby/4.x semantics (no funny undef stuff)
      • If a 4.x function is found it wins
      • If a 3.x function is found the arguments are transformed to the 3.x convention (i.e. undef funny stuff).
      • in a 3.x version, 4.x. functions are only found if parser==future.

      The solution is to add a method call_function on Scope. It accepts the name of a function as its first argument, followed by all arguments in an Array. It also accepts a Ruby block (as a lambda).

      ORIGINAL REPORT


      escaped and reported from outside:
      https://groups.google.com/forum/#!msg/puppet-users/Ht_5d2Wb6VA/v1lVtw031DkJ

        Attachments

          Issue Links

            Activity

            erict Eric Thompson created issue -
            erict Eric Thompson made changes -
            Field Original Value New Value
            Link This issue relates to PUP-4695 [ PUP-4695 ]
            erict Eric Thompson made changes -
            Hide
            henrik.lindberg Henrik Lindberg added a comment -

            I have a solution to this problem - by adding a method to Scope named call_function, a user can agnostically call a 3.x or 4.x function. Arguments are given in an Array, and it accepts a ruby block (to enable calling 4.x iterative functions).

            It is the caller's responsability to catch an ArgumentError if there is desire to provide better error than that the function does not exist - which ultimately ends up being reported against the original caller - e.g. "Failed to parse inline template" (which is a horrible error message that is shown for all failures - yeah, do use EPP instead), or that the failure relates to the call to the function that in turned called a non existing function.

            The solution to hand this over to the user is that the API would otherwise require to know things like _FILE, and __LINE_ and would look incredibly ugly.

            The new method also works in 3.x without the future parser, but then only calling 3.x functions.

            The arguments should have 4.x semantics, i.e. give nil and not :undef or empty string as undef. Any needed transformations if calling a 3.x function will be done by the implementation.

            Unforrtunately this adds yet one more place where calls are made. When refactoring the call support, this must be taken into account .

            PR coming shortly.

            Show
            henrik.lindberg Henrik Lindberg added a comment - I have a solution to this problem - by adding a method to Scope named call_function , a user can agnostically call a 3.x or 4.x function. Arguments are given in an Array, and it accepts a ruby block (to enable calling 4.x iterative functions). It is the caller's responsability to catch an ArgumentError if there is desire to provide better error than that the function does not exist - which ultimately ends up being reported against the original caller - e.g. "Failed to parse inline template" (which is a horrible error message that is shown for all failures - yeah, do use EPP instead), or that the failure relates to the call to the function that in turned called a non existing function. The solution to hand this over to the user is that the API would otherwise require to know things like _ FILE , and __LINE _ and would look incredibly ugly. The new method also works in 3.x without the future parser, but then only calling 3.x functions. The arguments should have 4.x semantics, i.e. give nil and not :undef or empty string as undef. Any needed transformations if calling a 3.x function will be done by the implementation. Unforrtunately this adds yet one more place where calls are made. When refactoring the call support, this must be taken into account . PR coming shortly.
            henrik.lindberg Henrik Lindberg made changes -
            Sprint Language 2015-06-24 [ 920 ]
            Story Points 2
            Description escaped and reported from outside:
            https://groups.google.com/forum/#!msg/puppet-users/Ht_5d2Wb6VA/v1lVtw031DkJ

            It is not possible to call a 4.x function from a 3.x function or ERB template because that logic does not have ready access to a mechanism to perform such a call correctly.

            We should add a new method of scope that works the same way as how the puppet language evaluator performs the call. i.e.:
            * The call is made with arguments using ruby/4.x semantics (no funny undef stuff)
            * If a 4.x function is found it wins
            * If a 3.x function is found the arguments are transformed to the 3.x convention (i.e. undef funny stuff).

            Since
            ORIGINAL REPORT
            ----
            escaped and reported from outside:
            https://groups.google.com/forum/#!msg/puppet-users/Ht_5d2Wb6VA/v1lVtw031DkJ

            Scope Change Reason found by community - painful to work around
            Scope Change Category Found [ 11410 ]
            Fix Version/s PUP 3.8.2 [ 13415 ]
            Fix Version/s PUP 4.2.0 [ 13200 ]
            henrik.lindberg Henrik Lindberg made changes -
            Remote Link This issue links to "PR-4033 (Web Link)" [ 31061 ]
            henrik.lindberg Henrik Lindberg made changes -
            Status Open [ 1 ] Ready for Review [ 10004 ]
            Assignee Henrik Lindberg [ henrik.lindberg ]
            henrik.lindberg Henrik Lindberg made changes -
            Status Ready for Review [ 10004 ] Ready for Merge [ 10002 ]
            henrik.lindberg Henrik Lindberg made changes -
            Description It is not possible to call a 4.x function from a 3.x function or ERB template because that logic does not have ready access to a mechanism to perform such a call correctly.

            We should add a new method of scope that works the same way as how the puppet language evaluator performs the call. i.e.:
            * The call is made with arguments using ruby/4.x semantics (no funny undef stuff)
            * If a 4.x function is found it wins
            * If a 3.x function is found the arguments are transformed to the 3.x convention (i.e. undef funny stuff).

            Since
            ORIGINAL REPORT
            ----
            escaped and reported from outside:
            https://groups.google.com/forum/#!msg/puppet-users/Ht_5d2Wb6VA/v1lVtw031DkJ

            It is not possible to call a 4.x function from a 3.x function or ERB template because that logic does not have ready access to a mechanism to perform such a call correctly.

            We should add a new method of scope that works the same way as how the puppet language evaluator performs the call. i.e.:
            * The call is made with arguments using ruby/4.x semantics (no funny undef stuff)
            * If a 4.x function is found it wins
            * If a 3.x function is found the arguments are transformed to the 3.x convention (i.e. undef funny stuff).
            * in a 3.x version, 4.x. functions are only found if parser==future.

            The solution is to add a method {{call_function}} on Scope. It accepts the name of a function as its first argument, followed by all arguments in an Array. It also accepts a Ruby block (as a lambda).

            ORIGINAL REPORT
            ----
            escaped and reported from outside:
            https://groups.google.com/forum/#!msg/puppet-users/Ht_5d2Wb6VA/v1lVtw031DkJ

            thomas.hallgren Thomas Hallgren made changes -
            Assignee Thomas Hallgren [ thomas.hallgren ]
            Hide
            thomas.hallgren Thomas Hallgren added a comment -

            Merged to 3.x at b2d5834, to stable at 9d31f55, and to master at 4b8c043.

            Show
            thomas.hallgren Thomas Hallgren added a comment - Merged to 3.x at b2d5834 , to stable at 9d31f55 , and to master at 4b8c043 .
            thomas.hallgren Thomas Hallgren made changes -
            Status Ready for Merge [ 10002 ] Ready for CI [ 10003 ]
            henrik.lindberg Henrik Lindberg made changes -
            Comment [ Merged to 3.x at: b2d5834 ]
            thomas.hallgren Thomas Hallgren made changes -
            Assignee Thomas Hallgren [ thomas.hallgren ]
            kylo Kylo Ginsberg made changes -
            Release Notes Bug Fix [ 11407 ]
            kylo Kylo Ginsberg made changes -
            Status Ready for CI [ 10003 ] Ready for Test [ 10029 ]
            Assignee QA [ qa ]
            Hide
            henrik.lindberg Henrik Lindberg added a comment -

            For functional review - this is easiest to test with an inline template:

            With and without future parser on 3.x and on 4.x

            notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>"))
            

            Only on 4.x

            notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) |x,y| {x+y}  %>"))
            

            Show
            henrik.lindberg Henrik Lindberg added a comment - For functional review - this is easiest to test with an inline template: With and without future parser on 3.x and on 4.x notice(inline_template( "<%= scope().call_function('fqdn_rand', [30]) %>" )) Only on 4.x notice(inline_template( "<%= scope().call_function('reduce', [[1,2,3]]) |x,y| {x+y} %>" ))
            henrik.lindberg Henrik Lindberg made changes -
            Component/s DOCS [ 11226 ]
            kurt.wall Kurt Wall made changes -
            Assignee qa [ qa ] Kurt Wall [ kurt.wall ]
            Hide
            kurt.wall Kurt Wall added a comment -

            Thanks Henrik! I fought this all day before remembering inline_template. SIgh. In master at SHA=6a4ee36c43bd1a3b18516be8ae0a7c1c74616be4:

            $ gl
            6a4ee36 Merge branch 'stable'
            4c1272d Merge branch '3.x' into stable
            5e7fb72 (PUP-3088) Make spec test provider agnostic
            a2d1716 Merge branch 'stable'
            2bc7a9e Merge branch '3.x' into stable
            721d7b0 Merge pull request #4000 from hunner/fix_zone
            f4b50ee (PUP-3088) Fix provider logging calls
            4ed8531 Merge remote-tracking branch 'origin/stable'
            8dd3f64 Merge pull request #4035 from peterhuene/maint
             
            $ cat x.pp
            notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>"))
            notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) |x,y| {x+y}  %>"))
             
            $ be puppet --version
            4.2.0
             
            $ be puppet apply x.pp
            Notice: Scope(Class[main]): 21
            Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')'
            ...nction('reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.fo...
            ...                               ^
            (erb):1: syntax error, unexpected '}', expecting =>
            ...reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.force_enco...
            ...                               ^
            

            Same code on 3.x, SHA=5e7fb72e4ea6ae801a70d7aff582dc9ba4fdfdff:

            $ gl
            5e7fb72 (PUP-3088) Make spec test provider agnostic
            721d7b0 Merge pull request #4000 from hunner/fix_zone
            f4b50ee (PUP-3088) Fix provider logging calls
            b2d5834 Merge pull request #4033 from hlindberg/PUP-4753_add-agnostic-call-function-to-scope
            2ae5598 (PUP-4753) Add test of call_function
            6058f95 (PUP-4753) Fix faulty code - operating on wrong object
            6019339 (PUP-4753) Fix typo in comment
            e738bda (PUP-4753) Add ability to call 3.x and 4.x function from scope
            a9e8b2f Merge pull request #4031 from geoffnichols/RE-4831/3.x/update_pinned_ruby_shas_for_openssl_100s
            $ be puppet --version
            3.8.1
            $ be puppet apply x.pp
            Notice: Scope(Class[main]): 21
            Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')'
            ...nction('reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.fo...
            ...                               ^
            (erb):1: syntax error, unexpected '}', expecting =>
            ...reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.force_enco...
            ...                               ^
             
            $ be puppet apply --parser=future x.pp
            Notice: Scope(Class[main]): 21
            Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')'
            ...nction('reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.fo...
            ...                               ^
            (erb):1: syntax error, unexpected '}', expecting =>
            ...reduce', [[1,2,3]]) |x,y| {x+y}  ).to_s); _erbout.force_enco...
            ...                               ^
            

            Show
            kurt.wall Kurt Wall added a comment - Thanks Henrik! I fought this all day before remembering inline_template . SIgh. In master at SHA=6a4ee36c43bd1a3b18516be8ae0a7c1c74616be4: $ gl 6a4ee36 Merge branch 'stable' 4c1272d Merge branch '3.x' into stable 5e7fb72 (PUP-3088) Make spec test provider agnostic a2d1716 Merge branch 'stable' 2bc7a9e Merge branch '3.x' into stable 721d7b0 Merge pull request #4000 from hunner/fix_zone f4b50ee (PUP-3088) Fix provider logging calls 4ed8531 Merge remote-tracking branch 'origin/stable' 8dd3f64 Merge pull request #4035 from peterhuene/maint   $ cat x.pp notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>")) notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) |x,y| {x+y} %>"))   $ be puppet --version 4.2.0   $ be puppet apply x.pp Notice: Scope(Class[main]): 21 Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')' ...nction('reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.fo... ... ^ (erb):1: syntax error, unexpected '}', expecting => ...reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.force_enco... ... ^ Same code on 3.x, SHA=5e7fb72e4ea6ae801a70d7aff582dc9ba4fdfdff: $ gl 5e7fb72 (PUP-3088) Make spec test provider agnostic 721d7b0 Merge pull request #4000 from hunner/fix_zone f4b50ee (PUP-3088) Fix provider logging calls b2d5834 Merge pull request #4033 from hlindberg/PUP-4753_add-agnostic-call-function-to-scope 2ae5598 (PUP-4753) Add test of call_function 6058f95 (PUP-4753) Fix faulty code - operating on wrong object 6019339 (PUP-4753) Fix typo in comment e738bda (PUP-4753) Add ability to call 3.x and 4.x function from scope a9e8b2f Merge pull request #4031 from geoffnichols/RE-4831/3.x/update_pinned_ruby_shas_for_openssl_100s $ be puppet --version 3.8.1 $ be puppet apply x.pp Notice: Scope(Class[main]): 21 Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')' ...nction('reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.fo... ... ^ (erb):1: syntax error, unexpected '}', expecting => ...reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.force_enco... ... ^   $ be puppet apply --parser=future x.pp Notice: Scope(Class[main]): 21 Error: Could not run: (erb):1: syntax error, unexpected ',', expecting ')' ...nction('reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.fo... ... ^ (erb):1: syntax error, unexpected '}', expecting => ...reduce', [[1,2,3]]) |x,y| {x+y} ).to_s); _erbout.force_enco... ... ^
            kurt.wall Kurt Wall made changes -
            Assignee Kurt Wall [ kurt.wall ]
            Hide
            kylo Kylo Ginsberg added a comment -

            Henrik Lindberg or Thomas Hallgren: any ideas on the error above? It seems to be Henrik's second example, including a block.

            Show
            kylo Kylo Ginsberg added a comment - Henrik Lindberg or Thomas Hallgren : any ideas on the error above? It seems to be Henrik's second example, including a block.
            Hide
            henrik.lindberg Henrik Lindberg added a comment -

            It fails because it should be a Ruby block (args inside) and not a puppet one (args outside)

            • henrik
            Show
            henrik.lindberg Henrik Lindberg added a comment - It fails because it should be a Ruby block (args inside) and not a puppet one (args outside) henrik
            erict Eric Thompson made changes -
            Assignee Eric Thompson [ eric.thompson ]
            Hide
            erict Eric Thompson added a comment - - edited

            validated on ubuntu14.04 at 3.x SHA: 5e7fb72

            [root@aux1ftmae9spuxm ~]# cat x.pp
            notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>"))
            [root@aux1ftmae9spuxm ~]# puppet --version
            3.8.1
            [root@aux1ftmae9spuxm ~]# puppet apply x.pp
            Notice: Scope(Class[main]): 10
            Notice: Compiled catalog for aux1ftmae9spuxm.delivery.puppetlabs.net in environment production in 0.03 seconds
            Notice: Finished catalog run in 0.01 seconds
             
            [root@aux1ftmae9spuxm ~]# puppet apply x.pp --parser future
            Notice: Scope(Class[main]): 10
            Notice: Compiled catalog for aux1ftmae9spuxm.delivery.puppetlabs.net in environment production in 0.42 seconds
            Notice: Finished catalog run in 0.01 seconds
            

            Show
            erict Eric Thompson added a comment - - edited validated on ubuntu14.04 at 3.x SHA: 5e7fb72 [root@aux1ftmae9spuxm ~]# cat x.pp notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>")) [root@aux1ftmae9spuxm ~]# puppet --version 3.8.1 [root@aux1ftmae9spuxm ~]# puppet apply x.pp Notice: Scope(Class[main]): 10 Notice: Compiled catalog for aux1ftmae9spuxm.delivery.puppetlabs.net in environment production in 0.03 seconds Notice: Finished catalog run in 0.01 seconds   [root@aux1ftmae9spuxm ~]# puppet apply x.pp --parser future Notice: Scope(Class[main]): 10 Notice: Compiled catalog for aux1ftmae9spuxm.delivery.puppetlabs.net in environment production in 0.42 seconds Notice: Finished catalog run in 0.01 seconds
            Hide
            erict Eric Thompson added a comment -

            validated on ubuntu14.04 at stable SHA: d93f591

            [root@joc3xk50poh0nx2 ~]# cat x.pp
            notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>"))
            notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) {|x,y| x+y}  %>"))
            [root@joc3xk50poh0nx2 ~]# puppet --version
            4.2.0
            [root@joc3xk50poh0nx2 ~]# puppet apply x.pp
            Notice: Scope(Class[main]): 20
            Notice: Scope(Class[main]): 6
            Notice: Compiled catalog for joc3xk50poh0nx2.delivery.puppetlabs.net in environment production in 0.42 seconds
            Notice: Applied catalog in 0.01 seconds
            

            Show
            erict Eric Thompson added a comment - validated on ubuntu14.04 at stable SHA: d93f591 [root@joc3xk50poh0nx2 ~]# cat x.pp notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>")) notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) {|x,y| x+y} %>")) [root@joc3xk50poh0nx2 ~]# puppet --version 4.2.0 [root@joc3xk50poh0nx2 ~]# puppet apply x.pp Notice: Scope(Class[main]): 20 Notice: Scope(Class[main]): 6 Notice: Compiled catalog for joc3xk50poh0nx2.delivery.puppetlabs.net in environment production in 0.42 seconds Notice: Applied catalog in 0.01 seconds
            Hide
            erict Eric Thompson added a comment -

            validated on windows2012r2 at stable SHA: d93f591

            Administrator@dqmvaynv21auy6c ~
            $ cmd /c puppet apply x.pp
            Notice: Scope(Class[main]): 26
            Notice: Scope(Class[main]): 6
            Notice: Compiled catalog for dqmvaynv21auy6c.delivery.puppetlabs.net in environment production in 0.44 seconds
            Notice: Applied catalog in 0.14 seconds
             
            Administrator@dqmvaynv21auy6c ~
            $ cat x.pp
            notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>"))
            notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) {|x,y| x+y}  %>"))
            

            Show
            erict Eric Thompson added a comment - validated on windows2012r2 at stable SHA: d93f591 Administrator@dqmvaynv21auy6c ~ $ cmd /c puppet apply x.pp Notice: Scope(Class[main]): 26 Notice: Scope(Class[main]): 6 Notice: Compiled catalog for dqmvaynv21auy6c.delivery.puppetlabs.net in environment production in 0.44 seconds Notice: Applied catalog in 0.14 seconds   Administrator@dqmvaynv21auy6c ~ $ cat x.pp notice(inline_template("<%= scope().call_function('fqdn_rand', [30]) %>")) notice(inline_template("<%= scope().call_function('reduce', [[1,2,3]]) {|x,y| x+y} %>"))
            erict Eric Thompson made changes -
            Resolution Fixed [ 1 ]
            Assignee Eric Thompson [ eric.thompson ]
            Status Ready for Test [ 10029 ] Resolved [ 5 ]
            erict Eric Thompson made changes -
            Remote Link This issue links to "test case: should be able to call 4.x functions from 3.x functions and ERB templates (Web Link)" [ 31520 ]
            erict Eric Thompson made changes -
            QA Status Reviewed [ 10700 ]
            josh Josh Cooper made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            derek.austin Dee Austin made changes -
            Workflow Scrum Team Workflow [ 381713 ] Scrum Teams Workflow [ 471589 ]
            kenn Kenn Hussey made changes -
            Workflow Scrum Teams Workflow [ 471589 ] Scrum Team Workflow [ 564754 ]
            Hide
            rkool Roland Kool added a comment -

            It woud be nice to catch this in the puppet preview tool as well.

            Show
            rkool Roland Kool added a comment - It woud be nice to catch this in the puppet preview tool as well.
            Hide
            henrik.lindberg Henrik Lindberg added a comment -

            Roland Kool that is incredibly difficult as ERB is just Ruby logic. Also, being able to do anything that would report calls that use old APIs would require a new version of 3.x and 3x is now End of Life so don't see that happening.

            What we could possibly do on 4.x is to warn about certain kinds of calls. OTOH that has the potential of creating lots of warnings. If you think that is of value, please open a separate ticket with such a request.

            Show
            henrik.lindberg Henrik Lindberg added a comment - Roland Kool that is incredibly difficult as ERB is just Ruby logic. Also, being able to do anything that would report calls that use old APIs would require a new version of 3.x and 3x is now End of Life so don't see that happening. What we could possibly do on 4.x is to warn about certain kinds of calls. OTOH that has the potential of creating lots of warnings. If you think that is of value, please open a separate ticket with such a request.

              People

              • Assignee:
                Unassigned
                Reporter:
                erict Eric Thompson
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Zendesk Support