Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-7738

"bundle exec puppet module install" fails on Puppet 5.0.0

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: PUP 5.0.0
    • Fix Version/s: PUP 5.0.1
    • Component/s: None
    • Labels:
      None
    • Environment:

      Puppet 5.0 gem on Centos7 with librarian puppet running with "bundle exec"

    • Template:
    • Team:
      Agent
    • Story Points:
      2
    • Sprint:
      Agent 2017-07-12, Agent 2017-07-26
    • Method Found:
      Customer Feedback
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Hide
      Puppet 5.0.0 introduced a regression when running "puppet module install" via bundler. This issue only occurred if the Gemfile directly or indirectly expressed a dependency on the "semantic_puppet" gem. A common way for this to happen was if the module's Gemfile relied on the "metadata-json-lint" gem, which depends on "semantic_puppet". This fix ensures puppet works correctly when using either the external "semantic_puppet" gem or the vendored version in Puppet.
      Show
      Puppet 5.0.0 introduced a regression when running "puppet module install" via bundler. This issue only occurred if the Gemfile directly or indirectly expressed a dependency on the "semantic_puppet" gem. A common way for this to happen was if the module's Gemfile relied on the "metadata-json-lint" gem, which depends on "semantic_puppet". This fix ensures puppet works correctly when using either the external "semantic_puppet" gem or the vendored version in Puppet.
    • QA Risk Assessment:
      Automate
    • QA Risk Assessment Reason:
      partially covered by unit tests. should still test in acceptance with semantic_puppet gem installed

      Description

      We have a Puppet 4.10.x pipeline running librarian puppet to install modules from Puppetfile.

      This now failes on Puppet 5.0 with:

      10:51:05 10:51:05 Error executing puppet module install. Check that this command succeeds:
      10:51:05 10:51:05 puppet module install --version 4.17.1 --target-dir /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 --module_repository https://forgeapi.puppetlabs.com --modulepath /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 --module_working_dir /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 --ignore-dependencies puppetlabs-stdlib
      10:51:05 10:51:05 Error:
      10:51:05 10:51:05 Warning: Support for ruby version 2.0.0 is deprecated and will be removed in a future release. See https://docs.puppet.com/puppet/latest/system_requirements.html#ruby for a list of supported ruby versions.
      10:51:05 10:51:05    (at /yyyyy/gems/ruby/gems/puppet-5.0.0/lib/puppet.rb:181:in `<module:Puppet>')
      10:51:05 10:51:05 Error: wrong number of arguments (2 for 1)
      10:51:05 10:51:05 Error: Try 'puppet help module install' for usage
      

        Issue Links

          Activity

          Hide
          kenn Kenn Hussey added a comment -
          Show
          kenn Kenn Hussey added a comment - Geoff Nichols FYI
          Hide
          ben.ford Ben Ford added a comment - - edited

          Robert Heinzmann Can you check your quoting, escaping, etc? What does your pipeline look like? Does your command work when run by hand?

          It appears to work here on CentOS 7,

          root@master:~ # /usr/local/bin/puppet --version
          5.0.0
          root@master:~ # /usr/local/bin/puppet module install --version 4.17.1 --target-dir /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 --module_repository https://forgeapi.puppetlabs.com --modulepath /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 --module_working_dir /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 --ignore-dependencies puppetlabs-stdlib
          Warning: Support for ruby version 2.0.0 is deprecated and will be removed in a future release. See https://docs.puppet.com/puppet/latest/system_requirements.html#ruby for a list of supported ruby versions.
             (at /usr/local/share/gems/gems/puppet-5.0.0/lib/puppet.rb:181:in `<module:Puppet>')
          Notice: Preparing to install into /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 ...
          Notice: Downloading from https://forgeapi.puppetlabs.com ...
          Notice: Installing -- do not interrupt ...
          /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1
          └── puppetlabs-stdlib (v4.17.1)
          

          Show
          ben.ford Ben Ford added a comment - - edited Robert Heinzmann Can you check your quoting, escaping, etc? What does your pipeline look like? Does your command work when run by hand? It appears to work here on CentOS 7, root@master:~ # /usr/local/bin/puppet --version 5.0.0 root@master:~ # /usr/local/bin/puppet module install --version 4.17.1 --target-dir /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 --module_repository https://forgeapi.puppetlabs.com --modulepath /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 --module_working_dir /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 --ignore-dependencies puppetlabs-stdlib Warning: Support for ruby version 2.0.0 is deprecated and will be removed in a future release. See https://docs.puppet.com/puppet/latest/system_requirements.html#ruby for a list of supported ruby versions. (at /usr/local/share/gems/gems/puppet-5.0.0/lib/puppet.rb:181:in `<module:Puppet>') Notice: Preparing to install into /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 ... Notice: Downloading from https://forgeapi.puppetlabs.com ... Notice: Installing -- do not interrupt ... /xxxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/puppetlabs-stdlib/4.17.1 └── puppetlabs-stdlib (v4.17.1)
          Hide
          josh Josh Cooper added a comment -

          Robert Heinzmann if you can reproduce outside of librarian puppet, can you add --trace to the command line, and add the backtrace to this ticket?

          Show
          josh Josh Cooper added a comment - Robert Heinzmann if you can reproduce outside of librarian puppet, can you add --trace to the command line, and add the backtrace to this ticket?
          Hide
          elconas Robert Heinzmann added a comment - - edited

          Hello,
          it seems to be only a problem when run with "bundle exec". This worked for puppet 4.10.4.

          In our pipeline librarian is called with "bundle exec librarian-puppet install". This failes if

          • librarian-puppet is in the Gemfile: Error: wrong number of arguments (2 for 1)
          • json is missin in Gemfile: cannot load such file – json

          OK:

          gem 'json_pure'
          gem 'puppet', '~> 5.0'
          

          Failes with Error: wrong number of arguments (2 for 1):

          gem 'librarian-puppet'
          gem 'json_pure'
          gem 'puppet', '~> 5.0'
          

          Failes with Error: json missing:

          gem 'puppet', '~> 5.0'
          

          Debugging (bundle exec gem list) shows that the failing Gem is puppet_forge:

          Failes with Error: wrong number of arguments (2 for 1)

          gem 'puppet_forge'
          gem 'json_pure'
          gem 'puppet', '~> 5.0'
          

          Notice: Preparing to install into /xxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/AlexCline-dirtree/0.2.1 ...
          Notice: Downloading from https://forgeapi.puppetlabs.com ...
          Error: wrong number of arguments (2 for 1)
          Error: Try 'puppet help module install' for usage
          

          Trace:

          Notice: Preparing to install into /xxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/AlexCline-dirtree/0.2.1 ...
          Notice: Downloading from https://forgeapi.puppetlabs.com ...
          Error: wrong number of arguments (2 for 1)
          /usr/local/share/gems/gems/semantic_puppet-0.1.4/lib/semantic_puppet/version_range.rb:26:in `parse'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/module_tool/applications/installer.rb:209:in `build_single_module_graph'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/module_tool/applications/installer.rb:86:in `run'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/module_tool/applications/application.rb:11:in `run'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/face/module/install.rb:131:in `block (3 levels) in <top (required)>'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/interface/action.rb+eval[wrapper]:242:in `install'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/application/face_base.rb:247:in `main'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/application.rb:366:in `run_command'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/application.rb:358:in `block in run'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/util.rb:666:in `exit_on_fail'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/application.rb:358:in `run'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/util/command_line.rb:132:in `run'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/util/command_line.rb:72:in `execute'
          /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/bin/puppet:5:in `<top (required)>'
          /home/xxxxx/bin/puppet:23:in `load'
          /home/xxxxx/bin/puppet:23:in `<top (required)>'
          /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli/exec.rb:74:in `load'
          /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli/exec.rb:74:in `kernel_load'
          /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli/exec.rb:27:in `run'
          /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli.rb:360:in `exec'
          /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
          /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
          /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/vendor/thor/lib/thor.rb:369:in `dispatch'
          /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli.rb:20:in `dispatch'
          /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/vendor/thor/lib/thor/base.rb:444:in `start'
          /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli.rb:10:in `start'
          /usr/local/share/gems/gems/bundler-1.15.0/exe/bundle:35:in `block in <top (required)>'
          /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/friendly_errors.rb:121:in `with_friendly_errors'
          /usr/local/share/gems/gems/bundler-1.15.0/exe/bundle:27:in `<top (required)>'
          /usr/local/bin/bundle:23:in `load'
          /usr/local/bin/bundle:23:in `<main>'
          Error: Try 'puppet help module install' for usage
          

          Gemspec https://github.com/puppetlabs/forge-ruby/blob/master/puppet_forge.gemspec

          Show
          elconas Robert Heinzmann added a comment - - edited Hello, it seems to be only a problem when run with "bundle exec". This worked for puppet 4.10.4. In our pipeline librarian is called with "bundle exec librarian-puppet install". This failes if librarian-puppet is in the Gemfile: Error: wrong number of arguments (2 for 1) json is missin in Gemfile: cannot load such file – json OK: gem 'json_pure' gem 'puppet', '~> 5.0' Failes with Error: wrong number of arguments (2 for 1): gem 'librarian-puppet' gem 'json_pure' gem 'puppet', '~> 5.0' Failes with Error: json missing: gem 'puppet', '~> 5.0' Debugging (bundle exec gem list) shows that the failing Gem is puppet_forge: Failes with Error: wrong number of arguments (2 for 1) gem 'puppet_forge' gem 'json_pure' gem 'puppet', '~> 5.0' Notice: Preparing to install into /xxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/AlexCline-dirtree/0.2.1 ... Notice: Downloading from https://forgeapi.puppetlabs.com ... Error: wrong number of arguments (2 for 1) Error: Try 'puppet help module install' for usage Trace: Notice: Preparing to install into /xxx/.tmp/librarian/cache/source/puppet/forge/forgeapi_puppetlabs_com/AlexCline-dirtree/0.2.1 ... Notice: Downloading from https://forgeapi.puppetlabs.com ... Error: wrong number of arguments (2 for 1) /usr/local/share/gems/gems/semantic_puppet-0.1.4/lib/semantic_puppet/version_range.rb:26:in `parse' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/module_tool/applications/installer.rb:209:in `build_single_module_graph' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/module_tool/applications/installer.rb:86:in `run' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/module_tool/applications/application.rb:11:in `run' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/face/module/install.rb:131:in `block (3 levels) in <top (required)>' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/interface/action.rb+eval[wrapper]:242:in `install' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/application/face_base.rb:247:in `main' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/application.rb:366:in `run_command' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/application.rb:358:in `block in run' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/util.rb:666:in `exit_on_fail' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/application.rb:358:in `run' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/util/command_line.rb:132:in `run' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/util/command_line.rb:72:in `execute' /home/xxxxx/.gem/ruby/gems/puppet-5.0.0/bin/puppet:5:in `<top (required)>' /home/xxxxx/bin/puppet:23:in `load' /home/xxxxx/bin/puppet:23:in `<top (required)>' /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli/exec.rb:74:in `load' /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli/exec.rb:74:in `kernel_load' /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli/exec.rb:27:in `run' /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli.rb:360:in `exec' /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run' /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command' /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/vendor/thor/lib/thor.rb:369:in `dispatch' /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli.rb:20:in `dispatch' /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/vendor/thor/lib/thor/base.rb:444:in `start' /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/cli.rb:10:in `start' /usr/local/share/gems/gems/bundler-1.15.0/exe/bundle:35:in `block in <top (required)>' /usr/local/share/gems/gems/bundler-1.15.0/lib/bundler/friendly_errors.rb:121:in `with_friendly_errors' /usr/local/share/gems/gems/bundler-1.15.0/exe/bundle:27:in `<top (required)>' /usr/local/bin/bundle:23:in `load' /usr/local/bin/bundle:23:in `<main>' Error: Try 'puppet help module install' for usage Gemspec https://github.com/puppetlabs/forge-ruby/blob/master/puppet_forge.gemspec
          Hide
          elconas Robert Heinzmann added a comment - - edited

          Debugged some more and it seems puppet 5.0 now comes with a bundled SemanticPuppet gem that has a new API "/home/xxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/vendor/semantic_puppet/lib/semantic_puppet.rb

          So if semantic_puppet is used as a gem by some other module (e.g. puppet_forge) this breaks load path.

          Why not fix / release and use the external gem ? This would allow tool developers to follow puppet internal semver versioning ?

          Show
          elconas Robert Heinzmann added a comment - - edited Debugged some more and it seems puppet 5.0 now comes with a bundled SemanticPuppet gem that has a new API "/home/xxxx/.gem/ruby/gems/puppet-5.0.0/lib/puppet/vendor/semantic_puppet/lib/semantic_puppet.rb So if semantic_puppet is used as a gem by some other module (e.g. puppet_forge) this breaks load path. Why not fix / release and use the external gem ? This would allow tool developers to follow puppet internal semver versioning ?
          Hide
          thomas.hallgren Thomas Hallgren added a comment - - edited

          Robert Heinzmann, the external gem is both fixed and released. According to the trace, you're using version 0.1.4. The current version of semantic_puppet is 1.0.0.

          That said, upgrading will not help you in this case, since the module tool relies on the vendored version of semantic_puppet where we've added code to cater for backward compatibility issues. As part of that, an additional (but optional) argument was added to the VersionRange#parse method. The released gem does not expect that argument.

          Show
          thomas.hallgren Thomas Hallgren added a comment - - edited Robert Heinzmann , the external gem is both fixed and released. According to the trace, you're using version 0.1.4. The current version of semantic_puppet is 1.0.0 . That said, upgrading will not help you in this case, since the module tool relies on the vendored version of semantic_puppet where we've added code to cater for backward compatibility issues. As part of that, an additional (but optional) argument was added to the VersionRange#parse method. The released gem does not expect that argument.
          Hide
          elconas Robert Heinzmann added a comment -

          Agreed

          ...where we've added code to cater for backward compatibility issues...

          Shouldn't this also be backported to semantic_puppet then, so that puppet_forge module can pick up the changes and then librarian will work again ?

          Show
          elconas Robert Heinzmann added a comment - Agreed ...where we've added code to cater for backward compatibility issues... Shouldn't this also be backported to semantic_puppet then, so that puppet_forge module can pick up the changes and then librarian will work again ?
          Hide
          thomas.hallgren Thomas Hallgren added a comment -

          That could be a solution but it's debatable since it introduces broken semver behavior that is undesirable for all other use-cases. Would it be possible to change librarian to instead use the vendored version?

          Show
          thomas.hallgren Thomas Hallgren added a comment - That could be a solution but it's debatable since it introduces broken semver behavior that is undesirable for all other use-cases. Would it be possible to change librarian to instead use the vendored version?
          Hide
          elconas Robert Heinzmann added a comment - - edited

          It seems that librarian-puppet (https://github.com/voxpupuli/librarian-puppet/blob/master/librarian-puppet.gemspec) uses puppet_forge (https://github.com/puppetlabs/forge-ruby/blob/master/puppet_forge.gemspec) a gem from puppetlabs and that one uses semantic_puppet.

          So fixing puppet_forge (on puppet side) would solve the issue

          In general https://github.com/puppetlabs/forge-ruby still uses 0.x version, while 1.0 is out "fixing" things, so toughing this code may valuable anyway.

          Show
          elconas Robert Heinzmann added a comment - - edited It seems that librarian-puppet ( https://github.com/voxpupuli/librarian-puppet/blob/master/librarian-puppet.gemspec ) uses puppet_forge ( https://github.com/puppetlabs/forge-ruby/blob/master/puppet_forge.gemspec ) a gem from puppetlabs and that one uses semantic_puppet. So fixing puppet_forge (on puppet side) would solve the issue In general https://github.com/puppetlabs/forge-ruby still uses 0.x version, while 1.0 is out "fixing" things, so toughing this code may valuable anyway.
          Hide
          kenn Kenn Hussey added a comment -
          Show
          kenn Kenn Hussey added a comment - Craig Gomes ^^^
          Hide
          hunter Hunter (Hunner) Haugen added a comment -

          In retrospect, to "vendor" something means to include a released codebase in your own codebase, but the codebase present in https://github.com/puppetlabs/puppet/tree/5.0.0/lib/puppet/vendor/semantic_puppet/lib/semantic_puppet is not a pre-existing codebase.

          Show
          hunter Hunter (Hunner) Haugen added a comment - In retrospect, to "vendor" something means to include a released codebase in your own codebase, but the codebase present in https://github.com/puppetlabs/puppet/tree/5.0.0/lib/puppet/vendor/semantic_puppet/lib/semantic_puppet is not a pre-existing codebase.
          Hide
          josh Josh Cooper added a comment -

          So it's complicated. Originally it was semver and was implemented in puppet, see https://projects.puppetlabs.com/issues/7204. Then we used that as the basis for constructing an external semantic library (since it was needed by puppet, r10k, etc). But we renamed it to semantic_puppet because there was already a semantic gem on the forge. Up until recently, puppet still vendored the old crusty version of semantic_puppet 0.1.4, and we updated that to 1.0.0 for puppet 5. The intention has always been to unvendor semantic_puppet from puppet, but we ran into puppetserver packaging issues.

          Show
          josh Josh Cooper added a comment - So it's complicated. Originally it was semver and was implemented in puppet, see https://projects.puppetlabs.com/issues/7204 . Then we used that as the basis for constructing an external semantic library (since it was needed by puppet, r10k, etc). But we renamed it to semantic_puppet because there was already a semantic gem on the forge. Up until recently, puppet still vendored the old crusty version of semantic_puppet 0.1.4 , and we updated that to 1.0.0 for puppet 5. The intention has always been to unvendor semantic_puppet from puppet, but we ran into puppetserver packaging issues.
          Hide
          josh Josh Cooper added a comment -

          This issue is occurring because Puppet's module tool calls:

                  range = SemanticPuppet::VersionRange.parse(version, @strict_semver)
          

          Normally, that invokes the vendored semantic_puppet library in puppet whose parse method accepts 2 arguments. However, when running in bundler and semantic_puppet is a gem dependency, then bundler will put the external gem ahead in the $LOAD_PATH. So then the PMT tries to call the wrong version of the method, and it fails with:

          Error: wrong number of arguments (2 for 1)
          

          The immediate fix is going to be to fix the external semantic_puppet gem to accept an optional argument and release a new version of the gem. So closing this as a dup of MODULES-5155.

          Show
          josh Josh Cooper added a comment - This issue is occurring because Puppet's module tool calls: range = SemanticPuppet::VersionRange.parse(version, @strict_semver ) Normally, that invokes the vendored semantic_puppet library in puppet whose parse method accepts 2 arguments. However, when running in bundler and semantic_puppet is a gem dependency, then bundler will put the external gem ahead in the $LOAD_PATH . So then the PMT tries to call the wrong version of the method, and it fails with: Error: wrong number of arguments (2 for 1) The immediate fix is going to be to fix the external semantic_puppet gem to accept an optional argument and release a new version of the gem. So closing this as a dup of MODULES-5155 .
          Hide
          thomas.hallgren Thomas Hallgren added a comment -

          Reopening this since we decided that the puppet module tool needs to check the arity of the VersionRange#parse method, or it will still fail when using other versions of the gem.

          Although this will fix almost all use-cases, there is still one that will remain. And that is if the semantic_puppet version 1.0.x is used. Its method has an arity of 1 and when called, it will always enforce strict semver semantics. We can only fix this partially by providing a 1.0.1 version of the gem where the patch is applied but the problem will still remain for the 1.0.0. version.

          Show
          thomas.hallgren Thomas Hallgren added a comment - Reopening this since we decided that the puppet module tool needs to check the arity of the VersionRange#parse method, or it will still fail when using other versions of the gem. Although this will fix almost all use-cases, there is still one that will remain. And that is if the semantic_puppet version 1.0.x is used. Its method has an arity of 1 and when called, it will always enforce strict semver semantics. We can only fix this partially by providing a 1.0.1 version of the gem where the patch is applied but the problem will still remain for the 1.0.0. version.
          Hide
          elconas Robert Heinzmann added a comment -

          Sorry to interfere from the outside (more or less), but I still do not get (as more of a ops then a dev guy) why to bundle semantic_puppet gem within the puppet gem, while semantic_puppet has initially been created to avoid code duplicates. Shouldn't you just depend on if in gemspec and bundle it in Puppet AIO in a tested version ?

          Show
          elconas Robert Heinzmann added a comment - Sorry to interfere from the outside (more or less), but I still do not get (as more of a ops then a dev guy) why to bundle semantic_puppet gem within the puppet gem, while semantic_puppet has initially been created to avoid code duplicates. Shouldn't you just depend on if in gemspec and bundle it in Puppet AIO in a tested version ?
          Hide
          thomas.hallgren Thomas Hallgren added a comment -

          Robert Heinzmann, I understand your concern and we are going to remove the bundled version. We did however decide to postpone that for Puppet 6 (see PUP-7115). The reason is partly because we want to retain a somewhat broken behavior that was present in 0.1.4 but has been corrected in 1.0.0. Incidentally, this is also what's causing the current problems with the extra argument. We didn't really anticipate that users were using the gem and that doing so actually overrides the vendored gem.

          Another reason for not removing it right now is that we're concerned that users might depend on the gem being vendored and thus be left with broken configurations if it's no longer present.

          Show
          thomas.hallgren Thomas Hallgren added a comment - Robert Heinzmann , I understand your concern and we are going to remove the bundled version. We did however decide to postpone that for Puppet 6 (see PUP-7115 ). The reason is partly because we want to retain a somewhat broken behavior that was present in 0.1.4 but has been corrected in 1.0.0. Incidentally, this is also what's causing the current problems with the extra argument. We didn't really anticipate that users were using the gem and that doing so actually overrides the vendored gem. Another reason for not removing it right now is that we're concerned that users might depend on the gem being vendored and thus be left with broken configurations if it's no longer present.
          Hide
          lazyfrosch Markus Frosch added a comment -

          Thanks for dealing with that!

          Show
          lazyfrosch Markus Frosch added a comment - Thanks for dealing with that!
          Hide
          dhollinger David Hollinger added a comment -

          Josh Cooper,

          Any idea when such a fix might be pushed into the external gem and release. Right now this issue is preventing users of the VP metadata-json-lint tool from using Puppet 5 as it breaks puppet module install with the external gem and the metadata-json-lint gem is broken without the external gem.

          We're looking into workarounds on our end, but it'd be preferential to not have to hack together some crazy logic depending on Puppet version.

          Show
          dhollinger David Hollinger added a comment - Josh Cooper , Any idea when such a fix might be pushed into the external gem and release. Right now this issue is preventing users of the VP metadata-json-lint tool from using Puppet 5 as it breaks puppet module install with the external gem and the metadata-json-lint gem is broken without the external gem. We're looking into workarounds on our end, but it'd be preferential to not have to hack together some crazy logic depending on Puppet version.
          Hide
          josh Josh Cooper added a comment -

          Hey David Hollinger, I'm reviewing the changes now, hope to get that merged today. But next Monday and Tues are US/company holidays, so it's going to be next week before we can get a fix published.

          To repro, create a Gemfile containing:

          source "https://rubygems.org"
          gem "puppet"
          gem "semantic_puppet"
          

          $ bundle install --path .bundle
          Fetching gem metadata from https://rubygems.org/..........
          Fetching version metadata from https://rubygems.org/.
          Resolving dependencies...
          Installing CFPropertyList 2.2.8
          Installing fast_gettext 1.1.0
          Installing locale 2.1.2
          Installing text 1.3.1
          Installing hiera 3.4.0
          Using bundler 1.14.6
          Installing facter 2.4.6 (universal-darwin)
          Installing gettext 3.2.3
          Installing gettext-setup 0.25
          Installing puppet 5.0.0 (universal-darwin)
          Installing semantic_puppet 1.0.0
          Bundle complete! 2 Gemfile dependencies, 11 gems now installed.
          Bundled gems are installed into ./.bundle.
          

          Trying to puppet module install fails with cryptic error:

          $ bundle exec puppet module install puppetlabs-apache
          Notice: Preparing to install into /Users/josh/.puppetlabs/etc/code/modules ...
          Notice: Downloading from https://forgeapi.puppet.com ...
          Error: Could not install module 'puppetlabs-apache' (???)
            No version of 'puppetlabs-apache' can satisfy all dependencies
              Use `puppet module install --ignore-dependencies` to install only this module
          

          Retry with --ignore-dependencies, fails with wrong number of arguments error.

          $ puppet module install puppetlabs-apache --ignore-dependencies --trace
          Notice: Preparing to install into /Users/josh/.puppetlabs/etc/code/modules ...
          Notice: Downloading from https://forgeapi.puppet.com ...
          Error: wrong number of arguments (given 2, expected 1)
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/semantic_puppet-1.0.0/lib/semantic_puppet/version_range.rb:64:in `parse'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/module_tool/applications/installer.rb:209:in `build_single_module_graph'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/module_tool/applications/installer.rb:86:in `run'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/module_tool/applications/application.rb:11:in `run'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/face/module/install.rb:131:in `block (3 levels) in <top (required)>'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/interface/action.rb+eval[wrapper]:242:in `install'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/application/face_base.rb:247:in `main'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/application.rb:366:in `run_command'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/application.rb:358:in `block in run'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/util.rb:666:in `exit_on_fail'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/application.rb:358:in `run'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/util/command_line.rb:132:in `run'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/util/command_line.rb:72:in `execute'
          /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/bin/puppet:5:in `<top (required)>'
          

          Show
          josh Josh Cooper added a comment - Hey David Hollinger , I'm reviewing the changes now, hope to get that merged today. But next Monday and Tues are US/company holidays, so it's going to be next week before we can get a fix published. To repro, create a Gemfile containing: source "https://rubygems.org" gem "puppet" gem "semantic_puppet" $ bundle install --path .bundle Fetching gem metadata from https://rubygems.org/.......... Fetching version metadata from https://rubygems.org/. Resolving dependencies... Installing CFPropertyList 2.2.8 Installing fast_gettext 1.1.0 Installing locale 2.1.2 Installing text 1.3.1 Installing hiera 3.4.0 Using bundler 1.14.6 Installing facter 2.4.6 (universal-darwin) Installing gettext 3.2.3 Installing gettext-setup 0.25 Installing puppet 5.0.0 (universal-darwin) Installing semantic_puppet 1.0.0 Bundle complete! 2 Gemfile dependencies, 11 gems now installed. Bundled gems are installed into ./.bundle. Trying to puppet module install fails with cryptic error: $ bundle exec puppet module install puppetlabs-apache Notice: Preparing to install into /Users/josh/.puppetlabs/etc/code/modules ... Notice: Downloading from https://forgeapi.puppet.com ... Error: Could not install module 'puppetlabs-apache' (???) No version of 'puppetlabs-apache' can satisfy all dependencies Use `puppet module install --ignore-dependencies` to install only this module Retry with --ignore-dependencies , fails with wrong number of arguments error. $ puppet module install puppetlabs-apache --ignore-dependencies --trace Notice: Preparing to install into /Users/josh/.puppetlabs/etc/code/modules ... Notice: Downloading from https://forgeapi.puppet.com ... Error: wrong number of arguments (given 2, expected 1) /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/semantic_puppet-1.0.0/lib/semantic_puppet/version_range.rb:64:in `parse' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/module_tool/applications/installer.rb:209:in `build_single_module_graph' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/module_tool/applications/installer.rb:86:in `run' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/module_tool/applications/application.rb:11:in `run' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/face/module/install.rb:131:in `block (3 levels) in <top (required)>' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/interface/action.rb+eval[wrapper]:242:in `install' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/application/face_base.rb:247:in `main' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/application.rb:366:in `run_command' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/application.rb:358:in `block in run' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/util.rb:666:in `exit_on_fail' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/application.rb:358:in `run' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/util/command_line.rb:132:in `run' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/lib/puppet/util/command_line.rb:72:in `execute' /Users/josh/work/test_semantic/.bundle/ruby/2.3.0/gems/puppet-5.0.0-universal-darwin/bin/puppet:5:in `<top (required)>'
          Hide
          josh Josh Cooper added a comment -

          The issue James Pogran mentioned is due to our semantic_puppet implementation (in both the external gem and in our vendored implementation) is MODULES-5159.

          Show
          josh Josh Cooper added a comment - The issue James Pogran mentioned is due to our semantic_puppet implementation (in both the external gem and in our vendored implementation) is MODULES-5159 .
          Show
          josh Josh Cooper added a comment - Merged to 5.0.x in https://github.com/puppetlabs/puppet/commit/7f53ea252a1f25189b9264e0f8dd5c5089f96753
          Hide
          josh Josh Cooper added a comment - - edited

          To watchers on the ticket, can you try modifying your Gemfile as:

          gem "puppet", :git => "https://github.com/puppetlabs/puppet", :branch => "5.0.x"
          

          And make sure this fix resolves the issue you're seeing?

          Show
          josh Josh Cooper added a comment - - edited To watchers on the ticket, can you try modifying your Gemfile as: gem "puppet" , :git => "https://github.com/puppetlabs/puppet" , :branch => "5.0.x" And make sure this fix resolves the issue you're seeing?
          Hide
          tdevelioglu Taylan Develioglu added a comment -

          Josh Cooper Can confirm this fixes the issue.

          Show
          tdevelioglu Taylan Develioglu added a comment - Josh Cooper Can confirm this fixes the issue.
          Hide
          kenn Kenn Hussey added a comment -

          Josh Cooper please provide release notes for this issue, if needed.

          Show
          kenn Kenn Hussey added a comment - Josh Cooper please provide release notes for this issue, if needed.
          Hide
          josh Josh Cooper added a comment -

          Awesome, thanks Taylan Develioglu!

          Show
          josh Josh Cooper added a comment - Awesome, thanks Taylan Develioglu !
          Hide
          erict Eric Thompson added a comment - - edited

          Erik Dasher, this looks like a prime first suspect for automation.

          bundle install with the Gemfile above (possibly JUST with semantic_puppet?) and bundle exec puppet module install with a version specified and strict_semver turned on.

          looks like this needs a testrail entry

          this is partially covered by unit tests. but needs to test external gems don't mess with internal function namespacing.

          Show
          erict Eric Thompson added a comment - - edited Erik Dasher , this looks like a prime first suspect for automation. bundle install with the Gemfile above (possibly JUST with semantic_puppet?) and bundle exec puppet module install with a version specified and strict_semver turned on. looks like this needs a testrail entry this is partially covered by unit tests. but needs to test external gems don't mess with internal function namespacing.
          Hide
          kenn Kenn Hussey added a comment -

          Eric Thompson Erik Dasher please create a separate ticket to track addition of automated tests per Eric Thompson's comment.

          Show
          kenn Kenn Hussey added a comment - Eric Thompson Erik Dasher please create a separate ticket to track addition of automated tests per Eric Thompson 's comment.
          Hide
          erict Eric Thompson added a comment -

          validated using a bundle install of semantic-puppet and puppet as above on ubuntu1404 (puppet gem from source ( 5.0.x@bcda912)):

          root@yr8hnm5adwkpdpg:~# bundle exec puppet module install puppetlabs-apache
          Warning: Support for ruby version 1.9.3 is deprecated and will be removed in a future release. See https://docs.puppet.com/puppet/latest/system_requirements.html#ruby for a list of supported ruby versions.
             (at /root/.bundle/ruby/1.9.1/gems/puppet-5.0.0/lib/puppet.rb:181:in `<module:Puppet>')
          Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...
          Notice: Downloading from https://forgeapi.puppet.com ...
          Error: Could not install module 'puppetlabs-apache' (???)
            No version of 'puppetlabs-apache' can satisfy all dependencies
              Use `puppet module install --ignore-dependencies` to install only this module
           
          root@yr8hnm5adwkpdpg:~# vi Gemfile
           
          root@yr8hnm5adwkpdpg:~# bundle update
          Fetching https://github.com/puppetlabs/puppet
          Fetching gem metadata from https://rubygems.org/..........
          Fetching version metadata from https://rubygems.org/.
          Resolving dependencies...
          Rubygems 1.8.23 is not threadsafe, so your gems will be installed one at a time. Upgrade to Rubygems 2.1.0 or higher to enable parallel gem installation.
          Fetching CFPropertyList 2.3.5
          Installing CFPropertyList 2.3.5
          Using bundler 1.15.2
          Using facter 2.5.0
          Using fast_gettext 1.1.0
          Using locale 2.1.2
          Using text 1.3.1
          Using hiera 3.4.0
          Fetching hocon 1.2.5
          Installing hocon 1.2.5
          Using gettext 3.2.3
          Using gettext-setup 0.26
          Using puppet 5.0.1 from https://github.com/puppetlabs/puppet (at 5.0.x@bcda912)
          Using semantic_puppet 1.0.1
          Bundle updated!
          root@yr8hnm5adwkpdpg:~# bundle exec puppet module install puppetlabs-apache
          Warning: Support for ruby version 1.9.3 is deprecated and will be removed in a future release. See https://docs.puppet.com/puppet/latest/system_requirements.html#ruby for a list of supported ruby versions.
             (at /root/.bundle/ruby/1.9.1/bundler/gems/puppet-bcda9125e43a/lib/puppet.rb:181:in `<module:Puppet>')
          Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...
          Notice: Downloading from https://forgeapi.puppet.com ...
          Notice: VersionRanges will always be strict when using non-vendored SemanticPuppet gem, version 1.0.1
          Notice: Installing -- do not interrupt ...
          /etc/puppetlabs/code/environments/production/modules
          └─┬ puppetlabs-apache (v1.11.0)
            ├── puppetlabs-concat (v2.2.1)
            └── puppetlabs-stdlib (v4.17.1)
          

          Show
          erict Eric Thompson added a comment - validated using a bundle install of semantic-puppet and puppet as above on ubuntu1404 (puppet gem from source ( 5.0.x@bcda912)): root@yr8hnm5adwkpdpg:~# bundle exec puppet module install puppetlabs-apache Warning: Support for ruby version 1.9.3 is deprecated and will be removed in a future release. See https://docs.puppet.com/puppet/latest/system_requirements.html#ruby for a list of supported ruby versions. (at /root/.bundle/ruby/1.9.1/gems/puppet-5.0.0/lib/puppet.rb:181:in `<module:Puppet>') Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ... Notice: Downloading from https://forgeapi.puppet.com ... Error: Could not install module 'puppetlabs-apache' (???) No version of 'puppetlabs-apache' can satisfy all dependencies Use `puppet module install --ignore-dependencies` to install only this module   root@yr8hnm5adwkpdpg:~# vi Gemfile   root@yr8hnm5adwkpdpg:~# bundle update Fetching https://github.com/puppetlabs/puppet Fetching gem metadata from https://rubygems.org/.......... Fetching version metadata from https://rubygems.org/. Resolving dependencies... Rubygems 1.8.23 is not threadsafe, so your gems will be installed one at a time. Upgrade to Rubygems 2.1.0 or higher to enable parallel gem installation. Fetching CFPropertyList 2.3.5 Installing CFPropertyList 2.3.5 Using bundler 1.15.2 Using facter 2.5.0 Using fast_gettext 1.1.0 Using locale 2.1.2 Using text 1.3.1 Using hiera 3.4.0 Fetching hocon 1.2.5 Installing hocon 1.2.5 Using gettext 3.2.3 Using gettext-setup 0.26 Using puppet 5.0.1 from https://github.com/puppetlabs/puppet (at 5.0.x@bcda912) Using semantic_puppet 1.0.1 Bundle updated! root@yr8hnm5adwkpdpg:~# bundle exec puppet module install puppetlabs-apache Warning: Support for ruby version 1.9.3 is deprecated and will be removed in a future release. See https://docs.puppet.com/puppet/latest/system_requirements.html#ruby for a list of supported ruby versions. (at /root/.bundle/ruby/1.9.1/bundler/gems/puppet-bcda9125e43a/lib/puppet.rb:181:in `<module:Puppet>') Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ... Notice: Downloading from https://forgeapi.puppet.com ... Notice: VersionRanges will always be strict when using non-vendored SemanticPuppet gem, version 1.0.1 Notice: Installing -- do not interrupt ... /etc/puppetlabs/code/environments/production/modules └─┬ puppetlabs-apache (v1.11.0) ├── puppetlabs-concat (v2.2.1) └── puppetlabs-stdlib (v4.17.1)

            People

            • Assignee:
              Unassigned
              Reporter:
              elconas Robert Heinzmann
            • Votes:
              2 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Agile