Uploaded image for project: 'Puppet Server'
  1. Puppet Server
  2. SERVER-2763

Some Dir methods broken with puppetserver's custom classpath

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: SERVER 6.10.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Template:
      PUP Bug Template
    • Team:
      Froyo
    • Method Found:
      Needs Assessment
    • QA Risk Assessment:
      Needs Assessment

      Description

      It looks like there was a regression in 9.2.10 that broke globbing from jars in some circumstances. There was a fix in 9.2.11, see https://github.com/jruby/jruby/pull/6084. However the way we specify the "puppetserver-lib" location for some configurations (maybe just cli tools??) cannot be correctly expanded and is causing this issue.

      Specifically with rdoc/rubygems. rdoc tries to load extensions by calling a rubygems function called "find_files" which searches the $LOAD_PATH for a specified glob. It is effectively this:

      $LOAD_PATH.flat_map {|path| Dir.glob('rdoc/discover.rb', base: path) } 

      Here's an example from lein irb:

      irb(main):006:0> Dir.glob("*", base: "./src/ruby/puppetserver-lib")
      => ["puppet"]
       
      irb(main):007:0> $LOAD_PATH
      => ["classpath:/puppetserver-lib", "./ruby/puppet/lib", "./ruby/facter/lib", "./ruby/hiera/lib", "./ruby/resource_api/lib", "uri:classloader:/META-INF/jruby.home/lib/ruby/2.5/site_ruby", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib"]
       
      irb(main):008:0> Dir.glob "*", base:  "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib"
      => ["drb", "racc", "digest", ...snip... "rdoc", "pathname.rb"]
       
      irb(main):009:0> Dir.glob "*", base:  "classpath:/puppetserver-lib"
      Traceback (most recent call last):
             16: from org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:325)
             15: from org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:205)
             14: from org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:396)
             13: from org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)
             12: from org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)
             11: from org.jruby.RubyDir$INVOKER$s$0$2$glob.call(RubyDir$INVOKER$s$0$2$glob.gen)
             10: from org.jruby.RubyDir.glob(RubyDir.java:310)
              9: from org.jruby.util.Dir.push_glob(Dir.java:310)
              8: from org.jruby.util.Dir.push_braces(Dir.java:449)
              7: from org.jruby.util.Dir.push_globs(Dir.java:481)
              6: from org.jruby.util.Dir.glob_helper(Dir.java:693)
              5: from org.jruby.util.Dir.glob_helper(Dir.java:745)
              4: from org.jruby.util.JRubyFile.createResource(JRubyFile.java:82)
              3: from org.jruby.util.JRubyFile.createResource(JRubyFile.java:119)
              2: from org.jruby.util.JRubyFile.create(JRubyFile.java:62)
              1: from org.jruby.util.JRubyFile.createNoUnicodeConversion(JRubyFile.java:143)
      Java::JavaLang::IllegalArgumentException (Neither current working directory (classpath:/puppetserver-lib) nor pathname (.) led to an absolute path) 

       

      This is most apparent when trying to generate rdoc content for gems. Generating rdoc content can be attempted automatically in some versions of Rubygems when installing gems. In those cases this bug causes gem installation to fail as well. When that happens it looks like:

      # puppetserver gem install bundler
      Successfully installed bundler-2.1.4
      LoadError: load error: rdoc/rdoc -- java.lang.IllegalArgumentException: Neither current working directory (classpath:/puppetserver-lib) nor pathname (rdoc/discover) led to an absolute path
                       require at org/jruby/RubyKernel.java:974
                       require at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:54
                     load_rdoc at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rdoc/rubygems_hook.rb:73
                         setup at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rdoc/rubygems_hook.rb:238
                      generate at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rdoc/rubygems_hook.rb:151
               generation_hook at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rdoc/rubygems_hook.rb:56
                          each at org/jruby/RubyArray.java:1809
               generation_hook at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rdoc/rubygems_hook.rb:55
                 install_hooks at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/request_set.rb:323
                          each at org/jruby/RubyArray.java:1809
                 install_hooks at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/request_set.rb:322
                       install at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/request_set.rb:221
                   install_gem at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/commands/install_command.rb:212
                  install_gems at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/commands/install_command.rb:264
                          each at org/jruby/RubyArray.java:1809
                  install_gems at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/commands/install_command.rb:258
                       execute at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/commands/install_command.rb:165
        invoke_with_build_args at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/command.rb:321
                  process_args at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/command_manager.rb:184
                           run at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/command_manager.rb:148
                           run at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/gem_runner.rb:59
                        <main> at /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/bin/jgem:21
                          load at org/jruby/RubyKernel.java:1009
                        <main> at /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/bin/gem:4
                          load at org/jruby/RubyKernel.java:1009
                        <main> at -e:1
      

      Notice that the gem install does not actually fail. It just explodes noisily when installing docs after succeeding to install the gem itself.
       

      As a workaround for this particular manifestation, the --no-document flag can be passed to the gem install command.

      It also looks like a similar issue is occuring during a function in CD4PE's IA feature when loading logging/plugins during a catalog compilation. There isn't a known workaround for that issue besides disabling Impact Analysis.

      To fix this, we most likely need to audit how we're constructing the classpath for JRuby to ensure JRuby's standard library, Puppet's libdir and Puppet Server's vendored Ruby code are all accessible and each entry is usable from Dir.glob("...", base: <our values>).

      Before that we should also see if upgrading to JRuby 9.2.13.0 resolves the issue.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              maggie Maggie Dreyer
              Reporter:
              maggie Maggie Dreyer
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:

                  Zendesk Support