[SERVER-2763] Some Dir methods broken with puppetserver's custom classpath - Puppet Tickets

Details

Type: Bug
Status: Resolved
Priority: Normal
Resolution: Fixed
Affects Version/s: SERVER 6.10.0
Fix Version/s: SERVER 6.14.0
Component/s: None
Labels:
- customer0

Template:
PUP Bug Template customfield_10700 351336
Team:
Froyo
Story Points:
2
Method Found:
Needs Assessment

Release Notes:
Bug Fix
Release Notes Summary:
Puppet Server's JRuby load path can now be used with `Dir.glob`. Notably, this re-enables installing gems with docs via `puppetserver gem`.

QA Risk Assessment:
Needs Assessment

Description

It looks like there was a regression in 9.2.10 that broke globbing from jars in some circumstances. There was a fix in 9.2.11, see https://github.com/jruby/jruby/pull/6084. However the way we specify the "puppetserver-lib" location for some configurations (maybe just cli tools??) cannot be correctly expanded and is causing this issue.

Specifically with rdoc/rubygems. rdoc tries to load extensions by calling a rubygems function called "find_files" which searches the $LOAD_PATH for a specified glob. It is effectively this:

$LOAD_PATH.flat_map {|path| Dir.glob('rdoc/discover.rb', base: path) }

Here's an example from lein irb:

irb(main):006:0> Dir.glob("*", base: "./src/ruby/puppetserver-lib")

=> ["puppet"]

irb(main):007:0> $LOAD_PATH

=> ["classpath:/puppetserver-lib", "./ruby/puppet/lib", "./ruby/facter/lib", "./ruby/hiera/lib", "./ruby/resource_api/lib", "uri:classloader:/META-INF/jruby.home/lib/ruby/2.5/site_ruby", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib"]

irb(main):008:0> Dir.glob "*", base:  "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib"

=> ["drb", "racc", "digest", ...snip... "rdoc", "pathname.rb"]

irb(main):009:0> Dir.glob "*", base:  "classpath:/puppetserver-lib"

Traceback (most recent call last):

       16: from org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:325)

       15: from org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:205)

       14: from org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:396)

       13: from org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)

       12: from org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)

       11: from org.jruby.RubyDir$INVOKER$s$0$2$glob.call(RubyDir$INVOKER$s$0$2$glob.gen)

       10: from org.jruby.RubyDir.glob(RubyDir.java:310)

        9: from org.jruby.util.Dir.push_glob(Dir.java:310)

        8: from org.jruby.util.Dir.push_braces(Dir.java:449)

        7: from org.jruby.util.Dir.push_globs(Dir.java:481)

        6: from org.jruby.util.Dir.glob_helper(Dir.java:693)

        5: from org.jruby.util.Dir.glob_helper(Dir.java:745)

        4: from org.jruby.util.JRubyFile.createResource(JRubyFile.java:82)

        3: from org.jruby.util.JRubyFile.createResource(JRubyFile.java:119)

        2: from org.jruby.util.JRubyFile.create(JRubyFile.java:62)

        1: from org.jruby.util.JRubyFile.createNoUnicodeConversion(JRubyFile.java:143)

Java::JavaLang::IllegalArgumentException (Neither current working directory (classpath:/puppetserver-lib) nor pathname (.) led to an absolute path)

This is most apparent when trying to generate rdoc content for gems. Generating rdoc content can be attempted automatically in some versions of Rubygems when installing gems. In those cases this bug causes gem installation to fail as well. When that happens it looks like:

# puppetserver gem install bundler

Successfully installed bundler-2.1.4

LoadError: load error: rdoc/rdoc -- java.lang.IllegalArgumentException: Neither current working directory (classpath:/puppetserver-lib) nor pathname (rdoc/discover) led to an absolute path

                 require at org/jruby/RubyKernel.java:974

                 require at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:54

               load_rdoc at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rdoc/rubygems_hook.rb:73

                   setup at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rdoc/rubygems_hook.rb:238

                generate at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rdoc/rubygems_hook.rb:151

         generation_hook at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rdoc/rubygems_hook.rb:56

                    each at org/jruby/RubyArray.java:1809

         generation_hook at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rdoc/rubygems_hook.rb:55

           install_hooks at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/request_set.rb:323

                    each at org/jruby/RubyArray.java:1809

           install_hooks at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/request_set.rb:322

                 install at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/request_set.rb:221

             install_gem at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/commands/install_command.rb:212

            install_gems at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/commands/install_command.rb:264

                    each at org/jruby/RubyArray.java:1809

            install_gems at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/commands/install_command.rb:258

                 execute at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/commands/install_command.rb:165

  invoke_with_build_args at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/command.rb:321

            process_args at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/command_manager.rb:184

                     run at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/command_manager.rb:148

                     run at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/gem_runner.rb:59

                  <main> at /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/bin/jgem:21

                    load at org/jruby/RubyKernel.java:1009

                  <main> at /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/bin/gem:4

                    load at org/jruby/RubyKernel.java:1009

                  <main> at -e:1

Notice that the gem install does not actually fail. It just explodes noisily when installing docs after succeeding to install the gem itself.

As a workaround for this particular manifestation, the --no-document flag can be passed to the gem install command.

It also looks like a similar issue is occuring during a function in CD4PE's IA feature when loading logging/plugins during a catalog compilation. There isn't a known workaround for that issue besides disabling Impact Analysis.

To fix this, we most likely need to audit how we're constructing the classpath for JRuby to ensure JRuby's standard library, Puppet's libdir and Puppet Server's vendored Ruby code are all accessible and each entry is usable from Dir.glob("...", base: <our values>).

Before that we should also see if upgrading to JRuby 9.2.13.0 resolves the issue.

Attachments

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

Attachments

versioned-fs.log
197 kB
2020/07/21 2:05 PM

Issue Links

relates to

SERVER-2758 Update JRuby past 9.2.8.0

Resolved

Some Dir methods broken with puppetserver's custom classpath

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates

Zendesk Support