Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-3032

Setting to cache `load_library` failures



    • Improvement
    • Status: Closed
    • Normal
    • Resolution: Fixed
    • None
    • PUP 3.7.0
    • Docs
    • None
    • 2
    • 2014-08-20, 2014-09-03


      See related PE ticket; this is just a placeholder to make sure we try to get something in for this in time for 3.7.0.

      In `rubygems.rb`, there are some calls to `Gem.clear_paths`. This is potentially a very expensive operation as it can cause ruby to have to examine all of the files in the entire gem load_path in order to rebuild its internal directory cache.

      This code is currently triggered via every call to the `load_library` method that is used to load features. Because we don't (always) cache failed attempts to load features, this means that for a feature that is not present, we will trigger this code every time we check for the feature.

      The `msgpack` feature is checked on every single HTTP request, which means that we currently call Gem.clear_paths and have to rebuild the gem cache on ever request, if `msgpack` is not available on the master.

      It'd be nice if we could get rid of the call to `Gem.clear_paths` entirely, but Josh Cooper explained that it is important in order to support the ability to load things from a gem during a single agent run (e.g., the gem is not initially installed, but is installed by the agent as part of a catalog run, and then later during the run the agent wants to check for a feature that was bundled in the gem), so this code may be necessary.

      It doesn't seem like it should be necessary to check this repeatedly on the master, though. Therefore, a simple proposal that Josh Cooper came up with would be to simply introduce a setting that would indicate that we always want to cache failures when calling `load_library`. Then we could simply set this setting on the master, and the msgpack feature check would then only trigger the call to `Gem.clear_paths` once. This could be a substantial performance improvement since it would no longer need to happen on every request.


        Issue Links



              britt Britt Gresham
              chris Chris Price
              Eric Thompson Eric Thompson
              0 Vote for this issue
              7 Start watching this issue



                Zendesk Support