Uploaded image for project: 'Puppet Development Kit'
  1. Puppet Development Kit
  2. PDK-1177

[feature] Support for per-project additional gems

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      All PDK supported platforms

    • Template:
    • QA Risk Assessment:
      Needs Assessment

      Description

      Overview
      I would like a way to manage loading of Ruby Gems available via PDK generated Gemfile.lock on a per-project basis in a way that is safe to commit to git.

      What is being proposed
      Modify the PDK Template Gemfile to change:

      # Evaluate Gemfile.local and ~/.gemfile if they exist
      extra_gemfiles = [
        "#{__FILE__}.local",
        File.join(Dir.home, '.gemfile'),
      ]
      

      to

      # Evaluate Gemfile.local and ~/.gemfile if they exist
      extra_gemfiles = [
        "#{__FILE__}.project"
        "#{__FILE__}.local",
        File.join(Dir.home, '.gemfile'),
      ]
      

      This would allow per-project gems to be loaded via Gemfile.project which can then be added to git for consistent Ruby Gem loading.

      Why do you want to that?
      I maintain the PDQTest project and would like to call out to run PDK commands at various points in my own code. The exact reasons and mechanism that I use for this right now are captured in the manual https://github.com/declarativesystems/pdqtest/blob/master/doc/pdk.md

      How have you tried to integrate so far

      • Added the pdqtest gem to Gemfile.local - works
      • Added the pdqtest gem to the bottom of Gemfile - works but now
        Gemfile is different to what PDK generates
      • .sync.yml - observed gem being written to a group but didn't test

      What is the problem with the current approaches?

      Gemfile.local
      PDK generates .gitignore that includes Gemfile.local. Gemfile.local must be in git for reproducible tests since it belongs to the project. It's possible and encouraged for different projects to run with different PDQTest versions, so a system-wide gem version at at ~/.gemfile doesn't work.

      The reason to exclude Gemfile.local appears to be because its just a way to add Gems for the workstation, but I need to add gems to the project.

      Gemfile
      Adding directly to Gemfile creates either a manual step or an automated incompatibility with PDK, neither of which are desirable.

      .sync.yml
      .sync.yml is another way of getting Ruby Gems into Gemfile but it has issues that prevent me using it:

      • After changing .sync.yml, its required to run pdk update to regenerate an updated Gemfile. This creates an intrinsic link between pdqtest upgrade and pdk update which is not desirable. I want users to be able to upgrade/update PDK and PDQTest independently.
      • The templates only allow access to specific fields in Gemfile. At present it is only allowed to write a new Bundler group using this mechanism. This also means more typing when running bundle, although my own project alleviates this somewhat with scripts.
      • Already have a method for partially updating Gemfile as part of the pdqtest upgrade command. Fixing this to use Gemfile.project was trivial, switching to .sync.yml would require a rewrite and would also mean maintaining two separate update systems if I were to use this tool with things that aren't Puppet.

      This leads me to the simple solution of creating a file that works the same way as Gemfile.local but a way that is compatible with the PDK's default files.

      What about creating custom templates?
      PDK users can provide custom integration by providing their own set of templates forked from https://github.com/puppetlabs/pdk-templates why not do this?

      This would certainly work, but this presents other problems:

      • I don't have the resources to keep these templates aligned with upstream
      • Potential for additional customisation that diverge projects from PDK established patterns
      • Anyone else requiring similar functionality needs to fork and maintain their own template repository
      • If this is the only option, then given the extent of the required changes I would probably just write an additional line to Gemfile using some other means.

      Resource to make requested change?
      I'm happy to PR the template repository with this change

      Impact on existing users?
      None, unless they create Gemfile.project.

      Background on PDK Ruby Gem loading
      PDK controls loading of Ruby Gems via Gemfile.lock which is generated by running:

      pdk bundle install
      

      On Linux you can pretty much just use the regular bundle install command to generate this file and although this definitely isn't recommended, it kinda works today.

      On Windows its a different story. PDK forces Gemfile.lock to use it's vendored libraries in order to support the platform and running bundle install will fail and completely break PDK/Ruby until pdk bundle install has been run again.

      Versioned Ruby Gems need to be present in Gemfile to be available at runtime.

        Attachments

          Activity

            jsd-sla-details-panel

              People

              • Assignee:
                Unassigned
                Reporter:
                geoff@declarativesystems.com Geoff Williams
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: