Uploaded image for project: 'Puppet Development Kit'
  1. Puppet Development Kit
  2. PDK-1192

PDK scans non-puppet directories and chokes on files it should ignore

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: PDK 1.8.0
    • Component/s: None
    • Environment:
    • Template:
    • Method Found:
      Needs Assessment
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      PDK validate and unit test commands will now ignore files in the vendor directory.
    • QA Risk Assessment:
      Needs Assessment

      Description

      Overview
      PDK scans directories it should ignore and fails validation on any .pp files these directories contain.

      This is important if you want to be able to run the "real" PDK validate commands in CI systems such as Travis using a custom .travis.yml file

      Analysis
      Uploading a .travis.yml to run pdk validate metadata,puppet in debug mode revealed gems being installed to ./vendor

      Two failure modes were observed:
      1. puppet-syntax: UTF-8: Could not parse for environment production: invalid byte sequence
      2. Attempt to redefine entity 'http://puppet.com/2016.1/runtime/type/puppet::ast'

      steps to reproduce
      1. pdk new module
      2. mkdir -p vendor/bundle
      3. gem install -i vendor/bundle XXX
      4. pdk validate metadata,puppet

      UTF-8 Error
      Caused by installing puppet-5.3.5 gem in vendor/bundle

      Redefine entity
      Caused by installing puppet-6.0.0 gem in vendor/bundle

      Testcase output

      geoff@computer:~/tmp$ pdk new module --skip-interview pentity
      pdk (INFO): Creating new module: pentity
      pdk (INFO): Module 'pentity' generated at path '/home/geoff/tmp/pentity', from template 'file:///opt/puppetlabs/pdk/share/cache/pdk-templates.git'.
      pdk (INFO): In your module directory, add classes with the 'pdk new class' command.
      geoff@computer:~/tmp$ cd pentity/
      geoff@computer:~/tmp/pentity$ mkdir -p vendor/bundle
      geoff@computer:~/tmp/pentity$ gem install -i vendor/bundle/ puppet
      Fetching: facter-2.5.1.gem (100%)
      Successfully installed facter-2.5.1
      Fetching: hiera-3.4.5.gem (100%)
      Successfully installed hiera-3.4.5
      Fetching: semantic_puppet-1.0.2.gem (100%)
      Successfully installed semantic_puppet-1.0.2
      Fetching: fast_gettext-1.1.2.gem (100%)
      Successfully installed fast_gettext-1.1.2
      Fetching: locale-2.1.2.gem (100%)
      Successfully installed locale-2.1.2
      Fetching: multi_json-1.13.1.gem (100%)
      Successfully installed multi_json-1.13.1
      Fetching: httpclient-2.8.3.gem (100%)
      Successfully installed httpclient-2.8.3
      Fetching: hocon-1.2.5.gem (100%)
      Successfully installed hocon-1.2.5
      Fetching: puppet-resource_api-1.6.0.gem (100%)
      Successfully installed puppet-resource_api-1.6.0
      Fetching: puppet-6.0.0.gem (100%)
      Successfully installed puppet-6.0.0
      10 gems installed
      geoff@computer:~/tmp/pentity$ pdk validate metadata,pupppet
      pdk (WARN): Unknown validator 'pupppet'. Available validators: metadata, puppet, ruby.
      pdk (INFO): Using Ruby 2.4.4
      pdk (INFO): Using Puppet 5.5.6
      [✔] Checking metadata syntax (metadata.json tasks/*.json).
      [✔] Checking module metadata style (metadata.json).
      info: task-metadata-lint: ./: Target does not contain any files to validate (tasks/*.json).
      geoff@computer:~/tmp/pentity$ pdk validate metadata,puppet
      pdk (INFO): Using Ruby 2.4.4
      pdk (INFO): Using Puppet 5.5.6
      [✔] Checking metadata syntax (metadata.json tasks/*.json).
      [✔] Checking module metadata style (metadata.json).
      [✖] Checking Puppet manifest syntax (**/**.pp).
      info: task-metadata-lint: ./: Target does not contain any files to validate (tasks/*.json).
      Error: puppet-syntax: Attempt to redefine entity 'http://puppet.com/2016.1/runtime/type/puppet::ast'. Originally set at file:/home/geoff/.pdk/cache/ruby/2.4.0/gems/puppet-5.5.6/lib/puppet/pops/model/ast.rb?line=1.
       
      geoff@computer:~/tmp/pentity$ gem install -i vendor/bundle/ r10k
      geoff@computer:~/tmp/pentity$ gem install -i vendor/bundle/ puppet --version 5.5.3
      Fetching: puppet-5.5.3.gem (100%)
      Successfully installed puppet-5.5.3
      1 gem installed
      geoff@computer:~/tmp/pentity$ pdk validate metadata,puppet
      pdk (INFO): Using Ruby 2.4.4
      pdk (INFO): Using Puppet 5.5.6
      [✔] Checking metadata syntax (metadata.json tasks/*.json).
      [✔] Checking module metadata style (metadata.json).
      [✖] Checking Puppet manifest syntax (**/**.pp).
      info: task-metadata-lint: ./: Target does not contain any files to validate (tasks/*.json).
      Error: puppet-syntax: UTF-8: Could not parse for environment production: invalid byte sequence
      
      

      Workaround
      Vendor directory can't be removed - it's needed for running tests
      Since tests fails are only caused by .pp files which themselves are only testcases, remove them as part of travis run:

      find vendor/ -iname '*.pp' -exec rm {} \;
      

        Attachments

          Issue Links

            Activity

              jsd-sla-details-panel

                People

                • Assignee:
                  tim.sharpe Tim Sharpe
                  Reporter:
                  geoff@declarativesystems.com Geoff Williams
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: