Uploaded image for project: 'Puppet Development Kit'
  1. Puppet Development Kit
  2. PDK-950

pdk test fails trying to compile native gems with beaker on Windows

    Details

    • Template:
    • Team:
      Puppet Developer Experience
    • Method Found:
      Needs Assessment
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Hide
      To address this specific issue, PDK will package the beaker dependencies that require native compilation. Modules that include beaker in their Gemfile can now install and use the packaged PDK without having to alter their Gemfile. Gems that require native compilation that aren't part of the dependency tree of Beaker, will still require compilation.

      ===================================

      See PDK-951

      You might not be able to use PDK with a module if that module's Gemfile requires the beaker acceptance testing library or certain other Ruby gems, particularly when running on Windows.

      PDK packages cannot always install gems that include native extensions, which must be compiled before they can be used by a Ruby application. Beaker has several dependencies that include these native extensions.

      When you run pdk validate or pdk test unit on a module, PDK tries to install any missing module dependencies before it runs validations or tests. On some platforms, PDK can install gems with native extensions, if you already have the required compilation tools and libraries installed. On Windows, however, the Ruby installations managed by PDK are not configured to support native extension compilation, even if the necessary tools are present.

      If you encounter this issue on a platform other than Windows, you might be able to resolve it by researching and installing the required dependencies for the gem that is failing to install.

      If you encounter this issue on a Windows platform, you must remove or comment out the Gemfile dependencies that include native extensions or that have dependencies that include native extensions.
      Show
      To address this specific issue, PDK will package the beaker dependencies that require native compilation. Modules that include beaker in their Gemfile can now install and use the packaged PDK without having to alter their Gemfile. Gems that require native compilation that aren't part of the dependency tree of Beaker, will still require compilation. =================================== See PDK-951 You might not be able to use PDK with a module if that module's Gemfile requires the beaker acceptance testing library or certain other Ruby gems, particularly when running on Windows. PDK packages cannot always install gems that include native extensions, which must be compiled before they can be used by a Ruby application. Beaker has several dependencies that include these native extensions. When you run pdk validate or pdk test unit on a module, PDK tries to install any missing module dependencies before it runs validations or tests. On some platforms, PDK can install gems with native extensions, if you already have the required compilation tools and libraries installed. On Windows, however, the Ruby installations managed by PDK are not configured to support native extension compilation, even if the necessary tools are present. If you encounter this issue on a platform other than Windows, you might be able to resolve it by researching and installing the required dependencies for the gem that is failing to install. If you encounter this issue on a Windows platform, you must remove or comment out the Gemfile dependencies that include native extensions or that have dependencies that include native extensions.
    • QA Risk Assessment:
      Needs Assessment

      Description

      I was trying to PDK convert the puppetlabs-windows module (a metadata only module) and I couldn't run pdk unit because the bundling process was failing;

      Also affects
      puppetlabs-motd
      puppetlabs-stdlib

      C:\Source\puppetlabs-windows [modules-7047-pdkify]> pdk test unit
      [X] Installing missing Gemfile dependencies.
      pdk (FATAL): The dependency puppet-module-posix-default-r2.1 (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for x64-mingw32 but the dependency is only for ruby. To add those platforms to the bundle, run `bundle lock --add-platform ruby`.
      The dependency puppet-module-posix-dev-r2.1 (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for x64-mingw32 but the dependency is only for ruby. To add those platforms to the bundle, run `bundle lock --add-platform ruby`.
      The dependency puppet-module-posix-system-r2.1 (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for x64-mingw32 but the dependency is only for ruby. To add those platforms to the bundle, run `bundle lock --add-platform ruby`.
      Fetching gem metadata from https://rubygems.org/..........
      Fetching version metadata from https://rubygems.org/..
      Resolving dependencies..................
      Using rake 12.3.1 (was 12.3.0)
      Using ffi 1.9.18 (x64-mingw32)
      Using mini_portile2 2.1.0
      Using metaclass 0.0.4
      Using parallel 1.12.1
      Using coderay 1.1.2
      Using method_source 0.9.0 (was 0.8.2)
      Using fast_gettext 1.1.2
      Using hiera 3.4.2
      Using locale 2.1.2
      Using minitar 0.6.1
      Using unf_ext 0.0.7.5 (x64-mingw32)
      Using mime-types 2.99.3
      Using netrc 0.11.0
      Fetching puppet-lint 2.3.5 (was 2.3.3)
      Fetching puppet_facts 0.2.1
      Using rspec-support 3.7.1
      Using diff-lcs 1.3
      Fetching rainbow 2.1.0 (was 2.2.2)
      Installing puppet-lint 2.3.5 (was 2.3.3)
      Installing rainbow 2.1.0 (was 2.2.2)
      Installing puppet_facts 0.2.1
      Using ast 2.4.0
      Using powerpack 0.1.1
      Using ruby-progressbar 1.9.0
      Fetching unicode-display_width 1.3.2 (was 1.3.0)
      Installing unicode-display_width 1.3.2 (was 1.3.0)
      Using net-ssh 4.2.0
      Using net-telnet 0.1.1
      Using sfl 2.3
      Using beaker-i18n_helper 1.1.0
      Fetching beaker-abs 0.5.0
      Installing beaker-abs 0.5.0
      Fetching json 1.8.6 (was 1.8.1)
      Using stringify-hash 0.0.2
      Fetching excon 0.62.0
      Installing json 1.8.6 (was 1.8.1) with native extensions
      Installing excon 0.62.0
      Using multi_json 1.13.1
      Using public_suffix 3.0.2
      Using multipart-post 2.0.0
      Using jwt 2.1.0
      Using little-plugger 1.1.4
      Using memoist 0.16.0
      Using os 0.9.6
      Using httpclient 2.8.3
      Using hurley 0.2
      Using declarative 0.0.10
      Using declarative-option 0.1.0
      Using uber 0.1.0
      Using retriable 3.1.1
      Using deep_merge 1.2.1
      Using builder 3.2.3
      Using formatador 0.2.5
      Using ipaddress 0.8.3
      Using in-parallel 0.1.17
      Using ansi 1.5.0
      Using CFPropertyList 2.3.6
      Using trollop 2.1.2
      Using hocon 1.2.5
      Using inifile 3.0.0
      Using minitest 5.11.3
      Using open_uri_redirections 0.2.1
      Fetching byebug 9.0.6
      Installing byebug 9.0.6 with native extensions
      Using rb-readline 0.5.5
      Using rsync 1.0.9
      Using thor 0.20.0
      Fetching beaker-task_helper 1.4.0
      Using require_all 1.3.3
      Using bundler 1.15.1
      Using puppet-syntax 2.4.1
      Using win32-dir 0.4.9
      Using win32-eventlog 0.6.5
      Using win32-process 0.7.5
      Using win32-security 0.2.5
      Using win32-service 0.8.8
      Using facter 2.5.1 (x64-mingw32)
      Using nokogiri 1.7.2 (x64-mingw32)
      Using mocha 1.1.0
      Fetching parallel_tests 2.21.3 (was 2.14.2)
      Installing beaker-task_helper 1.4.0
      Installing parallel_tests 2.21.3 (was 2.14.2)
      Fetching pry 0.11.3 (was 0.10.4)
      Using unf 0.1.4
      Installing pry 0.11.3 (was 0.10.4)
      Using rspec-core 3.7.1
      Using rspec-expectations 3.7.0
      Using rspec-mocks 3.7.0
      Fetching parser 2.5.1.0 (was 2.5.0.4)
      Using net-scp 1.2.1
      Using beaker-hiera 0.1.1
      Fetching beaker-vagrant 0.4.0
      Installing beaker-vagrant 0.4.0
      Using beaker-vmpooler 1.2.0
      Using addressable 2.5.2
      Fetching faraday 0.15.0
      Installing parser 2.5.1.0 (was 2.5.0.4)
      Installing faraday 0.15.0
      Using logging 2.2.2
      Using representable 3.0.4
      Fetching beaker-hostgenerator 1.1.12
      Fetching ruby-ll 2.1.2
      Installing ruby-ll 2.1.2 with native extensions
      Installing beaker-hostgenerator 1.1.12
      Using fission 0.5.0
      Fetching docker-api 1.34.2
      Using fog-core 1.45.0
      Installing docker-api 1.34.2
      Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
       
      C:/PROGRA~1/PUPPET~1/DevelopmentKit/private/ruby/2.1.9/bin/ruby.exe -r
      ./siteconf20180425-6272-1aht7jn.rb extconf.rb
      creating Makefile
      C:/PROGRA~1/PUPPET~1/DevelopmentKit/private/ruby/2.1.9/lib/ruby/2.1.0/mkmf.rb:1814:in
      `popen': No such file or directory - cygpath (Errno::ENOENT)
      from
      C:/PROGRA~1/PUPPET~1/DevelopmentKit/private/ruby/2.1.9/lib/ruby/2.1.0/mkmf.rb:1814:in
      `mkintpath'
      from
      C:/PROGRA~1/PUPPET~1/DevelopmentKit/private/ruby/2.1.9/lib/ruby/2.1.0/mkmf.rb:1841:in
      `block in configuration'
      from
      C:/PROGRA~1/PUPPET~1/DevelopmentKit/private/ruby/2.1.9/lib/ruby/2.1.0/mkmf.rb:1841:in
      `gsub'
      from
      C:/PROGRA~1/PUPPET~1/DevelopmentKit/private/ruby/2.1.9/lib/ruby/2.1.0/mkmf.rb:1841:in
      `configuration'
      from
      C:/PROGRA~1/PUPPET~1/DevelopmentKit/private/ruby/2.1.9/lib/ruby/2.1.0/mkmf.rb:2192:in
      `create_makefile'
              from extconf.rb:4:in `<main>'
       
      extconf failed, exit code 1
       
      Gem files will remain installed in
      C:/Users/glenn.sarti/AppData/Local/PDK/cache/ruby/2.1.0/gems/json-1.8.6 for
      inspection.
      Results logged to
      C:/Users/glenn.sarti/AppData/Local/PDK/cache/ruby/2.1.0/extensions/x64-mingw32/2.1.0/json-1.8.6/gem_make.out
       
      An error occurred while installing json (1.8.6), and Bundler cannot continue.
      Make sure that `gem install json -v '1.8.6'` succeeds before bundling.
       
      In Gemfile:
        puppet-module-win-system-r2.1 was resolved to 0.3.1, which depends on
          beaker-module_install_helper was resolved to 0.1.7, which depends on
            beaker was resolved to 3.34.0, which depends on
              beaker-aws was resolved to 0.4.0, which depends on
                aws-sdk-v1 was resolved to 1.67.0, which depends on
                  json
       
       
      [X] Cleaning up after running unit tests.
      pdk (ERROR): The spec_clean rake task failed with the following error(s):
       
      ruby: No such file or directory -- C:/Source/puppetlabs-windows/bin/rake (LoadError)
       
      pdk (FATAL): Failed to clean up after running unit tests
      C:\Source\puppetlabs-windows [modules-7047-pdkify]>
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                glenn.sarti Glenn Sarti
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Zendesk Support