Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-4659

environment_scenario test failures when master has agent role

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: PUP 4.1.0
    • Fix Version/s: PUP 4.2.0
    • Component/s: None
    • Labels:
      None
    • Template:
    • Story Points:
      2
    • Sprint:
      Client 2015-06-10
    • Release Notes:
      Not Needed

      Description

      When we upgraded the Puppet Server acceptance pipeline to using Puppet 4.1.0 / Puppet Agent 1.1.0, we started encountering some failures with the environment_scenario tests. I believe these tests are not failing when run on the platform pipeline only because some of the test steps are only run when the agent under test resides on the master. According to Josh Cooper, the master has not been run with the agent role in the platform AIO pipeline for some time.

      An example of one failing matrix is here:

      https://jenkins.puppetlabs.com/view/Puppet%20Server/view/All/job/platform_puppet-server_integration-system_no-conditional_full-master/64/#showFailuresLink

      Note that the same failures are occurring across all OSes under test.

      Here's a quick analysis of what it looks to me that may be going on with the failures:

      environment_scenario-bad.rb

      Same pattern like the one below for “puppet_config”, “puppet_apply”, “puppet_module_install”, and “puppet_module_uninstall”:

      -------------------------------------------------------------------------------
      puppet_config: *UNEXPECTED FAILURE*
      -------------------------------------------------------------------------------
      Expected the output:
      /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/environments.rb:38:in `get!': Could not find a directory environment named 'testing' anywhere in the path: /doesnotexist. Does the directory exist? (Puppet::Environments::EnvironmentNotFound)
      	from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application_support.rb:29:in `push_application_context'
      	from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:337:in `run'
      	from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:124:in `run'
      	from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'
      	from /opt/puppetlabs/bin/puppet:5:in `<main>'
       
      To match: (?-mix:Could not find a directory environment named 'testing' anywhere in the path.*\/etc\/puppetlabs\/code)
      

      The test sets the “environmentpath” in the “main” section of the puppet.conf file to “/doesnotexist” but the test appears to be failing because it expects the master’s “codedir” setting to appear in the message - “/etc/puppetlabs/code” - rather than the “environmentpath”. The “environmentpath" actually seems to me like the right value to include in the message here. I can reproduce this locally by just running “puppet config … —environment testing” (with no master involved).

      environment_scenario-default.rb

      Same pattern like the one below for “puppet_config”, “puppet_module_install”, and “puppet_module_uninstall”.

      -------------------------------------------------------------------------------
      puppet_config: *UNEXPECTED FAILURE*
      -------------------------------------------------------------------------------
      Expected the output:
      basemodulepath = /etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules
      modulepath = /etc/puppetlabs/code/environments/production/modules:/etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules
      manifest = /etc/puppetlabs/code/environments/production/manifests
      config_version = 
       
      To match: (?-mix:manifest.*\/etc\/puppetlabs\/code\/environments\/\/manifests$)
      And match: (?-mix:modulepath.*\/etc\/puppetlabs\/code\/environments\/\/modules:.+)
      

      In each case, it appears that the test is expecting the target directory to not include a directory environment name where the result of the test does - i.e., “production”. Where directory environments are mandated by default in Puppet v4, I wonder if this is just a case where the default expectation needs to include the “production” directory environment name.

      environment_scenario-master_environmentpath.rb

      For the “puppet_apply” test, the following output appeared:

      -------------------------------------------------------------------------------
      puppet_apply: *UNEXPECTED FAILURE*
      -------------------------------------------------------------------------------
      Expected the output:
      Notice: Compiled catalog for fd7ncmpi1suilmb.delivery.puppetlabs.net in environment testing in 0.33 seconds
      Notice: include directory testing environment testing_mod
      Notice: /Stage[main]/Testing_mod/Notify[include directory testing environment testing_mod]/message: defined 'message' as 'include directory testing environment testing_mod'
      Notice: Applied catalog in 0.08 seconds
       
      To match: (?-mix:include default environment testing_mod)
      ------
      

      It seems like the expectation that the output would use “default environment” is incorrect since the test is actually trying to use the “testing” directory environment.

      For the “puppet_module_install” test, the following output appeared:

      -------------------------------------------------------------------------------
      puppet_module_install: *UNEXPECTED FAILURE*
      -------------------------------------------------------------------------------
      Expected the output:
      Notice: Preparing to install into /etc/puppetlabs/code/environments/testing/modules ...
      Notice: Downloading from https://forgeapi.puppetlabs.com ...
      Notice: Installing -- do not interrupt ...
      /etc/puppetlabs/code/environments/testing/modules
      └── pmtacceptance-nginx (v0.0.1)
       
      To match: (?-mix:Preparing to install into \/etc\/puppetlabs\/code\/modules)
      ------
      Notes: Runs in user mode and doesn't see the master environmenetpath setting.
      

      Again, the expectation that the message does not include “…environments/testing” in the path seems wrong since the test is specifying that the “testing” directory environment be used.

      For the “puppet_module_uninstall” test, the following output appeared:

      -------------------------------------------------------------------------------
      puppet_module_uninstall: *UNEXPECTED FAILURE*
      -------------------------------------------------------------------------------
      Expected the output:
      Notice: Preparing to uninstall 'pmtacceptance-nginx' ...
      Removed 'pmtacceptance-nginx' (v0.0.1) from /etc/puppetlabs/code/environments/testing/modules
       
      To match: (?-mix:Removed.*pmtacceptance-nginx.*from \/etc\/puppetlabs\/code\/modules)
      ------
      

      Same comment as above for the “puppet_module_install” test.

      all tests

      It appears that across all of these tests that there’s a problem in the test cleanup where the occurrence of a failure is causing the test to abort with a stack trace:

      #<NoMethodError: undefined method `empty?' for nil:NilClass>
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/ruby/puppet/acceptance/lib/puppet/acceptance/environment_utils.rb:355
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/ruby/puppet/acceptance/lib/puppet/acceptance/environment_utils.rb:354
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/ruby/puppet/acceptance/lib/puppet/acceptance/environment_utils.rb:354
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/ruby/puppet/acceptance/tests/environment/environment_scenario-bad.rb:53
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/gems/beaker-2.12.0/lib/beaker/test_case.rb:124
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/gems/beaker-2.12.0/lib/beaker/test_case.rb:124
      /usr/local/rvm/rubies/ruby-1.9.3-p484/lib/ruby/1.9.1/benchmark.rb:295
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/gems/beaker-2.12.0/lib/beaker/test_case.rb:121
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/gems/beaker-2.12.0/lib/beaker/test_suite.rb:286
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/gems/beaker-2.12.0/lib/beaker/test_suite.rb:283
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/gems/beaker-2.12.0/lib/beaker/test_suite.rb:283
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/gems/beaker-2.12.0/lib/beaker/test_suite.rb:325
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/gems/beaker-2.12.0/lib/beaker/cli.rb:151
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/gems/beaker-2.12.0/lib/beaker/cli.rb:93
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/gems/beaker-2.12.0/bin/beaker:6
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/bin/beaker:23
      /var/lib/jenkins/workspace/platform_puppet-server_integration-system_no-conditional_full-master/LAYOUT/redhat6-64ma-64a/LDAP_TYPE/PLATFORM/label/beaker/vendor/bundler/ruby/1.9.1/bin/beaker:23
      Unstubbing address forge.puppetlabs.com to IP 10.32.23.69 on machine fd7ncmpi1suilmb.delivery.puppetlabs.net
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              jeremy.barlow Jeremy Barlow
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support