Uploaded image for project: 'OS Images'
  1. OS Images
  2. IMAGES-359

ssh to Fedora VMs can fail following host.reboot

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Template:
    • Acceptance Criteria:
      Hide

      The pxp-agent acceptance test tests.restart_host_run_puppet.rb should be re-enabled on Fedora platforms and passing stably in CI

      Show
      The pxp-agent acceptance test tests.restart_host_run_puppet.rb should be re-enabled on Fedora platforms and passing stably in CI
    • Team:
      Platform OS
    • Story Points:
      3

      Description

      Important Note

      • When this is resolved, the pxp-agent acceptance test tests.restart_host_run_puppet.rb should be re-enabled on Fedora platforms

      Seen in:

      https://jenkins.puppetlabs.com/view/puppet-agent/view/master/view/puppet-agent/job/platform_puppet-agent_intn-van-sys_suite-daily-pxp-agent-master/26/SLAVE_LABEL=beaker,TEST_TARGET=fedora22-64a/testReport/junit/(root)/tests/restart_host_run_puppet_rb/

      It appears that host.reboot succeeded, but the next part of the test that attempted ssh to the instance failed

      Error message:

      Cannot connect to fedora22-64-1

      Test output:

      C94777 - Ensure pxp-agent functions after agent host restart
        
        * Ensure each agent host has pxp-agent service running and enabled
          
          ylnffy4ctarmu2s.delivery.puppetlabs.net (fedora22-64-1) 17:00:46$ puppet resource service pxp-agent ensure=stopped
            Notice: /Service[pxp-agent]/ensure: ensure changed 'running' to 'stopped'
            service { 'pxp-agent':
              ensure => 'stopped',
            }
          
          ylnffy4ctarmu2s.delivery.puppetlabs.net (fedora22-64-1) executed in 0.95 seconds
          
          ylnffy4ctarmu2s.delivery.puppetlabs.net (fedora22-64-1) 17:00:47$ puppet config print ssldir
            /etc/puppetlabs/puppet/ssl
          
          ylnffy4ctarmu2s.delivery.puppetlabs.net (fedora22-64-1) executed in 0.80 seconds
          localhost $ scp /tmp/beaker20160827-17399-5azj79 fedora22-64-1:/etc/puppetlabs/pxp-agent/pxp-agent.conf {:ignore => }
          
          ylnffy4ctarmu2s.delivery.puppetlabs.net (fedora22-64-1) 17:00:47$ puppet resource service pxp-agent ensure=running enable=true
            Notice: /Service[pxp-agent]/ensure: ensure changed 'stopped' to 'running'
            service { 'pxp-agent':
              ensure => 'running',
              enable => 'true',
            }
          
          ylnffy4ctarmu2s.delivery.puppetlabs.net (fedora22-64-1) executed in 1.11 seconds
        
        * restart each agent
          
          ylnffy4ctarmu2s.delivery.puppetlabs.net (fedora22-64-1) 17:00:50$ /sbin/shutdown -r now
            Warning: ssh channel on fedora22-64-1 received exception post command execution IOError - closed stream
            Warning: ssh.close: connection is already closed, no action needed
            ssh connection to fedora22-64-1 has been terminated
          
          ylnffy4ctarmu2s.delivery.puppetlabs.net (fedora22-64-1) executed in 0.02 seconds
          
          * wait until pxp-agent is back up and associated on ylnffy4ctarmu2s.delivery.puppetlabs.net
            
            ylnffy4ctarmu2s.delivery.puppetlabs.net (fedora22-64-1) 17:01:00$ #<Beaker::Command:0x000000043d8408>
              Trying command 30 times.
            .        Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 1 -- Host ylnffy4ctarmu2s.delivery.puppetlabs.net unreachable: Errno::ETIMEDOUT - Connection timed out - connect(2) for "ylnffy4ctarmu2s.delivery.puppetlabs.net" port 22
              Warning: Trying again in 3 seconds
              Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 2 -- Host ylnffy4ctarmu2s.delivery.puppetlabs.net unreachable: Errno::EHOSTUNREACH - No route to host - connect(2) for "ylnffy4ctarmu2s.delivery.puppetlabs.net" port 22
              Warning: Trying again in 5 seconds
              Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 3 -- Host ylnffy4ctarmu2s.delivery.puppetlabs.net unreachable: Errno::EHOSTUNREACH - No route to host - connect(2) for "ylnffy4ctarmu2s.delivery.puppetlabs.net" port 22
              Warning: Trying again in 8 seconds
              Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 4 -- Host ylnffy4ctarmu2s.delivery.puppetlabs.net unreachable: Errno::EHOSTUNREACH - No route to host - connect(2) for "ylnffy4ctarmu2s.delivery.puppetlabs.net" port 22
              Warning: Trying again in 13 seconds
              Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 5 -- Host ylnffy4ctarmu2s.delivery.puppetlabs.net unreachable: Errno::EHOSTUNREACH - No route to host - connect(2) for "ylnffy4ctarmu2s.delivery.puppetlabs.net" port 22
              Warning: Trying again in 21 seconds
              Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 6 -- Host ylnffy4ctarmu2s.delivery.puppetlabs.net unreachable: Errno::EHOSTUNREACH - No route to host - connect(2) for "ylnffy4ctarmu2s.delivery.puppetlabs.net" port 22
              Warning: Trying again in 34 seconds
              Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 7 -- Host ylnffy4ctarmu2s.delivery.puppetlabs.net unreachable: Errno::EHOSTUNREACH - No route to host - connect(2) for "ylnffy4ctarmu2s.delivery.puppetlabs.net" port 22
              Warning: Trying again in 55 seconds
              Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 8 -- Host ylnffy4ctarmu2s.delivery.puppetlabs.net unreachable: Errno::EHOSTUNREACH - No route to host - connect(2) for "ylnffy4ctarmu2s.delivery.puppetlabs.net" port 22
              Warning: Trying again in 89 seconds
              Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 9 -- Host ylnffy4ctarmu2s.delivery.puppetlabs.net unreachable: Errno::EHOSTUNREACH - No route to host - connect(2) for "ylnffy4ctarmu2s.delivery.puppetlabs.net" port 22
              Warning: Trying again in 144 seconds
              Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 10 -- Host ylnffy4ctarmu2s.delivery.puppetlabs.net unreachable: Errno::EHOSTUNREACH - No route to host - connect(2) for "ylnffy4ctarmu2s.delivery.puppetlabs.net" port 22
              Warning: Trying again in 233 seconds
              Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 11 -- Host ylnffy4ctarmu2s.delivery.puppetlabs.net unreachable: Errno::EHOSTUNREACH - No route to host - connect(2) for "ylnffy4ctarmu2s.delivery.puppetlabs.net" port 22
              Warning: Trying again in 377 seconds
              Attempting ssh connection to ylnffy4ctarmu2s.delivery.puppetlabs.net, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Failed to connect to ylnffy4ctarmu2s.delivery.puppetlabs.net, after 12 attempts
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 1 -- Host fedora22-64-1 unreachable: SocketError - getaddrinfo: Name or service not known
              Warning: Trying again in 3 seconds
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 2 -- Host fedora22-64-1 unreachable: SocketError - getaddrinfo: Name or service not known
              Warning: Trying again in 5 seconds
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 3 -- Host fedora22-64-1 unreachable: SocketError - getaddrinfo: Name or service not known
              Warning: Trying again in 8 seconds
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 4 -- Host fedora22-64-1 unreachable: SocketError - getaddrinfo: Name or service not known
              Warning: Trying again in 13 seconds
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 5 -- Host fedora22-64-1 unreachable: SocketError - getaddrinfo: Name or service not known
              Warning: Trying again in 21 seconds
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 6 -- Host fedora22-64-1 unreachable: SocketError - getaddrinfo: Name or service not known
              Warning: Trying again in 34 seconds
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 7 -- Host fedora22-64-1 unreachable: SocketError - getaddrinfo: Name or service not known
              Warning: Trying again in 55 seconds
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 8 -- Host fedora22-64-1 unreachable: SocketError - getaddrinfo: Name or service not known
              Warning: Trying again in 89 seconds
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 9 -- Host fedora22-64-1 unreachable: SocketError - getaddrinfo: Name or service not known
              Warning: Trying again in 144 seconds
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 10 -- Host fedora22-64-1 unreachable: SocketError - getaddrinfo: Name or service not known
              Warning: Trying again in 233 seconds
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Try 11 -- Host fedora22-64-1 unreachable: SocketError - getaddrinfo: Name or service not known
              Warning: Trying again in 377 seconds
              Attempting ssh connection to fedora22-64-1, user: root, opts: {:config=>false, :paranoid=>false, :auth_methods=>["publickey"], :port=>22, :forward_agent=>true, :keys=>["~/.ssh/id_rsa-acceptance"], :user_known_hosts_file=>"/var/lib/jenkins/.ssh/known_hosts", :keepalive=>true, :user=>"root"}
              Warning: Failed to connect to fedora22-64-1, after 12 attempts
              Failed to connect to fedora22-64-1, attempted vmhostname (ylnffy4ctarmu2s.delivery.puppetlabs.net), hostname (fedora22-64-1)
              RuntimeError: Cannot connect to fedora22-64-1
              /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-pxp-agent-master/SLAVE_LABEL/beaker/TEST_TARGET/fedora22-64a/pxp-agent/acceptance/tests/restart_host_run_puppet.rb:52
              /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-pxp-agent-master/SLAVE_LABEL/beaker/TEST_TARGET/fedora22-64a/pxp-agent/acceptance/tests/restart_host_run_puppet.rb:47
              /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-pxp-agent-master/SLAVE_LABEL/beaker/TEST_TARGET/fedora22-64a/pxp-agent/acceptance/tests/restart_host_run_puppet.rb:33
              /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-pxp-agent-master/SLAVE_LABEL/beaker/TEST_TARGET/fedora22-64a/pxp-agent/acceptance/tests/restart_host_run_puppet.rb:33
              /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-pxp-agent-master/SLAVE_LABEL/beaker/TEST_TARGET/fedora22-64a/pxp-agent/acceptance/tests/restart_host_run_puppet.rb:32
              /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-pxp-agent-master/SLAVE_LABEL/beaker/TEST_TARGET/fedora22-64a/pxp-agent/acceptance/tests/restart_host_run_puppet.rb:3
              /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-pxp-agent-master/SLAVE_LABEL/beaker/TEST_TARGET/fedora22-64a/pxp-agent/acceptance/.bundle/gems/gems/beaker-2.50.0/bin/beaker:6
              Begin teardown
              End teardown

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              james.stocks James Stocks
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support