Uploaded image for project: 'Modules'
  1. Modules
  2. MODULES-10581

reboot : last_boot_time task fails via PCP because the host isn't ready in time

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Needs Information
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Template:
      MODULES Bug Template
    • Acceptance Criteria:
      Hide

      Most users can run the reboot plan over PCP without failures.

      Show
      Most users can run the reboot plan over PCP without failures.
    • Team:
      Modules
    • Method Found:
      Customer Feedback
    • Zendesk Ticket IDs:
      38285
    • Zendesk Ticket Count:
      1
    • QA Risk Assessment:
      Needs Assessment

      Description

      Basic Info
      Module Version: 2.4.0
      Puppet Version: 2019.2.2, 2019.3
      OS Name/Version: RHEL 7, Oracle Linux 7 **

      Desired Behavior:

      The reboot plan works for most users when run via PE using default settings.

      Actual Behavior:

      Running the reboot plan reboots the system but fails on the last_boot_time task:

      # puppet plan run reboot --params '{"nodes":["redacted.example.com"]}'
      Starting plan job ...
      New Plan Job ID: 38
       
      Started run_task reboot::last_boot_time
      New job ID: 147
      Nodes: 1
       
      Started on redacted.example.com ...
      Finished on node redacted.example.com
      STDOUT:
      reboot system boot 4.14.35-1902.10. Wed Feb 26 12:50:45 2020 - Tue Mar 3 09:27:49 2020 (5+20:37)
       
      wtmp begins Thu Nov 23 13:59:46 2017
       
      Job completed. 1/1 nodes succeeded.
      Duration: 0 sec
       
      Started run_task reboot
      New job ID: 148
      Nodes: 1
       
      Started on redacted.example.com ...
      Finished on node redacted.example.com
      status : queued
      timeout : 3
       
      Job completed. 1/1 nodes succeeded.
      Duration: 1 sec
       
      Started run_task reboot::last_boot_time
      New job ID: 149
      Nodes: 1
       
      Started on redacted.example.com ...
      Failed on redacted.example.com
      Error: This node (redacted.example.com) is either disconnected or does not have a connection type that supports running tasks/plans.
       
      Job failed. 1 node failed, 0 nodes skipped, 0 nodes succeeded.
      Duration: 0 sec
       
      Started wait_until_available
      New job ID: 150
      Nodes: 1
       
      Started on redacted.example.com ...
      Failed on redacted.example.com
      Error: Timed out waiting for target: redacted.example.com
       
      Job failed. 1 node failed, 0 nodes skipped, 0 nodes succeeded.
      Duration: 3 min, 0 sec
       
      {
        "msg": "Plan aborted: wait_until_available failed on 1 nodes",
        "kind": "bolt/run-failure",
        "details": {
          "action": "wait_until_available",
          "object": null,
          "result_set": [
            {
              "node": "redacted.example.com",
              "action": "wait",
              "object": null,
              "result": {
                "_error": {
                  "msg": "Timed out waiting for target: redacted.example.com",
                  "kind": "puppetlabs.tasks/exception-error",
                  "details": {
                    "class": "Bolt::Executor::TimeoutError"
                  },
                  "issue_code": "EXCEPTION"
                }
              },
              "status": "failure",
              "target": "redacted.example.com"
            }
          ]
        }
      }

      To work around this, increase the plan's disconnect_wait value until it works.

      puppet plan run reboot --params '{"nodes":["redacted.example.com"],"disconnect_wait":60}'

      This only affects execution via PCP/Orchestrator. The task and plan do not appear to be affected when run via Bolt/SSH.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              sheena Sheena Tharakanparampil
              Reporter:
              garrett.guillotte Garrett Guillotte
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:

                  Zendesk Support