Uploaded image for project: 'Puppet Server'
  1. Puppet Server
  2. SERVER-1055

Revisit automated upgrade tests

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: SERVER 5.1.0
    • Component/s: None
    • Labels:
      None
    • Template:
    • Acceptance Criteria:
      Hide

      Have a CI job that installs an old version of Puppet Server and do an upgrade to the new version and run a smoke test.

      Show
      Have a CI job that installs an old version of Puppet Server and do an upgrade to the new version and run a smoke test.
    • Team:
      Systems Engineering
    • Sub-team:
    • Story Points:
      5
    • Sprint:
      Server 2017-06-28, Server 2017-07-11
    • Release Notes:
      Not Needed

      Description

      At one point in time, we had a Jenkins job which was intended to be used for upgrading from a previously released version of Puppet Server to a package built from latest code on a branch. There were various problems with that approach which ultimately led to us to setting it aside / disabling the tests. SERVER-256 documents one of the problems, a Beaker issue, but not sure if there were others.

      In lieu of these tests running in CI, we've instead been doing manual upgrade smoke testing prior to release. We don't do this very frequently, though, and often don't cover many OS variants in the process. It would be much better if we could have this covered in an automated test suite which runs frequently.

      I would be happy with at least having the basic scenario that we cover in manual smoke testing covered as a starting point:

      1) Puppet Server installed on the latest release version.
      2) Puppet Server upgraded to a new candidate version based on the build from branch latest.
      3) Validate that Puppet Server can be started successfully.
      4) Validate that an agent can be run against the newly upgraded Puppet Server.

      We may ultimately want to go beyond this and run through at least the equivalent of the "20: Acceptance Smoke Test" suite job against the Puppet Server upgrade pipeline - if not also the "25: Full Acceptance Test Suite" (perhaps at a significantly reduced frequency than "on every merge" like we have with the standard pipelines today.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              adrien Adrien Thebo
              Reporter:
              jeremy.barlow Jeremy Barlow
              QA Contact:
              Erik Dasher Erik Dasher
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support