Uploaded image for project: 'Puppet Server'
  1. Puppet Server
  2. SERVER-1055

Revisit automated upgrade tests



    • Type: Task
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: SERVER 5.1.0
    • Component/s: None
    • Labels:
    • Template:
    • Acceptance Criteria:

      Have a CI job that installs an old version of Puppet Server and do an upgrade to the new version and run a smoke test.

      Have a CI job that installs an old version of Puppet Server and do an upgrade to the new version and run a smoke test.
    • Team:
      Systems Engineering
    • Sub-team:
    • Story Points:
    • Sprint:
      Server 2017-06-28, Server 2017-07-11
    • Release Notes:
      Not Needed


      At one point in time, we had a Jenkins job which was intended to be used for upgrading from a previously released version of Puppet Server to a package built from latest code on a branch. There were various problems with that approach which ultimately led to us to setting it aside / disabling the tests. SERVER-256 documents one of the problems, a Beaker issue, but not sure if there were others.

      In lieu of these tests running in CI, we've instead been doing manual upgrade smoke testing prior to release. We don't do this very frequently, though, and often don't cover many OS variants in the process. It would be much better if we could have this covered in an automated test suite which runs frequently.

      I would be happy with at least having the basic scenario that we cover in manual smoke testing covered as a starting point:

      1) Puppet Server installed on the latest release version.
      2) Puppet Server upgraded to a new candidate version based on the build from branch latest.
      3) Validate that Puppet Server can be started successfully.
      4) Validate that an agent can be run against the newly upgraded Puppet Server.

      We may ultimately want to go beyond this and run through at least the equivalent of the "20: Acceptance Smoke Test" suite job against the Puppet Server upgrade pipeline - if not also the "25: Full Acceptance Test Suite" (perhaps at a significantly reduced frequency than "on every merge" like we have with the standard pipelines today.


          Issue Links



              adrien Adrien Thebo
              jeremy.barlow Jeremy Barlow
              QA Contact:
              Erik Dasher Erik Dasher
              0 Vote for this issue
              2 Start watching this issue



                  Zendesk Support