[PUP-9651] Delete low risk acceptance tests Created: 2019/04/22  Updated: 2019/10/07  Resolved: 2019/09/19

Status: Resolved
Project: Puppet
Component/s: None
Affects Version/s: None
Fix Version/s: PUP 5.5.17, PUP 6.4.4, PUP 6.10.0

Type: Task Priority: Normal
Reporter: John Duarte Assignee: John Duarte
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
relates to PUP-7732 Audit puppet acceptance tests to remo... Closed
Acceptance Criteria:

Given a checkout of puppet
And I am in the acceptance directory
When I run bundle exec rake ci:test:aio OPTIONS='--preserve-hosts=always --test-tag-or='audit:low,risk:low' SHA=latest RELEASE_STREAM=puppet6 HOSTS=redhat7-64ma
Then all tests should be skipped due to the lack of matching low risk tests.

Epic Link: Puppet component pipeline acceptance 2
Team: Coremunity
Sprint: Platform Core KANBAN
Release Notes: Not Needed
Release Notes Summary: This only changes tests
QA Risk Assessment: Needs Assessment


As a puppet tester
I want the acceptance test suite to mitigate risk commensurate to its cost of execution
So that I can get test feedback faster
Without increasing my product risk

Acceptance tests are expensive.

The puppet acceptance test suite was audited in July 2017 to evaluate the relative risk that each test mitigated. Tests deemed as mitigating a low risk via the inclusion of the either the audit:low or risk:low beaker tags should be removed from the test suite.

Comment by John Duarte [ 2019/04/26 ]

PR 7497 opened to delete acceptance tests that mitigate low risk regressions.

Rob Braden, please route to your team members as needed.

Comment by Jacob Helwig [ 2019/04/29 ]

Merged to master branch in 14878c4e11. John Duarte, OK to move to "Ready for CI" or were there going to be follow-on PRs for this ticket?

Comment by John Duarte [ 2019/04/29 ]

The merged PR removes all of the low risk tests. No follow-on PRs are needed. I have updated the state to "Ready for CI". Thanks Jacob Helwig!

Comment by John Duarte [ 2019/09/17 ]

Opened PR 7729 to cherry pick these test deletions back to 5.5.x.

Comment by Josh Cooper [ 2019/09/17 ]

Merged to 5.5.x in https://github.com/puppetlabs/puppet/commit/b60a6703c19b9fa04167dd4a17a1499e42416867

Comment by Kris Bosland [ 2019/09/19 ]

Passed CI in 7f5d209009940ec97c5ec7e25538aff4d2a17654.

Generated at Thu Jul 16 10:59:07 PDT 2020 using Jira 8.5.2#805002-sha1:a66f9354b9e12ac788984e5d84669c903a370049.