Uploaded image for project: 'Puppet Development Kit'
  1. Puppet Development Kit
  2. PDK-1347

Coverage report does not report failures in PDK

    XMLWordPrintable

    Details

    • Template:
    • Acceptance Criteria:
      • `pdk test unit` should report failure if code coverage falls below configured threshold
    • Team:
      Puppet Developer Experience
    • Method Found:
      Customer Feedback
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Upstream project rspec-puppet has been fixed to correctly report insufficient resource coverage (if configured) as an error.

      Description

      PDK does show the output from the coverage reports and shows the test failure, but marks the Running unit tests as successful despite failure in coverage reports. See the checkbox in the output below.

      $ pdk test unit
      pdk (INFO): Using Ruby 2.5.3
      pdk (INFO): Using Puppet 6.2.0
      [✔] Preparing to run the unit tests.
      [✔] Running unit tests.
      Run options: exclude {:bolt=>true}
        Evaluated 7 tests in 10.245895 seconds: 1 failures, 0 pending.
       
      Total resources:   1
      Touched resources: 0
      Resource coverage:  0.00%
       
      Untouched resources:
        File[/tmp/test]
      

      The expected behavior is that the

      {Running unit tests}

      would have a

      {x}

      in it due to the failed tests and PDK would return a non zero exit code.

      Reproduction Steps
      1. Create a new module with PDK that contains resources and no custom tests
      2. PDK update with the following .sync.yml

      ---
      spec/spec_helper.rb:
        coverage_report: true
        minimum_code_coverage_percentage: 90
      

      3. Run `pdk test unit`.

      Expected Behavior
      The tests should complete with a failure denoted in PDK other than the test output.

        Attachments

          Activity

            People

            Assignee:
            tim.sharpe Tim Sharpe
            Reporter:
            jarret.lavallee Jarret Lavallee
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Zendesk Support