Uploaded image for project: 'Puppet Development Kit'
  1. Puppet Development Kit
  2. PDK-1347

Coverage report does not report failures in PDK

    XMLWordPrintable

Details

      • `pdk test unit` should report failure if code coverage falls below configured threshold
    • Puppet Developer Experience
    • Customer Feedback
    • Bug Fix
    • Upstream project rspec-puppet has been fixed to correctly report insufficient resource coverage (if configured) as an error.

    Description

      PDK does show the output from the coverage reports and shows the test failure, but marks the Running unit tests as successful despite failure in coverage reports. See the checkbox in the output below.

      $ pdk test unit
      pdk (INFO): Using Ruby 2.5.3
      pdk (INFO): Using Puppet 6.2.0
      [✔] Preparing to run the unit tests.
      [✔] Running unit tests.
      Run options: exclude {:bolt=>true}
        Evaluated 7 tests in 10.245895 seconds: 1 failures, 0 pending.
       
      Total resources:   1
      Touched resources: 0
      Resource coverage:  0.00%
       
      Untouched resources:
        File[/tmp/test]
      

      The expected behavior is that the

      {Running unit tests}

      would have a

      {x}

      in it due to the failed tests and PDK would return a non zero exit code.

      Reproduction Steps
      1. Create a new module with PDK that contains resources and no custom tests
      2. PDK update with the following .sync.yml

      ---
      spec/spec_helper.rb:
        coverage_report: true
        minimum_code_coverage_percentage: 90
      

      3. Run `pdk test unit`.

      Expected Behavior
      The tests should complete with a failure denoted in PDK other than the test output.

      Attachments

        Activity

          People

            tim.sharpe Tim Sharpe
            jarret.lavallee Jarret Lavallee
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Zendesk Support