Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-8338

Puppet apply doesn't stream file content

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: PUP 6.14.0
    • Component/s: None
    • Labels:
      None
    • Template:
      PUP Bug Template
    • Epic Link:
    • Team:
      Coremunity
    • Sprint:
      Platform Core KANBAN
    • Method Found:
      Needs Assessment
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Hide
      This changes "puppet apply" to retrieve content for file resources with a "source" parameter in the same way that "puppet agent" does. Previously, "puppet apply" read the entire file contents into memory and cached it for the duration of the run. It also did not follow HTTP redirects or handle HTTP errors in the same way, such as HTTP service unavailable (503). Now "puppet apply" uses the same code path as "puppet agent" to stream file content from "puppet://" and "http(s)://" file sources. As a result, memory footprint should be reduced when using "puppet apply".
      Show
      This changes "puppet apply" to retrieve content for file resources with a "source" parameter in the same way that "puppet agent" does. Previously, "puppet apply" read the entire file contents into memory and cached it for the duration of the run. It also did not follow HTTP redirects or handle HTTP errors in the same way, such as HTTP service unavailable (503). Now "puppet apply" uses the same code path as "puppet agent" to stream file content from "puppet://" and "http(s)://" file sources. As a result, memory footprint should be reduced when using "puppet apply".
    • QA Risk Assessment:
      Needs Assessment

      Description

      Puppet Version: Puppet 4 (at least) and up
      Puppet Server Version: n/a
      OS Name/Version: all

      When running puppet apply with a file resource and source parameter, then puppet will load the entire file into memory, instead of streaming it.

      This was noticed while researching why the bug described in PUP-7482 only affected puppet agent but not puppet apply.

      Desired Behavior:

      When managing files with a source parameter, puppet apply should always stream file content from all of the following sources

      <absolute path> local file
      file://<URI-encoded path> local file
      puppet:///modules/mymodule local file
      http?://host:port/path/to/file remote file

      Actual Behavior:

      puppet apply does not stream file content from http-based sources. See https://github.com/puppetlabs/puppet/blob/5.3.3/lib/puppet/type/file/source.rb#L274-L276.

      At first glance it appears the Puppet::Type::File::ParameterSource#content method can be deleted entirely, and should always rely on chunk_file_from_disk or chunk_file_from_source.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              josh Josh Cooper
              Reporter:
              josh Josh Cooper
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support