Uploaded image for project: 'Puppet Server'
  1. Puppet Server
  2. SERVER-269

Puppet Server aggressively coerces request data to UTF-8

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: SERVER 1.0.0
    • Fix Version/s: SERVER 1.0.2
    • Component/s: Puppet Server
    • Labels:
    • Template:
    • Sub-team:
    • Story Points:
      2
    • Sprint:
      SERVER 2014/12/31, Server 2015-01-21, Server 2015-02-04

      Description

      The Clojure code that processes requests for delivery to JRuby aggressively transforms the request body to UTF-8. This sort of processing is lossy, alters the data contained in the request body and is not appropriate for all requests submitted to a Puppet Master. Notably, file bucket uploads will completely fail if the file content being saved is not strictly UTF-8.

      Reproduction Case

      Install Puppet Server and configure /etc/puppet/manifests/site.pp with a server-side filebucket. Also set up a file File default for backups and a file resource that purges anything placed into a directory:

      # /etc/puppet/manifests/site.pp
      filebucket { 'main':
        server => 'poss-head-master.puppetdebug.vlan',
        path   => false,
      }
       
      File { backup => 'main' }
       
      node default {
        # Everything in /tmp/bukket will be copied to the remote filebucket and
        # then deleted.
        file{'/tmp/bukket':
          ensure => directory,
          purge => true,
          recurse => true,
        }
      }
      

      Place a file containing non-UTF-8 content, such as an ICU4C sample file, into the purge directory and run the puppet agent.

      Outcome

      The first time the agent runs, the file is backed up using the wrong hash. The second time the agent runs, backup fails due to a hash mismatch:

      # cd /tmp/bukket/
      # curl -O http://source.icu-project.org/repos/icu/icu/tags/milestone-54-0-1/source/extra/uconv/samples/iso8859-1.txt
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
      101  1215  101  1215    0     0    237      0  0:00:05  0:00:05 --:--:-- 22924
      # md5sum iso8859-1.txt 
      4730fc9e7d7d0e1a1070b6e8c16c728d  iso8859-1.txt
       
      # puppet agent -t --server=$(hostname -f)
      Info: Retrieving pluginfacts
      Info: Retrieving plugin
      Info: Caching catalog for poss-head-master.puppetdebug.vlan
      Info: Applying configuration version '1419035270'
      Info: Computing checksum on file /tmp/bukket/iso8859-1.txt
      Info: /Stage[main]/Main/Node[default]/File[/tmp/bukket/iso8859-1.txt]: Filebucketed /tmp/bukket/iso8859-1.txt to main with sum 4730fc9e7d7d0e1a1070b6e8c16c728d
      Notice: /Stage[main]/Main/Node[default]/File[/tmp/bukket/iso8859-1.txt]/ensure: removed
      Info: Creating state file /var/lib/puppet/state/state.yaml
      Notice: Finished catalog run in 0.17 seconds
       
      # ls -R /var/lib/puppet/bucket/
      /var/lib/puppet/bucket/:
      f
       
      /var/lib/puppet/bucket/f:
      c
       
      /var/lib/puppet/bucket/f/c:
      8
       
      /var/lib/puppet/bucket/f/c/8:
      9
       
      /var/lib/puppet/bucket/f/c/8/9:
      8
       
      /var/lib/puppet/bucket/f/c/8/9/8:
      6
       
      /var/lib/puppet/bucket/f/c/8/9/8/6:
      a
       
      /var/lib/puppet/bucket/f/c/8/9/8/6/a:
      8
       
      /var/lib/puppet/bucket/f/c/8/9/8/6/a/8:
      fc8986a80772ba0822da161b685d9898
       
      /var/lib/puppet/bucket/f/c/8/9/8/6/a/8/fc8986a80772ba0822da161b685d9898:
      contents  paths
       
      # curl -O http://source.icu-project.org/repos/icu/icu/tags/milestone-54-0-1/source/extra/uconv/samples/iso8859-1.txt
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
      101  1215  101  1215    0     0    238      0  0:00:05  0:00:05 --:--:-- 27000
       
      # puppet agent -t --server=$(hostname -f)
      Info: Retrieving pluginfacts
      Info: Retrieving plugin
      Info: Caching catalog for poss-head-master.puppetdebug.vlan
      Info: Applying configuration version '1419035270'
      Info: Computing checksum on file /tmp/bukket/iso8859-1.txt
      Error: Could not back up /tmp/bukket/iso8859-1.txt: Error 400 on SERVER: Got passed new contents for sum {md5}fc8986a80772ba0822da161b685d9898
      Error: Could not back up /tmp/bukket/iso8859-1.txt: Error 400 on SERVER: Got passed new contents for sum {md5}fc8986a80772ba0822da161b685d9898
      Error: /Stage[main]/Main/Node[default]/File[/tmp/bukket/iso8859-1.txt]/ensure: change from file to absent failed: Could not back up /tmp/bukket/iso8859-1.txt: Error 400 on SERVER: Got passed new contents for sum {md5}fc8986a80772ba0822da161b685d9898
      Notice: Finished catalog run in 0.12 seconds
      

      Expected Outcome

      Puppet Server handles arbitrary character encodings, including raw binary.

        Attachments

          Issue Links

            Activity

              jsd-sla-details-panel

                People

                • Assignee:
                  Unassigned
                  Reporter:
                  chuck Charlie Sharpsteen
                  QA Contact:
                  Erik Dasher
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Zendesk Support