Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-2427

Pluginsync will download every file every time if digest_algorithms do not agree

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Duplicate
    • Affects Version/s: PUP 3.6.0
    • Fix Version/s: None
    • Component/s: Networking
    • Labels:
      None
    • Template:
    • Story Points:
      3

      Description

      Due to PUP-1840, if agent and master do not agree on digest_algorithm, e.g. agent is using md5, server is using sha256, then the agent pluginsync will make a file_metadatas request with the client's checksum_type:

      Debug: connecting to https://puppetmaster.solar.lan:8140/production/file_metadatas/plugins?links=manage&recurse=true&ignore=.svn&ignore=CVS&ignore=.git&checksum_type=md5
      

      The file serving code ignores the requested checksum_type, and instead uses the checksum as specified in digest_algorithm. Note sha256 in the JSON response:

      $ curl -k --cert ~/.puppet/ssl/certs/arcturus.corp.puppetlabs.net.pem --key ~/.puppet/ssl/private_keys/arcturus.corp.puppetlabs.net.pem 'https://puppetmaster.solar.lan:8140/production/file_metadatas/plugins?links=manage&recurse=true&ignore=.svn&ignore=CVS&ignore=.git&checksum_type=md5'
      ...
      {"data":{"group":0,"mode":292,"destination":null,"relative_path":"puppet/parser/functions/delete_at.rb","checksum":{"value":"{sha256}b6ac4f3cb6500f9b09ce7db246801493cad7ce57a5c543d326004b7b409ff00e","type":"sha256"},"type":"file","links":"manage","path":"/etc/puppet/modules/stdlib/lib","owner":0},"metadata":{"api_version":1},"document_type":"FileMetadata"}]
      

      Since current != desired, puppet will re-download the file (for every pluginsync'ed file, every time the agent runs):

      Notice: /File[/Users/josh/.puppet/var/lib/facter/pe_version.rb]/content: content changed '{md5}4a9353952963b011759f3e6652a10da5' to
       '{sha256}27498d481b63291c84537447ce5d0545a3406a458321a25324e7040fd163013d'
      

      The problem is that the master needs to handle checksum types like it does serialization formats. The master should have a "preferred_digest_algorithm" setting that accepts multiple values, e.g. sha256, sha1, but not md5. Then the client needs to send the checksum it accepts in the file_metadata request. The file serving code needs to validate the checksum from the request, and generate the metadata based on the requested checksum.

      There is a secondary issue in that if you specify the source parameter:

      file { '/tmp/newfile':
        ensure => file,
        source => 'puppet://fileserver.solar.lan/modules/stdlib/newfile.txt',
        checksum => 'sha256',
      }
      

      Then applying it on the agent, will result in a file_metadata (singular) request that doesn't include the checksum_type. As a result, the server has no idea which checksum type to use:

      Debug: connecting to https://fileserver.solar.lan:8140/production/file_metadata/modules/stdlib/newfile.txt?links=manage&source_permissions=use
      

        Attachments

          Issue Links

            Activity

              jsd-sla-details-panel

                People

                • Assignee:
                  Unassigned
                  Reporter:
                  josh Josh Cooper
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: