Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-7951

agent - server setup - file bucketing for binary files fails in case same file is bucketed for 2 different paths

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • PUP 4.10.7
    • PUP 4.10.9, PUP 5.3.3
    • None
    • None
    • Red Hat Enterprise Linux Server release 7.2 (Maipo) x86_64

    • Platform Core
    • 1
    • Platform Core KANBAN
    • Needs Assessment
    • Bug Fix
    • Hide
      Prior to PUP-7951, when running the master via puppet server, backing up the same file content to filebucket more than once could result in a mistaken error warning that while the files had the same checksum value, they were different (ie a potential hash collision). PUP-7951 addresses this issue so that this is correctly treated as a duplicate file by filebucket.
      Show
      Prior to PUP-7951 , when running the master via puppet server, backing up the same file content to filebucket more than once could result in a mistaken error warning that while the files had the same checksum value, they were different (ie a potential hash collision). PUP-7951 addresses this issue so that this is correctly treated as a duplicate file by filebucket.
    • No Action
    • updated tests to cover

    Description

      Scenario: Puppet Agent is bucketing file to Puppetserver

      Following scenario (but in agent-server) configuration should show the error:
      puppet apply -e 'file

      { "/tmp/d1/file1": source => "/usr/bin/at" }

      '

      puppet apply -e 'file

      { "/tmp/d1/file1": source => "/usr/bin/bash" }

      '

      1. file1 already exists, so it is bucketed before rewritten by contents of 'bash'

      puppet apply -e 'file

      { "/tmp/d2/file2": source => "/usr/bin/at" }

      '

      puppet apply -e 'file

      { "/tmp/d2/file2": source => "/usr/bin/bash" }

      '

      1. file2 already exists, so it is bucketed before rewritten by contents of 'bash'. It is bucketed to same directory on server because md5 hash of file2 is same as md5 hash of bucketed file1. And bucketing fails.

      In theory this should work. And for local bucketing it works.
      There are 2 artefacts on server for each bucketed file - contents and paths. Paths contains all paths where contents was stored on origin system.

      When file is bucketed there are multiple checks - size, md5 hash and file to be bucketed is byte-to-byte compared with already bucketed file.

      In case that given file is bucketed (file with md5sum hash exists on server) and path on origin system is different to path stored in 'paths' than there are 2 checks - size of bucketed file and checksum. Size is compared between data of file received over network from agent and existing 'contents' on server.

      The problem is, that this comparison usually fails because encoding of data received over network are 'UTF-8' encoded and contents is "like" plain bytes. So size doesn't match and it is considered error and processing of file resource fails.

      For puppet-agent-1.10.4-1.el7.x86_64 rpm:
      Comparison is in: method Puppet::FileBucketFile::File.verify_identical_file()
      bucket_file is: Puppet::FileBucket::File::StringContents

      Solution should be to ensure size of data received over network to be converted to something like "ASCII-8BIT" encoding even for 'size' operation.

      Attachments

        Activity

          People

            Unassigned Unassigned
            standa11111 Standa eM
            Eric Delaney Eric Delaney
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Zendesk Support