Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-10158

Filename with non-UTF8-encoding corrupts state.yaml

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Accepted
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: PUP 6.10.1
    • Fix Version/s: None
    • Component/s: Windows
    • Labels:
      None
    • Template:
      PUP Bug Template
    • Agent OS:
      Windows 7 (x86, x64)
    • Master OS:
      Other
    • Team:
      Coremunity
    • Method Found:
      Needs Assessment
    • QA Risk Assessment:
      Needs Assessment

      Description

      Puppet Version: Agent 5.5.14
      Puppet Server Version: 6.7.2-1stretch
      OS Name/Version: Windows 7 **

      I'm not an encoding nor puppet expert, this description is what happens from my point of view:

      I use a simple file resource to copy a directory test_from.zip to a windows client like this:

      file { 'c:/test_to':
        ensure => directory,
        backup => false,
        force => true,
        purge => true,
        recurse => true,
        source => "c:/test_from",
        }

      Unfortunately, the source contains some files with non-unicode characters in their filenames (ASCII-encoded?).

      These filenames are written to state.yaml and transactionstore.yaml during the agent run.

      During the next run, puppet tries to read the yaml files (in /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/yaml.rb Puppet::Util::Yaml.safe_load_file) and gives an error as shown below. This may be because safe_load_file assumes the content of the state files to be utf-8 (Puppet::(FileSystem.read(filename, :encoding => 'bom|utf-8')).

      I know it is/may be mainly an issue of the wrongly encoded filenames, but maybe it is possible to make puppet a little bit more robust.

      Desired Behavior:

      Agent run should finish without errors.

      Actual Behavior:

      First agent run finishes without errors.

      From the second run, I get this error:

      c:\ProgramData\PuppetLabs\puppet\cache\state>"c:\Program Files\Puppet Labs\Puppet\bin\puppet.bat" agent --test
      Error: Checksumfile C:/ProgramData/PuppetLabs/puppet/cache/state/state.yaml is corrupt ((C:/ProgramData/PuppetLabs/puppet/cache/state/state.yaml): invalid leading UTF-8 octet at line 1 column 1); replacing
      Info: Retrieving pluginfacts
      Info: Retrieving plugin
      Info: Retrieving locales
      Info: Loading facts
      Info: Caching catalog for roloffvm.net.adk.de
      Error: Transaction store file C:/ProgramData/PuppetLabs/puppet/cache/state/transactionstore.yaml is corrupt ((C:/ProgramData/PuppetLabs/puppet/cache/state/transactionstore.yaml): invalid leading UTF-8 octet at line 1 column 1); replacing
      Wrapped exception:
      (C:/ProgramData/PuppetLabs/puppet/cache/state/transactionstore.yaml): invalid leading UTF-8 octet at line 1 column 1

      Workaroud

      Find all files with "wrong" filenames in source directory and rename them. I copied the state.yaml to a linux machine and issued something like:

      :~$ grep -axv '.*' state.yaml 
       
      File[c:/test_from/Nat. Bek. Freih�ndige Vergabe.doc]:
      File[c:/test_from/Nat. Bek. Freih�ndige Vergabe.rtf]:
      

      Then, I renamed these files.

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            troloff Torsten Roloff
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:

                Zendesk Support