Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-6188

Document Puppet::FileSystem.open / exclusive_open encoding usage, and make existing code explicitly use encodings

    XMLWordPrintable

Details

    • Hide
      • Verify exclusive open fails before applying the change.
      • Verify passes after change.
      • Double-check Forge for modules that make calls to filesystem methods and notify authors if they are using filesystem methods that will be not be backwards compatible with our changes.
      • Validate whether PUP-1583 is fixed by this.
      Show
      Verify exclusive open fails before applying the change. Verify passes after change. Double-check Forge for modules that make calls to filesystem methods and notify authors if they are using filesystem methods that will be not be backwards compatible with our changes. Validate whether PUP-1583 is fixed by this.
    • Agent
    • 3
    • AP 2016-10-19, AP 2016-11-02, AP 2016-11-16, AP 2016-11-30, AP 2016-12-14
    • Not Needed
    • No Action
    • N/A [mass update]: Various build, documentation, infra, or test-related issues

    Description

      The current FileSystem.open implementation is a bit peculiar at https://github.com/puppetlabs/puppet/blob/master/lib/puppet/file_system/file_impl.rb#L30-L32:

      def open(path, mode, options, &block)
      		 
        ::File.open(path, options, mode, &block)
      		 
      end

      The signature of the underlying Ruby method is open(filename [, mode [, perm]] [, opt], &block) - which means that we have shuffled and renamed some of Rubys parameters:

      • Rubys perm in our call is renamed as mode, and is moved from position 2 to position 1
      • Rubys mode in our call is renamed as options, and is moved from position 1 to position 2
      • We don't support the passing in of Rubys opt which is where one would set :encoding => 'utf-8'

      Fortunately there are only a few callsites which use FileSystem.open that are currently impacted:

      Note that both the calls to FileSystem.open and FileSystem.exclusive_open are impacted.

      At the time this PR goes up, another scan should occur to make sure that all callsites are corrected.

      Attachments

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. PUP-6876 Puppet::Settings::IniFile doesn't handle Unicode keys / sections properly - can cause `config set` / `config print` failures

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. PUP-6875 [Spike] Investigate usage of ReGex \w character class in Puppet

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Ready for Engineering

          Bug - A problem which impairs or prevents the functions of the product. PUP-6876 Puppet::Settings::IniFile doesn't handle Unicode keys / sections properly - can cause `config set` / `config print` failures

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Closed

          Task - A task that needs to be done. PUP-6260 SPIKE - PUP-6188 determine how we can test 6188

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. PUP-6959 Puppet::FileSystem.open / exclusive_open / replace_file should use Windows file system support for setting `mode`

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Ready for Engineering

          Improvement - An improvement or enhancement to an existing feature or task. PUP-6904 Evaluate FileType and ParsedFile classes for UTF-8 compatibility

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. PUP-6925 Convert Ruby ::File calls for open to Puppet::FileSystem and explicitly specify encoding

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. PUP-7134 Convert Ruby ::File calls for open to Puppet::FileSystem and explicitly specify encoding for SSL files

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. PUP-1583 Non ASCII comments in puppet.conf causes exception

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. PUP-5879 Ensure Puppet uses FileSystem.read where applicable to read JSON, settings and other files as UTF-8

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Closed

          Activity

            People

              ethan Ethan Brown
              ethan Ethan Brown
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Zendesk Support