Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-7021

Prevent incompatible encodings reaching Puppet::Parameter#format from ruby Etc module

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: PUP 4.10.0
    • Component/s: None
    • Labels:
      None
    • Template:
    • Acceptance Criteria:
      Hide

      Given a linux host in a non-utf8 locale, such as ja_JP.EUC-JP, a user can have its groups modified from a utf-8 to a different utf-8 name

      Show
      Given a linux host in a non-utf8 locale, such as ja_JP.EUC-JP, a user can have its groups modified from a utf-8 to a different utf-8 name
    • Epic Link:
    • Team:
      Agent
    • Story Points:
      2
    • Sprint:
      AP 2017-01-25, AP 2017-02-08, AP 2017-02-22, AP 2017-03-08, Agent 2017-03-22, Agent 2017-04-05
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Hide
      Prior to PUP 4.10.0, Puppet user management on some *nix platforms could experience various errors related to the handling of UTF-8 characters read in by the ruby Etc module (which parses /etc/passwd and /etc/group for user and group information among other things). As of Puppet 4.10.0 and later, Puppet will attempt to convert values read in by the ruby Etc module to UTF-8 if they are not already UTF-8. If the values cannot be cleanly converted to UTF-8, they are left as-is. These unconvertible values may still cause issues later during the lifecycle of a run, so it is recommended that Puppet ben run in UTF-8 environments.
      Show
      Prior to PUP 4.10.0, Puppet user management on some *nix platforms could experience various errors related to the handling of UTF-8 characters read in by the ruby Etc module (which parses /etc/passwd and /etc/group for user and group information among other things). As of Puppet 4.10.0 and later, Puppet will attempt to convert values read in by the ruby Etc module to UTF-8 if they are not already UTF-8. If the values cannot be cleanly converted to UTF-8, they are left as-is. These unconvertible values may still cause issues later during the lifecycle of a run, so it is recommended that Puppet ben run in UTF-8 environments.
    • QA Risk Assessment:
      No Action
    • QA Risk Assessment Reason:
      Build and infra changes not risk-assessed

      Description

      Puppet::Parameter#format currently does not check if the strings it is interpolating are of incompatible encodings.

      This method is used, among other places, in the resource harness when setting the message for failed change events. If the strings contain incompatible encodings (i.e., they cannot be concatenated) the format method will fail, the event will not contain a message, and then it will fail to be logged.

      This also results in error messages that look like this:

      Error: /Stage[main]/Main/Node[ubuntu16.localdomain]/User[foo]: Could not evaluate: Puppet::Util::Log requires a message

      Notes

      • After diving further into this, this form of problem is scattered all through puppet - the concatenation of strings without regard to encoding. The deeper problem is that strings are entering the system in incompatible encodings to begin with. The problem-area that gave rise to this specific bug is the nameservice provider, and specifically it's reliance on the Etc module in ruby, which can return strings in default external encoding which are then sent around puppet to be compared and munged with utf-8 strings. Thus this ticket is now re-scoped from it's original (smaller context) to include a broader fix of setting strings incoming from Etc to utf-8.

      In Scope

      • Anywhere we take strings in the from the Etc module in a provider, these strings should have UTF-8 encoding

        Attachments

          Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. PUP-7498 Puppet merges two users in different encodings into one

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. PUP-7031 Unicode characters in 'comment' don't round-trip on Windows

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. PUP-1473 user resource fails on UTF-8 comment

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. PUP-6777 regression: incompatible character encodings: UTF-8 and ASCII-8BIT

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Closed
          links to

          Web Link Redmine #22703

            Activity

              People

              Assignee:
              moses Moses Mendoza
              Reporter:
              moses Moses Mendoza
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support