Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-7021

Prevent incompatible encodings reaching Puppet::Parameter#format from ruby Etc module

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: PUP 4.10.0
    • Component/s: None
    • Labels:
      None
    • Template:
    • Acceptance Criteria:
      Hide

      Given a linux host in a non-utf8 locale, such as ja_JP.EUC-JP, a user can have its groups modified from a utf-8 to a different utf-8 name

      Show
      Given a linux host in a non-utf8 locale, such as ja_JP.EUC-JP, a user can have its groups modified from a utf-8 to a different utf-8 name
    • Team:
      Agent
    • Story Points:
      2
    • Sprint:
      AP 2017-01-25, AP 2017-02-08, AP 2017-02-22, AP 2017-03-08, Agent 2017-03-22, Agent 2017-04-05
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Hide
      Prior to PUP 4.10.0, Puppet user management on some *nix platforms could experience various errors related to the handling of UTF-8 characters read in by the ruby Etc module (which parses /etc/passwd and /etc/group for user and group information among other things). As of Puppet 4.10.0 and later, Puppet will attempt to convert values read in by the ruby Etc module to UTF-8 if they are not already UTF-8. If the values cannot be cleanly converted to UTF-8, they are left as-is. These unconvertible values may still cause issues later during the lifecycle of a run, so it is recommended that Puppet ben run in UTF-8 environments.
      Show
      Prior to PUP 4.10.0, Puppet user management on some *nix platforms could experience various errors related to the handling of UTF-8 characters read in by the ruby Etc module (which parses /etc/passwd and /etc/group for user and group information among other things). As of Puppet 4.10.0 and later, Puppet will attempt to convert values read in by the ruby Etc module to UTF-8 if they are not already UTF-8. If the values cannot be cleanly converted to UTF-8, they are left as-is. These unconvertible values may still cause issues later during the lifecycle of a run, so it is recommended that Puppet ben run in UTF-8 environments.
    • QA Risk Assessment:
      No Action
    • QA Risk Assessment Reason:
      Build and infra changes not risk-assessed

      Description

      Puppet::Parameter#format currently does not check if the strings it is interpolating are of incompatible encodings.

      This method is used, among other places, in the resource harness when setting the message for failed change events. If the strings contain incompatible encodings (i.e., they cannot be concatenated) the format method will fail, the event will not contain a message, and then it will fail to be logged.

      This also results in error messages that look like this:

      Error: /Stage[main]/Main/Node[ubuntu16.localdomain]/User[foo]: Could not evaluate: Puppet::Util::Log requires a message
      

      Notes

      • After diving further into this, this form of problem is scattered all through puppet - the concatenation of strings without regard to encoding. The deeper problem is that strings are entering the system in incompatible encodings to begin with. The problem-area that gave rise to this specific bug is the nameservice provider, and specifically it's reliance on the Etc module in ruby, which can return strings in default external encoding which are then sent around puppet to be compared and munged with utf-8 strings. Thus this ticket is now re-scoped from it's original (smaller context) to include a broader fix of setting strings incoming from Etc to utf-8.

      In Scope

      • Anywhere we take strings in the from the Etc module in a provider, these strings should have UTF-8 encoding

        Attachments

          Issue Links

            Activity

              jsd-sla-details-panel

                People

                • Assignee:
                  moses Moses Mendoza
                  Reporter:
                  moses Moses Mendoza
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Zendesk Support