Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-7021

Prevent incompatible encodings reaching Puppet::Parameter#format from ruby Etc module

          Details

          • Type: Bug
          • Status: Closed
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: None
          • Fix Version/s: PUP 4.10.0
          • Component/s: None
          • Labels:
            None
          • Template:
          • Acceptance Criteria:
            Hide

            Given a linux host in a non-utf8 locale, such as ja_JP.EUC-JP, a user can have its groups modified from a utf-8 to a different utf-8 name

            Show
            Given a linux host in a non-utf8 locale, such as ja_JP.EUC-JP, a user can have its groups modified from a utf-8 to a different utf-8 name
          • Epic Link:
          • Team:
            Agent
          • Story Points:
            2
          • Sprint:
            AP 2017-01-25, AP 2017-02-08, AP 2017-02-22, AP 2017-03-08, Agent 2017-03-22, Agent 2017-04-05
          • Release Notes:
            Bug Fix
          • Release Notes Summary:
            Hide
            Prior to PUP 4.10.0, Puppet user management on some *nix platforms could experience various errors related to the handling of UTF-8 characters read in by the ruby Etc module (which parses /etc/passwd and /etc/group for user and group information among other things). As of Puppet 4.10.0 and later, Puppet will attempt to convert values read in by the ruby Etc module to UTF-8 if they are not already UTF-8. If the values cannot be cleanly converted to UTF-8, they are left as-is. These unconvertible values may still cause issues later during the lifecycle of a run, so it is recommended that Puppet ben run in UTF-8 environments.
            Show
            Prior to PUP 4.10.0, Puppet user management on some *nix platforms could experience various errors related to the handling of UTF-8 characters read in by the ruby Etc module (which parses /etc/passwd and /etc/group for user and group information among other things). As of Puppet 4.10.0 and later, Puppet will attempt to convert values read in by the ruby Etc module to UTF-8 if they are not already UTF-8. If the values cannot be cleanly converted to UTF-8, they are left as-is. These unconvertible values may still cause issues later during the lifecycle of a run, so it is recommended that Puppet ben run in UTF-8 environments.
          • QA Risk Assessment:
            No Action
          • QA Risk Assessment Reason:
            Build and infra changes not risk-assessed

            Description

            Puppet::Parameter#format currently does not check if the strings it is interpolating are of incompatible encodings.

            This method is used, among other places, in the resource harness when setting the message for failed change events. If the strings contain incompatible encodings (i.e., they cannot be concatenated) the format method will fail, the event will not contain a message, and then it will fail to be logged.

            This also results in error messages that look like this:

            Error: /Stage[main]/Main/Node[ubuntu16.localdomain]/User[foo]: Could not evaluate: Puppet::Util::Log requires a message

            Notes

            • After diving further into this, this form of problem is scattered all through puppet - the concatenation of strings without regard to encoding. The deeper problem is that strings are entering the system in incompatible encodings to begin with. The problem-area that gave rise to this specific bug is the nameservice provider, and specifically it's reliance on the Etc module in ruby, which can return strings in default external encoding which are then sent around puppet to be compared and munged with utf-8 strings. Thus this ticket is now re-scoped from it's original (smaller context) to include a broader fix of setting strings incoming from Etc to utf-8.

            In Scope

            • Anywhere we take strings in the from the Etc module in a provider, these strings should have UTF-8 encoding

              Attachments

                Issue Links

                relates to

                Bug - A problem which impairs or prevents the functions of the product. PUP-7498 Puppet merges two users in different encodings into one

                • Critical - Crashes, loss of data, severe memory leak.
                • Closed

                Bug - A problem which impairs or prevents the functions of the product. PUP-7031 Unicode characters in 'comment' don't round-trip on Windows

                • Major - Major loss of function.
                • Closed

                Bug - A problem which impairs or prevents the functions of the product. PUP-1473 user resource fails on UTF-8 comment

                • Normal - Regular issue, some loss of functionality under specific circumstances.
                • Closed

                Bug - A problem which impairs or prevents the functions of the product. PUP-6777 regression: incompatible character encodings: UTF-8 and ASCII-8BIT

                • Normal - Regular issue, some loss of functionality under specific circumstances.
                • Closed
                links to

                Web Link Redmine #22703

                  Activity

                    People

                    • Assignee:
                      moses Moses Mendoza
                      Reporter:
                      moses Moses Mendoza
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      4 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved:

                        Zendesk Support