[PUP-6777] regression: incompatible character encodings: UTF-8 and ASCII-8BIT Created: 2016/10/04  Updated: 2017/02/02  Resolved: 2016/12/28

Status: Closed
Project: Puppet
Component/s: None
Affects Version/s: PUP 4.6.1
Fix Version/s: PUP 4.9.0

Type: Bug Priority: Normal
Reporter: Klavs Klavsen Assignee: Moses Mendoza
Resolution: Fixed Votes: 1
Labels: i18n, utf-8
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to PUP-7031 Unicode characters in 'comment' don't... Closed
relates to PUP-4633 User resource fails with UTF-8 commen... Closed
relates to PUP-7021 Prevent incompatible encodings reachi... Closed
Template:
Acceptance Criteria:
  • Tests
  • Fix passes CI all platforms
Epic Link: Phase 1 Puppet Unicode Adoption Blockers
Team: Agent
Story Points: 3
Sprint: AP 2016-12-14, AP 2017-01-11
Release Notes: Bug Fix
Release Notes Summary: Same as PUP-7031.
QA Risk Assessment: Automate
QA Risk Assessment Reason: Regression

 Description   

This bug - seemingly fixed in https://github.com/puppetlabs/puppet/pull/3977#issuecomment-251466761 seems to have cropped up again:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=782494

This bug has cropped up again in puppet-4.6.1 (puppet-agent 1.6.1) - since we upgraded from puppet-3.7.1 on an agent. it only happens when we run puppet in cron - and the workaround LANG=en_US.utf8 in front of puppet agent -t - works.



 Comments   
Comment by Angel L. Mateo [ 2016/12/02 ]

I'm having this same problem with puppet running on ubuntu server. I'm using now puppet-agent 4.8.1, but 4.8.0 has the problem too.

Comment by Moses Mendoza [ 2016/12/02 ]

thanks for the note Angel L. Mateo

Comment by Moses Mendoza [ 2016/12/09 ]

PR raised, but I think we're going to want acceptance tests added. Working on those.

Comment by Ethan Brown [ 2016/12/21 ]

Merged to master in https://github.com/puppetlabs/puppet/commit/6f945619d49fd38c7a1d12f3262c02430c6ce34f

Comment by Moses Mendoza [ 2016/12/27 ]

this has failed CI on arista hosts, which seems to be shipped with stripped down locale support.

failure:

18:30:40           Begin tests/resource/user/utf8_user_comments.rb
18:30:40           
18:30:40           PUP-6777 Manage users with UTF-8 comments
18:30:40             
18:30:40             * ensure user can be created with UTF-8 comment (with UTF-8 environment on *nix)
18:30:40               
18:30:40               gyfkbhj30z5lyvk.delivery.puppetlabs.net (arista4-32-1) 18:30:40$ mktemp -t apply_manifest.pp.XXXXXX
18:30:40                 /tmp/apply_manifest.pp.3CebKh
18:30:40               
18:30:40               gyfkbhj30z5lyvk.delivery.puppetlabs.net (arista4-32-1) executed in 0.01 seconds
18:30:40               localhost $ scp /tmp/beaker20161223-29130-ks592z arista4-32-1:/tmp/apply_manifest.pp.3CebKh {:ignore => }
18:30:40               
18:30:40               gyfkbhj30z5lyvk.delivery.puppetlabs.net (arista4-32-1) 18:30:40$ env LANG="en_US.UTF-8" puppet apply --verbose --detailed-exitcodes /tmp/apply_manifest.pp.3CebKh
18:30:40                 2016-12-24                                 02:30:40.813793                                 WARN                                                  puppetlabs.facter                 -                 locale environment variables were bad; continuing with LANG=C LC_ALL=C
18:30:42                 Notice: Compiled catalog for gyfkbhj30z5lyvk.delivery.puppetlabs.net in environment production in 0.11 seconds
18:30:42                 Info: Applying configuration version '1482546642'
18:30:43                 Error: Could not set comment on user[foo9606]: Execution of '/usr/sbin/usermod -c Aۿ foo9606' returned 6: usermod: user 'foo9606' does not exist in /etc/passwd
18:30:43                 Error: /Stage[main]/Main/User[foo9606]/comment: change from Unknown User (foo9606) to Aۿ failed: Could not set comment on user[foo9606]: Execution of '/usr/sbin/usermod -c Aۿ foo9606' returned 6: usermod: user 'foo9606' does not exist in /etc/passwd
18:30:43                 Notice: Applied catalog in 0.25 seconds
18:30:43               
18:30:43               gyfkbhj30z5lyvk.delivery.puppetlabs.net (arista4-32-1) executed in 2.38 seconds
18:30:43               Exited: 4
18:30:43               Beaker::Host::CommandFailure: Host 'gyfkbhj30z5lyvk.delivery.puppetlabs.net' exited with 4 running:
18:30:43                env LANG="en_US.UTF-8" puppet apply --verbose --detailed-exitcodes /tmp/apply_manifest.pp.3CebKh
18:30:43               Last 10 lines of output were:
18:30:43               	2016-12-24 02:30:40.813793 WARN  puppetlabs.facter - locale environment variables were bad; continuing with LANG=C LC_ALL=C
18:30:43               	Notice: Compiled catalog for gyfkbhj30z5lyvk.delivery.puppetlabs.net in environment production in 0.11 seconds
18:30:43               	Info: Applying configuration version '1482546642'
18:30:43               	Error: Could not set comment on user[foo9606]: Execution of '/usr/sbin/usermod -c Aۿ foo9606' returned 6: usermod: user 'foo9606' does not exist in /etc/passwd
18:30:43               	Error: /Stage[main]/Main/User[foo9606]/comment: change from Unknown User (foo9606) to Aۿ failed: Could not set comment on user[foo9606]: Execution of '/usr/sbin/usermod -c Aۿ foo9606' returned 6: usermod: user 'foo9606' does not exist in /etc/passwd
18:30:43               	Notice: Applied catalog in 0.25 seconds
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/host.rb:366
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/dsl/helpers/host_helpers.rb:83
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/shared/host_manager.rb:127
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/dsl/patterns.rb:37
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/dsl/helpers/host_helpers.rb:63
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/dsl/helpers/puppet_helpers.rb:487
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/shared/host_manager.rb:127
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/shared/host_manager.rb:115
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/shared/host_manager.rb:114
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/shared/host_manager.rb:114
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/dsl/patterns.rb:37
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/dsl/helpers/puppet_helpers.rb:416
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/tests/resource/user/utf8_user_comments.rb:44
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/dsl/structure.rb:43
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/tests/resource/user/utf8_user_comments.rb:36
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/dsl/structure.rb:58
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/tests/resource/user/utf8_user_comments.rb:10
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/test_case.rb:133
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/test_case.rb:133
18:30:43               /usr/local/rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/benchmark.rb:308
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/test_case.rb:130
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/test_suite.rb:325
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/test_suite.rb:322
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/test_suite.rb:322
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/test_suite.rb:371
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/cli.rb:180
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/lib/beaker/cli.rb:106
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/gems/beaker-3.5.0/bin/beaker:6
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/bin/beaker:23
18:30:43               /var/lib/jenkins/workspace/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL/beaker/TEST_TARGET/arista4-32a/puppet/acceptance/.bundle/gems/bin/beaker:23

(https://jenkins.puppetlabs.com/view/puppet-agent/view/master/view/puppet-agent/job/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/SLAVE_LABEL=beaker,TEST_TARGET=arista4-32a/296/consoleFull)

On an arista host:

-bash-4.1# locale -a
C
POSIX

I'm going to take a couple minutes and see if we can't massage this into working on arista but if not I'll add a confine to except that platform

Comment by Moses Mendoza [ 2016/12/27 ]

Raised another PR to confine against arista. See ARISTA-42 and PUP-7049 - the arista user provider is effectively non-functional for our purposes so the new tests aren't going to work

Comment by Moses Mendoza [ 2016/12/28 ]

this has passed CI, here: https://jenkins.puppetlabs.com/view/puppet-agent/view/master/view/puppet-agent/job/platform_puppet-agent_intn-van-sys_suite-daily-puppet-master/297/

Comment by Kenn Hussey [ 2017/01/13 ]

Moses Mendoza please provide release notes for this issue.

Generated at Thu Nov 21 02:41:19 PST 2019 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.