-
Type:
Bug
-
Status: Closed
-
Priority:
Normal
-
Resolution: Cannot Reproduce
-
Affects Version/s: PUP 3.6.0
-
Fix Version/s: None
-
Component/s: None
-
Labels:None
-
Environment:
3.6.0 master, 3.4.2 and 3.6.0 agents (Ubuntu 12.04 and CentOS 6.2)
-
Template:customfield_10700 36987
We've seen errors like
Tue May 27 08:38:34 -0700 2014 /Stage[main]/Puppet::Puppetize_sh/File[/root/puppetize.sh] (err): Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/puppet/puppetize.sh
occasionally for a long time now. Out of 4000 or so hosts, we would see maybe 10-20 of these per day. The files seem to be randomly selected from those for which we use source => "puppet://...". I have assumed this was related to some occasional rails failure, as seems to be common with the framework.
After a partial upgrade to 3.6.0, we're now seeing hundreds per hour. This makes it a lot more difficult to find real errors! To take one particular example:
Tue May 27 08:29:59 -0700 2014 /Stage[main]/Shellprofile::Base/File[/etc/profile] (err): Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/shellprofile/darwin-profile
Tue May 27 08:29:59 -0700 2014 /Stage[main]/Buildslave::Startup::Launchd/File[/usr/local/bin/run-buildslave.sh] (err): Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/buildslave/darwin-run-buildslave.sh
the puppet master's access_log shows a bunch of successful runs, followed by 404's:
10.12.51.130 - - [27/May/2014:07:28:04 -0700] "GET /production/file_metadata/modules/shellprofile/darwin-profile?links=manage HTTP/1.1" 200 320 "" ""
10.12.51.130 - - [27/May/2014:07:28:04 -0700] "GET /production/file_metadata/modules/buildslave/darwin-run-buildslave.sh?links=manage HTTP/1.1" 200 328 "" ""
10.12.51.130 - - [27/May/2014:07:39:35 -0700] "GET /production/file_metadata/modules/shellprofile/darwin-profile?links=manage HTTP/1.1" 200 320 "" ""
10.12.51.130 - - [27/May/2014:07:39:35 -0700] "GET /production/file_metadata/modules/buildslave/darwin-run-buildslave.sh?links=manage HTTP/1.1" 200 328 "" ""
10.12.51.130 - - [27/May/2014:08:29:59 -0700] "GET /production/file_metadata/modules/shellprofile/darwin-profile?links=manage HTTP/1.1" 404 75 "" ""
10.12.51.130 - - [27/May/2014:08:29:59 -0700] "GET /production/file_metadata/modules/buildslave/darwin-run-buildslave.sh?links=manage HTTP/1.1" 404 83 "" ""
The files in question were present before, during, and after that time period.