Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-2665

Increased rate of 404's for legitimate file_metadata resources with 3.6.0

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Cannot Reproduce
    • Affects Version/s: PUP 3.6.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      3.6.0 master, 3.4.2 and 3.6.0 agents (Ubuntu 12.04 and CentOS 6.2)

    • Template:

      Description

      We've seen errors like

      Tue May 27 08:38:34 -0700 2014 /Stage[main]/Puppet::Puppetize_sh/File[/root/puppetize.sh] (err): Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/puppet/puppetize.sh

      occasionally for a long time now. Out of 4000 or so hosts, we would see maybe 10-20 of these per day. The files seem to be randomly selected from those for which we use source => "puppet://...". I have assumed this was related to some occasional rails failure, as seems to be common with the framework.

      After a partial upgrade to 3.6.0, we're now seeing hundreds per hour. This makes it a lot more difficult to find real errors! To take one particular example:

      Tue May 27 08:29:59 -0700 2014 /Stage[main]/Shellprofile::Base/File[/etc/profile] (err): Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/shellprofile/darwin-profile
      Tue May 27 08:29:59 -0700 2014 /Stage[main]/Buildslave::Startup::Launchd/File[/usr/local/bin/run-buildslave.sh] (err): Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/buildslave/darwin-run-buildslave.sh

      the puppet master's access_log shows a bunch of successful runs, followed by 404's:

      10.12.51.130 - - [27/May/2014:07:28:04 -0700] "GET /production/file_metadata/modules/shellprofile/darwin-profile?links=manage HTTP/1.1" 200 320 "" ""
      10.12.51.130 - - [27/May/2014:07:28:04 -0700] "GET /production/file_metadata/modules/buildslave/darwin-run-buildslave.sh?links=manage HTTP/1.1" 200 328 "" ""
      10.12.51.130 - - [27/May/2014:07:39:35 -0700] "GET /production/file_metadata/modules/shellprofile/darwin-profile?links=manage HTTP/1.1" 200 320 "" ""
      10.12.51.130 - - [27/May/2014:07:39:35 -0700] "GET /production/file_metadata/modules/buildslave/darwin-run-buildslave.sh?links=manage HTTP/1.1" 200 328 "" ""
      10.12.51.130 - - [27/May/2014:08:29:59 -0700] "GET /production/file_metadata/modules/shellprofile/darwin-profile?links=manage HTTP/1.1" 404 75 "" ""
      10.12.51.130 - - [27/May/2014:08:29:59 -0700] "GET /production/file_metadata/modules/buildslave/darwin-run-buildslave.sh?links=manage HTTP/1.1" 404 83 "" ""

      The files in question were present before, during, and after that time period.

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            djmitche Dustin J. Mitchell
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Zendesk Support