Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-7451

Puppet HTTP client should respect Retry-After headers

    XMLWordPrintable

    Details

    • Template:
    • Acceptance Criteria:
      Hide
      • If a HTTP request made through the Puppet::Network::HTTP library returns a 503 "Service Unavailable" or 429 "Too Many Requests" response code that includes a Retry-After header, then the agent sleeps for the number of seconds specified in the header and then retry the request.
      • Should be able to specify a maximum number of retries.
      Show
      If a HTTP request made through the Puppet::Network::HTTP library returns a 503 "Service Unavailable" or 429 "Too Many Requests" response code that includes a Retry-After header, then the agent sleeps for the number of seconds specified in the header and then retry the request. Should be able to specify a maximum number of retries.
    • Team:
      Platform Core
    • Story Points:
      5
    • Sprint:
      Platform Core KANBAN
    • CS Priority:
      Major
    • CS Frequency:
      2 - 5-25% of Customers
    • CS Severity:
      4 - Major
    • CS Business Value:
      4 - $$$$$
    • CS Impact:
      Poor handling of thundering herds is an issue for many customers running large numbers of nodes. This would effectively allow the server to re-splay nodes and avoid many or maybe all calls about thundering herd related performance issues.
    • Release Notes:
      New Feature
    • Release Notes Summary:
      Hide
      The HTTP client used by the Puppet Agent now respects Retry-After header fields included with 503 or 429 responses. This behavior can be combined with the max-queued-requests and max-retry-delay settings introduced in Puppet Server 5.1.0 to automatically re-splay agents that have formed a thundering herd.
      Show
      The HTTP client used by the Puppet Agent now respects Retry-After header fields included with 503 or 429 responses. This behavior can be combined with the max-queued-requests and max-retry-delay settings introduced in Puppet Server 5.1.0 to automatically re-splay agents that have formed a thundering herd.
    • QA Risk Assessment:
      No Action

      Description

      If a HTTP request made through the Puppet::Network::HTTP library returns a 503 "Service Unavailable" or 429 "Too Many Requests" response code that includes a Retry-After header, then the agent should sleep for the number of seconds specified in the header and then retry the request.

      This would enable Puppet Server to automatically re-splay thundering herds of agents as discussed in SERVER-1767.

      Relevant HTTP standards:

      The 503 (Service Unavailable) status code indicates that the server
      is currently unable to handle the request due to a temporary overload
      or scheduled maintenance, which will likely be alleviated after some
      delay. The server MAY send a Retry-After header field
      (Section 7.1.3) to suggest an appropriate amount of time for the
      client to wait before retrying the request.

      http://tools.ietf.org/html/rfc7231#section-6.6.4

      The 429 status code indicates that the user has sent too many
      requests in a given amount of time ("rate limiting").

      The response representations SHOULD include details explaining the
      condition, and MAY include a Retry-After header indicating how long
      to wait before making a new request.

      http://tools.ietf.org/html/rfc6585#section-4

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              chuck Charlie Sharpsteen
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support