Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-10780

Optimize environment cache eviction

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Accepted
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Template:
    • QA Risk Assessment:
      Needs Assessment

      Description

      Puppetserver calls Puppet::Environment::Cached#get! for each puppet/v3 REST request (eg node, catalog, file_metadata(s), report), and each call causes the cache to check the eviction status of N other environments. So if puppetserver is handling large numbers of environments, then we'll get quadratic behavior.

      I think it'd be better if the environment cache kept a list of environments sorted on expiration time (in ascending order) and if puppetserver called directly into the cache to mark expired environments, rather than waiting for puppet to callback later, one environment at a time. Whenever an environment is touched, we just need to remove it from the sorted list and add it back with the new expiration time.

      Cache eviction then becomes "evict all marked environments" and "evict environments from the sorted list, and stop when we reach the first non-expired environment or end of list". We can also optimize for the case where the marked set is empty and the first entry in the expiration list is in the future.

      /cc Nick Walker, Maggie Dreyer, Justin Stoller

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              josh Josh Cooper
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:

                  Zendesk Support