Uploaded image for project: 'PuppetDB'
  1. PuppetDB
  2. PDB-3751

PDB Sync should allow a separate setting for the maximum amount of reports to sync

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: PDB 5.2.13, PDB 6.9.1
    • Component/s: None
    • Labels:
    • Template:
    • Team:
      HA
    • Story Points:
      2
    • Sprint:
      HA Team 2020-01-29, HA Team 2020-02-12, HA Team 2020-02-26
    • CS Priority:
      Normal
    • CS Frequency:
      2 - 5-25% of Customers
    • CS Severity:
      3 - Serious
    • CS Business Value:
      4 - $$$$$
    • CS Impact:
      Hide
      Customers with a large number of reports, or who store more than the default days of reports will take a long time to sync those reports if they lose connectivity between PDB nodes. This would get their sync running again much faster.

      This is analogous to situations where customers run out of disk space on the postgres node. They can truncate the reports table to get running again if they don't need old reports.
      Show
      Customers with a large number of reports, or who store more than the default days of reports will take a long time to sync those reports if they lose connectivity between PDB nodes. This would get their sync running again much faster. This is analogous to situations where customers run out of disk space on the postgres node. They can truncate the reports table to get running again if they don't need old reports.
    • Release Notes:
      New Feature
    • Release Notes Summary:
      Adds the initial-report-threshold config option to the sync settings. On startup, PuppetDB will only sync reports newer than the threshold. Older reports will still be transferred on subsequent periodic syncs.
    • QA Risk Assessment:
      Needs Assessment

      Description

      The Problem

      By default, PDB sync will try to sync all reports in the database with the other PuppetDB. Syncing all of the reports in the database is generally not valuable and syncing a smaller portion and then growing the new sync node as new reports come in would likely work great for 90% of users.

      In this case, you could set the sync-report-ttl to say 1d while leaving report-ttl at 14d. When setting up a new sync node you'd only sync 1d of reports and over the next 13 days the new reports coming in would bring the sync node up to the 14d of report-ttl.

      This also allows for sync to go down for multiple days and then come back up relatively quickly depending on how you decide to configure these settings.

      Having this as a configurable setting provides a release valve when you're trying to get sync working and you don't want to wait for 14 days of reports to sync. You just want sync working now and you don't care about the reports syncing all the way back in time.

        Attachments

          Activity

            People

            Assignee:
            austin.blatt Austin Blatt
            Reporter:
            nick.walker Nick Walker
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Zendesk Support