Uploaded image for project: 'PuppetDB'
  1. PuppetDB
  2. PDB-3742

Sync should skip records that are already present in the command queue

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: PDB 5.1.3
    • Fix Version/s: PDB 5.2.9, PDB 6.0.4, PDB 6.3.1
    • Component/s: None
    • Labels:
      None
    • Template:
    • Team:
      PuppetDB
    • CS Priority:
      Normal
    • CS Frequency:
      2 - 5-25% of Customers
    • CS Severity:
      3 - Serious
    • CS Business Value:
      4 - $$$$$
    • CS Impact:
      Hide
      This can lead to long sync times for customers which have a lot of data to sync. In those cases this causes the sync process to consume many more resource than it should. Such as database connections, CPU usage, network traffic, all to process sync data that is already sitting in the command queue waiting to get into the database.
      Show
      This can lead to long sync times for customers which have a lot of data to sync. In those cases this causes the sync process to consume many more resource than it should. Such as database connections, CPU usage, network traffic, all to process sync data that is already sitting in the command queue waiting to get into the database.
    • Release Notes Summary:
      Hide
      Allow PupppetDB to take already enqueued commands into account when deciding which records need to by pulled from a remote PuppetDB during HA Sync. This fixes an issue where duplicate commands could be added to the command queue during sync, which could result in extra work and more downtime before the PuppetDB doing the sync could respond to new commands.
      Show
      Allow PupppetDB to take already enqueued commands into account when deciding which records need to by pulled from a remote PuppetDB during HA Sync. This fixes an issue where duplicate commands could be added to the command queue during sync, which could result in extra work and more downtime before the PuppetDB doing the sync could respond to new commands.
    • QA Risk Assessment:
      Needs Assessment

      Description

      PuppetDB sync compares the contents of two databases and transfers over any data that isn't present in the local database. If a large amount of data is transferred, a large command queue will be created which has to be worked off. This can result in several sync cycles transferring the same data before the first copy reaches the front of the command queue and is inserted into the database.

      PuppetDB sync should list the IDs of records in the queue at the beginning of sync and skip transferring any remote records that are included in the list.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zachary.kent Zachary Kent
                Reporter:
                chuck Charlie Sharpsteen
              • Votes:
                1 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Zendesk Support