[PDB-3684] PDB performance issues with large structured facts Created: 2017/09/20  Updated: 2018/02/13  Resolved: 2018/01/04

Status: Closed
Project: PuppetDB
Component/s: PuppetDB
Affects Version/s: PDB 4.4.1
Fix Version/s: PDB 5.2.0

Type: Bug Priority: Major
Reporter: Adam Bottchen Assignee: Unassigned
Resolution: Fixed Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
relates to PDB-3630 Configurable facts blacklist Closed
relates to PDB-3611 Fact filter from overview page uses n... Closed
relates to PDB-3249 Reduplicate fact values Closed
relates to PDB-3723 Use JSONB for fact storage Closed
Sprint: Hopper/Triage, Data Platform 2017-10-18
Method Found: Customer Feedback
CS Priority: Critical
CS Frequency: 4 - 50-90% of Customers
CS Severity: 4 - Major
CS Business Value: 5 - $$$$$$
CS Impact: This is a very common and troubleshooting it has consumed a huge amount of support time. The workaround is to stop sending large structured facts which in some cases impacts their use of puppet.

In one example a custom structured fact collected rpm data which they were using in their code somehow so turning it off wasn't really a workaround but a reduction in functionality.

This can be very common on Solaris where there is a mount per user. A 20-30KB structured fact can be easily reached on solaris and cause this issue.
Release Notes: Not Needed


As outlined in PDB-2631, PuppetDB encounters severe performance degradation in the presence of large structured facts. In support, we have seen this impact with mountpoint or partitions facts of only 20-30 KiB, which is not terribly uncommon. As noted in that ticket, the expectation is that PDB will be able to handle anything Facter might throw at it.

This issue was resolved in PDB-3249, but the fix was removed by PDB-3611. We need a solution that will allow large structured facts to be stored in PDB without causing causing query performance issues.

Comment by Nick Walker [ 2017/10/18 ]

A configurable facts blacklist in PDB would mitigate but not solve this issue.


Comment by Wyatt Alt [ 2017/12/06 ]

We believe these issues will be resolved with https://github.com/puppetlabs/puppetdb/pull/2410

Comment by Wyatt Alt [ 2017/12/13 ]

The above has been merged.

Comment by Nick Walker [ 2018/01/04 ]

Russell Mull is this resolved by PDB-3723? If so we can close this out?

Comment by Russell Mull [ 2018/01/04 ]

Fixed by PDB-3723

Generated at Sat Aug 08 07:08:26 PDT 2020 using Jira 8.5.2#805002-sha1:a66f9354b9e12ac788984e5d84669c903a370049.