[PDB-3924] Avoid temp file flood during fact path gc Created: 2018/05/15  Updated: 2019/01/17  Resolved: 2019/01/08

Status: Closed
Project: PuppetDB
Component/s: None
Affects Version/s: None
Fix Version/s: PDB 5.2.7, PDB 6.0.2, PDB 6.2.0

Type: Improvement Priority: Normal
Reporter: Rob Browning Assignee: Rob Browning
Resolution: Fixed Votes: 0
Labels: resolved-issue-added
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to PDB-3857 Fix fact path GC incompatibility with... Closed
relates to PDB-4198 Investigate rampant (often tiny) temp... Accepted
Support
Template:
Team: PuppetDB
Sprint: Hopper/Triage
Release Notes: Bug Fix
Release Notes Summary: PuppetDB should no longer cause PostgreSQL to create a inordinate number of temporary files during its periodic garbage collections. Among other things, this could flood the PostgreSQL's log if its log_temp_files option was set to a small enough value.
QA Risk Assessment: Needs Assessment

 Comments   
Comment by Charlie Sharpsteen [ 2018/11/13 ]

We also observed this during an upgrade when migration 66 was being applied:

https://github.com/puppetlabs/puppetdb/blob/6.0.1/src/puppetlabs/puppetdb/scf/migrate.clj#L1472-L1481

Postgresql work_mem had to be temporarily boosted to 2 GB in order to get through the migration — otherwise the 4 hour service startup timeout was tripped and the upgrade failed. If there is a general solution to this problem, then migration 66 should be re-written to avoid tempfile generation.

Comment by Rob Browning [ 2019/01/10 ]

Kenn Hussey I'm not sure, but note that we currently use "Ready for Merge" as the criteria for the diffusing column on our Kanban board, and that column is where we keep track of things that have been committed to at least one branch, but haven't yet landed everywhere they're destined – in the typical case, they haven't been fully merged up (but there can be cases where we need to cherry-pick).

If there's some other state you'd prefer we use for that, then I assume we'd be happy to.

Thanks

Generated at Sun Aug 18 00:22:03 PDT 2019 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.