Resolution: Won't Fix
Affects Version/s: PDB 2.3.3
Fix Version/s: None
CentOS 7, Postgres 9.2.7
Just as an aside to
PDB-1448, we're also seeing PuppetDB run out of heap.
Looking at the memory dumps, we see a very large number of Postgres Prepared Statements, taking up huge amounts of memory.
For example, from today we had:
4,881 instances of "org.postgresql.jdbc4.Jdbc4PreparedStatement" with retained size of 500,866,312 bytes
and one from a month ago:
4,614 instances of "org.postgresql.jdbc4.Jdbc4PreparedStatement" with retained size of 617,294,101 bytes.
Our heap size is 750M, and 109 puppet agents.
My guess is that when the 'replace facts' fails with the foreign key constraint violation, it doesn't clean up the prepared statement objects.
For now, we're restarting PuppetDB when it becomes unresponsive about once a week (usually don't wait for a heap dump to happen) while we wait for the fix for
PDB-1448 to be released.