Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-2799

Fork improvements



    • Type: Improvement
    • Status: Closed
    • Priority: Normal
    • Resolution: Won't Do
    • Affects Version/s: PUP 3.6.2
    • Fix Version/s: None
    • Component/s: Catalog Application
    • Labels:
    • Template:
    • Story Points:


      This experimentation was based on a blog post comparing the performance of config management systems. It used the equivalent of a bunch of execs, which turns out to be a degenerate case for Puppet. So that was what I tested with; specifically, 1000 echo commands. There were two things I remember from the first time, so I briefly tried to reproduce them now, to make sure I was recalling correctly, and to see if they’re still valid (using Ruby 1.9.3).

      Baseline run time for the catalog was ~60 seconds.

      1) fork+exec seems to trigger a GC in the child process. It’s unnecessary since that memory space is about to go away with the exec, and Ruby’s non-COW-friendly GC causes an awful lot of pointless copying to happen. Disabling the GC before forking and enabling it in the parent after forking.

      Adding the disable and enable calls reduced run time to ~21 seconds.

      2) Closing file descriptors is very slow, since it’s done by attempting to close each one and then rescuing the exception that gets raised. Since almost all FDs are not in use, it’s even worse.

      Adding the io-extra gem and using IO.closefrom (but removing the GC patch) reduced the run time to ~7 seconds (!!!).

      Reapplying the GC patch on top of the io-extra patch didn’t seem to improve things (maybe the GC was from GCing all the exceptions we threw away but don’t with the IO.closefrom call?).

      Then on a whim, I tried the same test on Ruby 2.1. The base runtime was ~11 seconds without either patch, ~9 seconds with the GC patch, ~8 seconds with the IO.closefrom patch, and ~7 seconds with both patches. So maybe the whole thing is soon to be moot.




            kylo Kylo Ginsberg
            QA Contact:
            Eric Thompson
            0 Vote for this issue
            4 Start watching this issue



                Zendesk Support