Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-9092

External process termination handling rate (including "exec" resource) is slow

    Details

    • Template:
    • Team:
      Coremunity
    • Sprint:
      Platform Core KANBAN
    • Method Found:
      Needs Assessment
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Hide
      Previously puppet took at least one second to execute external processes, even if the process completed more quickly than that. This fix significantly decreases the time it takes the puppet agent to apply a catalog if the catalog contains a large number of exec resources and each child process completes in less than 1 second.
      Show
      Previously puppet took at least one second to execute external processes, even if the process completed more quickly than that. This fix significantly decreases the time it takes the puppet agent to apply a catalog if the catalog contains a large number of exec resources and each child process completes in less than 1 second.
    • QA Risk Assessment:
      Needs Assessment

      Description

      Puppet Version: 4.10.5+
      Puppet Server Version: irrelevant
      OS Name/Version: Any supported Windows OS

      Executing external processes by puppet on Windows can be very slow, compared to the execution time of the process itself, if the execution time is below 1 second. When applying manifests that contain many "exec" resources, it may take many minutes to apply. When analyzing logs, I took note that even simple "reg query" process took 1.02/1.03 second to run. Windows doesn't spawn processes that slow, I wrote simple example in C# which showed that my machine is capable of executing about 140 of such commands per second (no parallelism, waiting for exit of each process before spawning another).
      I did a little bit of digging and found this piece of code. This code waits for at least one second, unless the process terminated immediately (which is rarely feasible).
      Replacing 'sleep(1)' with 'sleep(0)' is the simplest fix, but is not too efficient from a CPU consumption standpoint, so I propose a different fix in a PR#7033.

      I did a little more digging and found out that this 'sleep(1)' was added to properly handle signals, I did additional testing ("exec"-ed a slow process and tried to Ctrl-C an interactive puppet run), and it worked.

      Desired Behavior: Process termination is handled immediately

      Actual Behavior: Processes are considered to be running at least for 1 second, even if they exit immediately

       It would be great if this gets into 4.10.z release train, since at least our company is not yet ready for 5.x.

        Attachments

          Activity

            jsd-sla-details-panel

              People

              • Assignee:
                josh Josh Cooper
                Reporter:
                onyxmaster Aristarkh Zagorodnikov
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: