Uploaded image for project: 'Puppet Server'
  1. Puppet Server
  2. SERVER-1975

HTTP metrics for the /puppet API should only measure time spent in the JRuby handler

    XMLWordPrintable

Details

    • Major
    • 2 - 5-25% of Customers
    • 4 - Major
    • 4 - $$$$$
    • Hide
      Not having this can lengthen the time to resolution for performance issues. This has been seen both during internal performance debugging and with customers.

      This doesn't directly cause performance issues, but it's a barrier to debugging performance issues.
      Show
      Not having this can lengthen the time to resolution for performance issues. This has been seen both during internal performance debugging and with customers. This doesn't directly cause performance issues, but it's a barrier to debugging performance issues.
    • Enhancement
    • The jruby-metrics section of the status API now includes a borrow-timers dataset that provides separate measurements for each activity that borrows a JRuby instance (catalog compilation, file serving, report processing, etc.).
    • Needs Assessment

    Description

      Currently, the metrics for the puppet/v3 API are measured by the Comidi handler in a fashion that includes all Clojure handlers in addition to JRuby handlers — which includes time spent waiting for a free JRuby worker in addition to time spent by JRuby working on a request. This means that a performance issue in one JRuby route skews the results reported by all JRuby routes, thus making it look like all routes are slow and masking the source of the issue.

      We measure JRuby time spent to handle requests via the Puppet profiler:

      https://github.com/puppetlabs/puppet/blob/5.2.0/lib/puppet/network/http/handler.rb#L62

      However, we send those metrics to /dev/null instead of reporting them to JMX:

      https://github.com/puppetlabs/puppetserver/blob/5.1.0/src/java/com/puppetlabs/puppetserver/MetricsPuppetProfiler.java#L79-L83

      We should stop discarding HTTP metrics measured by the Puppet profiler so that performance issues affecting JRuby routes can be easily identified.

      Attachments

        Issue Links

          Activity

            People

              chuck Charlie Sharpsteen
              chuck Charlie Sharpsteen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Zendesk Support