Uploaded image for project: 'Puppet Server'
  1. Puppet Server
  2. SERVER-1975

HTTP metrics for the /puppet API should only measure time spent in the JRuby handler

    XMLWordPrintable

Details

    • Major
    • 2 - 5-25% of Customers
    • 4 - Major
    • 4 - $$$$$
    • Hide
      Not having this can lengthen the time to resolution for performance issues. This has been seen both during internal performance debugging and with customers.

      This doesn't directly cause performance issues, but it's a barrier to debugging performance issues.
      Show
      Not having this can lengthen the time to resolution for performance issues. This has been seen both during internal performance debugging and with customers. This doesn't directly cause performance issues, but it's a barrier to debugging performance issues.
    • Enhancement
    • The jruby-metrics section of the status API now includes a borrow-timers dataset that provides separate measurements for each activity that borrows a JRuby instance (catalog compilation, file serving, report processing, etc.).
    • Needs Assessment

    Description

      Currently, the metrics for the puppet/v3 API are measured by the Comidi handler in a fashion that includes all Clojure handlers in addition to JRuby handlers — which includes time spent waiting for a free JRuby worker in addition to time spent by JRuby working on a request. This means that a performance issue in one JRuby route skews the results reported by all JRuby routes, thus making it look like all routes are slow and masking the source of the issue.

      We measure JRuby time spent to handle requests via the Puppet profiler:

      https://github.com/puppetlabs/puppet/blob/5.2.0/lib/puppet/network/http/handler.rb#L62

      However, we send those metrics to /dev/null instead of reporting them to JMX:

      https://github.com/puppetlabs/puppetserver/blob/5.1.0/src/java/com/puppetlabs/puppetserver/MetricsPuppetProfiler.java#L79-L83

      We should stop discarding HTTP metrics measured by the Puppet profiler so that performance issues affecting JRuby routes can be easily identified.

      Attachments

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-2198 Include JRuby processing time in Puppet Server access log entries

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Resolved
          relates to

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. SERVER-1860 Improve Performance of File Serving in PuppetServer

          • Major - Major loss of function.
          • Closed

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. SERVER-792 Metrics improvements

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Closed

          Activity

            People

              chuck Charlie Sharpsteen
              chuck Charlie Sharpsteen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Zendesk Support