[SERVER-1770] Gatling perf testing with Jetty 9.4.x in Puppet Server Created: 2017/04/05  Updated: 2017/06/08  Resolved: 2017/06/08

Status: Closed
Project: Puppet Server
Component/s: None
Affects Version/s: None
Fix Version/s: SERVER 5.0.0

Type: Task Priority: Normal
Reporter: Jeremy Barlow Assignee: Adrien Thebo
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Epic Link: Drop Java 7
Team: Systems Engineering
Sub-team: Server
Story Points: 3
Sprint: Server 2017-05-31, Server 2017-06-14
Release Notes: Not Needed
QA Risk Assessment: Needs Assessment


For this ticket, we would do some preliminary perf testing around the use of Jetty 9.4.x in Puppet Server. This would involve:

  1. Assuming SERVER-1741 has not been completed yet, create a custom build of Puppet Server with its Jetty dependency bumped up to 9.4.x (via a preliminary tk-jetty9 2.x).
  2. Do a couple of moderate-length Gatling runs, one with Puppet Server having the same Jetty as in the OSS Puppet Server 2.7.2 release, 9.2.10, vs. one with Puppet Server having the latest Jetty 9.4.x release.

For these, we could just start with some simple 2 hour runs - simulating up to 1200 agents doing perf-control medium catalog compilations like the current oss-puppetserver-latest Jenkins job does. As we get closer to the Puppet Server 5 release, we should look into some longer-duration runs, e.g., a week long run, but I think the shorter A/B comparison run for now to get a pulse on Jetty performance is good enough.

Comment by Adrien Thebo [ 2017/06/08 ]

Between the perf oss-stable and oss-latest jobs there was almost no performance difference. The stable job had a mean response time of 33ms with a standard deviation of 189, while the latest job had a mean response time of 35ms with a standard deviation of 196. With respect to individual requests catalog compilation was slower on oss-latest but requests like file metadata was about equivalently fast, which makes me believe that any performance issues between stable are latest are unrelated to the jetty bump.

Generated at Fri Jun 05 20:05:12 PDT 2020 using Jira 8.5.2#805002-sha1:a66f9354b9e12ac788984e5d84669c903a370049.