Affects Version/s: SERVER 0.4.0
Fix Version/s: SERVER 1.0.0
This ticket was spun off from
SERVER-207. The intention of the ticket is just to create some documentation in the Puppet Server project about the issues discovered with trying to have Puppet Server send reports to a virtual ip address which is load balanced to multiple backend servers having different certificates. One of the comments from the ticket was cut/pasted below and could provide a lot of the raw material for the new documentation.
One difference between Puppet Server and the Ruby Puppet master is that Puppet Server's SSL client connections will attempt to resume an SSL session, using the session id provided from the server (PuppetDB, in this case), whereas a Ruby Puppet master will not.
For the case that a load balancer is sitting between the Puppet master and PuppetDB instances, a client connection directed to a server which has no registered session id for the session that the client is trying to resume, the SSL handshake would need to be renegotiated. My understanding is that the JDK, which underlies the Puppet Server master, added a check for uniqueness of the server certificate during a re-negotation following a session resumption. This check is done as a way to help mitigate the TLS triple handshake attack – see https://secure-resumption.com. The client connection is aborted if the check fails.
There appear to be a few options:
1) In order to avoid having the certificate check fail, all end servers could present the same certificate, as suggested in the original description for this ticket.
As it turns out, it appears to be possible for the server certificates to use certificates which are different but have some matching attributes – i.e., the ipAddress in the SubjectAltName extension on the certificates, the dNSName in the SubjectAltName extension on the certificates, and/or the subject and issuer on the certificates. See http://hg.openjdk.java.net/bsd-port/bsd-port/jdk/rev/eabde5c42157#l1.186. The approach of having certificates with selectively matched attributes may not be foolproof, though, in that the JDK implementation may strengthen the comparison in a later version to only accept an exact match for the full data in the certificate.
2) To the JAVA_ARGS variable value in the /etc/sysconfig/puppetserver configuration file, add -Djdk.tls.allowUnsafeServerCertChange=true.
The use of this property is also documented in http://hg.openjdk.java.net/bsd-port/bsd-port/jdk/rev/eabde5c42157#l1.50. While this would appear to allow for the certificate equivalence check to be bypassed entirely, this would also open up the Puppet master more fully to the TLS triple handshake attack. For this reason, this is probably not the best solution.
3) Add the ability via configuration to turn off SSL session caching to the Puppet master client and/or web server running PuppetDB (Jetty).
This last option would involve new implementation work, nothing that an end user could do without code changes. Separate tickets have been filed to enable this work:
TK-124- Disable SSL session caching in the Jetty server TK-125- Disable SSL session caching in the clj-http-client library that Puppet Server uses to make its client requests SERVER-216- Utilize work in TK-125to allow SSL session caching to be disabled for Puppet Server client requests
While this approach would provide mitigation against the triple handshake attack, allow different certificates to be used on each PuppetDB server, and would provide for more backward compatible behavior with the Ruby Puppet master, this approach would have performance tradeoffs. Not having the ability to use session resumption would force a full SSL handshake to occur on each connection for master requests. If the load balancer were aggressively redirecting every request to a different PuppetDB server, a full handshake would inevitably be occurring anyway. If the load balancer were to redirect at least some requests for different connections back to the same server, however, session resumption would otherwise have a performance benefit.