Certificate issued from Intermediate CA cert via Puppet Server CA service fails validation

I roughly went through the examples from this site to generate a Root CA and a subordinate Intermediate CA. I dropped the resulting files into the /etc/puppetlabs/puppet/ssl/ca directory on a Puppet Server:

• private key of the Intermediate CA - attached as ca_key.pem
• CA bundle (Root + Intermediate CA certificates) - attached as ca_crt.pem
• CA CRL (Root + Intermediate CRLs) - attached as ca_crl.pem

When I initially did a new agent run - without any prior ssl artifacts in place on the agent - against a master using the Root + Intermediate CA artifacts, I ran into SERVER-1315, which documents how having multiple CA certificates in the ca_crt.pem file causes either the Puppet Server autosigning or certificate_status endpoint requests to fail. I removed the Root CA from the ca_crt.pem file - resulting in contents like the attached ca_crt_intermediate.pem file - and tried a certificate_status signing request again. The signing process was successful. Subsequent agent runs against the master, however, still failed with the following error:

Error: /File[/opt/puppetlabs/puppet/cache/facts.d]: Failed to generate additional resources using 'eval_generate': SSL_connect SYSCALL returned=5 errno=0 state=unknown state

The error in this case appears to be due to the master not trusting the agent certificate. If I had instead signed the certificate via the command line with puppet cert sign, however, the resulting agent certificate could successfully be used to complete subsequent agent runs.

I’ve attached two certificates signed with the Intermediate CA certificate:

• agent5.pem (signed by calling certificate_status with a PUT request)
• agent6.pem (signed by running `puppet cert sign` from the command line)

I found via running the openssl verify command that the ‘agent5’ certificate was considered invalid:

openssl verify -CAfile ca_crt_root.pem -untrusted ca_crt_intermediate.pem agent5.pem

agent5.pem: CN = agent5

error 20 at 0 depth lookup:unable to get local issuer certificate

The ‘agent6’ certificate, however, was considered valid:

openssl verify -CAfile ca_crt_root.pem -untrusted ca_crt_intermediate.pem agent6.pem

agent6.pem: OK

I’m thinking that the critical difference between the two signed certificates is that the format of the Issuer field on the certificate signed via certificate_status does not match the Subject of the Intermediate CA whereas the Issuer field on the certificate signed via puppet cert sign is identical to the Intermediate CA subject:

> openssl x509 -in ca_crt_intermediate.pem -noout -subject

subject= /C=US/ST=Oregon/O=Puppet/CN=intermediateca.example.org/emailAddress=intermediateca@example.org

> openssl x509 -in agent5.pem -noout -issuer

issuer= /emailAddress=intermediateca@example.org/CN=intermediateca.example.org/O=Puppet/ST=Oregon/C=US

> openssl x509 -in agent6.pem -noout -issuer

issuer= /C=US/ST=Oregon/O=Puppet/CN=intermediateca.example.org/emailAddress=intermediateca@example.org

I think we’d need to look into why - only for certificates issued from Puppet Server's CA - the contents of the Issuer field is generated in reverse order from the Intermediate CA subject.