Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-7814

HTTPS file sources with non-puppet-trusted certs can't be used

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: PUP 5.5.14, PUP 6.5.0
    • Fix Version/s: PUP 6.17.0
    • Component/s: None
    • Labels:
    • Template:
      PUP Bug Template
    • Acceptance Criteria:
      Hide

      It should be possible to retrieve file metadata and content from the following http(s) sources:

      1. From the puppetserver (already works)
      2. From a server whose CA cert is contained in the puppet-agent CA bundle (already works)
      3. From a server whose CA cert is not in the puppet-agent CA bundle, but is in a file system location, eg /etc/pki/tls/cert.pem, and puppet is configured to use that file (new).

      To support this:

      1. Add a new puppet setting of type :file
      2. Default to nil
      3. Update Puppet::SSL::SSLProvider#create_system_context to call OpenSSL::X509::Store#add_file if the setting is not-nil.
      4. Update spec/integration/http/client_spec to ensure it makes an HTTPS connection to a test server when the CA cert is not referenced by SSL_CERT_FILE, but is referenced by the new setting.

      Show
      It should be possible to retrieve file metadata and content from the following http(s) sources: 1. From the puppetserver (already works) 2. From a server whose CA cert is contained in the puppet-agent CA bundle (already works) 3. From a server whose CA cert is not in the puppet-agent CA bundle, but is in a file system location, eg /etc/pki/tls/cert.pem, and puppet is configured to use that file (new). To support this: 1. Add a new puppet setting of type :file 2. Default to nil 3. Update Puppet::SSL::SSLProvider#create_system_context to call OpenSSL::X509::Store#add_file if the setting is not-nil. 4. Update spec/integration/http/client_spec to ensure it makes an HTTPS connection to a test server when the CA cert is not referenced by SSL_CERT_FILE , but is referenced by the new setting.
    • Team:
      Coremunity
    • Sprint:
      Platform Core KANBAN
    • Method Found:
      Needs Assessment
    • CS Priority:
      Reviewed
    • Zendesk Ticket IDs:
      32650,35349
    • Zendesk Ticket Count:
      2
    • Release Notes:
      New Feature
    • Release Notes Summary:
      Hide
      Puppet agents can be configured to trust additional CA certificates when connecting to HTTPS servers whose issuing CA certificate is neither the puppet CA nor in the CA cert bundle in the puppet-agent package. This can be accomplished by setting "ssl_trust_store" to a file containing CA cert(s) in PEM format concatenated together. For example, "/etc/pki/tls/cert.pem". Note the setting only applies when puppet makes an HTTPS request to non-puppet REST endpoints, such as when applying a file resource with an "https://" source parameter or running "puppet module install" to install modules from the forge. Puppet REST requests, such as retrieving a catalog, will only trust the puppet CA regardless of the ssl_trust_store setting. Previously the only way to trust additional CA certificates was to manually add the certificate to puppet-agent's CA bundle, however, those changes were lost as soon as the puppet-agent package was next updated.
      Show
      Puppet agents can be configured to trust additional CA certificates when connecting to HTTPS servers whose issuing CA certificate is neither the puppet CA nor in the CA cert bundle in the puppet-agent package. This can be accomplished by setting "ssl_trust_store" to a file containing CA cert(s) in PEM format concatenated together. For example, "/etc/pki/tls/cert.pem". Note the setting only applies when puppet makes an HTTPS request to non-puppet REST endpoints, such as when applying a file resource with an "https://" source parameter or running "puppet module install" to install modules from the forge. Puppet REST requests, such as retrieving a catalog, will only trust the puppet CA regardless of the ssl_trust_store setting. Previously the only way to trust additional CA certificates was to manually add the certificate to puppet-agent's CA bundle, however, those changes were lost as soon as the puppet-agent package was next updated.
    • QA Risk Assessment:
      Needs Assessment

      Description

      In PUP-1072 we added support for http and https URL schemes for file resources' source attributes:

      file { '/tmp/myfile':
        source => 'https://myserver/myfile'
      }
      

      UPDATED

      1. The above works provided the CA certificate that issued the server's certificate is in the CA bundle distributed in puppet-agent packages. There are workarounds to add certificates to puppet's cert bundle, but those workarounds are lost when the puppet-agent package is updated.

      2. As of Puppet 6.15.0 puppet will trust the puppet CA in addition to puppet's CA bundle when retrieving file metadata and content via https file sources. So the issue Charlie Sharpsteen described with source => "https://${server_facts['servername']}:8140//puppet-ca/v1/certificate/ca?environment=production" will be resolved.

      3. This ticket is about a) creating a puppet setting referring to a directory or file containing system CA certs, and b) modifying the Puppet::SSL::SSLProvider#create_system_context method so that the system CA certs are added to the X509 store via OpenSSL::X509::Store#add_path (for a directory) or OpenSSL::X509::Store#add_file (for a file).

      Original

      Somehow we failed to realize that the implementation of this feature requires that the remote webserver present a certificate that is trusted by the puppet agent's SSL configuration. In practice this is extremely unlikely and difficult to configure, and therefore the utility of this feature is pretty limited.

      In the code, this is because the request ultimately uses the same routines (in Puppet::Util::HttpProxy.get_http_object) that the agent itself uses, where the SSL verification mode is hardcoded but the certificate store is not.

      There should be some way to permit retrieval of these files. A couple of options that occured to me as I was tracing this down:

      1. make the ssl verification mode configurable or disabled by default for file-source retrieval (equivalent to curl -k)
      2. use the system-wide trust store instead of puppet's store
      3. expose the certificate trust and verification mode as parameters on the file type (this is truly awful)

      ... surely someone else has better ideas ...

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              josh Josh Cooper
              Reporter:
              eric.sorenson Eric Sorenson
              Votes:
              5 Vote for this issue
              Watchers:
              19 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support