Uploaded image for project: 'Puppet Server'
  1. Puppet Server
  2. SERVER-2558

Attempting to start a compile master fails with CRL error

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: SERVER 6.15.3, SERVER 7.1.0
    • Component/s: None
    • Labels:
      None
    • Template:
      PUP Bug Template
    • Team:
      Froyo
    • Story Points:
      2
    • Sprint:
      Froyo - 02/24/2021
    • Method Found:
      Needs Assessment
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      The Jetty webserver will now use the local copy of the CRL, from Puppet's SSL dir, rather than the CA's copy. This makes it easier to set up compilers, which have a disabled CA service and therefore no CRL at the CA path.
    • QA Risk Assessment:
      Needs Assessment

      Description

      Puppet Version: 6.4.2
      Puppet Server Version: 6.3.0
      OS Name/Version: Debian 9 (stretch)

      I'm attempting to set up a Puppet compile master as described in the docs, using DNS SRV records for service discovery:

      _x-puppet._tcp.dev-chrisb.zcode.net. 142 IN SRV	0 50 8140 puppet.dev-chrisb.zcode.net.
      _x-puppet-ca._tcp.dev-chrisb.zcode.net.	300 IN SRV 0 100 8140 puppet.dev-chrisb.zcode.net.
      

      I have successfully managed to configure a primary puppet server which is providing the CA services, and I have deployed a separate PuppetDB server for all the masters to connect to.

      However when I then try to set up the secondary compile masters without the CA service, I get an error about ssl-crl-path being inaccessible:

      Error: Systemd start for puppetserver failed!
      journalctl log for puppetserver:
      -- Logs begin at Thu 2019-06-06 19:10:37 BST, end at Fri 2019-06-07 11:37:24 BST. --
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]: Exception in thread "main" java.lang.IllegalArgumentException: Non-readable path specified for ssl-crl-path option: /etc/puppetlabs/puppet/ssl/ca/ca_crl.pem
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at clojure.lang.Reflector.invokeConstructor(Reflector.java:305)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_config$fn__27083$get_ssl_crl_path_BANG___27088$fn__27089.invoke(jetty9_config.clj:326)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_config$fn__27083$get_ssl_crl_path_BANG___27088.invoke(jetty9_config.clj:319)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_config$fn__27257$maybe_get_https_connector__27262$fn__27263.invoke(jetty9_config.clj:399)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_config$fn__27257$maybe_get_https_connector__27262.invoke(jetty9_config.clj:386)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_config$fn__27308$maybe_add_https_connector__27313$fn__27314.invoke(jetty9_config.clj:415)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_config$fn__27308$maybe_add_https_connector__27313.invoke(jetty9_config.clj:411)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_config$fn__27334$process_config__27339$fn__27340.invoke(jetty9_config.clj:436)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_config$fn__27334$process_config__27339.invoke(jetty9_config.clj:431)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_core$fn__28437$create_webserver__28442$fn__28443.invoke(jetty9_core.clj:655)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_core$fn__28437$create_webserver__28442.invoke(jetty9_core.clj:613)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_core$fn__28466$start_webserver_BANG___28471$fn__28472.invoke(jetty9_core.clj:693)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_core$fn__28466$start_webserver_BANG___28471.invoke(jetty9_core.clj:688)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_core$start_server_single_default.invokeStatic(jetty9_core.clj:940)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_core$start_server_single_default.invoke(jetty9_core.clj:937)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_core$fn__28919$start_BANG___28924$fn__28925.invoke(jetty9_core.clj:1037)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_core$fn__28919$start_BANG___28924.invoke(jetty9_core.clj:1032)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services.webserver.jetty9_service$reify__29314$service_fnk__4991__auto___positional$reify__29321.start(jetty9_service.clj:53)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services$fn__4828$G__4809__4831.invoke(services.clj:9)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.services$fn__4828$G__4808__4835.invoke(services.clj:9)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.internal$fn__13553$run_lifecycle_fn_BANG___13560$fn__13561.invoke(internal.clj:198)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.internal$fn__13553$run_lifecycle_fn_BANG___13560.invoke(internal.clj:181)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.internal$fn__13582$run_lifecycle_fns__13587$fn__13588.invoke(internal.clj:231)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.internal$fn__13582$run_lifecycle_fns__13587.invoke(internal.clj:208)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.internal$fn__14155$build_app_STAR___14164$fn$reify__14176.start(internal.clj:586)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.internal$fn__14203$boot_services_for_app_STAR__STAR___14210$fn__14211$fn__14213.invoke(internal.clj:612)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.internal$fn__14203$boot_services_for_app_STAR__STAR___14210$fn__14211.invoke(internal.clj:610)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.internal$fn__14203$boot_services_for_app_STAR__STAR___14210.invoke(internal.clj:604)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at clojure.core$partial$fn__5826.invoke(core.clj:2630)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.internal$fn__13627$initialize_lifecycle_worker__13638$fn__13639$fn__13789$state_machine__10791__auto____13814$fn__13817.invoke(internal.clj:251)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at puppetlabs.trapperkeeper.internal$fn__13627$initialize_lifecycle_worker__13638$fn__13639$fn__13789$state_machine__10791__auto____13814.invoke(internal.clj:251)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:973)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:972)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:977)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:975)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at clojure.core.async$ioc_alts_BANG_$fn__11006.invoke(async.clj:384)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at clojure.core.async$do_alts$fn__10946$fn__10949.invoke(async.clj:253)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at clojure.core.async.impl.channels.ManyToManyChannel$fn__6246$fn__6247.invoke(channels.clj:95)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at clojure.lang.AFn.run(AFn.java:22)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      Jun 07 11:32:49 puppet2.dev-chrisb.zcode.net puppetserver[22791]:         at java.lang.Thread.run(Thread.java:748)
      Jun 07 11:32:50 puppet2.dev-chrisb.zcode.net systemd[1]: puppetserver.service: Control process exited, code=exited status=1
      Jun 07 11:32:50 puppet2.dev-chrisb.zcode.net systemd[1]: Failed to start puppetserver Service.
      Jun 07 11:32:50 puppet2.dev-chrisb.zcode.net systemd[1]: puppetserver.service: Unit entered failed state.
      

      I have disabled the CA service on this host as described here

      Note that ssl-crl-path is not defined anywhere in the Puppet server's configuration, so it looks like this is a default value? It looks like the default doesn't cope with the CA service being disabled and/or there's something vital missing from the documentation on how to set up compile masters (or I somehow missed it).

      Edited to add:

      I was able to get the server to work by specifying the paths to the SSL files downloaded by puppet-agent in /etc/puppetlabs/puppetserver/conf.d/webserver.conf

      webserver: {
          access-log-config: /etc/puppetlabs/puppetserver/request-logging.xml
          client-auth: want
          ssl-host: 0.0.0.0
          ssl-port: 8140
          ssl-cert: /etc/puppetlabs/puppet/ssl/certs/puppet2.dev-chrisb.zcode.net.pem
          ssl-ca-cert: /etc/puppetlabs/puppet/ssl/certs/ca.pem
          ssl-key: /etc/puppetlabs/puppet/ssl/private_keys/puppet2.dev-chrisb.zcode.net.pem
          ssl-crl-path: /etc/puppetlabs/puppet/ssl/crl.pem
      }
      

      If this is required when setting up compile masters it should probably be mentioned in the docs, although ideally the defaults would just work…

        Attachments

          Activity

            People

            Assignee:
            maggie Maggie Dreyer
            Reporter:
            chrisbzc Chris Butler
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Zendesk Support