Uploaded image for project: 'Puppet Agent'
  1. Puppet Agent
  2. PA-101

AIO's OpenSSL cannot make SSL connection to apt.dockerproject.org

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: puppet-agent 1.3.0
    • Fix Version/s: puppet-agent 1.3.2
    • Component/s: None
    • Labels:
      None
    • Environment:
      • Ubuntu Trusty (14.04)
      • Puppet Agent v1.3.0 (1.3.0-1trusty)
      • puppetlabs/apt v2.2.0
    • Template:

      Description

      When trying to connect to HTTPS protected sites from the agent, e.g. when trying to download keys for apt, the agent's ruby refuses the connection since it cannot verify the server's certificate:

      apt::source {'docker':
        location => 'https://apt.dockerproject.org/repo',
        release => 'ubuntu-trusty',
        repos => 'main',
        key => '58118E89F3A912897C070ADBF76221572C52609D',
        key_source => 'https://apt.dockerproject.org/gpg'
      }
      

      leads to

      Notice: Compiled catalog for ip-172-31-52-7.ec2.internal in environment production in 0.35 seconds
      Error: Could not set 'present' on ensure: SSL_connect returned=1 errno=0 state=error: certificate verify failed at 77:/etc/puppetlabs/code/environments/production/modules/apt/manifests/key.pp
      Error: Could not set 'present' on ensure: SSL_connect returned=1 errno=0 state=error: certificate verify failed at 77:/etc/puppetlabs/code/environments/production/modules/apt/manifests/key.pp
      Wrapped exception:
      SSL_connect returned=1 errno=0 state=error: certificate verify failed
      Error: /Stage[main]/Main/Apt::Source[docker]/Apt::Key[Add key: 58118E89F3A912897C070ADBF76221572C52609D from Apt::Source docker]/Apt_key[Add key: 58118E89F3A912897C070ADBF76221572C52609D from Apt::Source docker]/ensure: change from absent to present failed: Could not set 'present' on ensure: SSL_connect returned=1 errno=0 state=error: certificate verify failed at 77:/etc/puppetlabs/code/environments/production/modules/apt/manifests/key.pp
      Notice: /Stage[main]/Main/Apt::Source[docker]/Apt::Key[Add key: 58118E89F3A912897C070ADBF76221572C52609D from Apt::Source docker]/Anchor[apt_key 58118E89F3A912897C070ADBF76221572C52609D present]: Dependency Apt_key[Add key: 58118E89F3A912897C070ADBF76221572C52609D from Apt::Source docker] has failures: true
      

      Internally apt::source uses apt_key, which in turn uses URI.read from open-uri. Use ruby -r open-uri -e 'puts URI::parse("https://apt.dockerproject.org/gpg").read' to reproduce this easily.

      It works with Ubuntu's default ruby:

      [...]
      [pid 21847] stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=198, ...}) = 0
      [pid 21847] open("/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = 5
      [pid 21847] open("/dev/urandom", O_RDONLY|O_NOCTTY|O_NONBLOCK) = 6
      [pid 21847] stat("/usr/lib/ssl/certs/5ad8a5d6.0", {st_mode=S_IFREG|0644, st_size=1261, ...}) = 0
      [pid 21847] open("/usr/lib/ssl/certs/5ad8a5d6.0", O_RDONLY) = 6
      [pid 21847] stat("/usr/lib/ssl/certs/5ad8a5d6.1", 0x7fff15448680) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/etc/localtime", O_RDONLY|O_CLOEXEC) = 6
      [pid 21847] open("/usr/local/lib/site_ruby/1.9.1/enc/trans/single_byte.rb", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/local/lib/site_ruby/1.9.1/x86_64-linux/enc/trans/single_byte.rb", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/local/lib/site_ruby/enc/trans/single_byte.rb", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/lib/ruby/vendor_ruby/1.9.1/enc/trans/single_byte.rb", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/lib/ruby/vendor_ruby/1.9.1/x86_64-linux/enc/trans/single_byte.rb", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/lib/ruby/vendor_ruby/enc/trans/single_byte.rb", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/lib/ruby/1.9.1/enc/trans/single_byte.rb", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/lib/ruby/1.9.1/x86_64-linux/enc/trans/single_byte.rb", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/local/lib/site_ruby/1.9.1/enc/trans/single_byte.so", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/local/lib/site_ruby/1.9.1/x86_64-linux/enc/trans/single_byte.so", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/local/lib/site_ruby/enc/trans/single_byte.so", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/lib/ruby/vendor_ruby/1.9.1/enc/trans/single_byte.so", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/lib/ruby/vendor_ruby/1.9.1/x86_64-linux/enc/trans/single_byte.so", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/lib/ruby/vendor_ruby/enc/trans/single_byte.so", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/lib/ruby/1.9.1/enc/trans/single_byte.so", O_RDONLY) = -1 ENOENT (No such file or directory)
      [pid 21847] open("/usr/lib/ruby/1.9.1/x86_64-linux/enc/trans/single_byte.so", O_RDONLY) = 6
      [pid 21847] open("/usr/lib/ruby/1.9.1/x86_64-linux/enc/trans/single_byte.so", O_RDONLY|O_CLOEXEC) = 6
      [pid 21847] open("/proc/self/maps", O_RDONLY|O_CLOEXEC) = 5
      -----BEGIN PGP PUBLIC KEY BLOCK-----
       
      mQINBFWln24BEADrBl5p99uKh8+rpvqJ48u4eTtjeXAWbslJotmC/CakbNSqOb9o
      ddfzRvGVeJVERt/Q/mlvEqgnyTQy+e6oEYN2Y2kqXceUhXagThnqCoxcEJ3+KM4R
      mYdoe/BJ/J/6rHOjq7Omk24z2qB3RU1uAv57iY5VGw5p45uZB4C4pNNsBJXoCvPn
      [...]
      

      but does not with the AIO agent's ruby:

      [pid 21854] stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=198, ...}) = 0
      [pid 21854] open("/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = 7
      [pid 21854] open("/dev/urandom", O_RDONLY|O_NOCTTY|O_NONBLOCK) = 8
      [pid 21854] stat("/opt/puppetlabs/puppet/ssl/certs/5ad8a5d6.0", 0x7fff9b0f7f80) = -1 ENOENT (No such file or directory)
      [pid 21854] stat("/opt/puppetlabs/puppet/ssl/certs/b85455c4.0", 0x7fff9b0f7f80) = -1 ENOENT (No such file or directory)
      /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:923:in `connect': SSL_connect returned=1 errno=0 state=error: certificate verify failed (OpenSSL::SSL::SSLError)
      	from /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:923:in `block in connect'
      	from /opt/puppetlabs/puppet/lib/ruby/2.1.0/timeout.rb:75:in `timeout'
      	from /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:923:in `connect'
      	from /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:863:in `do_start'
      	from /opt/puppetlabs/puppet/lib/ruby/2.1.0/net/http.rb:852:in `start'
      

      You can clearly see where the agent's ruby fails to load the ca bundle required to verify the certificate from /opt/puppetlabs/puppet/ssl/certs/, while the native ruby finds it in /usr/lib/ssl/certs/

      On my test system there are no certs in puppet-agent's certs dir:

      # find /opt/puppetlabs/puppet/ssl/certs/
      /opt/puppetlabs/puppet/ssl/certs/
      #
      

      This potentially impacts all modules and types/providers that use ruby's SSL to access external sites.

        Attachments

          Issue Links

            Activity

              jsd-sla-details-panel

                People

                • Assignee:
                  bradejr Rob Braden
                  Reporter:
                  david.schmitt David Schmitt
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  7 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: