[PUP-10102] Systemd provider is not correctly selected on Alpine Linux Created: 2019/10/15  Updated: 2020/01/08  Resolved: 2019/11/04

Status: Resolved
Project: Puppet
Component/s: None
Affects Version/s: PUP 6.8.0, PUP 6.9.0, PUP 6.8.1, PUP 6.10.0
Fix Version/s: PUP 5.5.18, PUP 6.4.5, PUP 6.11.0

Type: Bug Priority: Normal
Reporter: Tomasz Wolbach Assignee: Gheorghe Popescu
Resolution: Fixed Votes: 0
Labels: resolved-issue-added
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Template: PUP Bug Template
Team: Night's Watch
Story Points: 2
Sprint: NW - 2019-10-30, NW - 2019-11-13
Method Found: Needs Assessment
Release Notes: Bug Fix
Release Notes Summary: Fixed an edge-case issues that could occur when running puppet on distributions that do not have a default service provider. Puppet might try to evaluate runit and set defpath default before verifying that runit is suitable. Raising an error when setting defpath resulted in a state that should not be reached.
QA Risk Assessment: Needs Assessment

 Description   

Puppet Version: 6.8.0 - 6.10.0

Running unit tests (i.e pdk test unit) on Alpine Linux with puppet agents >= 6.8.0 fails:

 

pdk (INFO): Using Ruby 2.4.6
pdk (INFO): Using Puppet 6.8.0
Run options: exclude {:bolt=>true}
 Evaluated 83 tests in 30.530483644 seconds: 1 failures, 0 pending.
failed: rspec: ./spec/classes/common_spec.rb:9: Could not find the daemon directory (tested [/etc/sv,/var/lib/service])
 common should contain class: Ubuntu on ubuntu-18.04-x86_64 is expected to compile into a catalogue without dependency cycles
 Failure/Error:
 let(:facts) { os_facts }
it { is_expected.to compile.with_all_deps }
 end

 

This seems to be related to changes in:

https://tickets.puppetlabs.com/browse/PUP-7312
https://tickets.puppetlabs.com/browse/PUP-10016



 Comments   
Comment by Josh Cooper [ 2019/10/15 ]

Did this used to work and was broken by one of those changes?

Comment by Tomasz Wolbach [ 2019/10/16 ]

Yes, indeed.
I forgot to mention that testing is done in the docker and this is the main problem.
I made some tests with two docker images:

  • puppet/puppet-dev-tools:0.5 (centos 7)
  • alpine

and it fails on both with pdk test unit --puppet-version=6.8.0 (or higher). Tests with puppet-version=6.7.2 (od lower) are passing ok.
In docker images /proc/1/comm is set to "tail". 
 
https://github.com/puppetlabs/puppet/commit/c2ccdd4efe8ffc4a721ab47c88a834158a37251e
 

Comment by Mihai Buzgau [ 2019/10/16 ]

Hi Tomasz Wolbach,

Can you please provide the module that you are testing?
I was able to successfully run the tests on inifile module with:
PDK version: 1.14.0
Puppet Version: 6.10.1

On alipne docker image.

~/mod/puppetlabs-motd # pdk test unit
pdk (WARN): This module is compatible with an older version of PDK. Run `pdk update` to update it to your version of PDK.
pdk (INFO): Using Ruby 2.4.6
pdk (INFO): Using Puppet 6.10.1
[✔] Installing missing Gemfile dependencies.
[✔] Preparing to run the unit tests.
/usr/bin/ruby -I/root/.pdk/cache/ruby/2.4.0/gems/rspec-core-3.9.0/lib:/root/.pdk/cache/ruby/2.4.0/gems/rspec-support-3.9.0/lib /root/.pdk/cache/ruby/2.4.0/gems/rspec-core-3.9.0/exe/rspec --pattern spec/\{aliases,classes,defines,functions,hosts,integration,plans,tasks,type_aliases,types,unit\}/\*\*/\*_spec.rb
Run options: exclude {:bolt=>true}
 
motd
  On a non-linux system
    does not fail
    is expected not to contain File[/etc/motd]
    is expected not to contain File[/etc/issue]
    is expected not to contain File[/etc/issue.net]
  On Linux
    when neither template or source are specified
      is expected to contain File[/etc/motd] with ensure => "file", backup => "false", content  supplied string, owner => "root", group => "root" and mode => "0644"
    when both template and source are specified
      is expected to contain File[/etc/motd] with ensure => "file", backup => "false" and content  supplied string
    when a source is specified
      is expected to contain File[/etc/motd] with ensure => "file", backup => "false" and content  supplied string
    when an external template is specified
      is expected to contain File[/etc/motd] with ensure => "file", backup => "false" and content  supplied string
    when a template is specified for /etc/issue
      is expected to contain File[/etc/issue] with ensure => "file", backup => "false", content  supplied string, owner => "root", group => "root" and mode => "0644"
    when content is specified for /etc/issue
      is expected to contain File[/etc/issue] with ensure => "file", backup => "false" and content  supplied string
    when both content and template is specified for /etc/issue
      is expected to contain File[/etc/issue] with ensure => "file", backup => "false" and content  supplied string
    when a template is specified for /etc/issue.net
      is expected to contain File[/etc/issue.net] with ensure => "file", backup => "false", content  supplied string, owner => "root", group => "root" and mode => "0644"
    when content is specified for /etc/issue.net
      is expected to contain File[/etc/issue.net] with ensure => "file", backup => "false" and content  supplied string
    when both content and template is specified for /etc/issue.net
      is expected to contain File[/etc/issue.net] with ensure => "file", backup => "false" and content  supplied string
  On Debian based Operating Systems
    when dynamic motd is false
      is expected to contain File_line[dynamic_motd] with line => "session    optional     pam_motd.so  motd=/run/motd.dynamic noupdate"
    when dynamic motd is true
      is expected not to contain File_line[dynamic_motd]
  On Windows
    when neither template or source are specified
      is expected to contain Registry_value[HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\policies\system\legalnoticetext] with ensure => "present", type => "string" and data => "TestOS 5 x86_64\n\nFQDN:         test.example.com (123.23.243.1)\nProcessor:    intel awesome\nKernel:       windows\nMemory Size:  16.00 GB\n"
    when content is specified
      is expected to contain Registry_value[HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\policies\system\legalnoticecaption] with ensure => "present", type => "string" and data => "This is the title."
  On FreeBSD
    when neither template or source are specified
      is expected to contain File[/etc/motd] with ensure => "file", backup => "false" and content  supplied string
    when both template and source are specified
      is expected to contain File[/etc/motd] with ensure => "file", backup => "false" and content  supplied string
    when a source is specified
      is expected to contain File[/etc/motd] with ensure => "file", backup => "false" and content  supplied string
    when an external template is specified
      is expected to contain File[/etc/motd] with ensure => "file", backup => "false" and content  supplied string
  On AIX
    when neither template or source are specified
      is expected to contain File[/etc/motd] with ensure => "file", backup => "false", content  supplied string, owner => "bin", group => "bin" and mode => "0644"
    when a template is specified for /etc/issue
      is expected to contain File[/etc/issue] with ensure => "file", backup => "false", content  supplied string, owner => "bin", group => "bin" and mode => "0644"
    when a template is specified for /etc/issue.net
      is expected to contain File[/etc/issue.net] with ensure => "file", backup => "false", content  supplied string, owner => "bin", group => "bin" and mode => "0644"
 
Code coverage
  must cover at least 0% of resources
Total resources:   6
Touched resources: 6
Resource coverage: 100.00%
 
Finished in 1.13 seconds (files took 1.37 seconds to load)
26 examples, 0 failures

P.S. seems like this version of PDK has a bug. To get past it I've applied this patch: https://github.com/puppetlabs/pdk/pull/775/files

Comment by Mihai Buzgau [ 2019/10/16 ]

I've also tested pdk unit tests using the docker image we provide: puppet/puppet-dev-tools:0.5 (centos 7).
This image has:
PDK version 1.13.0
Puppet Version: 6.8.1

The tests for pupppetlabs-motd module passed:

[root@a59b05e8fca9 puppetlabs-motd-master]# pdk test unit
pdk (WARN): This module is compatible with an older version of PDK. Run `pdk update` to update it to your version of PDK.
pdk (INFO): Using Ruby 2.5.3
pdk (INFO): Using Puppet 6.8.1
[✔] Installing missing Gemfile dependencies.
[✔] Preparing to run the unit tests.
[✔] Running unit tests.
Run options: exclude {:bolt=>true}
  Evaluated 26 tests in 4.5599529 seconds: 0 failures, 0 pending.
 
Total resources:   6
Touched resources: 6
Resource coverage: 100.00%
[root@a59b05e8fca9 puppetlabs-motd-master]# pdk --version
1.13.0

Comment by Tomasz Wolbach [ 2019/10/17 ]

I've tested above module too and it is ok, so the problem must be somewhere in my module. Investigating..

Comment by Tomasz Wolbach [ 2019/10/17 ]

Mihai Buzgau: could you try to test module puppetlabs-ntp ? The problem occurs only when resource "service" is in manifest. On puppetlabs-ntp I have the same error

Comment by Mihai Buzgau [ 2019/10/17 ]

Tomasz Wolbach tests passed on puppetlabs-ntp master using:
Ruby 2.4.6
Puppet 6.10.1

Code coverage
  must cover at least 0% of resources
Total resources:   21
Touched resources: 17
Resource coverage: 80.95%
 
Untouched resources:
  File[/tmp/foo/ntp.keys]
  File[/var/log/foobar.log]
  Package[bos.net.tcp.client]
  Package[service/network/ntp]
 
Finished in 3 minutes 6 seconds (files took 1 minute 9.21 seconds to load)
2444 examples, 0 failures
 
~/mod/puppetlabs-ntp # pwd
/root/mod/puppetlabs-ntp

Comment by Mihai Buzgau [ 2019/10/17 ]

tests passed on with the v8.1.0 tag of puppetlabs-ntp

Comment by Tomasz Wolbach [ 2019/10/18 ]

I've created new module (pdk: 1.14.0- pdk new module, pdk new class...) with simple manifest, where I install package and start service and still I get:

Could not find the daemon directory (tested [/etc/sv,/var/lib/service])

when I add to service resource: provider => 'systemd' it's fine( also with puppet-agent < 6.8.0). Tested in docker (puppet/puppet-dev-tools:0.5).

cat /proc/1/comm shows: "tail". Don't know where the problem could be

Comment by Mihai Buzgau [ 2019/10/21 ]

Tomasz Wolbach in order to manage services with systemd you need to start your container with it as PID 1. You can do that by using the following docker image:

docker run -ti --privileged --volume /sys/fs/cgroup:/sys/fs/cgroup:ro waffleimage/ubuntu48.04

Please note that you need to enable privilidged mode for your docker container and you also need to mount cgroup to it.

Seems like you're running puppet/puppet-dev-tools:0.5 with tail as PID 1. This docker image has the systemd package installed on it but it's not managing any services, a simple systemctl status fails:

[root@e95ea3144091 repo]# systemctl status
Failed to get D-Bus connection: Operation not permitted

Also, I think your module unit tests should not be making system calls.

Comment by Mihai Buzgau [ 2019/10/25 ]

Tomasz Wolbach please re-open the ticket if the issue still persists.

Comment by Tomasz Wolbach [ 2019/10/25 ]

Mihai Buzgau: Thank you. I will inform you if I find something new. It is an OS related problem, I'm sure

Comment by Christoph Maser [ 2019/10/28 ]

Mihai Buzgau some tests run in docker containeres which don't have systemd installed, which don't have systemd as PID 1 and which don't need privileges.

 

Comment by Mihai Buzgau [ 2019/10/28 ]

Christoph Maser is the code open-source? can you point me to the tests that are failing?

Comment by Christoph Maser [ 2019/10/28 ]

Mihai Buzgau this basically happens with any unit test where you call a class containing a service. one example of our internal code failing:

class pocu_base::rsyslog {
  include rsyslog::client
...

# frozen_string_literal: true
 
require 'spec_helper'
 
describe 'pocu_base::rsyslog', type: 'class' do
  context 'default' do
    on_supported_os.each do |os, facts|
      context "on #{os}" do
        let(:facts) do
          facts
        end
 
        it { is_expected.to compile }
        it { is_expected.to contain_class('pocu_base::rsyslog') }
        it { is_expected.to contain_logrotate__rule('rsyslog-other') }
        it { is_expected.to contain_logrotate__rule('rsyslog') }
      end
    end
  end
end

This fails with:

ailures:
 
  1) pocu_base::rsyslog default on debian-8-x86_64 is expected to compile into a catalogue without dependency cycles
     Failure/Error: it { is_expected.to compile }
 
     RuntimeError:
       Could not find the daemon directory (tested [/etc/sv,/var/lib/service])
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/provider/service/runit.rb:50:in `defpath'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type/service.rb:184:in `block (3 levels) in <module:Puppet>'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type.rb:834:in `set_default'
     # ./vendor/bundle/ruby/2.4.0/gems/rspec-puppet-2.7.7/lib/rspec-puppet/monkey_patches.rb:87:in `call'
     # ./vendor/bundle/ruby/2.4.0/gems/rspec-puppet-2.7.7/lib/rspec-puppet/monkey_patches.rb:87:in `block in <class:Type>'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type.rb:2507:in `block in set_parameters'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type.rb:2506:in `each'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type.rb:2506:in `set_parameters'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type.rb:2395:in `initialize'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource.rb:478:in `new'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource.rb:478:in `to_ral'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource/catalog.rb:639:in `block in to_catalog'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource/catalog.rb:631:in `each'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource/catalog.rb:631:in `to_catalog'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource/catalog.rb:516:in `to_ral'
     # ./vendor/bundle/ruby/2.4.0/gems/rspec-puppet-2.7.7/lib/rspec-puppet/matchers/compile.rb:149:in `cycles_found?'
     # ./vendor/bundle/ruby/2.4.0/gems/rspec-puppet-2.7.7/lib/rspec-puppet/matchers/compile.rb:25:in `matches?'
     # ./spec/classes/rsyslog_spec.rb:13:in `block (5 levels) in <top (required)>'
 
  2) pocu_base::rsyslog default on debian-9-x86_64 is expected to compile into a catalogue without dependency cycles
     Failure/Error: it { is_expected.to compile }
 
     RuntimeError:
       Could not find the daemon directory (tested [/etc/sv,/var/lib/service])
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/provider/service/runit.rb:50:in `defpath'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type/service.rb:184:in `block (3 levels) in <module:Puppet>'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type.rb:834:in `set_default'
     # ./vendor/bundle/ruby/2.4.0/gems/rspec-puppet-2.7.7/lib/rspec-puppet/monkey_patches.rb:87:in `call'
     # ./vendor/bundle/ruby/2.4.0/gems/rspec-puppet-2.7.7/lib/rspec-puppet/monkey_patches.rb:87:in `block in <class:Type>'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type.rb:2507:in `block in set_parameters'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type.rb:2506:in `each'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type.rb:2506:in `set_parameters'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/type.rb:2395:in `initialize'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource.rb:478:in `new'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource.rb:478:in `to_ral'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource/catalog.rb:639:in `block in to_catalog'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource/catalog.rb:631:in `each'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource/catalog.rb:631:in `to_catalog'
     # ./vendor/bundle/ruby/2.4.0/gems/puppet-5.5.17/lib/puppet/resource/catalog.rb:516:in `to_ral'
     # ./vendor/bundle/ruby/2.4.0/gems/rspec-puppet-2.7.7/lib/rspec-puppet/matchers/compile.rb:149:in `cycles_found?'
     # ./vendor/bundle/ruby/2.4.0/gems/rspec-puppet-2.7.7/lib/rspec-puppet/matchers/compile.rb:25:in `matches?'
     # ./spec/classes/rsyslog_spec.rb:13:in `block (5 levels) in <top (required)>'
 
1 deprecation warning total
 
Finished in 2.65 seconds (files took 5.83 seconds to load)
8 examples, 2 failures

The failing service instance here is inherited from 'saz/rsyslog', '5.0.0'

Generated at Wed Jan 22 03:20:44 PST 2020 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.