Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-9719

Cannot run Puppet Agent as Administrator if first PA run is done as System

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: PUP 6.4.2
    • Fix Version/s: PUP 6.13.0
    • Component/s: None
    • Template:
      PUP Bug Template
    • Acceptance Criteria:
      Hide

      TBD

      Show
      TBD
    • Team:
      Night's Watch
    • Story Points:
      3
    • Sprint:
      PR - 2019-06-25, PR - 2019-07-10, PR - 2019-07-23, NW - 2019-08-07, NW - 2019-08-21, NW - 2019-09-03, NW - 2019-09-18, NW - 2019-10-02, NW - 2019-10-16, NW - 2019-10-30, NW - 2019-11-13, 2019-11-27, 2019-12-11, 2019-12-24, NW - 2020-01-22, NW - 2020-02-05
    • Method Found:
      Customer Feedback
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Hide
      If the first Puppet Agent run is done under SYSTEM account, following runs done by Administrator users were failing to send the report to master due to insufficient file permissions caused by the implementation of `Puppet::Util.replace_file`.
      Puppet now uses the new `Puppet::FileSystem.replace_file` method which correctly handles files permissions on Windows.
      Show
      If the first Puppet Agent run is done under SYSTEM account, following runs done by Administrator users were failing to send the report to master due to insufficient file permissions caused by the implementation of `Puppet::Util.replace_file`. Puppet now uses the new `Puppet::FileSystem.replace_file` method which correctly handles files permissions on Windows.
    • QA Risk Assessment:
      Needs Assessment

      Description

      Puppet Version: 6.4.2
      Puppet Server Version: N/A
      OS Name/Version: Windows Server 2016 x64

      When installing Puppet Agent (6.4.2/PE 2019.1.0) on Server 2016 using a Powershell script running as the SYSTEM account, Administrator users can't run Puppet. Daemon/service runs are performed as expected. Direct Puppet runs appear to occur but no report is sent to the master.
      UPDATE - It the first Run as SYSTEM that is the problem - so have changed ticket title.

      PUP-8939 had reportedly solved this issue.

      Desired Behavior:

      note - so far, this can only be replicated using scheduled tasks as psexec doesn't provide the same permissions environment :

      1. Install: schtasks /create /tn PuppetInstall /RL HIGHEST /RU SYSTEM /F /SC ONCE /ST 11:15 /TR 'cmd /c c:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe -sta -WindowStyle Hidden -ExecutionPolicy Bypass -NonInteractive -NoProfile -File c:\puppet.ps1 >> C:\windows\temp\puppet-ins.log 2>&1'
      2. Run PA schtasks /create /tn PuppetInstall /RL HIGHEST /RU SYSTEM /F /SC ONCE /ST 11:17 /TR 'cmd /c c:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe -sta -WindowStyle Hidden -ExecutionPolicy Bypass -NonInteractive -NoProfile -File c:\puppetrun.ps1 >> C:\windows\temp\puppet-run.log 2>&1'
      3. Populate csr_attributes.yml and server in the agent config as necessary.
      4. As an Administrator, run puppet agent --test on the newly installed agent.

       See https://gist.github.com/jcoconnor/79767ef986a3ec600de0cd84ec4ea600 for example puppet-ins.ps1 and puppet-run.ps1 scripts.

      Actual Behavior:

      Catalog retrieval fails; log has been redacted:

      2019-05-21 17:20:36 -0400 Puppet (debug): HTTP POST https://compiler.example.net:8140/puppet/v3/catalog/examplenode.example.net returned 200 OK
      2019-05-21 17:20:36 -0400 Puppet (debug): Caching connection for https://compiler.example.net:8140
      2019-05-21 17:20:36 -0400 Puppet (info): Caching catalog for examplenode.example.net
      2019-05-21 17:20:38 -0400 Puppet (err): ReplaceFile(C:/ProgramData/PuppetLabs/puppet/cache/client_data/catalog/examplenode.example.net.json, C:/ProgramData/PuppetLabs/puppet/cache/client_data/catalog/examplenode.example.net.json20190521-6580-blokpv):  Access is denied. 
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util/windows/file.rb:89:in `replace_file'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util.rb:636:in `replace_file'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/indirector/json.rb:17:in `save'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:200:in `find'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:466:in `block in retrieve_new_catalog'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util.rb:518:in `block in thinmark'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util.rb:517:in `thinmark'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:465:in `retrieve_new_catalog'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:75:in `retrieve_catalog'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:167:in `prepare_and_retrieve_catalog'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:342:in `run_internal'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:240:in `block in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet.rb:264:in `override'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:217:in `run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:59:in `block (5 levels) in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/2.5.0/timeout.rb:93:in `block in timeout'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/2.5.0/timeout.rb:103:in `timeout'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:58:in `block (4 levels) in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent/locker.rb:21:in `lock'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:52:in `block (3 levels) in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:130:in `with_client'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:49:in `block (2 levels) in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:87:in `run_in_fork'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:48:in `block in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/application.rb:179:in `controlled_run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:46:in `run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:371:in `onetime'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:353:in `run_command'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/application.rb:382:in `block in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util.rb:671:in `exit_on_fail'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/application.rb:382:in `run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:139:in `run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:77:in `execute'
      c:/Program Files/Puppet Labs/Puppet/puppet/bin/puppet:4:in `<main>'
      2019-05-21 17:20:38 -0400 Puppet (err): Could not retrieve catalog from remote server: ReplaceFile(C:/ProgramData/PuppetLabs/puppet/cache/client_data/catalog/examplenode.example.net.json, C:/ProgramData/PuppetLabs/puppet/cache/client_data/catalog/examplenode.example.net.json20190521-6580-blokpv):  Access is denied. 
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util/windows/file.rb:89:in `replace_file'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util.rb:636:in `replace_file'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/indirector/json.rb:17:in `save'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:200:in `find'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:466:in `block in retrieve_new_catalog'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util.rb:518:in `block in thinmark'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util.rb:517:in `thinmark'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:465:in `retrieve_new_catalog'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:75:in `retrieve_catalog'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:167:in `prepare_and_retrieve_catalog'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:342:in `run_internal'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:240:in `block in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet.rb:264:in `override'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:217:in `run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:59:in `block (5 levels) in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/2.5.0/timeout.rb:93:in `block in timeout'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/2.5.0/timeout.rb:103:in `timeout'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:58:in `block (4 levels) in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent/locker.rb:21:in `lock'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:52:in `block (3 levels) in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:130:in `with_client'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:49:in `block (2 levels) in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:87:in `run_in_fork'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:48:in `block in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/application.rb:179:in `controlled_run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:46:in `run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:371:in `onetime'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:353:in `run_command'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/application.rb:382:in `block in run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util.rb:671:in `exit_on_fail'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/application.rb:382:in `run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:139:in `run'
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:77:in `execute'
      c:/Program Files/Puppet Labs/Puppet/puppet/bin/puppet:4:in `<main>'
      

      Reporting also fails:

      2019-05-21 17:20:38 -0400 Puppet (debug): Using cached connection for https://compiler.example.net:8140
      2019-05-21 17:20:39 -0400 Puppet (debug): HTTP PUT https://compiler.example.net:8140/puppet/v3/report/examplenode.example.net returned 200 OK
      2019-05-21 17:20:39 -0400 Puppet (debug): Caching connection for https://compiler.example.net:8140
      2019-05-21 17:20:39 -0400 Puppet (err): Could not send report: ReplaceFile(C:/ProgramData/PuppetLabs/puppet/cache/state/last_run_report.yaml, C:/ProgramData/PuppetLabs/puppet/cache/state/last_run_report.yaml20190521-6580-1yfrb1r):  Access is denied. 
      c:/Program Files/Puppet Labs/Puppet/puppet/lib/ruby/vendor_ruby/puppet/util/windows/file.rb:89:in `replace_file'
      ...
      

      ACLs from C:\ProgramData\PuppetLabs\puppet\cache\ on the agent:

      PS C:\Windows\system32> Get-Acl C:\ProgramData\PuppetLabs\puppet\cache\ | Select *
       
      PSPath : Microsoft.PowerShell.Core\FileSystem::C:\ProgramData\PuppetLabs\puppet\cache\
      PSParentPath : Microsoft.PowerShell.Core\FileSystem::C:\ProgramData\PuppetLabs\puppet
      PSChildName : cache
      PSDrive : C
      PSProvider : Microsoft.PowerShell.Core\FileSystem
      CentralAccessPolicyId :
      CentralAccessPolicyName :
      Path : Microsoft.PowerShell.Core\FileSystem::C:\ProgramData\PuppetLabs\puppet\cache\
      Owner : NT AUTHORITY\SYSTEM
      Group : NT AUTHORITY\SYSTEM
      Access : {System.Security.AccessControl.FileSystemAccessRule,
      System.Security.AccessControl.FileSystemAccessRule,
      System.Security.AccessControl.FileSystemAccessRule}
      Sddl : O:SYG:SYD:AI(A;OICIID;FA;;;SY)(A;OICIID;FA;;;BA)(A;OICIID;FA;;;S-1-5-21-1092630797-2642148860
      -572411492-162715)
      AccessToString : NT AUTHORITY\SYSTEM Allow FullControl
      BUILTIN\Administrators Allow FullControl
      DEVAONNET\ah14740 Allow FullControl
      AuditToString :
      AccessRightType : System.Security.AccessControl.FileSystemRights
      AccessRuleType : System.Security.AccessControl.FileSystemAccessRule
      AuditRuleType : System.Security.AccessControl.FileSystemAuditRule
      AreAccessRulesProtected : False
      AreAuditRulesProtected : False
      AreAccessRulesCanonical : True
      AreAuditRulesCanonical : True
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              gheorghe.popescu Gheorghe Popescu
              Reporter:
              garrett.guillotte Garrett Guillotte
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support