Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-3022

Puppet module install deletes existing files before downloading new files. If the files cannot be downloaded the data is lost

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Cannot Reproduce
    • Affects Version/s: None
    • Fix Version/s: PUP 3.7.0
    • Component/s: PMT
    • Labels:
      None
    • Template:

      Description

      During a recent outage of the forge, this behavior of the module tool became apparent. While I am using r10k, it was determined by FriedBob in IRC that this was a PMT issue.

      Example:

      Starting directory:
      [root@puppet puppet-tutorial]# ls /etc/puppet/environments/

      {production,motd}

      /modules
      /etc/puppet/environments/motd/modules:
      apache base firewall git make ntp portage ruby stdlib ygrpms
      augeas concat gcc inifile motd pe_gem r10k ssh vcsrepo

      /etc/puppet/environments/production/modules:
      apache base firewall git make pe_gem r10k ssh vcsrepo
      augeas concat gcc inifile ntp portage ruby stdlib ygrpms

      During the outage, I re-deployed with r10k and vcsrepo was unavailable (output trimmed):

      [root@puppet rnelson0-base]# r10k deploy environment -pv
      [R10K::Task::Deployment::DeployEnvironments - INFO] Loading environments from all sources
      [R10K::Task::Environment::Deploy - NOTICE] Deploying environment dave
      [R10K::Task::Module::Sync - INFO] Deploying vcsrepo into /etc/puppet/environments/dave/modules
      [R10K::TaskRunner - ERROR] Task #<R10K::Task::Module::Sync:0x7fe6e070ca00> failed while running: Command puppet module --modulepath /etc/puppet/environments/dave/modules install --version=0.2.0 --ignore-dependencies puppetlabs/vcsrepo exited with 1: Error: Could not execute operation for 'puppetlabs/vcsrepo'
      The server being queried was https://forge.puppetlabs.com
      The HTTP response we received was '502 Bad Gateway'
      Check the author and module names are correct.
      [R10K::Task::Environment::Deploy - NOTICE] Deploying environment motd
      [R10K::Task::Puppetfile::Sync - INFO] Loading modules from Puppetfile into queue
      [R10K::Task::Module::Sync - INFO] Deploying vcsrepo into /etc/puppet/environments/motd/modules
      ^C[R10K::TaskRunner - ERROR] Aborted!

      Environment after deployment:

      [root@puppet rnelson0-base]# ls /etc/puppet/environments/production/modules/
      apache base firewall git make ntp portage ruby stdlib
      augeas concat gcc inifile motd pe_gem r10k ssh ygrpms
      [root@puppet rnelson0-base]# ls /etc/puppet/environments/dave/modules/
      apache base firewall git make ntp portage ruby stdlib
      augeas concat gcc inifile motd pe_gem r10k ssh ygrpms

      This appears to be undesireable behavior to me. A normal "puppet module install --environment <env> <module>" will copy the module to a temporary location before moving it to a permanent home, but in this case the --modulepath option appears to be changing that behavior slightly. It could also be that the target location is always overwritten before the download completes, too, but I don't have the ability to simulate a forge outage to test

      IRC chat logs from #puppet (~3:58-4:30PM Eastern 3/28/2014):
      <rnelson0> There an issue with the forge? getting random drops or 502 errors.
      <+FriedBob> rnelson0: Yes, and it is being worked on
      <rnelson0> ok cool. my r10k doesn't like it is all
      <+FriedBob> rnelson0: I think the forge should be back up now
      <rnelson0> yep it has been stable for a few
      <rnelson0> interesting failure mode though, r10k does not verify it can download the modules before deleting the local copy, which basically broke my puppet setup.
      <+finch> rnelson0: if you're installing a module from the forge, then that's the behavior of the puppet module tool
      <+finch> r10k itself tries very hard to not destroy modules but that only works for git
      <+finch> or svn
      <rnelson0> finch: seems like it might be best to place it in a temp folder and then move it.
      <+FriedBob> rnelson0: The PMT itself doesn't do that. But I'm sure we'd accept a PR
      <rnelson0> or at least try and fail with environments other than 'production' first
      <+finch> rnelson0: in the medium term I'll have a native implementation of the pmt, and when that's done all modules will be cached first and then installed from there

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                rnelson0@gmail.com Rob Nelson
                QA Contact:
                Eric Thompson
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Zendesk Support