Uploaded image for project: 'Modules'
  1. Modules
  2. MODULES-3429

Modulesync - document tarball format implications / module pipelines

    XMLWordPrintable

Details

    • Hide

      The rake task should include new code to verify the tarball index

      Show
      The rake task should include new code to verify the tarball index
    • Modules
    • 2

    Description

      There are new release to forge style pipelines that produce tarball artifacts within Jenkins and upload them - https://jenkins-modules.puppetlabs.com/view/release%20to%20forge/.

      The tar tool used to build a forge release has implications when it comes time to install the module (discussed in MODULES-3135). Depending on the format, the header may get written differently within the file, which may trigger Puppet bug PUP-5994 during extraction by minitar on a Windows platform (which doesn't extract full paths).

      This is of particular interest to the DSC module, which uses long file paths, and has had an emergency release in a different tar format to workaround this problem for old Puppet agents that don't include the PUP-5994 fix.

      In local testing, it appears that an archive produced on OSX with bsdtar 2.8.8 - libarchive 2.8.3, is a format that minitar will not properly extract. However, on the Jenkins node ci-slave01-prod.delivery.puppetlabs.net which is Debian 7.8/wheezy amd64, the tarball produced by tar (GNU tar) 1.26 for the same DSC module, will extract correctly by minitar, even without the PUP-5994 changes.

      This seemed initially like an important note that could be added to the README.md for modulesync, but it seems like this could also be programatically enforced as part of the rake build task used to produce the tarball – the task could be modified to load the archive with the minitar code and verify that the full file names are present in the index, as accessed by entry.name rather than entry.full_name

      Without some sort of automated verification on build (which would benefit the CI workflow and local user builds), there is no guarantee that we don't actually ship a bad tar archive in the future.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ethan Ethan Brown
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Zendesk Support