Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-10603

File Resource HTTP GET - Misuse of URL encoding

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: PUP 6.16.0, PUP 6.17.0
    • Fix Version/s: PUP 6.18.0
    • Component/s: Modules
    • Labels:
    • Template:
      PUP Bug Template
    • Agent OS:
      Windows 10 (x86, x64)
    • Master OS:
      Ubuntu 16.04 (amd64)
    • Team:
      Coremunity
    • Sprint:
      Platform Core KANBAN
    • Method Found:
      Needs Assessment
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Hide
      Puppet 6.16.0 introduced a regression when using http file sources and the URL request path or query contained special characters. This restores the behavior that existed previously. In order to specify a path or query that contains a space (for example), then it must be URL encoded in the manifest, e.g. source => 'http://example.com/path%20to%file?q=a%20b'
      Show
      Puppet 6.16.0 introduced a regression when using http file sources and the URL request path or query contained special characters. This restores the behavior that existed previously. In order to specify a path or query that contains a space (for example), then it must be URL encoded in the manifest, e.g. source => ' http://example.com/path%20to%file?q=a%20b'
    • QA Risk Assessment:
      Needs Assessment

      Description

      Puppet Version: 6.16.0
      Puppet Server Version: 6.
      OS Name/Version: Win10x64 WinServer2012R2

      After upgrading from agent 6.13.0 URL's that are pre-encoded are being re-encoded during the HTTP GET phase of file resource execution. The header HEAD phase is working as expected. This was not occurring in agent 6.13.0 but occurs in 6.16 and 6.15 versions of the agent

      When attempting to remove the encoding the HTTP HEAD uses the correct url unencoded, but when the GET goes to encode, it for some reason drops the trailing '=' (in fact no '=' are encoded at all) which is found in all Azure SAS tokens and storage keys.

      We store sensitive files in Azure private blob containers, and utilize a SAS token who's sensitive data is looked up via a hiera eyaml and appended to the https:// endpoint of the url as the file "source" parameter

      Desired Behavior:

      If already encoded the file resource should not re-encode during the GET phase

      If not encoded, the GET phase should encode the request but not drop the trailing '='.

      Actual Behavior:

      Dummy file resource being run in debug when specifying with the encoded section of the SAS token appended to the http request:

      Debug: Using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384
      Debug: HTTP HEAD https://anystorageaccount.blob.core.windows.net/puppet/test/test.txt?sv=2019-02-02&st=2020-07-28T20%3A18%3A53Z&se=2020-07-28T20%3A38%3A00Z&sr=b&sp=r&sig=xkd9E0xReTZ18zc1Woa1rQmhAwcaYYJRHZcac5yczQ8%3D returned 200 OK
      Debug: Caching connection for https://anystorageaccount.blob.core.windows.net:443
      Debug: Using cached connection for https://anystorageaccount.blob.core.windows.net:443
      Debug: HTTP GET https://anystorageaccount.blob.core.windows.net/puppet/test/test.txt?sv=2019-02-02&st=2020-07-28T20%253A18%253A53Z&se=2020-07-28T20%253A38%253A00Z&sr=b&sp=r&sig=xkd9E0xReTZ18zc1Woa1rQmhAwcaYYJRHZcac5yczQ8%253D returned 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
      Debug: Closing connection for https://anystorageaccount.blob.core.windows.net:443
      Error: Could not set 'file' on ensure: Error 403 on SERVER: ´╗┐<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
      RequestId:ce4f9ec1-501e-0114-7f1e-65a9cd000000
      Time:2020-07-28T20:34:34.9632157Z</Message><AuthenticationErrorDetail>Signature fields not well formed.
      

      same file resource, with the token portion of the string unencoded. The final '=' of the string is stripped off during the GET

      Debug: Using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384
      Debug: HTTP HEAD https://anystorageaccount.blob.core.windows.net/puppet/test/test.txt?sv=2019-02-02&st=2020-07-28T20:18:53Z&se=2020-07-28T21:03:00Z&sr=b&sp=r&sig=JaZhcqxT4akJcOwUdUGrQB2m1geUoh89iL8WMag8a8c= returned 200 OK
      Debug: Caching connection for https://anystorageaccount.blob.core.windows.net:443
      Debug: Using cached connection for https://anystorageaccount.blob.core.windows.net:443
      Debug: HTTP GET https://anystorageaccount.blob.core.windows.net/puppet/test/test.txt?sv=2019-02-02&st=2020-07-28T20%3A18%3A53Z&se=2020-07-28T21%3A03%3A00Z&sr=b&sp=r&sig=JaZhcqxT4akJcOwUdUGrQB2m1geUoh89iL8WMag8a8c returned 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
      Debug: Closing connection for https://anystorageaccount.blob.core.windows.net:443
      Error: Could not set 'file' on ensure: Error 403 on SERVER: ´╗┐<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
      RequestId:1a6c4010-101e-003e-7920-659add000000
      Time:2020-07-28T20:46:22.0326377Z</Message><AuthenticationErrorDetail>Signature fields not well formed.
      

      We noticed that puppet 6.15.0 introduced many changes to the file http resource according to the documentation.

       

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              josh Josh Cooper
              Reporter:
              chughesvf Christopher Hughes
              Votes:
              4 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Zendesk Support