Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-10603

File Resource HTTP GET - Misuse of URL encoding

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • High
    • Resolution: Fixed
    • PUP 6.16.0, PUP 6.17.0
    • PUP 6.18.0
    • Modules
    • Ubuntu 16.04 (amd64)
    • Coremunity
    • Platform Core KANBAN
    • Needs Assessment
    • Bug Fix
    • Hide
      Puppet 6.16.0 introduced a regression when using http file sources and the URL request path or query contained special characters. This restores the behavior that existed previously. In order to specify a path or query that contains a space (for example), then it must be URL encoded in the manifest, e.g. source => 'http://example.com/path%20to%file?q=a%20b'
      Show
      Puppet 6.16.0 introduced a regression when using http file sources and the URL request path or query contained special characters. This restores the behavior that existed previously. In order to specify a path or query that contains a space (for example), then it must be URL encoded in the manifest, e.g. source => ' http://example.com/path%20to%file?q=a%20b'
    • Needs Assessment

    Description

      Puppet Version: 6.16.0
      Puppet Server Version: 6.
      OS Name/Version: Win10x64 WinServer2012R2

      After upgrading from agent 6.13.0 URL's that are pre-encoded are being re-encoded during the HTTP GET phase of file resource execution. The header HEAD phase is working as expected. This was not occurring in agent 6.13.0 but occurs in 6.16 and 6.15 versions of the agent

      When attempting to remove the encoding the HTTP HEAD uses the correct url unencoded, but when the GET goes to encode, it for some reason drops the trailing '=' (in fact no '=' are encoded at all) which is found in all Azure SAS tokens and storage keys.

      We store sensitive files in Azure private blob containers, and utilize a SAS token who's sensitive data is looked up via a hiera eyaml and appended to the https:// endpoint of the url as the file "source" parameter

      Desired Behavior:

      If already encoded the file resource should not re-encode during the GET phase

      If not encoded, the GET phase should encode the request but not drop the trailing '='.

      Actual Behavior:

      Dummy file resource being run in debug when specifying with the encoded section of the SAS token appended to the http request:

      Debug: Using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384
      		 
      Debug: HTTP HEAD https://anystorageaccount.blob.core.windows.net/puppet/test/test.txt?sv=2019-02-02&st=2020-07-28T20%3A18%3A53Z&se=2020-07-28T20%3A38%3A00Z&sr=b&sp=r&sig=xkd9E0xReTZ18zc1Woa1rQmhAwcaYYJRHZcac5yczQ8%3D returned 200 OK
      		 
      Debug: Caching connection for https://anystorageaccount.blob.core.windows.net:443
      		 
      Debug: Using cached connection for https://anystorageaccount.blob.core.windows.net:443
      		 
      Debug: HTTP GET https://anystorageaccount.blob.core.windows.net/puppet/test/test.txt?sv=2019-02-02&st=2020-07-28T20%253A18%253A53Z&se=2020-07-28T20%253A38%253A00Z&sr=b&sp=r&sig=xkd9E0xReTZ18zc1Woa1rQmhAwcaYYJRHZcac5yczQ8%253D returned 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
      		 
      Debug: Closing connection for https://anystorageaccount.blob.core.windows.net:443
      		 
      Error: Could not set 'file' on ensure: Error 403 on SERVER: ´╗┐<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
      		 
      RequestId:ce4f9ec1-501e-0114-7f1e-65a9cd000000
      		 
      Time:2020-07-28T20:34:34.9632157Z</Message><AuthenticationErrorDetail>Signature fields not well formed.

      same file resource, with the token portion of the string unencoded. The final '=' of the string is stripped off during the GET

      Debug: Using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384
      		 
      Debug: HTTP HEAD https://anystorageaccount.blob.core.windows.net/puppet/test/test.txt?sv=2019-02-02&st=2020-07-28T20:18:53Z&se=2020-07-28T21:03:00Z&sr=b&sp=r&sig=JaZhcqxT4akJcOwUdUGrQB2m1geUoh89iL8WMag8a8c= returned 200 OK
      		 
      Debug: Caching connection for https://anystorageaccount.blob.core.windows.net:443
      		 
      Debug: Using cached connection for https://anystorageaccount.blob.core.windows.net:443
      		 
      Debug: HTTP GET https://anystorageaccount.blob.core.windows.net/puppet/test/test.txt?sv=2019-02-02&st=2020-07-28T20%3A18%3A53Z&se=2020-07-28T21%3A03%3A00Z&sr=b&sp=r&sig=JaZhcqxT4akJcOwUdUGrQB2m1geUoh89iL8WMag8a8c returned 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
      		 
      Debug: Closing connection for https://anystorageaccount.blob.core.windows.net:443
      		 
      Error: Could not set 'file' on ensure: Error 403 on SERVER: ´╗┐<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
      		 
      RequestId:1a6c4010-101e-003e-7920-659add000000
      		 
      Time:2020-07-28T20:46:22.0326377Z</Message><AuthenticationErrorDetail>Signature fields not well formed.

      We noticed that puppet 6.15.0 introduced many changes to the file http resource according to the documentation.

       

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. PUP-10613 Puppet::Util.uri_encode discards '='

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Needs Information

          Task - A task that needs to be done. PUP-10492 Handle special characters in http(s) source URLs

          • Normal - Regular issue, some loss of functionality under specific circumstances.
          • Needs Information

          Activity

            People

              josh Josh Cooper
              chughesvf Christopher Hughes
              Votes:
              4 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Zendesk Support