Uploaded image for project: 'Puppet'
  1. Puppet
  2. PUP-8561

Augeas unable to handle escaped quotes within quotes

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: PUP 5.5.1
    • Component/s: None
    • Labels:
      None
    • Release Notes:
      Bug Fix
    • Release Notes Summary:
      Hide
      The Augeas provider did not unescape quotes in quoted arguments for {{set}} and similar commands. Putting the command {{set thing "\"my stuff\""}} would lead to {{thing}} having the value {{\"my stuff\"}}, including the backslashes. Those backslashes are now properly removed so that the above command leads to {{thing}} having the value {{"my stuff"}}.
      Show
      The Augeas provider did not unescape quotes in quoted arguments for {{set}} and similar commands. Putting the command {{set thing "\"my stuff\""}} would lead to {{thing}} having the value {{\"my stuff\"}}, including the backslashes. Those backslashes are now properly removed so that the above command leads to {{thing}} having the value {{"my stuff"}}.
    • QA Risk Assessment:
      Needs Assessment

      Description

      Puppet Version: 5.4.0
      Puppet Server Version: 5.2.0
      OS Name/Version: CentOS 7.4.1708

      I am unable to find a way to set the "log_format" variable in nginx.conf using augeas. The problem seems to be the combination of quotes within quotes just doesn't seem to be possible.

      The desired value I am trying to set exactly is:

      timed_combined  '$remote_addr - $remote_user $http_x_unit_id [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time $upstream_response_time $pipe'

      Take careful note that I need to have the long single quoted string, but that this contains some double quoted strings. I have tried every possible combination I can think of for enclosing in single or double quoted strings in and for escaping then, but I have not successfully managed to set exactly what is required. I have tried so many different possible ways over the last 2 days that I am now convinced I haven't missed something and that this has to be a bug.

      To describe some of the more obvious steps I have tried, lets store the initial variables like as following:

      #The log title:
      $log_title = 'timed_combined'
       
      #Set-up the needed log_format string:
      $log_format_str = '$remote_addr - $remote_user $http_x_unit_id [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time $upstream_response_time $pipe'
      

      The initial most logical combination of options that I have tried are (elaborated here to try and make it clear):

      Attempt 1
      Escape the double quotes within the substring, then pass the variable in augeas with escaped quotes:

      # Create the combined string
      $use_log_format = regsubst("${log_title} '${log_format_str}'", '"', '\\"', 'G')
       
      augeas { 'set-up_log_format':
          context => "/files${conf_file}/http",
          changes => [
              "set log_format \"${use_log_format}\""
          ]
      }
      

      Produces:

      -    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
      -                      '$status $body_bytes_sent "$http_referer" '
      -                      '"$http_user_agent" "$http_x_forwarded_for"';
      +    log_format  timed_combined  '$remote_addr - $remote_user $http_x_unit_id [$time_local] \"$request\" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" $request_time $upstream_response_time $pipe';
      

      Result: Produces almost exactly what I want, but unfortunately the escapes have been passed through and retained. The result is confirmed when grep'ing the file:

      root@devsvr2 ~]# grep log_format /etc/nginx/nginx.conf
          log_format  timed_combined  '$remote_addr - $remote_user $http_x_unit_id [$time_local] \"$request\" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" $request_time $upstream_response_time $pipe';
      

      Maybe I do not need the escapes? Let's try without...

      Attempt 2
      Don't escape the double quotes within the substring, but still pass the variable in augeas with escaped quotes:

      # Create the combined string
      $use_log_format = "${log_title} '${log_format_str}'"
       
      augeas { 'set-up_log_format':
          context => "/files${conf_file}/http",
          changes => [
              "set log_format \"${use_log_format}\""
          ]
      }
      

      Produces:

      -    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
      -                      '$status $body_bytes_sent "$http_referer" '
      -                      '"$http_user_agent" "$http_x_forwarded_for"';
      +    log_format  timed_combined  '$remote_addr - $remote_user $http_x_unit_id [$time_local] ;
      

      Result: As you might expect, the unescaped quotes cause the value to be truncated completely. How about we wrap it in single quotes in the augeas command instead? Let's try it...

      Attempt 3
      Single quotes in the augeas command instead of the escaped doubles:

      # Create the combined string
      $use_log_format = "${log_title} '${log_format_str}'"
       
      augeas { 'set-up_log_format':
          context => "/files${conf_file}/http",
          changes => [
              "set log_format '${use_log_format}'"
          ]
      }
      

      Produces:

      -    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
      -                      '$status $body_bytes_sent "$http_referer" '
      -                      '"$http_user_agent" "$http_x_forwarded_for"';
      +    log_format  timed_combined  ;
      

      Result: Now we've lost the value altogether. Maybe I'd better escape those single quotes in the string too...

      Attempt 4
      Single quotes in the augeas command, but add escapes to the single quotes that define the substring in the variable

      # Create the combined string
      $use_log_format = "${log_title} \\'${log_format_str}\\'"
       
      augeas { 'set-up_log_format':
          context => "/files${conf_file}/http",
          changes => [
              "set log_format '${use_log_format}'"
          ]
      }
      

      Produces:

      -    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
      -                      '$status $body_bytes_sent "$http_referer" '
      -                      '"$http_user_agent" "$http_x_forwarded_for"';
      +    log_format  timed_combined  \'$remote_addr - $remote_user $http_x_unit_id [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time $upstream_response_time $pipe\';
      

      Result: Oh wonderful! Now we have a the reverse of attempt 1 - the escaped single quotes have now been passed through to the final output. Again, confirming this is real and not just in the displayed output:

      [root@devsvr2 ~]# grep log_format /etc/nginx/nginx.conf
          log_format  timed_combined  \'$remote_addr - $remote_user $http_x_unit_id [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time $upstream_response_time $pipe\';
      

      As you can see, this is how I began. I tried adding various extra levels of escape and switching back and forth between single and double quotes, but nothing worked.

      I then eventually tried to apply the same set-up manually using augtool, and it didn't take long to get a working result:
      Attempt 1:
      Set the line value, in double quotes, with no escapes:

      augtool> set /files/etc/nginx/nginx.conf/http/log_format "timed_combined  '$remote_addr - $remote_user $http_x_unit_id [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time $upstream_response_time $pipe'"
      

      Produces:

      augtool> print /files/etc/nginx/nginx.conf/http/log_format
      /files/etc/nginx/nginx.conf/http/log_format = "timed_combined  '$remote_addr - $remote_user $http_x_unit_id [$time_local] $request $status $body_bytes_sent $http_referer $http_user_agent $request_time $upstream_response_time $pipe'"
      

      Result: Missing the double quotes in the output. Okay, I'll try escaping them:

      Attempt 2:
      Set the line value, in double quotes, and escape the double quotes in the substring:

      augtool> set /files/etc/nginx/nginx.conf/http/log_format "timed_combined  '$remote_addr - $remote_user $http_x_unit_id [$time_local] \"$request\" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" $request_time $upstream_response_time $pipe'"
      

      Produces:

      augtool> print /files/etc/nginx/nginx.conf/http/log_format
      /files/etc/nginx/nginx.conf/http/log_format = "timed_combined  '$remote_addr - $remote_user $http_x_unit_id [$time_local] \"$request\" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" $request_time $upstream_response_time $pipe'"
      

      Result: Escapes are shown in the output value above, but when 'save' is run and we check the file on disk, we can see that it is actually correct:

      [root@devsvr2 ~]# grep log_format /etc/nginx/nginx.conf
          log_format  timed_combined  '$remote_addr - $remote_user $http_x_unit_id [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time $upstream_response_time $pipe';
      

      So, I guess the ultimate question is: Is this a bug, or is this the correct expected behaviour? If it is the latter, then this feature is becomes pretty unusable.

      Before posting, I did a fair amount of searching for someone else with the exact same issue, and though I found plenty of comments/tickets/posts from people with similar issues, none were the exact same issue as mine; but I apologise now if the above for the slightest moment wastes someone's time.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              skrussell Russell Knighton
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Zendesk Support