[PUP-4462] Single backslash before $ blocks interpolation in heredoc with no escapes enabled Created: 2015/04/24  Updated: 2015/05/20  Resolved: 2015/05/20

Status: Closed
Project: Puppet
Component/s: Docs, Language
Affects Version/s: PUP 3.7.5, PUP 4.0.0
Fix Version/s: PUP 3.8.1, PUP 4.1.0

Type: Bug Priority: Normal
Reporter: Nicholas Fagerlund Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Template:
Epic Link: 4.x Language
Story Points: 1
Sprint: Language 2015-04-29, Language 2015-05-13
Release Notes: Bug Fix
QA Contact: Eric Thompson

 Description   

Ran:

$thing = "hi there"
notice(@("END INTERP"))
This string should be able to interpolate variables, and all backslashes should
be interpreted as plain old literal backslashes.
Variable standing apart: ${thing}
Variable enjambed: hey${thing}
Variable with single backslash enjambed: \${thing}
Variable with double backslash enjambed: \\${thing}
END INTERP

The single backslash before the $ causes a problem. I'm pretty sure this is against the specification of heredocs.

Actual result:

Notice: Scope(Class[main]): This string should be able to interpolate variables, and all backslashes should
be interpreted as plain old literal backslashes.
Variable standing apart: hi there
Variable enjambed: heyhi there
Variable with single backslash enjambed: \${thing}
Variable with double backslash enjambed: \\hi there

Expected result:

Notice: Scope(Class[main]): This string should be able to interpolate variables, and all backslashes should
be interpreted as plain old literal backslashes.
Variable standing apart: hi there
Variable enjambed: heyhi there
Variable with single backslash enjambed: \hi there
Variable with double backslash enjambed: \\hi there

3.7.5 and 4.0.0 behave identically.

In addition, if any escape (other than '$' escapes) are turned on, it also turns on escaping of '$'. This is caused by the same faulty logic.

QA


risk: medium (validate only)
probability: low (single backslash directly before interpolation in pp heredoc)
severity: medium (obvious issue, workarounds)
test level: unit



 Comments   
Comment by Henrik Lindberg [ 2015/04/24 ]

Nicholas Fagerlund Yes, you found a bug. The specification says that @("END") means that interpolation is on, but not escapes. When turning on any escape, a literal backslash must always be escaped. You correctly described the expected result as per the specification.

It seems that the code assumes that turning on interpolation also turns on the $ escape, and subsequently that backslash must be escaped. That is against the spirit of heredoc (to be as close to verbatim text as possible).

Comment by Thomas Hallgren [ 2015/04/27 ]

Merged to 3.x f5974e2.

Comment by Thomas Hallgren [ 2015/04/27 ]

Merge to stable at 0f52fc9 with subsequent maintenance commit at 730d01e.

Comment by Thomas Hallgren [ 2015/04/27 ]

Merged to master at e034275.

Comment by Nicholas Fagerlund [ 2015/04/28 ]

Docs impact: for affected versions, remove the "known issue" note in the future parser page about heredocs.

Comment by Eric Thompson [ 2015/04/29 ]

Henrik Lindberg it looks like the spec tests for the PR for this is testing the escaped backslash case (two
) but not the single escaping of the interpolation (since with @("BLAH") escapes are turned off)?

Comment by Eric Thompson [ 2015/04/29 ]

validated on ubuntu14.04 at master SHA: 8964241

[root@fv1nemlp2jj6fpm puppet]# puppet apply interp_heredoc.pp
Notice: Scope(Class[main]): Variable standing apart: hi there
Variable enjambed: heyhi there
Variable with single backslash enjambed: \hi there
Variable with double backslash enjambed: \\hi there
 
Notice: Compiled catalog for fv1nemlp2jj6fpm.delivery.puppetlabs.net in environment production in 0.40 seconds
Notice: Applied catalog in 0.02 seconds
[root@fv1nemlp2jj6fpm puppet]# cat interp_heredoc.pp
$thing = "hi there"
notice(@("END INTERP"))
Variable standing apart: ${thing}
Variable enjambed: hey${thing}
Variable with single backslash enjambed: \${thing}
Variable with double backslash enjambed: \\${thing}
END INTERP

Comment by Henrik Lindberg [ 2015/04/29 ]

Reading the spec tests (and writing them) is tricky due to the multiple layers of string mangling, a double backslash may be needed to get once backslash char in the result. It took me several rounds of verification in irb before I got them right - too easy to believe testing one thing when actually testing another. I think I got it right.

It may lack combinations - was that what you meant?

The problem that we faced only had to do with interpolation - the interpolation logic needs to also handle finding the end of a string sequence - the added tests are specific to that.

Comment by Kurt Wall [ 2015/04/29 ]

Validated in stable at SHA=17b75e2104c4e5615af559ed2d231dc7047f9dc3 on Windows 2012r2:

C:\Program Files\Puppet Labs\Puppet\bin>puppet apply x.pp --verbose --parser fut
ure
Notice: Scope(Class[main]): Variable standing apart: hi there
Variable enjambed: heyhi there
Variable with single backslash enjambed: \hi there
Variable with double backslash enjambed: \\hi there
 
Notice: Compiled catalog for irzmc77aybd4u9s.delivery.puppetlabs.net in environm
ent production in 0.56 seconds
Info: Applying configuration version '1430344943'
Info: Creating state file C:/ProgramData/PuppetLabs/puppet/var/state/state.yaml
Notice: Finished catalog run in 0.02 seconds
C:\Program Files\Puppet Labs\Puppet\bin>

Comment by Eric Thompson [ 2015/04/29 ]

Henrik Lindberg i had hoped that was the case, that you needed to escape the ... escaping backslash in the heredoc. but it's matching on double backslash in the output, i think. but maybe that part is escaped as well?

the ticket was mostly just about the single backslash case.
eg:

$thing = "hi there"
notice(@("END INTERP"))
Variable with single backslash enjambed: \${thing}
END INTERP

so if you are saying that's covered by:

 it "parses interpolated heredoc expression containing escapes" do
    src = <<-CODE
    @("END")
    Hello \\$name
    |- END
    CODE
    dump(parse(src)).should == [
      "(@()",
      "  (sublocated (cat 'Hello \\' (str $name) ''))",
      ")"
    ].join("\n")
  end

then i'm good with the tests as is.

Comment by Henrik Lindberg [ 2015/04/29 ]

Yes, that is covered by those examples.

Comment by Eric Thompson [ 2015/04/29 ]

needs more validation against 3.x, and windows

Comment by Eric Thompson [ 2015/05/01 ]

validated on ubuntu14 at 3.x SHA: 17b75e2

[root@mupkgq3reracnf3 puppet]# puppet apply interp_heredoc.pp --parser future
Notice: Scope(Class[main]): Variable standing apart: hi there
Variable enjambed: heyhi there
Variable with single backslash enjambed: \hi there
Variable with double backslash enjambed: \\hi there

Comment by Eric Thompson [ 2015/05/01 ]

calling this done enough

Generated at Sat Dec 14 03:04:07 PST 2019 using JIRA 7.7.1#77002-sha1:e75ca93d5574d9409c0630b81c894d9065296414.