[PUP-9163] Lexer incorrectly identifying list starts inside heredocs Created: 2018/09/24  Updated: 2018/11/07  Resolved: 2018/09/26

Status: Closed
Project: Puppet
Component/s: None
Affects Version/s: PUP 4.10.0, PUP 6.0.0
Fix Version/s: PUP 5.5.7, PUP 6.0.1

Type: Bug Priority: Normal
Reporter: Alex Harvey Assignee: Unassigned
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
relates to PUP-9306 Regression in heredoc parser in 5.5.8 Closed
Template: PUP Bug Template
Team: Coremunity
Sprint: Platform Core KANBAN
Release Notes: Bug Fix
Release Notes Summary: When using interpolation inside a heredoc the position and location information for the interpolated expressions were wrong. This could lead to two problems;
* if using {{[]}} expressions a mysterious Syntax Error would be raised if an
seemingly arbitrary position after the interpolation contained white-space.
* If there were errors in the interpolation they could end up being reported for the wrong line and position on the line.

This is now fixed.
QA Risk Assessment: Needs Assessment


Square brackets that are:

  • Inside heredocs
  • Inside an interpolation

Are sometimes being incorrectly lexed as TOKEN_LISTSTART.

Simplest example

Create test.pp:

$xxxxxxx = @("END")

Puppet apply it.

What I expect

Some text to appear showing the OS family.

What I actually see

Error: Could not parse for environment production: Syntax error at '[' (file: /Users/alexharvey/git/home/puppet-test/test.pp, line: 2, column: 14)

More info

Note the space inside XXX XX. That space is matching in the lexer on this line here, leading to the '[' being lexed as TOKEN_LISTSTART.

Comment by Henrik Lindberg [ 2018/09/24 ]

I found a fix, have test to write. Expect PR later today.

Comment by Jacob Helwig [ 2018/09/24 ]

Merged to 5.5.x in 452dded9f0.

Comment by Josh Cooper [ 2018/09/26 ]

Passed CI in e2f7ccfb7

Generated at Sun May 31 20:14:51 PDT 2020 using Jira 8.5.2#805002-sha1:a66f9354b9e12ac788984e5d84669c903a370049.