fix: parse heredocs with an empty body (#309)

[livingstaccato]

Fixes #309.

A heredoc with an empty body failed to parse. The v8 rewrite turned the optional newline after the opening marker into a mandatory one, so a marker immediately followed by its closing delimiter no longer matched HEREDOC_TEMPLATE.

The louder half of the bug

The parse error is the rare case. Having failed to match at that position, the lexer scans on and matches a later delimiter, silently absorbing everything in between:

import hcl2

hcl2.loads('''locals {
  a = <<EOF
EOF

  b = <<EOF
real body
EOF
}
''')

8.1.2 returns a single attribute whose value runs to the end of b, with b absent from the output. No exception. Only a file whose empty heredoc has no later delimiter to latch onto surfaces as UnexpectedToken.

The fix

Make the body and its trailing newline one lazy optional group:

HEREDOC_TEMPLATE : /<<(?P<heredoc>[a-zA-Z][a-zA-Z0-9._-]+)\n(?:(?:.|\n)*?\n)??\s*(?P=heredoc)\n/

Two simpler-looking patches are both wrong, which seems worth recording:

• Restoring the bare \n? that the rewrite dropped fixes the empty case but keeps the run-on, because the body can still stretch to a later delimiter.
• Dropping the \n before \s* fixes both of those, but then \s* can match nothing and a word merely ending in the delimiter terminates the body: <<EOF\nsayEOF\nEOF truncates at sayEOF.

The laziness is load-bearing for the same reason: a greedy ? prefers to match a body. The delimiter still has to begin its own line, so the rule from #194 is preserved.

Tests

test/helpers/terraform-config/ carried an empty-heredoc fixture, added with #117 when this was fixed the first time. The v8 test reorganisation replaced that layout, and no zero-line heredoc appears anywhere in the current suite — which is why nothing caught this.

The specialized heredoc fixture now covers empty, empty-trimmed and blank-line-only heredocs, plus an attribute after them that proves the lexer stops where it should. Without the grammar change, 6 of the new tests fail. nose2 --config tox.ini: 1400 tests, OK. ruff clean.

---

This pull request, and the investigation behind it, were produced by an AI assistant (Claude) working on behalf of the author. Every reproduction, test run and benchmark cited was executed rather than inferred, but please review with that provenance in mind.