Skip to content

fix: accept HCL keywords as block labels and object keys (#148) - #356

Closed
jmkacz wants to merge 1 commit into
mainfrom
bugifx/support-in-blocks
Closed

fix: accept HCL keywords as block labels and object keys (#148)#356
jmkacz wants to merge 1 commit into
mainfrom
bugifx/support-in-blocks

Conversation

@jmkacz

@jmkacz jmkacz commented Sep 4, 2026

Copy link
Copy Markdown

Problem

HCL does not reserve its keywords, so in, for, true, etc. are legal identifiers. Two positions in the grammar rejected them.

1. Block labels

block : identifier (identifier | string)* ... where identifier : NAME, but in lexes as the dedicated IN terminal and never as NAME. In a body position the contextual lexer therefore only accepted IN as the start of an attribute, so this documented Snowflake provider data source failed to parse:

data "snowflake_schemas" "in" {
  in {
    database = "database"
  }
}
lark.exceptions.UnexpectedToken: Unexpected token Token('IN', 'in')
Expected one of:
	* EQ
Previous tokens: [Token('IN', 'in')]

2. Object keys — a regression of #148

object_elem_key : expression cannot reach keyword either. #148 was fixed once by #164, then lost when 8.x moved keywords out of identifier into their own keyword rule and wired it only into _attribute_name.

#164's regression fixture covered a block-body attribute, which is a different grammar path (_attribute_name) from the object element (object_elem_key) that the issue actually reports. So the fixture kept passing while the reported case broke.

The object path then worked only by accident: Lark's contextual lexer falls back to NAME only in states that do not accept the keyword terminal, which made a key's position in the object decide whether the file parsed.

{ in = "h", name = "n" }    # parsed
{ name = "n", in = "h" }    # failed  <- the shape in #148
{ for = 1 }                 # failed in every position

That second line is why the jsonencode OpenAPI body from #148 still raised on main — verified against a clean checkout, independent of this branch.

Fix

File Change
hcl2/hcl2.lark block labels route through a new _block_label : identifier | keyword | literal_value, mirroring _attribute_name
hcl2/hcl2.lark object_elem_key : expression | keyword
hcl2/transformer.py block() and object_elem_key() normalize KeywordRule / LiteralValueRule to IdentifierRule

LALR builds with no new conflicts: after reducing keyword, EQ still means attribute and NAME/"/{ means block.

The normalization is load-bearing beyond tidiness. _label_to_str (hcl2/query/blocks.py:14) falls through to str(label.serialize()), and LiteralValueRule serializes true to Python True — so a true-named block would otherwise query as "True".

Testing

unittest: 1534 passing (from 1518), on a freshly built grammar cache. ruff check, ruff format, and mypy pass.

Unittest/unit/test_api.py:

Both classes carry a guard that in still works as the for-expression keyword. Separately swept 14 for-expression and template-directive forms by hand — no regressions.

Integration — two new round-trip suites, resource_keyword_block and object_keyword_keys, each with all four golden files. Direct reconstruction (HCL → IR → Lark → HCL) is byte-identical to source for both.

Notes for the reviewer

  • hq 'data.snowflake_schemas.in.in.database' resolves correctly against the new fixture.
  • One case is covered in unit tests rather than a golden-file suite: a for-expression used as a value reconstructs with a stray space ([for i in x : i][ for i in x : i]). That reproduces on clean main and is unrelated to keywords, so I left it alone rather than bake it into a golden file. Happy to fix it separately if you'd like.

🤖 Generated with Claude Code

HCL does not reserve its keywords, so `in`, `for`, `true`, etc. are legal
identifiers. Two positions rejected them.

Block labels. `block : identifier ...` where `identifier : NAME`, but `in`
lexes as the dedicated `IN` terminal and never as `NAME`. In a body position
the contextual lexer therefore only accepted `IN` as the start of an
*attribute*, so the Snowflake provider's nested `in` block failed with
"Expected one of: * EQ" right after `Token('IN', 'in')`:

    data "snowflake_schemas" "in" {
      in {
        database = "database"
      }
    }

Object keys. `object_elem_key : expression` cannot reach `keyword` either,
which regressed #148 (fixed once by #164, then lost when 8.x moved keywords
out of `identifier` into their own rule and wired it only into
`_attribute_name`). #164's regression test covered a block-body attribute,
a different grammar path from the object element the issue actually reports.
The object path then worked only by accident: Lark's contextual lexer falls
back to `NAME` only in states that do not accept the keyword terminal, so a
key's *position* in the object decided whether the file parsed.

    { in = "h", name = "n" }    # parsed
    { name = "n", in = "h" }    # failed  <- the shape in #148
    { for = 1 }                 # failed in every position

Both now route through new `_block_label` / `keyword` alternatives, and the
transformer normalizes the resulting KeywordRule and LiteralValueRule nodes
to IdentifierRule, mirroring what `attribute()` already did. That
normalization is load-bearing beyond tidiness: `_label_to_str` in
hcl2/query/blocks.py falls through to `str(label.serialize())`, and
LiteralValueRule serializes `true` to Python `True`, so a `true`-named block
would otherwise query as "True".

Fixes #148

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jmkacz
jmkacz requested a review from a team as a code owner September 4, 2026 17:57
@jmkacz jmkacz closed this Sep 4, 2026
@jmkacz
jmkacz deleted the bugifx/support-in-blocks branch September 4, 2026 17:58
@jmkacz

jmkacz commented Sep 4, 2026

Copy link
Copy Markdown
Author

Closed automatically by a branch rename (bugifxbugfix, typo fix). Superseded by #357 — same commit, no content changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant