Concurrent `loads()` calls silently corrupt each other's values

[livingstaccato]

Summary

serialize() declares context=SerializationContext() as a default argument.
Python evaluates a default once, at import, so every rule in the process shares
one mutable SerializationContext — and the expression, function and indexing
rules mutate it in place.

A thread serializing a function call therefore changes what other threads see
mid-serialization. A tuple comes back as the string '[1, 2, 3]' instead of a
list, an object as '{a = 1}' instead of a dict. No exception is raised; a
caller doing schema validation downstream reports it as a type error in the
user's configuration.

Reproduction

from concurrent.futures import ThreadPoolExecutor
import hcl2

PLAIN = "x = [1, 2, 3]\ny = {a = 1}\n"
TOGGLES = "z = f([1, 2, 3], {a = 1})\n"   # serializing this sets the shared flag
EXPECT = {"x": [1, 2, 3], "y": {"a": 1}}

def work(i):
    if i % 2:
        hcl2.loads(TOGGLES)
        return None
    return hcl2.loads(PLAIN)

with ThreadPoolExecutor(max_workers=8) as pool:
    results = [r for r in pool.map(work, range(800)) if r is not None]

bad = [r for r in results if r != EXPECT]
print(f"{len(bad)} of {len(results)} corrupted")
print("example:", bad[0] if bad else None)

Output on 8.1.3:

400 of 400 corrupted
example: {'x': '[1, 2, 3]', 'y': '{a = 1}'}

The threads have to overlap for it to appear, so it is workload-dependent:

interleaved parses	corrupted
100	0 of 50
200	11 of 100
400	58 of 200
800	400 of 400
2000	1000 of 1000

Cause

hcl2/rules/base.py:42 and the other 49 serialize() signatures:

def serialize(self, options=SerializationOptions(), context=SerializationContext()) -> Any:

SerializationContext is mutated in place through
with context.modify(inside_dollar_string=True) in rules/expressions.py
(lines 155, 269, 306), rules/functions.py:95 and rules/indexing.py (94,
151).

Nothing supplies a context at any public entry point — api.serialize calls
tree.serialize(), and query/_base.py:95 calls
self._node.serialize(options=options) — so both land on the shared object.
Eight call sites inside rules/base.py also drop the context they were given
and re-default to it; the rules below them already thread it correctly.

Suggested shape of the fix

Have the structural rules in base.py build a fresh context when called without
one, and thread the context they were given into every child call:

def serialize(self, options=SerializationOptions(), context=None) -> Any:
    context = context if context is not None else SerializationContext()
    ...
    return self.body.serialize(options, context)

That is enough to make the shared default unreachable in practice: with it, the
reproduction above reports 0 of 400. The remaining serialize() signatures
keep the mutable default, which is no longer reachable but is still a trap for a
future call site — worth removing separately, since it is 50 signatures.

options=SerializationOptions() has the same shape but is only ever read, so it
is benign unless a caller mutates the object it passes in.

Happy to open a PR. The branch I have includes a regression test that
interleaves 800 parses across 8 threads; the count is high because the effect
needs roughly 400 interleaved parses to show at all, and a smaller test passes
against the unfixed code.

---

---

The investigation behind this report was produced with AI assistance, working on my behalf. Every reproduction, version comparison and measurement cited was executed rather than inferred; I reviewed it before filing.

---

This issue, and the investigation behind it, were produced by an AI assistant (Claude) working on behalf of the author. Please review with that provenance in mind.

[livingstaccato]

Update: opened as #334, which also closes #344 — the shared default is gone, and SerializationContext is frozen so a caller-supplied context is safe to share as well.

_{🤖 Drafted with Claude Code.}