Context
Part of the champion's "LOW-severity hole" — the finding that launched the whole B2 retraining study — is a generator artifact, not a capability gap. The malformed_date variant of FIELD_CORRUPTION writes a garbled date into statement.value_date, which is semantically indistinguishable from VALUE_DATE_MISMATCH unless the model has internalized the verifier's fixed priority order. Models get scored wrong for a defensible reading. B2 may have been partly chasing this artifact.
Evidence
forge/src/reconforge_forge/generator.py — FIELD_CORRUPTION variants; malformed_date targets statement.value_date. 12 of the 37 benchmark FIELD_CORRUPTION items are malformed_date (re-derive from the seed-777 pool by regenerating with the eval seed and counting variants).- Champion sends 11–12 FIELD_CORRUPTION items to VALUE_DATE_MISMATCH; DeepSeek sends 7 (confusion matrices in
docs/validation/bench-eval-full-740.json / bench-deepseek-full.json).
- Champion FIELD_CORRUPTION exact-type recall: 0.270 [0.133, 0.421] (n=37) — the worst class after DUPLICATE.
forge/src/reconforge_forge/verifier.py — fixed single-exception priority order (decision F1) is what makes the gold label FIELD_CORRUPTION rather than VALUE_DATE_MISMATCH.
Task
- Quantify precisely: for the seed-777 benchmark, list which FIELD_CORRUPTION items are
malformed_date-on-value_date, and recompute per-type recall and R_w under a "collision-tolerant" scoring where FIELD_CORRUPTION↔VALUE_DATE_MISMATCH confusions on those items count as caught-and-typed-correct. Report both scorings.
- Decide the fix and record it in DECISIONS.md:
- Generator fix (preferred):
malformed_date corrupts a field that has no dedicated exception class (e.g. currency string garbling, reference format), so the collision disappears from future pools; or
- Taxonomy fix: define the priority rule in the task prompt itself so the label is derivable from the input; or
- Scoring fix: permanent dual-label acceptance for the ambiguous variant.
- Re-examine the B2 conclusion in BENCHMARK.md §B2: how much of the FIELD_CORRUPTION "improvement target" was this artifact? Add a corrective note.
- Note the interaction with the severity weights: these items are LOW-weight, so the R_w headline moves little — state the number rather than asserting it.
Acceptance criteria
- Both scorings published; ambiguity resolved in generator/taxonomy/scoring with a DECISIONS entry.
- B2 section carries the corrective note.
Cost tier
T1-cheap — regeneration of the eval pool locally + re-scoring committed exports; no model runs.
Context
Part of the champion's "LOW-severity hole" — the finding that launched the whole B2 retraining study — is a generator artifact, not a capability gap. The
malformed_datevariant of FIELD_CORRUPTION writes a garbled date intostatement.value_date, which is semantically indistinguishable from VALUE_DATE_MISMATCH unless the model has internalized the verifier's fixed priority order. Models get scored wrong for a defensible reading. B2 may have been partly chasing this artifact.Evidence
forge/src/reconforge_forge/generator.py— FIELD_CORRUPTION variants;malformed_datetargetsstatement.value_date. 12 of the 37 benchmark FIELD_CORRUPTION items aremalformed_date(re-derive from the seed-777 pool by regenerating with the eval seed and counting variants).docs/validation/bench-eval-full-740.json/bench-deepseek-full.json).forge/src/reconforge_forge/verifier.py— fixed single-exception priority order (decision F1) is what makes the gold label FIELD_CORRUPTION rather than VALUE_DATE_MISMATCH.Task
malformed_date-on-value_date, and recompute per-type recall and R_w under a "collision-tolerant" scoring where FIELD_CORRUPTION↔VALUE_DATE_MISMATCH confusions on those items count as caught-and-typed-correct. Report both scorings.malformed_datecorrupts a field that has no dedicated exception class (e.g. currency string garbling, reference format), so the collision disappears from future pools; orAcceptance criteria
Cost tier
T1-cheap — regeneration of the eval pool locally + re-scoring committed exports; no model runs.