Problem
The accuracy suite uses a hand-parsed YAML-like format that can silently accept incomplete
input. Its 18 cases contain expected diagnostics from only eight of the 38 catalog codes,
which is not enough evidence for per-rule release thresholds.
Expected behavior
Replace the hand parser with one strict versioned format supported by the standard library
or an existing dependency. Reject unknown and missing fields. Expand independently labeled
positive and negative cases with rule, source format, scientific domain, provenance, and
license or synthetic status.
Run every case through the public analysis path and keep expectations human-authored. Do
not add a canary differential or precision/recall gate until enough independently labeled
data and baseline variance exist. Align future corpus work with #198–#201.
Problem
The accuracy suite uses a hand-parsed YAML-like format that can silently accept incomplete
input. Its 18 cases contain expected diagnostics from only eight of the 38 catalog codes,
which is not enough evidence for per-rule release thresholds.
Expected behavior
Replace the hand parser with one strict versioned format supported by the standard library
or an existing dependency. Reject unknown and missing fields. Expand independently labeled
positive and negative cases with rule, source format, scientific domain, provenance, and
license or synthetic status.
Run every case through the public analysis path and keep expectations human-authored. Do
not add a canary differential or precision/recall gate until enough independently labeled
data and baseline variance exist. Align future corpus work with #198–#201.