Skip to content

feat(check): fold downstream compile errors into the breaking site they come from (UPT-99) - #222

Merged
lucasbraum merged 4 commits into
lucasbraum/upt-95-ground-truth-generatorsfrom
lucasbraum/upt-99-fold-cascade-diagnostics
Oct 18, 2026
Merged

lucasbraum merged 4 commits into
lucasbraum/upt-95-ground-truth-generatorsfrom
lucasbraum/upt-99-fold-cascade-diagnostics

Conversation

@lucasbraum

Copy link
Copy Markdown
Collaborator

Closes UPT-99. Stacked on #218 (base: lucasbraum/upt-95-ground-truth-generators).

Rule first. The folding rule went into docs/concepts.md ("One root cause, one finding") and docs/packs.md (Scoring) in 2cb4eef, before any scoring run. The engine change is 7f66699.

What changes

A new compile error whose value the compiler traces to another new error's expression is that site's consequence. It folds into the breaking finding there as a downstream error. It no longer counts as a breaking site of its own.

  • Tracing is compiler-only (flowOrigin, adapters/typescript/cause.ts). It starts at the expression at the error and follows the value back through:

    • variable and destructured declarations;
    • a callback whose parameter is typed only by an any receiver (errors.map(e => …));
    • members of returned object literals;
    • casts to a type whose import is itself a new error (as SafeParseReturnType<…>);
    • calls of repository functions.

    A call is followed only when every return whose type changed between the installed program and the target leads to such an expression. Types are compared by an order-free shape (members, unions and intersections sorted), because the two programs print the same Prisma payload types in a different member order. There is no line-distance fallback: no trace means no origin.

  • Folding (foldConsequences, check/root-cause.ts) runs per workspace after the pack has run. The root is the innermost breaking finding (not a cause cluster) whose span holds the origin. Chains go to the first root, and loops don't fold. A diagnostic with no breaking finding at its origin stays as it was. The verdict is recounted.

  • Output:

    • default view: 49 sites, +54 downstream errors on the rule's line;
    • --details: each folded error listed under its root;
    • --json: the root finding's downstream, plus plan[].downstream.

    Downstream errors are not sites, so fix selects only the root. Cause clusters no longer leak their downstream onto their expanded sites.

  • fix verification is unchanged: it still compiles the whole workspace and compares every diagnostic with the baseline. Folding changes what is reported, never what has to compile.

  • Fixtures (fixtures/deps/issues-v1/-v2, cause.test.ts):

    • a .errors read whose map(e => e.message) (TS7006) and lost narrowing (TS18048) fold into the root;
    • a removed import in the same file (TS2305) stays;
    • a call whose return also changed somewhere untraceable is not followed.

    root-cause.test.ts covers the fold: chains, orphans, non-breaking roots, loops and the plan count.

No ground-truth, gate or threshold change, and no verification.json rewrite.

Before / after

"Before" is #218's CI run 39076300528 at 34419e2 (pack-test artifacts). "After" is a local re-score of this branch: zod first, then one pack at a time, each with its own cache, deleted after. The columns are #212's: breaking ground-truth sites found by a finding at any severity, and found as breaking.

Pack Breaking sites Any severity, before Any severity, after As breaking, before As breaking, after Breaking FPs before → after Status before → after
@sentry/browser 22 82% (18) 82% (18) 82% (18) 82% (18) 0 → 0 verified → verified
@sentry/nextjs 8 100% (8) 100% (8) 100% (8) 100% (8) 0 → 0 verified → verified
@sentry/react 29 72% (21) 72% (21) 72% (21) 72% (21) 3 → 3 candidate → candidate
ai 11 82% (9) 82% (9) 82% (9) 82% (9) 0 → 0 verified → verified
express 86 86% (74) 86% (74) 86% (74) 86% (74) 0 → 0 verified → verified
ioredis 2 0% (0) 0% (0) 0% (0) 0% (0) 0 → 0 candidate → candidate
joi 108 100% (108) 100% (108) 100% (108) 100% (108) 0 → 0 verified → verified
lucide-react 11 100% (11) 100% (11) 100% (11) 100% (11) 0 → 0 verified → verified
next 34 88% (30) 88% (30) 88% (30) 88% (30) 0 → 0 verified → verified
openai 20 15% (3) 15% (3) 15% (3) 15% (3) 0 → 0 verified → verified
react 130 95% (123) 95% (123) 95% (123) 95% (123) 0 → 0 verified → verified
react-router 18 89% (16) 89% (16) 78% (14) 78% (14) 0 → 0 verified → verified
redis 20 85% (17) 85% (17) 75% (15) 75% (15) 0 → 0 verified → verified
stripe 25 92% (23) 92% (23) 92% (23) 92% (23) 0 → 0 verified → verified
typescript 1399 99% (1379) 99% (1379) 99% (1379) 99% (1379) 0 → 0 verified → verified
vitest 32 97% (31) 97% (31) 97% (31) 97% (31) 0 → 0 verified → verified
zod 249 93% (232) 87% (216) 22% (55) 47% (117) 0 → 0 verified → verified

No pack loses a true positive. Site by site, every true-positive site of every pack in the "before" run is still found, with one exception: zod's 17, which #218 itself loses (see below).

zod, with vzdc-artcc compiling

The CI "before" is not the state this issue is about. On #218's CI, vzdc-artcc's prisma generate failed. The artifact reports generator failed: prisma generate … ECONNREFUSED 127.0.0.1:9: Prisma 7 fetches schema-engine on the Linux runner, and #218's engine-download block refuses it. So vzdc-artcc did not compile there, and zod stayed at its pre-generator numbers. Locally the engines are already in ~/.cache/prisma, so the project generates and compiles. #218's own local re-score at the same commit (34419e2) is the meaningful zod "before":

zod Any severity As breaking Breaking TP / FP / FN Status
#218 CI (generator failed) 93% (232) 22% (55) 55 / 0 / 17 verified
#218 local (generator ran) 87% (216) 47% (117) 117 / 58 / 33 candidate (gate fails)
this PR, local 87% (216) 47% (117) 117 / 0 / 33 verified (gate passes)

All 58 vzdc-artcc false positives fold:

  • 54 under the error-issues sites (return {errors: result.error.errors} in the server actions), from the forms' errors.map(e => e.message) (TS7006) and the lost narrowing (savedAirport possibly undefined, TS18048).
  • 4 under the SafeParseReturnType import in actions/event.ts/actions/eventPosition.ts (removed-exports). Those actions cast the parse result to it, so their .errors is any without an error of its own.

The 17 vzdc misses are the same before and after. They are #218's: the 15 { message } generic sites that compile clean under zod 4, plus the two .errors reads behind that cast.

The CI generator failure is #218's. Once this PR is merged into #218's branch, CI will still score vzdc-artcc without the generated client. The zod gate will pass there either way. The folding is only exercised where prisma generate succeeds.

Local notes

  • @sentry/react ran with the same pnpm-12 shim and LFS pass-through env as UPT-64/UPT-95. It exits 1 on its existing 3 false positives, as before.
  • Tests:
    • new and related suites pass: cause, root-cause, unattributed, option-cascades, format-check, format-check-details.
    • Failing on this branch and on its base alike (fixture installs, scaffold, zod contract against the generated vzdc project, the check verdict JSON): 20 tests.
    • Under a full parallel run, some feedback.test.ts and fix.test.ts cases time out on both. Run alone, all 89 pass on this branch.

🤖 Generated with Claude Code

@lucasbraum
lucasbraum force-pushed the lucasbraum/upt-95-ground-truth-generators branch from 34419e2 to daacbbe Compare October 18, 2026 00:59
@lucasbraum
lucasbraum force-pushed the lucasbraum/upt-99-fold-cascade-diagnostics branch from 6f92c13 to f86fe3c Compare October 18, 2026 00:59
lucasbraum and others added 4 commits October 17, 2026 21:26
… from (UPT-99)

Declares the rule before any re-score: a new compile error whose value the
compiler traces (declarations, destructuring, any receivers, returned
object members, repository function calls) to the expression of another
new, breaking error folds into that finding as a downstream error. The
default view counts them, --details and --json list them, fix still
verifies against every diagnostic, and pack test scores them as neither
hit nor false positive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
…ey come from (UPT-99)

A new compile error whose value the compiler traces to another new
error's expression is that site's consequence. flowOrigin (cause.ts)
follows the value back through variable and destructured declarations,
a callback's any receiver, returned object members, casts to a type
whose import is itself a new error, and calls of repository functions
whose changed returns all lead to such an expression. Types are compared
by an order-free shape, since the two programs list members and union
or intersection constituents differently. There is no line-distance
fallback: no trace, no origin.

check keeps the origin on the unattributed finding and, once the pack
has run, foldConsequences moves it into the innermost breaking finding
at that position as a downstream error, recounting the verdict. The plan
counts them per group (+N downstream errors), --details lists them under
their root, --json carries them as downstream. They are not sites, so
fix selects only the root. fix verification still compiles everything.

Fixtures: issues-v1/v2, where a .errors read folds its map(e => e.message)
and narrowing errors, a removed import in the same file stays, and a call
whose return changed elsewhere is not followed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
…UPT-99)

pnpm uptide pack test zod --write, then pnpm docs:packs. With #218's
generator step and this branch's folding: breaking found at any severity
216/249, as breaking 117/249, 0 breaking false positives, verified, the
numbers CI measured for this branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
@lucasbraum
lucasbraum force-pushed the lucasbraum/upt-95-ground-truth-generators branch from daacbbe to f64e72f Compare October 18, 2026 04:30
@lucasbraum
lucasbraum force-pushed the lucasbraum/upt-99-fold-cascade-diagnostics branch from f86fe3c to d68c342 Compare October 18, 2026 04:30
@lucasbraum
lucasbraum merged commit d2f42e8 into lucasbraum/upt-95-ground-truth-generators Oct 18, 2026
28 checks passed
@lucasbraum
lucasbraum deleted the lucasbraum/upt-99-fold-cascade-diagnostics branch October 18, 2026 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant