Repository navigation
docs(packs): add "breaking found as breaking" to the pack tables - #212
Merged
Merged
Conversation
Base automatically changed from
lucasbraum/upt-92-zod-pack-remaining-misses
to
main
October 17, 2026 20:25
Record breakingFound in verification.json (breaking sites a breaking-severity finding landed on) and show it beside recall, which leaves a site found only as a deprecation out of both counts. The README sentence now says what each column measures. Scorer and gate unchanged. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
…e same sites Record breakingFoundAny, breakingFound and breakingScored (all scored repositories, fixtures included) and show the first two as shares of the third in place of the hybrid recall column. The README says what each column covers. Scorer and gate unchanged. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
…ebasing on main Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
lucasbraum
force-pushed
the
lucasbraum/docs-packs-breaking-severity
branch
from
October 17, 2026 21:12
473d23e to
83e9a1a
Compare
… within its line budget The Found definitions move to docs/compatibility.md. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
Merged
lucasbraum
added a commit
that referenced
this pull request
Oct 18, 2026
…ey come from (UPT-99) (#222) Closes UPT-99. Stacked on #218 (base: `lucasbraum/upt-95-ground-truth-generators`). **Rule first.** The folding rule went into docs/concepts.md ("One root cause, one finding") and docs/packs.md (Scoring) in 2cb4eef, before any scoring run. The engine change is 7f66699. ## What changes A new compile error whose value the compiler traces to another new error's expression is that site's consequence. It folds into the breaking finding there as a **downstream error**. It no longer counts as a breaking site of its own. - **Tracing is compiler-only (`flowOrigin`, adapters/typescript/cause.ts).** It starts at the expression at the error and follows the value back through: - variable and destructured declarations; - a callback whose parameter is typed only by an `any` receiver (`errors.map(e => …)`); - members of returned object literals; - casts to a type whose import is itself a new error (`as SafeParseReturnType<…>`); - calls of repository functions. A call is followed only when every return whose type changed between the installed program and the target leads to such an expression. Types are compared by an order-free shape (members, unions and intersections sorted), because the two programs print the same Prisma payload types in a different member order. There is no line-distance fallback: no trace means no origin. - **Folding (`foldConsequences`, check/root-cause.ts)** runs per workspace after the pack has run. The root is the innermost breaking finding (not a cause cluster) whose span holds the origin. Chains go to the first root, and loops don't fold. A diagnostic with no breaking finding at its origin stays as it was. The verdict is recounted. - **Output:** - default view: `49 sites, +54 downstream errors` on the rule's line; - `--details`: each folded error listed under its root; - `--json`: the root finding's `downstream`, plus `plan[].downstream`. Downstream errors are not sites, so `fix` selects only the root. Cause clusters no longer leak their `downstream` onto their expanded sites. - **`fix` verification is unchanged:** it still compiles the whole workspace and compares every diagnostic with the baseline. Folding changes what is reported, never what has to compile. - **Fixtures** (`fixtures/deps/issues-v1`/`-v2`, `cause.test.ts`): - a `.errors` read whose `map(e => e.message)` (TS7006) and lost narrowing (TS18048) fold into the root; - a removed import in the same file (TS2305) stays; - a call whose return also changed somewhere untraceable is not followed. `root-cause.test.ts` covers the fold: chains, orphans, non-breaking roots, loops and the plan count. No ground-truth, gate or threshold change, and no `verification.json` rewrite. ## Before / after "Before" is #218's CI run [39076300528](https://github.com/uptide-dev/uptide/actions/runs/39076300528) at 34419e2 (pack-test artifacts). "After" is a local re-score of this branch: zod first, then one pack at a time, each with its own cache, deleted after. The columns are #212's: breaking ground-truth sites found by a finding at any severity, and found as breaking. | Pack | Breaking sites | Any severity, before | Any severity, after | As breaking, before | As breaking, after | Breaking FPs before → after | Status before → after | | --- | ---: | ---: | ---: | ---: | ---: | ---: | --- | | @sentry/browser | 22 | 82% (18) | 82% (18) | 82% (18) | 82% (18) | 0 → 0 | verified → verified | | @sentry/nextjs | 8 | 100% (8) | 100% (8) | 100% (8) | 100% (8) | 0 → 0 | verified → verified | | @sentry/react | 29 | 72% (21) | 72% (21) | 72% (21) | 72% (21) | 3 → 3 | candidate → candidate | | ai | 11 | 82% (9) | 82% (9) | 82% (9) | 82% (9) | 0 → 0 | verified → verified | | express | 86 | 86% (74) | 86% (74) | 86% (74) | 86% (74) | 0 → 0 | verified → verified | | ioredis | 2 | 0% (0) | 0% (0) | 0% (0) | 0% (0) | 0 → 0 | candidate → candidate | | joi | 108 | 100% (108) | 100% (108) | 100% (108) | 100% (108) | 0 → 0 | verified → verified | | lucide-react | 11 | 100% (11) | 100% (11) | 100% (11) | 100% (11) | 0 → 0 | verified → verified | | next | 34 | 88% (30) | 88% (30) | 88% (30) | 88% (30) | 0 → 0 | verified → verified | | openai | 20 | 15% (3) | 15% (3) | 15% (3) | 15% (3) | 0 → 0 | verified → verified | | react | 130 | 95% (123) | 95% (123) | 95% (123) | 95% (123) | 0 → 0 | verified → verified | | react-router | 18 | 89% (16) | 89% (16) | 78% (14) | 78% (14) | 0 → 0 | verified → verified | | redis | 20 | 85% (17) | 85% (17) | 75% (15) | 75% (15) | 0 → 0 | verified → verified | | stripe | 25 | 92% (23) | 92% (23) | 92% (23) | 92% (23) | 0 → 0 | verified → verified | | typescript | 1399 | 99% (1379) | 99% (1379) | 99% (1379) | 99% (1379) | 0 → 0 | verified → verified | | vitest | 32 | 97% (31) | 97% (31) | 97% (31) | 97% (31) | 0 → 0 | verified → verified | | **zod** | 249 | 93% (232) | 87% (216) | 22% (55) | 47% (117) | 0 → 0 | verified → verified | No pack loses a true positive. Site by site, every true-positive site of every pack in the "before" run is still found, with one exception: zod's 17, which #218 itself loses (see below). ### zod, with vzdc-artcc compiling The CI "before" is not the state this issue is about. **On #218's CI, vzdc-artcc's `prisma generate` failed.** The artifact reports `generator failed: prisma generate … ECONNREFUSED 127.0.0.1:9`: Prisma 7 fetches `schema-engine` on the Linux runner, and #218's engine-download block refuses it. So vzdc-artcc did not compile there, and zod stayed at its pre-generator numbers. Locally the engines are already in `~/.cache/prisma`, so the project generates and compiles. #218's own local re-score at the same commit (34419e2) is the meaningful zod "before": | zod | Any severity | As breaking | Breaking TP / FP / FN | Status | | --- | ---: | ---: | --- | --- | | #218 CI (generator failed) | 93% (232) | 22% (55) | 55 / 0 / 17 | verified | | #218 local (generator ran) | 87% (216) | 47% (117) | 117 / **58** / 33 | candidate (gate fails) | | this PR, local | 87% (216) | 47% (117) | 117 / **0** / 33 | **verified** (gate passes) | All 58 vzdc-artcc false positives fold: - 54 under the `error-issues` sites (`return {errors: result.error.errors}` in the server actions), from the forms' `errors.map(e => e.message)` (TS7006) and the lost narrowing (`savedAirport` possibly undefined, TS18048). - 4 under the `SafeParseReturnType` import in `actions/event.ts`/`actions/eventPosition.ts` (`removed-exports`). Those actions cast the parse result to it, so their `.errors` is `any` without an error of its own. The 17 vzdc misses are the same before and after. They are #218's: the 15 `{ message }` generic sites that compile clean under zod 4, plus the two `.errors` reads behind that cast. **The CI generator failure is #218's.** Once this PR is merged into #218's branch, CI will still score vzdc-artcc without the generated client. The zod gate will pass there either way. The folding is only exercised where `prisma generate` succeeds. ## Local notes - @sentry/react ran with the same pnpm-12 shim and LFS pass-through env as UPT-64/UPT-95. It exits 1 on its existing 3 false positives, as before. - Tests: - new and related suites pass: cause, root-cause, unattributed, option-cascades, format-check, format-check-details. - Failing on this branch and on its base alike (fixture installs, scaffold, zod contract against the generated vzdc project, the check verdict JSON): 20 tests. - Under a full parallel run, some `feedback.test.ts` and `fix.test.ts` cases time out on both. Run alone, all 89 pass on this branch. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Lucas Braum <lucasbraum@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
lucasbraum
added a commit
that referenced
this pull request
Oct 18, 2026
…UPT-95) (#218) Closes UPT-95. **Method first.** The docs/packs.md change was committed before any scoring run (694b141). The engine-download rule (dbdbac7) was also committed before the final re-score. After the install and before `check`, ground-truth preparation (`ensureRepo`, so both `packs:fetch` and `pack test`) runs UPT-90's `generateClients` on the installed project. That means `prisma generate` for a `schema.prisma`, and GraphQL codegen for a `codegen.ts`/`yml` with a local schema. It applies to every pack and every repository. `check` still runs no scripts. Fixtures are synthetic and installed in the tree, so they get no generator step. - The outcome of each generator is recorded in the cache marker. A cache installed before this change runs its generators on next use, with no re-fetch or re-install, and also when offline. - `pack test` reports a generator that failed, timed out or was refused as `generator failed: <command> (<reason>)` (`generatorFailed` in `--json`). That repository is scored without the generated code, as before. - Ground truth sets `PRISMA_ENGINES_MIRROR` to an address nothing listens on, passed through a new optional `env` parameter of `generateClients`. Prisma 5/6 download their engine from binaries.prisma.sh when the install script did not run. Checked on a clean `prisma@5.22.0` install with scripts off: `generate` fails with `ECONNREFUSED 127.0.0.1:9`. Prisma 7's client generator needs no engine. - No ground truth, gate or threshold is changed. No `verification.json` is rewritten. ## Before / after Before is CI run [39069556978](https://github.com/uptide-dev/uptide/actions/runs/39069556978), at this PR's base 213e1a5. After is a local re-score at this branch, one pack at a time, each with its own cache (as CI), deleted after. The columns are #212's, computed the same way from the `pack test --json` sites: breaking ground-truth sites of every scored repository, found by a finding at any severity, and found as breaking. | Pack | Breaking sites | Any severity, before | Any severity, after | As breaking, before | As breaking, after | Status before → after | | --- | ---: | ---: | ---: | ---: | ---: | --- | | ai | 11 | 82% (9) | 82% (9) | 82% (9) | 82% (9) | verified → verified | | express | 86 | 86% (74) | 86% (74) | 86% (74) | 86% (74) | verified → verified | | ioredis | 2 | 0% (0) | 0% (0) | 0% (0) | 0% (0) | candidate → candidate | | joi | 108 | 100% (108) | 100% (108) | 100% (108) | 100% (108) | verified → verified | | lucide-react | 11 | 100% (11) | 100% (11) | 100% (11) | 100% (11) | verified → verified | | next | 34 | 88% (30) | 88% (30) | 88% (30) | 88% (30) | verified → verified | | openai | 20 | 15% (3) | 15% (3) | 15% (3) | 15% (3) | verified → verified | | react | 130 | 95% (123) | 95% (123) | 95% (123) | 95% (123) | verified → verified | | react-router | 18 | 89% (16) | 89% (16) | 78% (14) | 78% (14) | verified → verified | | redis | 20 | 85% (17) | 85% (17) | 75% (15) | 75% (15) | verified → verified | | @sentry/browser | 22 | 82% (18) | 82% (18) | 82% (18) | 82% (18) | verified → verified | | @sentry/nextjs | 8 | 100% (8) | 100% (8) | 100% (8) | 100% (8) | verified → verified | | @sentry/react | 29 | 72% (21) | 72% (21) | 72% (21) | 72% (21) | candidate → candidate | | stripe | 25 | 92% (23) | 92% (23) | 92% (23) | 92% (23) | verified → verified | | typescript | 1399 | 99% (1379) | 99% (1379) | 99% (1379) | 99% (1379) | verified → verified | | vitest | 32 | 97% (31) | 97% (31) | 97% (31) | 97% (31) | verified → verified | | **zod** | 249 | 93% (232) | **87% (216)** | 22% (55) | **47% (117)** | verified → **candidate** | **Generators that ran (11 repositories, 9 packs).** - Generated: vzdc-artcc/website (zod), cubicsteak/keep (lucide-react), wardby/wardby (openai) and maximunited/yomu (@sentry/nextjs), all on Prisma 7. - Prisma 5/6, generated locally from an engine already in `~/.cache/prisma`: HiroshiIgarash/koremina (next), csnyder256/ux-struggle-detector (openai) and UNN-Devotek/FCM-Fallout-Chat-Mod (redis). On a fresh runner the download block makes these fail and be reported. Their scores are the same either way. - Failed, reported, scored as before: - simondietschi/dokunc and kazumadev619-dev/FishingConditionsApp (ioredis): `prisma.config.ts` needs `DATABASE_URL` / `DATABASE_URL_DIRECT`. - getlago/lago-front (react-router): `Query root type must be provided`. - m-lyon/recipe (typescript): invalid codegen config, no `schema`. Only zod moves. ## zod: the vzdc-artcc sites, and a new gate failure All 63 `error-issues`/`error-params` sites in vzdc-artcc/website were `unverified` before. Now: - **61 move to breaking** (48 `error-issues`, all 13 `error-params`). - **2 are no longer reported**: `actions/event.ts:165` and `actions/eventPosition.ts:263`. The parse result there is cast `as SafeParseReturnType<any, any>`, so `result.error.errors` is `any` and nothing fires. This is a real finder gap, not an environment one. Other vzdc-artcc truth sites: - 15 `generic` breaking sites (`{ message: … }` params) were hit only by the file-level `unverified` finding. They now compile clean under zod 4 (`message` is deprecated, not removed), so they are misses. That is the any-severity drop, 232 → 216. - 3 `generic` sites move from `unverified` to `deprecated`. - 1 `error-issues` site is newly found as breaking. **58 new breaking false positives, all `generic`, all in vzdc-artcc.** These are compile diagnostics in the form components, such as `components/Airports/AirportForm.tsx:17,21,24`. They come from `errors.map(e => e.message)` over a `ZodError.errors` result that is `any` at zod 4, and from the narrowing that falls out of it (`savedAirport` possibly undefined). The truth entry says it leaves these out on purpose ("the 36 implicit-any diagnostics (7006) in components that map over a ZodError.errors result typed any at zod 4"). Before this change they were hidden because the files did not compile at all. So zod now measures **candidate** (gate: "58 false positives among breaking findings"). Its recorded `verified` makes `pack test zod` exit 1, and **the zod pack job will fail on this PR**. I didn't change the ground truth, the gate or the record, per the issue's rules. Three ways forward, your call: 1. Accept candidate (`pack test zod --write`). 2. Review the vzdc-artcc truth for these downstream diagnostics in a separate truth PR, scored after. 3. Keep that repository's downstream diagnostics out of the breaking tally in the engine. That is a scorer change, so a separate issue. ## Local notes - @sentry/react's first local pass errored on 6 repositories at install: corepack 0.34 can't start pnpm 12, and BotFleet needs git-lfs. With the same local shim and LFS env as UPT-64 it matches CI exactly. These are local environment issues, unrelated to this change. - Tests: `truth.test.ts` covers a cached install that gets generated once, offline, and reports a failing generator. `generate.test.ts` passes. Two tooling tests fail locally on origin/main too (the scaffold formatter check, and "passes every registered pack: each companion it names carries its source"). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Lucas Braum <lucasbraum@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
lucasbraum
pushed a commit
that referenced
this pull request
Oct 18, 2026
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## uptide@0.15.1 ### Patch Changes - [#220](#220) [`cf86d65`](cf86d65) Thanks [@lucasbraum](https://github.com/lucasbraum)! - With an assisting agent, a root pin Uptide cannot update safely (a nested override such as `overrides > pkg > peerDependencies > react`) no longer stops `fix` with exit 2: the run continues as an agent-led draft that leaves the pin as written and names it, with its exact path and value, at the top of the PR body. The agent-led peer reason now names each blocker's first release that accepts the target, or says none does. An agent-led draft must pass the repository's clean install (`npm ci` or the manager's frozen install) before delivery; if it does not, the PR says so at the top. Without an agent, with `--no-llm`, or on a safety refusal, `fix` still exits 2. - [#232](#232) [`e4a2885`](e4a2885) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide fix` spends less wall time on test runs and dependency copies: - The agent's own `run_tests` check runs only its runner's related tests when the `test` script builds first and then runs a script that is just the runner (`pnpm run build && pnpm run test:unit`). The gate after the agent's edits and the final verification run exactly what they ran before. - The upgrade's install moves the installed `node_modules` out of its temporary worktree instead of copying it. - Each phase of the fix report lists the repository's commands it ran (installs, generators, builds, type-checks, tests, lint, format) with their count and summed wall time. An agent-led run's clean install has its own phase. - [#223](#223) [`2dc2162`](2dc2162) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix` no longer stops with "baseline tests modified the working tree" when the test run only writes new untracked files, such as a junit `test-report.xml` that is not gitignored. Those files are recorded as test artifacts (`testArtifacts` in the report), kept out of every commit, the agent's diff and the pending-changes check, and removed before each later test run, `uptide verify` included. A change to a tracked file during baseline tests still refuses the run. - [#215](#215) [`abddcb3`](abddcb3) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `check` no longer lists possible impact (type changes nothing confirmed as breaking) on its first screen: each package says it in one line, `+N possible impact in M files, not counted as breaking`, and `--details` and `--json` still have every site. A package whose types were partly verified no longer repeats its coverage line in the heading, and a monorepo's unbuilt workspace chains are one count on that line. - [#213](#213) [`2de3a18`](2de3a18) Thanks [@lucasbraum](https://github.com/lucasbraum)! - A companion pack that moves with the lead's fix (react beside next) now gets its own scan context and inventory, and the PR body carries a section with its review notes. - [#212](#212) [`733cfbe`](733cfbe) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `pack test --write` records how many breaking sites a finding landed on at any severity and as breaking, out of all breaking sites it scored, and the pack tables show both ("Found, any severity", "Found as breaking") in place of the single recall column. Packs recorded before it show `—`. - [#224](#224) [`09f68df`](09f68df) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix` on npm no longer stops when the lock-only install hits ERESOLVE from the old lockfile. It retries without the lockfile entries of the moved packages, their companions and the packages whose peers they satisfy, and only then with the whole lockfile re-resolved; the report says which step worked, and the lockfile check reports the churn as usual. It never uses `--legacy-peer-deps` or `--force`. When every step fails, `fix` ends blocked, naming the conflicting packages from npm's ERESOLVE report. `fix --json` now prints JSON on every exit, a failure included; when the run already has its report (a PR body over GitHub's size limit), that report comes with the error. - [#221](#221) [`ff8de8e`](ff8de8e) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix` no longer rejects an agent patch for test files the parent commit already failed, whichever test scope ran them: when the test script builds first and the build failed on the error being fixed, the start had no test results to compare with, so a correct patch was reverted every round. The patch's tests also run with the pack's follow-ups in place (a test asserting the old Stripe API version), which `fix` still applies and reports after the agent. When a PR body cannot fit, the stored report.json the error points to is now written first. `check` and `fix` now also analyze files that reach a package's wrapper through a barrel (`import { getStripeClient } from '../stripe'` with `export * from './client'`). Edited files are formatted with oxfmt and dprint as well as biome and prettier, from the workspace that holds them, with its own configuration, else the root's. The stripe pack adds `request-options`: request options passed in the params slot (`subscriptions.cancel(id, { idempotencyKey })`), which SDK 22 rejects, move to the third argument. It runs assisted until a ground-truth repository exercises it. - [#226](#226) [`448c3cb`](448c3cb) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide fix` judges consumed workspace builds on the final head, not on the tree right after the bump. A build that failed after the bump and passes once the edits land no longer starts a feedback round, adds an "Agent-led draft" reason or shows "Build failed" in the PR body. The builds run again only when the tree changed since their last result, and not at all when the verification build already built those workspaces. The fix report's `buildRuns` gives the number of builds the run executed and their wall time. - [#218](#218) [`13e343b`](13e343b) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `check` folds a compile error into the breaking site it comes from when the compiler traces its value there (an `errors.map(e => e.message)` under a `result.error.errors` that zod 4 no longer types, and the narrowing that depended on it). The rule's line shows `+N downstream errors`, `--details` lists them under their site, and `--json` carries them as that finding's `downstream`. They are no longer separate breaking sites. `fix` still verifies against every error. - [#214](#214) [`195ae10`](195ae10) Thanks [@lucasbraum](https://github.com/lucasbraum)! - A rewrite rule below 80% precision over at least 5 scored ground-truth sites now runs as assisted in `fix` (react-router `dom-imports`); smaller samples (next `middleware-proxy`) stay mechanical and are flagged in the compatibility table. Pack status is unchanged. - [#218](#218) [`13e343b`](13e343b) Thanks [@lucasbraum](https://github.com/lucasbraum)! - pack test runs a ground-truth repository's code generators (prisma generate, local graphql-codegen) after its install, as fix does. It fetches the Prisma engines the locked version needs itself, so generators stay off the network. A repository whose generator still fails is incomplete and fails the pack, unless its ground truth declares `withoutGenerated` - [#227](#227) [`ae175a9`](ae175a9) Thanks [@lucasbraum](https://github.com/lucasbraum)! - A migration whose evidence is larger than GitHub's PR description limit no longer stops `fix` with "Required migration evidence exceeds the PR body budget" and loses the run. Long required lists get a compact form when they do not fit. Lockfile churn shows counts per kind of change and the first packages. Manual sites are grouped by rule, with counts and the first locations. Remaining type errors show the first entries. Every full list stays in report.json and, in the Uptide Action, in the job summary, which the description links to. If even the compact form does not fit, the description is a short summary that says so at the top: the verdict, the checks, the counts and where the full report is. The branch and report.json are always kept. An agent's decisions under "Decisions for you" name the files the agent changed, five at most, and count the rest, instead of every file in the task's scope. A verification-feedback task scoped to the whole repository made each of its decisions a list of hundreds of files. - [#219](#219) [`12dc612`](12dc612) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix zod` rewrites `z.string().uuid()` to `z.guid()` by rule without `--include-deprecated`: zod 4's `uuid()` also checks the RFC version and variant, so ids zod 3 accepted stop validating, and `check` now counts the site as breaking. `fix` no longer reports false pre-existing type errors on a TypeScript 7 repository whose `lib` names a library newer than its `target` (`lib: ["ES2023"]` under `ES2022`). Co-authored-by: uptide-release[bot] <338861287+uptide-release[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #209 (stacked on its branch; retarget to main once #209 merges).
Columns. The pack tables (README, docs/compatibility.md) drop the hybrid "Recall" and show Found, any severity and Found as breaking, both a share of the same set: every breaking ground-truth site of the repositories
pack testscores. That is all scored repositories, including the synthetic fixture repo (zod's storefront); it is not the public-only "Breaking sites" column. "Any severity" counts a site a finding landed on whatever the severity (breaking, deprecated, unverified); "as breaking" only a breaking-severity finding.pack test --writerecordsbreakingFoundAny,breakingFoundandbreakingScored; scorer and gate unchanged. Sites are line-level (file:line), as the scorer has them; nothing is folded.zod: 93% any severity, 22% as breaking (55 of 249). Other gaps: react-router 89→78%, redis 85→75%. All verified packs and ioredis re-recorded (only these fields changed); joi, openai, @sentry/react and sentry-node are not rewritten and show
—.The
unverifiederror-issues/error-params sites (zod). 63 sites, all in vzdc-artcc/website (50error-issues, 13error-params).checkattached: "57 of 95 files cannot resolve their imports at the installed version; compile signal skipped" and "unverified: compile check skipped". The unresolved imports are the repository's generated Prisma client (@/generated/prisma/client,@/generated/prisma/browser, ~210 imports): its output isprisma generate, which the repo runs in its build script, and installs here run with scripts off. The scoring environment is not changed here: that is a method change for all repositories, a separate issue to declare before scoring. PATH has no such sites.joi: an offline
--writeflipped joi candidate→verified; reverted, not promoted. The committed record is held at candidate on purpose (repos: 0, older truth digest); the run measured 2 repos, 108 TP / 0 FP, 108 breaking sites, which are a joi 17.7.0 typing change, not 18.🤖 Generated with Claude Code