From 5c3214261231e9fd9991799dfb04a5323ced5c1e Mon Sep 17 00:00:00 2001 From: Lucas Braum Date: Thu, 15 Oct 2026 15:09:20 -0700 Subject: [PATCH 1/3] feat(packs): verified typescript 5 to 6 and express 4 to 5 packs pack test --write on main under UPT-48's gate, one pack at a time, each with its own cache. Ground truth unchanged. - typescript: 7 repos, 1,399 real breaking sites, 0 breaking FP; unexercised downlevel-iteration, import-assertions, module-keyword (run assisted). - express: 4 repos, 86 real breaking sites, 0 breaking FP; none unexercised. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Lucas Braum --- .changeset/promote-typescript-express.md | 5 +++ README.md | 4 +-- packages/core/src/packs/express/RESEARCH.md | 28 +++++++++------ .../core/src/packs/express/verification.json | 2 +- .../core/src/packs/typescript/RESEARCH.md | 34 +++++++++---------- .../src/packs/typescript/verification.json | 2 +- 6 files changed, 44 insertions(+), 31 deletions(-) create mode 100644 .changeset/promote-typescript-express.md diff --git a/.changeset/promote-typescript-express.md b/.changeset/promote-typescript-express.md new file mode 100644 index 00000000..532c3d14 --- /dev/null +++ b/.changeset/promote-typescript-express.md @@ -0,0 +1,5 @@ +--- +"uptide": minor +--- + +The typescript 5 to 6 and express 4 to 5 packs are now verified, so `check`, `list` and `fix` use them by default. Both measure verified under the stricter gate on unchanged ground truth: typescript across 7 public repositories (1,399 real breaking sites, no breaking false positive; its `downlevel-iteration`, `import-assertions` and `module-keyword` rewrites stay assisted until a real site exercises them), express across 4 (86 real breaking sites, no breaking false positive). diff --git a/README.md b/README.md index 1a990fed..6501701c 100644 --- a/README.md +++ b/README.md @@ -57,7 +57,7 @@ The contract: [migration packs](docs/packs.md). | Package | Range | Precision | Recall | Breaking sites | Ground-truth repositories | Status | | --- | --- | ---: | ---: | ---: | --- | --- | | `ai` | 6.x → 7.x | 100% | 82% | 11 | vercel/chatbot, miurla/morphic | verified | -| `express` | 4.x → 5.x | 100% | 85% | 86 | netlify/cli, bcgov/tenant-management-system, amiantos/lurker, openhab/openhab-cloud | candidate | +| `express` | 4.x → 5.x | 100% | 85% | 86 | netlify/cli, bcgov/tenant-management-system, amiantos/lurker, openhab/openhab-cloud | verified | | `ioredis` | 5.x → 6.x | n/a | 0% | 2 | ferni-ai/voiceai, syzygyhack/open-foundry, simondietschi/dokunc, safe-global/safe-client-gateway, kazumadev619-dev/FishingConditionsApp | candidate | | `joi` | 17.x → 18.x | n/a | n/a | — | edtech4good/edtech-lms-api, senacor/azure-function-middleware | candidate | | `lucide-react` | 0.x → 1.x | 100% | 100% | 11 | tradeblocks-org/tradeblocks, spizeck/SFPCA, cubicsteak/keep | verified | @@ -70,7 +70,7 @@ The contract: [migration packs](docs/packs.md). | `@sentry/nextjs` | 10.x → 11.x | 100% | 100% | 8 | cohstats/coh3-stats, xdoubleu/tools.xdoubleu.com, maximunited/yomu | verified | | `@sentry/react` | 10.x → 11.x | 100% | 100% | 2 | owine/youth-activity-scheduler, successible/cleanslate | candidate | | `stripe` | 14 and newer | 100% | 92% | 21 | unkeyed/unkey, nextjs/saas-starter, starl3xx/wallet-to-social, AtlasDevHQ/atlas | verified | -| `typescript` | 5.x → 6.x | 100% | 17% | 1399 | m-lyon/recipe, robrechtme/recipes, JosephMaynard/dependency-radar, CroffZ/github-release-notifier, SignalK/signalk-derived-data, shogo82148/typescript-action, Borewit/music-metadata | candidate | +| `typescript` | 5.x → 6.x | 100% | 17% | 1399 | m-lyon/recipe, robrechtme/recipes, JosephMaynard/dependency-radar, CroffZ/github-release-notifier, SignalK/signalk-derived-data, shogo82148/typescript-action, Borewit/music-metadata | verified | | `vitest` | 4.x → 5.x | 100% | 50% | 32 | votingworks/vxsuite, vitorvasc/opentelemetry-ecosystem-explorer | verified | | `zod` | 3.x → 4.x | 100% | 64% | 219 | ugurkocde/AwesomeIntune, howardyang2009/PATH, band-ai/band-sdk-typescript, vzdc-artcc/website, CANShift/canshift-core | verified | diff --git a/packages/core/src/packs/express/RESEARCH.md b/packages/core/src/packs/express/RESEARCH.md index eed60404..13c08361 100644 --- a/packages/core/src/packs/express/RESEARCH.md +++ b/packages/core/src/packs/express/RESEARCH.md @@ -129,10 +129,11 @@ Searched read-only (blobless clones, `git log`/`show`, `gh search`): ## Measured pack test -`pnpm uptide pack test express --write` with per-pack caches (`UPTIDE_GROUND_TRUTH_CACHE`, -`UPTIDE_CACHE_DIR`), rebased on UPT-48 (#155) and UPT-63 (#157), 2026-10-15: **passes, measures -verified; recorded candidate by choice** (below). 4 public repositories, 86 expected breaking sites, -no unexercised rewrite. +`pnpm uptide pack test express --write` on main (`530f90c`, after UPT-48 #155 and UPT-63 #157) +with its own `UPTIDE_CACHE_DIR`, 2026-10-15: **passes, measures verified, recorded verified.** 4 +public repositories, 86 expected breaking sites, no unexercised rewrite. The ground truth is +unchanged (same digest as the candidate record), and the numbers match the run that first measured +verified. | Rule / scoring group | Precision | Recall | TP | FP | FN | | --- | ---: | ---: | ---: | ---: | ---: | @@ -153,9 +154,11 @@ ground truth was not changed: - openhab: body-parser and serve-static moved to 2.x, the repository's choice (neither depends on express). -**Why candidate.** Kept a candidate on purpose until a person reviews it: the http-proxy-middleware -peer conflict means `fix` would leave an install the netlify upgrade never had, and every rule -outside `route-syntax` and `params-string-array` is measured by fixtures only. +**Known gaps of the verified record.** The http-proxy-middleware peer conflict means `fix` would +leave an install the netlify upgrade never had (`check` names the conflict, it does not move the +package), and every rule outside `route-syntax` and `params-string-array` is measured by fixtures +only. The pack was held as a candidate on these grounds until UPT-48's gate was on main; the gate +does not cover either, so they are stated here. **Recall misses, all kept.** Lurker's handler annotations and bcgov's destructure are where the upgrade fixed a params type, not where the compiler reports it. netlify keeps three route paths as @@ -177,9 +180,14 @@ them rewrites, so under UPT-48 none is an unexercised rewrite. ## UPT-48 gate -Scored under the stricter gate (#155): 86 expected breaking sites across 4 repositories, per-rule -TP/FP/FN above, `unexercised: []` (no rule rewrites). It measures verified; the record says -candidate by choice. +Verified under the stricter gate (#155): + +- Repositories: 4 public (gate: ≥2). +- Real breaking sites: 86 expected (gate: ≥5). +- Breaking false positives: 0 (gate: 0). Breaking precision 100%, recall 85% (73 of 86). +- Behavior notes: 2 expected `req-body-undefined` sites, none predicted (recall 0%, no false + positive). +- Unexercised: none (`unexercised: []`). No rule rewrites, so no rewrite runs unconfirmed. ## What the engine cannot express diff --git a/packages/core/src/packs/express/verification.json b/packages/core/src/packs/express/verification.json index 07b4c2f0..2a4a5a81 100644 --- a/packages/core/src/packs/express/verification.json +++ b/packages/core/src/packs/express/verification.json @@ -1,5 +1,5 @@ { - "status": "candidate", + "status": "verified", "truth": "sha256-d07b7e14c0c525dae477ce9085f51713a46a89ecebdb260f8f9c4d48eef88e99", "repos": 4, "breaking": { diff --git a/packages/core/src/packs/typescript/RESEARCH.md b/packages/core/src/packs/typescript/RESEARCH.md index af1cabda..16664136 100644 --- a/packages/core/src/packs/typescript/RESEARCH.md +++ b/packages/core/src/packs/typescript/RESEARCH.md @@ -217,26 +217,26 @@ The recall figure measures the original truth's convention (every error line a s the engine's (one site per edit). It is not a count of breaks `check` misses: every folded error is listed under its tsconfig finding. -## The verified gate, now and under UPT-48 - -Under today's gate the run measures **verified**: seven public repositories, no breaking false -positive, 1,379 breaking sites predicted. The task asked to hold the claim until the stricter gate -(UPT-48) lands, so `verification.json` keeps the candidate record and is not written by this pull -request. After rebasing on UPT-48, `pack test --write` re-scores and records whatever that gate -says. - -Against UPT-48 as specified: - -- ≥2 repositories: 7. -- ≥5 expected breaking sites: 1,399. +## Verified under UPT-48 + +`pnpm uptide pack test typescript --write` on main (`530f90c`, UPT-48's gate), with its own +`UPTIDE_CACHE_DIR`, 2026-10-15: **PASS, measures verified, recorded verified.** The ground truth +is unchanged (same digest as the candidate record). + +- Repositories: 7 public (gate: ≥2). +- Real breaking sites: 1,399 expected (gate: ≥5). +- Breaking false positives: 0 (gate: 0). Breaking precision 100%, recall 17% (240 of 1,399; the + folded `types` errors above). +- Behavior notes: precision 20%, recall 50% (2 TP, 8 FP, 2 FN), all `changed-defaults` anchors at + `tsconfig*.json:1`. Notes do not count against a breaking-class gate. - Per-rule real-world true positives: `base-url` 4, `module-resolution-node10` 2, `target-es5` 1, `root-dir` 1 (all assisted). -- The three mechanical rewrites (`downlevel-iteration`, `module-keyword`, `import-assertions`) - met **no** real-world site. Under UPT-48 they are unexercised and run as assisted. The fixtures - are their only evidence. +- **Unexercised** (recorded in `verification.json`): `downlevel-iteration`, `import-assertions`, + `module-keyword`. No real site exercises their rewrites, so `check` and `fix` run them assisted; + the fixtures are their only evidence. - `module-resolution-classic`, `module-format-removed`, `out-file`, `interop-false` and - `always-strict-false` are detection only and also unexercised. Each matches only the exact value - 6.0 reports. + `always-strict-false` are detection only with no real site either. Each matches only the exact + value 6.0 reports. ## Known limits diff --git a/packages/core/src/packs/typescript/verification.json b/packages/core/src/packs/typescript/verification.json index 9d225d18..ecb2dc18 100644 --- a/packages/core/src/packs/typescript/verification.json +++ b/packages/core/src/packs/typescript/verification.json @@ -1,5 +1,5 @@ { - "status": "candidate", + "status": "verified", "truth": "sha256-b463fcdce83820c229ab46ab51a2afb62c3da5f4f5e6a2b53fee63f302755e68", "repos": 7, "breaking": { From f2f12c248351392c785bee997e1864ddf89a7c08 Mon Sep 17 00:00:00 2001 From: Lucas Braum Date: Thu, 15 Oct 2026 18:42:22 -0700 Subject: [PATCH 2/3] test(cli): list snapshot shows typescript as verified Co-Authored-By: Claude Opus 5.5 Signed-off-by: Lucas Braum --- .../__snapshots__/list-design.test.ts.snap | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/packages/cli/src/__snapshots__/list-design.test.ts.snap b/packages/cli/src/__snapshots__/list-design.test.ts.snap index 983f9e83..47bb8813 100644 --- a/packages/cli/src/__snapshots__/list-design.test.ts.snap +++ b/packages/cli/src/__snapshots__/list-design.test.ts.snap @@ -29,7 +29,7 @@ PACKAGES cookie-plugin 1.0.0 → 2.0.0 major 1 file 1 ref TOOLING 19 packages, used by scripts and config · --all - typescript 5.0.0 → 6.0.0 major 0 files + typescript 5.0.0 → 6.0.0 major 0 files verified compiler major: check build and tsconfig webpack 5.0.0 → 6.0.0 major 0 files bundler major: check build config @@ -53,27 +53,27 @@ exports[`Nest terminal design, TTY=true 1`] = ` GROUPS upgrade together @nestjs/* 10 packages → 12.x npx uptide check --group nestjs - @nestjs/cli 10.0.0 → 12.0.0 2 majors behind 0 files + @nestjs/cli 10.0.0 → 12.0.0 2 majors behind 0 files   known configuration or build tool - @nestjs/common 10.4.0 → 12.0.0 2 majors behind 1 file 2 calls - @nestjs/core 10.4.0 → 12.0.0 2 majors behind 1 file 1 call - @nestjs/platform-express 10.3.0 → 12.0.0 2 majors behind 1 file 1 ref - @nestjs/platform-fastify 10.4.0 → 12.0.0 2 majors behind 1 file 1 call - @nestjs/schematics 10.0.0 → 12.0.0 2 majors behind 0 files + @nestjs/common 10.4.0 → 12.0.0 2 majors behind 1 file 2 calls  + @nestjs/core 10.4.0 → 12.0.0 2 majors behind 1 file 1 call  + @nestjs/platform-express 10.3.0 → 12.0.0 2 majors behind 1 file 1 ref  + @nestjs/platform-fastify 10.4.0 → 12.0.0 2 majors behind 1 file 1 call  + @nestjs/schematics 10.0.0 → 12.0.0 2 majors behind 0 files   known configuration or build tool  referenced by configuration - @nestjs/swagger 7.4.2 → 12.0.0 5 majors behind 1 file 1 call + @nestjs/swagger 7.4.2 → 12.0.0 5 majors behind 1 file 1 call  @fastify/static 6.12.0 → 10.1.5 4 majors behind peer of @nestjs/platform-fastify nodemailer 6.9.0 → 8.0.0 2 majors behind peer of @nestjs/core reflect-metadata 0.1.0 → 1.0.0 major peer of @nestjs/common, @nestjs/core PACKAGES - cookie-plugin 1.0.0 → 2.0.0 major 1 file 1 ref + cookie-plugin 1.0.0 → 2.0.0 major 1 file 1 ref  TOOLING 19 packages, used by scripts and config · --all - typescript 5.0.0 → 6.0.0 major 0 files + typescript 5.0.0 → 6.0.0 major 0 files verified  compiler major: check build and tsconfig - webpack 5.0.0 → 6.0.0 major 0 files + webpack 5.0.0 → 6.0.0 major 0 files   bundler major: check build config POSSIBLY UNUSED 2 packages, no usage found by Uptide's scan; verify before removing · --all From 90c45cb53ed7fe1af844475299f8276f109255d7 Mon Sep 17 00:00:00 2001 From: Lucas Braum Date: Thu, 15 Oct 2026 20:14:12 -0700 Subject: [PATCH 3/3] fix(packs): pack test counts truth sites listed as folded evidence as covered (#165) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Stacked on #160 (base is its branch; retarget to main once #160 merges). ## What `predictedSites` (packs/tooling/pack-test.ts) now takes the repository's expected sites. When an anchored finding lists one as `downstream` evidence, that site counts as covered, at the truth's own rule and severity. Examples: TypeScript 6 `types: []` errors folded into one `changed-defaults` finding per tsconfig, and #136's TS7006 option cascades. So covered sites add true positives to the breaking and overall tallies and to their own rule. They don't inflate the behavior-note tally. Evidence the truth doesn't name stays unscored, as before. No ground-truth changes. ## Re-score: typescript (`--write`, once) | | Before | After | |---|---|---| | Breaking precision | 100% | 100% | | Breaking recall | 17% (240 / 1,399) | **99% (1,379 / 1,399)** | | Breaking FP | 0 | 0 | | Behavior notes P / R | 20% / 50% | 20% / 50% (unchanged) | | Gate | verified | verified | The 20 remaining breaking misses are the second-round and same-change lines RESEARCH.md already lists. README table, RESEARCH.md and verification.json are updated; the change to docs/packs.md is one sentence. Packs whose records include anchored evidence may show changed numbers on the next run. The scorer only adds true positives, so no false positive count or gate verdict can change. ## Checks - `vitest run` in core: 1,857 passed. The `predictedSites` evidence test now also covers a named evidence site. - `pnpm lint` (pre-existing warnings only), `typecheck`, `docs:packs --check`, `public:check` pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Lucas Braum Co-authored-by: Claude Opus 5.5 --- .changeset/score-folded-evidence.md | 5 +++++ README.md | 2 +- docs/packs.md | 4 +++- .../core/src/packs/tooling/pack-test.test.ts | 9 +++++++++ packages/core/src/packs/tooling/pack-test.ts | 18 ++++++++++++++++-- packages/core/src/packs/typescript/RESEARCH.md | 9 ++++++++- .../src/packs/typescript/verification.json | 6 +++--- 7 files changed, 45 insertions(+), 8 deletions(-) create mode 100644 .changeset/score-folded-evidence.md diff --git a/.changeset/score-folded-evidence.md b/.changeset/score-folded-evidence.md new file mode 100644 index 00000000..92411bab --- /dev/null +++ b/.changeset/score-folded-evidence.md @@ -0,0 +1,5 @@ +--- +"uptide": patch +--- + +`uptide pack test` counts a ground-truth site that a folded finding lists as its evidence as covered by that finding, so recall reflects what `check` reports. The typescript pack's breaking recall goes from 17% to 99% on unchanged ground truth: its TypeScript 6 `types` default errors fold into one tsconfig finding each. diff --git a/README.md b/README.md index 6501701c..42346382 100644 --- a/README.md +++ b/README.md @@ -70,7 +70,7 @@ The contract: [migration packs](docs/packs.md). | `@sentry/nextjs` | 10.x → 11.x | 100% | 100% | 8 | cohstats/coh3-stats, xdoubleu/tools.xdoubleu.com, maximunited/yomu | verified | | `@sentry/react` | 10.x → 11.x | 100% | 100% | 2 | owine/youth-activity-scheduler, successible/cleanslate | candidate | | `stripe` | 14 and newer | 100% | 92% | 21 | unkeyed/unkey, nextjs/saas-starter, starl3xx/wallet-to-social, AtlasDevHQ/atlas | verified | -| `typescript` | 5.x → 6.x | 100% | 17% | 1399 | m-lyon/recipe, robrechtme/recipes, JosephMaynard/dependency-radar, CroffZ/github-release-notifier, SignalK/signalk-derived-data, shogo82148/typescript-action, Borewit/music-metadata | verified | +| `typescript` | 5.x → 6.x | 100% | 99% | 1399 | m-lyon/recipe, robrechtme/recipes, JosephMaynard/dependency-radar, CroffZ/github-release-notifier, SignalK/signalk-derived-data, shogo82148/typescript-action, Borewit/music-metadata | verified | | `vitest` | 4.x → 5.x | 100% | 50% | 32 | votingworks/vxsuite, vitorvasc/opentelemetry-ecosystem-explorer | verified | | `zod` | 3.x → 4.x | 100% | 64% | 219 | ugurkocde/AwesomeIntune, howardyang2009/PATH, band-ai/band-sdk-typescript, vzdc-artcc/website, CANShift/canshift-core | verified | diff --git a/docs/packs.md b/docs/packs.md index 50d1f888..7299d565 100644 --- a/docs/packs.md +++ b/docs/packs.md @@ -140,7 +140,9 @@ site is reported is part of the contract (`reported`): findings are claimed and folds compiler errors that one edit elsewhere resolves into a finding at that edit, as their evidence. The typescript pack folds the "Cannot find name" errors of the 6.0 `types: []` default into one at the tsconfig that leaves - `types` unset. + `types` unset. `pack test` counts a ground-truth site listed as such evidence as + covered, at the site's own rule and severity; evidence the truth does not name is not + scored. - `decision`: the pull request lists it under "Decisions for you", with the files and lines. - `test-follow-up`: an assertion is updated only after the migrated code made that test fail, and the test then has to pass (zod's reworded default messages). diff --git a/packages/core/src/packs/tooling/pack-test.test.ts b/packages/core/src/packs/tooling/pack-test.test.ts index 14840b73..b250af51 100644 --- a/packages/core/src/packs/tooling/pack-test.test.ts +++ b/packages/core/src/packs/tooling/pack-test.test.ts @@ -142,6 +142,15 @@ it('scores an anchored finding at its anchor; the call sites listed as evidence expect(predictedSites(report, aiPack).map((s) => `${s.file}:${s.line} ${s.rule}`)).toEqual([ 'packages/lib/hook.ts:6 tool-context', ]); + // Evidence the ground truth names is covered by the anchor, at the truth's rule and severity; + // evidence it does not name (x.tsx:11) stays unscored. + const expected = [ + { file: 'apps/b/src/y.tsx', line: 59, rule: 'generic', severity: 'breaking' as const }, + ]; + expect(predictedSites(report, aiPack, expected)).toEqual([ + { file: 'apps/b/src/y.tsx', line: 59, rule: 'generic', severity: 'breaking' }, + { file: 'packages/lib/hook.ts', line: 6, rule: 'tool-context', severity: 'breaking' }, + ]); }); it('counts a required move as moved, and a move check reports only a new major makes as explained', () => { diff --git a/packages/core/src/packs/tooling/pack-test.ts b/packages/core/src/packs/tooling/pack-test.ts index df7d7d2c..2b4992d3 100644 --- a/packages/core/src/packs/tooling/pack-test.ts +++ b/packages/core/src/packs/tooling/pack-test.ts @@ -141,8 +141,14 @@ const key = (s: { file: string; line: number }): string => `${s.file}:${s.line}` /** * The sites `check` shows for the package, from its plan: the findings worth acting on, * grouped under the rule `fix` would apply. Repository-relative, as ground truth is written. + * An `expected` site an anchored finding lists as evidence is shown too (see below). */ -export function predictedSites(report: CheckReport, pack: Pack): ScoredSite[] { +export function predictedSites( + report: CheckReport, + pack: Pack, + expected: ScoredSite[] = [], +): ScoredSite[] { + const wanted = new Map(expected.map((e) => [key(e), e])); const own = new Set([...pack.rules.map((r) => r.id), ...pack.behavior.map((b) => b.id)]); const sites = new Map(); for (const p of report.packages) { @@ -184,6 +190,14 @@ export function predictedSites(report: CheckReport, pack: Pack): ScoredSite[] { // One site, one rule: the more severe group wins, as the plan lists it first. if (!sites.has(key(site))) sites.set(key(site), site); } + // Evidence the ground truth names is covered: `check` reports the line under its anchor + // (TypeScript 6's `types` errors under one tsconfig finding, #136's option cascades), so + // it scores as the site the truth expects, at that site's rule and severity. Evidence the + // truth does not name stays unscored. + for (const k of evidence) { + const site = wanted.get(k); + if (site && !sites.has(k)) sites.set(k, { ...site }); + } } return [...sites.values()].sort((a, b) => a.file.localeCompare(b.file) || a.line - b.line); } @@ -350,7 +364,7 @@ async function scoreRepo( if (installed.length > 0 && !installed.includes(entry.from)) base.installedMismatch = `the lockfile has ${installed.join(', ')}, the ground truth says ${entry.from}`; const own = report.packages.filter((p) => p.name === pack.name); - const predicted = predictedSites(report, pack); + const predicted = predictedSites(report, pack, expected); return { ...base, ...companionOutcome(own, entry.with, (name) => dependsOn(project, dir, name, pack.name)), diff --git a/packages/core/src/packs/typescript/RESEARCH.md b/packages/core/src/packs/typescript/RESEARCH.md index 16664136..b1769097 100644 --- a/packages/core/src/packs/typescript/RESEARCH.md +++ b/packages/core/src/packs/typescript/RESEARCH.md @@ -217,6 +217,13 @@ The recall figure measures the original truth's convention (every error line a s the engine's (one site per edit). It is not a count of breaks `check` misses: every folded error is listed under its tsconfig finding. +**Scoring the folded evidence.** Since then `pack test` counts a ground-truth site that an +anchored finding lists as evidence as covered by that finding, at the site's own rule and +severity (`predictedSites`, as for #136's option cascades). The ground truth did not change. +Re-scored 2026-10-15: breaking recall **99% (1,379 of 1,399)**, 0 breaking false positives, +the same as before the fold. The 20 breaking misses are the second-round and same-change lines +listed under "Measured `pack test`". + ## Verified under UPT-48 `pnpm uptide pack test typescript --write` on main (`530f90c`, UPT-48's gate), with its own @@ -226,7 +233,7 @@ is unchanged (same digest as the candidate record). - Repositories: 7 public (gate: ≥2). - Real breaking sites: 1,399 expected (gate: ≥5). - Breaking false positives: 0 (gate: 0). Breaking precision 100%, recall 17% (240 of 1,399; the - folded `types` errors above). + folded `types` errors above), 99% (1,379) once the scorer counts folded evidence (above). - Behavior notes: precision 20%, recall 50% (2 TP, 8 FP, 2 FN), all `changed-defaults` anchors at `tsconfig*.json:1`. Notes do not count against a breaking-class gate. - Per-rule real-world true positives: `base-url` 4, `module-resolution-node10` 2, `target-es5` 1, diff --git a/packages/core/src/packs/typescript/verification.json b/packages/core/src/packs/typescript/verification.json index ecb2dc18..6382c7a4 100644 --- a/packages/core/src/packs/typescript/verification.json +++ b/packages/core/src/packs/typescript/verification.json @@ -4,10 +4,10 @@ "repos": 7, "breaking": { "precision": 1, - "recall": 0.1716, + "recall": 0.9857, "falsePositives": 0, - "truePositives": 240, - "falseNegatives": 1159 + "truePositives": 1379, + "falseNegatives": 20 }, "behavior": { "precision": 0.2,