Repository navigation
feat(packs): add a candidate ioredis 5 to 6 pack - #129
Merged
Merged
Conversation
This was referenced Oct 14, 2026
lucasbraum
added a commit
that referenced
this pull request
Oct 15, 2026
…UPT-52) (#143) Fixes [UPT-52](https://linear.app/uptide/issue/UPT-52/pack-scorer-counts-missed-behavior-note-sites-as-breaking). ## What and why `pack test` counted every missed expected site as a **breaking** false negative unless a *rule* with that id was `deprecated`. `BehaviorNote.severity` was never read, so a missed behavior-note site that only needs a look counted as a missed break. Found by the ioredis pack ([#129](#129)): "Breaking: 0 predicted, 0 FP, 6 FN", where four of the six misses are `client-defaults`, a note declared `severity: 'deprecated'`. This PR: - **Scores each expected site at the severity of the rule or note it belongs to** (`siteSeverity`): a rule's `severity`; a note's `severity`, or `breaking` when the note reports findings without one (what `check` reports for it); `behavior` for a note `check` never reports as a finding (`decision`, `test-follow-up`); `breaking` for `generic`. Breaking recall counts only misses whose severity is `breaking`. - **Reports behavior-note precision/recall on its own**: a `behavior notes (sites)` row in `pack test`, `behavior` in `--json`, and `behavior` in `verification.json`. - **Records the counts behind each rate** (`truePositives`, `falseNegatives`) in `verification.json`, so a zero denominator prints **`n/a` instead of 100%** in `pack test` and in the README packs table (`rateLabel`). Backward compatible: both new fields are optional, a record without them prints as before, and the runtime gate (`recordedStatus`) reads nothing new. - False negatives print their severity, as false positives already did. - Scoring moves into a pure `scoreTruth(pack, repos)`, with no change in behavior, so it can be tested without fetching repositories. - CONTRIBUTING gets **"Run pack test locally"**: corepack for Yarn only, `COREPACK_NPM_REGISTRY` for Yarn 4, per-pack cache dirs and eviction. `check` and `fix` are untouched: no file under `check/`, `fix/` or the adapters changes. ### One decision to review: notes declared `breaking` stay breaking A note's own `severity` decides, not the fact that it is a note. Two reasons: - vitest (verified) has 13 truth sites under `coverage-patterns`, a note declared `severity: 'breaking'` with `detect`. `check` shows those findings as breaking, so their false positives must keep counting against the verified gate. Treating every note site as non-breaking would hide them and change vitest's verified numbers. - So ioredis on #129 goes from **6 breaking FN to 2**, not 0: the two left are `handshake-auth` sites (`log.test.ts:20`, `:28`), and that note declares `severity: 'breaking'`. If the note should only need a look, that is a one-word change in the pack on #129, not a scorer change. ## How it was verified **Failing test first.** `packages/core/src/packs/tooling/behavior-score.test.ts` uses synthetic sites modeled on the ioredis ground truth (voiceai, open-foundry, dokunc, plus the FishingConditionsApp control). On main's scorer it reproduces the bug exactly: ``` FAIL behavior-note sites > count as breaking misses only when their note is declared breaking - "falseNegatives": 2, + "falseNegatives": 6, ``` All six of its tests failed before the change and pass after. `packages/cli/src/pack.test.ts` covers the new row, `n/a`, and the false-negative severity. **End to end on #129 (local only, nothing pushed).** #129's head, scored before and after cherry-picking this commit, on the same five public repositories: | ioredis | Breaking P / R (TP/FP/FN) | Behavior notes P / R (TP/FP/FN) | | --- | --- | --- | | #129 as is | 100%* / 0% (0/0/6) | not reported | | #129 + this PR | n/a / 0% (0/0/2) | 30% / 33% (3/7/6) | \* printed as 100% before this PR, with 0 predicted. <details><summary><code>pack test ioredis</code> with this PR</summary> ``` rule precision recall tp fp fn client-defaults 30% 43% 3 7 4 handshake-auth n/a 0% 0 0 2 overall (sites) 30% 33% 3 7 6 behavior notes (sites) 30% 33% 3 7 6 breaking (sites) n/a 0% 0 0 2 false positives (7) ferni-ai/voiceai@c59ed0982f96 src/memory/storage/redis-cache/redis-cache.ts:101 client-defaults (deprecated) ferni-ai/voiceai@c59ed0982f96 src/services/data/redis-pubsub.ts:140 client-defaults (deprecated) ferni-ai/voiceai@c59ed0982f96 src/services/platform/rate-limiter.ts:57 client-defaults (deprecated) ferni-ai/voiceai@c59ed0982f96 src/services/workflows/jobs/redis-job-queue.ts:92 client-defaults (deprecated) syzygyhack/open-foundry@73910018b5e8 packages/api/src/server.ts:138 client-defaults (deprecated) simondietschi/dokunc@83681ed0ccc5 apps/web/src/lib/redis.ts:54 client-defaults (deprecated) kazumadev619-dev/FishingConditionsApp@8ba3f0d24bc7 src/lib/cache/client.ts:13 client-defaults (deprecated) false negatives (6) ferni-ai/voiceai@c59ed0982f96 src/memory/storage/redis-cache/redis-cache.ts:98 client-defaults (deprecated) ferni-ai/voiceai@c59ed0982f96 src/services/workflows/jobs/redis-job-queue.ts:89 client-defaults (deprecated) ferni-ai/voiceai@c59ed0982f96 src/memory/vectors/embedding-cache.ts:134 client-defaults (deprecated) syzygyhack/open-foundry@73910018b5e8 packages/api/src/server.ts:136 client-defaults (deprecated) simondietschi/dokunc@83681ed0ccc5 apps/web/src/lib/log.test.ts:20 handshake-auth (breaking) simondietschi/dokunc@83681ed0ccc5 apps/web/src/lib/log.test.ts:28 handshake-auth (breaking) PASS ioredis: candidate ``` </details> **Re-score of every registered pack**, against main at `9ba005a` (includes [#139](#139)). Each pack was fetched once (scripts off), scored on main and on this branch, then its caches were evicted: | Pack | Status | Breaking P / R (TP/FP/FN), main @ 9ba005a | Breaking, this PR | Behavior notes, this PR | pack test | | --- | --- | --- | --- | --- | --- | | `ai` | verified → verified | 100% / 82% (9/0/2) | 100% / 82% (9/0/2) same | n/a / n/a (0/0/0) | pass → pass | | `lucide-react` | verified → verified | 100% / 100% (11/0/0) | 100% / 100% (11/0/0) same | n/a / n/a (0/0/0) | pass → pass | | `next` | verified → verified | 100% / 85% (11/0/2) | 100% / 85% (11/0/2) same | n/a / n/a (0/0/0) | pass → pass | | `react` | verified → verified | 100% / 69% (89/0/40) | 100% / 69% (89/0/40) same | n/a / n/a (0/0/0) | pass → pass | | `sentry-browser` | verified → verified | 100% / 100% (3/0/0) | 100% / 100% (3/0/0) same | n/a / n/a (0/0/0) | pass → pass | | `sentry-nextjs` | verified → verified | 100% / 100% (8/0/0) | 100% / 100% (8/0/0) same | n/a / n/a (0/0/0) | pass → pass | | `sentry-react` | verified → verified | 100% / 100% (2/0/0) | 100% / 100% (2/0/0) same | n/a / n/a (0/0/0) | pass → pass | | `stripe` | verified → verified | 100% / 90% (18/0/2) | 100% / 90% (18/0/2) same | n/a / n/a (0/0/0) | pass → pass | | `vitest` | verified → verified | 100% / 50% (16/0/16) | 100% / 50% (16/0/16) same | n/a / 0% (0/0/13) | pass → pass | | `zod` | verified → verified | 100% / 74% (59/0/21) | 100% / 74% (59/0/21) same | n/a / n/a (0/0/0) | pass → pass | Breaking precision, recall and false positives are **identical for every pack**, and so is each status. The only difference in the records is additive: `behavior` plus the `truePositives`/`falseNegatives` counts (commit 2). vitest is the one pack with behavior-note sites in its truth: 13 `coverage-patterns` sites, a note declared `breaking`. They stay breaking misses, as before, and are now also visible as behavior-note recall 0%. How the runs were made: Yarn 4 through `corepack enable yarn` (tldraw) and `COREPACK_NPM_REGISTRY`, as the new CONTRIBUTING section says. A first pass with a blanket `corepack enable` could not install the repositories that pin pnpm (wortkarte, saas-starter, both sentry-react repositories) on main and on this branch alike. I re-scored those three packs on all checkouts with the documented setup; the table is that complete run. `pnpm lint`, `pnpm typecheck` and `pnpm test` pass (core 1650 passed / 32 skipped, CLI 453 passed). `pnpm docs:packs --check` and `pnpm docs:links` pass. ## Checklist - [x] Tests for the change (a bug fix starts with a failing test) - [x] `pnpm lint && pnpm typecheck && pnpm test` pass - [x] README and docs updated if behavior changed (docs/packs.md "Scoring", CONTRIBUTING; the README table is generated and unchanged in content) - [x] Changeset added if a user would notice (patch) - [x] No new dependency, or it was agreed in an issue - [x] No private code, names or paths in fixtures, tests or docs (test sites are synthetic) - [x] The rules in CONTRIBUTING.md ("Rules that do not bend") still hold 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_011t9c3ZLsM9B3KiYGPFDyyK --- _Generated by [Claude Code](https://claude.ai/code/session_011t9c3ZLsM9B3KiYGPFDyyK)_ --------- Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com>
lucasbraum
added a commit
that referenced
this pull request
Oct 15, 2026
…nd-truth site (UPT-53) (#144) Fixes [UPT-53](https://linear.app/uptide/issue/UPT-53/score-added-only-lines-as-ground-truth-sites). **Stacked on [#143](#143 (UPT-52): review only the last commit; this PR retargets to `main` once that one merges. ## What and why docs/packs.md counted no pure insertion as a site. So an upgrade that only **adds** lines inside an existing call could not be written as ground truth. The typical example is an option added to a constructor: open-foundry pinned `protocol: 2` inside `new RedisClient(redisUrl, {` at `packages/api/src/server.ts:138`. A pack that reports the same constructor can never score a true positive there; its finding shows as a false positive. Found by the ioredis pack ([#129](#129)). This PR: - **Defines the site.** A pure insertion is a site when it adds to a call or object the upgrade otherwise left unchanged. The site is the start line of the innermost call (or `new`) whose parentheses hold the inserted lines, through object and array literals, or of the object literal when no call takes it. It is not a site when it falls between statements, inside a function body (even one passed to a call), or inside a call the upgrade also changed (those changed lines are already its site). - **`pnpm packs:truth` drafts these anchors**, for review like every other drafted site. The JSON draft gets `anchors: [{ file, line, text, within, inserted }]`; `changed` and `inserted` are as before. The diff reading moves from `scripts/packs-truth.ts` into `packages/core/src/packs/tooling/truth-diff.ts` (`diffLines`, `enclosingLiteral`, `insertionAnchors`), where it is unit-tested; the script's output for `changed` and `inserted` is unchanged. - **Docs**: docs/packs.md "Ground truth" (the anchor rule, what `packs:truth` prints) and CONTRIBUTING "Write a pack" step 5. Scoring needs no change: sites are file and line, so an anchored site scores like any other. No pack's ground truth changes here. Adding the anchors to ioredis's truth is #129's call. ## How it was verified **Failing test first.** `packages/core/src/packs/tooling/truth-diff.test.ts` is modeled on open-foundry `server.ts:135–145`, with synthetic code and the same diff shape: lines 135–137 rewritten (the truth's site at `:136`) plus a pure insertion after `:138`, inside the constructor call. Before this change there was no anchor (the module did not exist), and the prediction at `:138` scores as a false positive. After: the insertion is drafted at `:138`, and with that site in the truth the prediction is a true positive. Negative cases cover insertions between statements, into a callback body, after the closing `)`, and into a call whose lines also changed. Writing the tests also caught a real bug before it shipped: an argument added after a trailing comma was missed. **On the real repositories** (read only, scripts off), `pnpm packs:truth ioredis …` on this branch drafts: | Repository | Anchor | Inserted | ioredis pack today | | --- | --- | --- | --- | | syzygyhack/open-foundry | `packages/api/src/server.ts:138` `redisClient = new RedisClient(redisUrl, {` | `protocol: 2,` | false positive at `:138` | | ferni-ai/voiceai | `src/services/data/redis-pubsub.ts:140` `this.publisher = new Redis(redisUrl, {` | `protocol: 2,` | false positive at `:140` | | ferni-ai/voiceai | `src/services/platform/rate-limiter.ts:57` `redisClient = new Redis(redisUrl, {` | `protocol: 2,` | false positive at `:57` | | simondietschi/dokunc | `apps/web/src/lib/redis.ts:54` `const client = new Redis(url, {` | `retryStrategy: reconnectDelay,` | false positive at `:54` | These are exactly the four "real constructions the upgrade changed, but by pure insertion" that #129's RESEARCH.md lists as unscorable. voiceai also drafts one object anchor (`src/utils/__tests__/error-scrub.test.ts:44`, `return {` gaining `status: 'ok'`) from the @sentry/node bump in the same commit. The reviewer drops it, as the drafting flow intends. **Re-score of every registered pack.** Base: this branch's parent (#143), itself on main at `9ba005a`. Same fetch, scripts off, caches evicted per pack: | Pack | Status | Breaking P / R (TP/FP/FN), #143 | Breaking, this PR | Behavior notes, this PR | pack test | | --- | --- | --- | --- | --- | --- | | `ai` | verified → verified | 100% / 82% (9/0/2) | 100% / 82% (9/0/2) same | n/a / n/a (0/0/0) | pass → pass | | `lucide-react` | verified → verified | 100% / 100% (11/0/0) | 100% / 100% (11/0/0) same | n/a / n/a (0/0/0) | pass → pass | | `next` | verified → verified | 100% / 85% (11/0/2) | 100% / 85% (11/0/2) same | n/a / n/a (0/0/0) | pass → pass | | `react` | verified → verified | 100% / 69% (89/0/40) | 100% / 69% (89/0/40) same | n/a / n/a (0/0/0) | pass → pass | | `sentry-browser` | verified → verified | 100% / 100% (3/0/0) | 100% / 100% (3/0/0) same | n/a / n/a (0/0/0) | pass → pass | | `sentry-nextjs` | verified → verified | 100% / 100% (8/0/0) | 100% / 100% (8/0/0) same | n/a / n/a (0/0/0) | pass → pass | | `sentry-react` | verified → verified | 100% / 100% (2/0/0) | 100% / 100% (2/0/0) same | n/a / n/a (0/0/0) | pass → pass | | `stripe` | verified → verified | 100% / 90% (18/0/2) | 100% / 90% (18/0/2) same | n/a / n/a (0/0/0) | pass → pass | | `vitest` | verified → verified | 100% / 50% (16/0/16) | 100% / 50% (16/0/16) same | n/a / 0% (0/0/13) | pass → pass | | `zod` | verified → verified | 100% / 74% (59/0/21) | 100% / 74% (59/0/21) same | n/a / n/a (0/0/0) | pass → pass | Identical for every pack: this PR changes drafting and docs, not scoring. `pack test --write` changed no record. `pnpm lint`, `pnpm typecheck` and `pnpm test` pass (core 1656 passed / 32 skipped, CLI 453 passed). `pnpm docs:links` passes. ## Checklist - [x] Tests for the change (a bug fix starts with a failing test) - [x] `pnpm lint && pnpm typecheck && pnpm test` pass - [x] README and docs updated if behavior changed (docs/packs.md, CONTRIBUTING) - [x] Changeset added if a user would notice (patch) - [x] No new dependency, or it was agreed in an issue (uses core's existing `ts-morph`) - [x] No private code, names or paths in fixtures, tests or docs (test code is synthetic; the table cites public repositories already in #129) - [x] The rules in CONTRIBUTING.md ("Rules that do not bend") still hold 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_011t9c3ZLsM9B3KiYGPFDyyK --- _Generated by [Claude Code](https://claude.ai/code/session_011t9c3ZLsM9B3KiYGPFDyyK)_ --------- Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com>
lucasbraum
marked this pull request as ready for review
October 15, 2026 01:46
lucasbraum
force-pushed
the
claude/dreamy-tesla-7g3b9g
branch
from
October 15, 2026 02:09
ffd6ad2 to
e16104b
Compare
Rules, notes, fixtures and ground truth drafted from the ioredis 6 sources and five public upgrades, committed before the pack is scored against them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017BYqQRCpXk6jz9vcCnx21H Signed-off-by: Claude <noreply@anthropic.com>
pack test against the five public upgrades: no breaking false positive and no breaking prediction, since none of them has a breaking site. Records the candidate status, the research ledger, the blocked queue entry and the evidence and decision tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017BYqQRCpXk6jz9vcCnx21H Signed-off-by: Claude <noreply@anthropic.com>
…everities Rebased on main (#143, #144). The ground truth gains the four ioredis insertion anchors packs:truth now drafts (protocol: 2 and retryStrategy added inside unchanged constructor calls); no earlier site moved. Behavior notes 70% precision, 54% recall; still a candidate with no breaking false positive and no breaking prediction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017BYqQRCpXk6jz9vcCnx21H Signed-off-by: Claude <noreply@anthropic.com>
lucasbraum
force-pushed
the
claude/dreamy-tesla-7g3b9g
branch
from
October 15, 2026 02:47
e16104b to
30579da
Compare
Merged
This was referenced Oct 15, 2026
lucasbraum
added a commit
that referenced
this pull request
Oct 15, 2026
…8) (#151) ## Pack `redis` (node-redis), `>=4 <6` → `>=5 <7`, for [UPT-58](https://linear.app/uptide/issue/UPT-58/pack-redis-node-redis-4-5-and-5-6). ioredis is a separate pack (#129). **One pack, two rule sets.** 4 → 5 and 5 → 6 are separate rule sets, and an upgrade gets the sets of the majors it crosses: 4 → 6 gets both, 5 → 6 never sees a 4 → 5 site. It is one pack and not two because pack lookup is one pack per package, and because 4.7 → 6.x in one step is the most common real upgrade (Dependabot offers it to every 4.x user; 3 of the 6 upgrades found did it). The gate is `appliesTo` in `index.ts`: detection, `ruleOf` and notes check the crossed majors, and `transform` refuses a 4 → 5 rewrite on code already on 5. **Status: candidate.** `verification.json` records candidate; since #154 pack test accepts a record that claims less than the run measures. ## Sources Pinned at node-redis `b64bacb`, read 2026-10-14: `docs/v4-to-v5.md`, `docs/v5-to-v6.md`, `docs/RESP.md`; release notes `redis@5.0.0` and `redis@6.0.0`; published declarations of @redis/client 1.6.1 / 5.12.1 / 6.3.0, @redis/search and @redis/time-series. Release group: the five `@redis/*` packages `redis` pins at its own version. `@redis/graph` is gone in 5. ## Rules and behavior notes **4 → 5** - `legacy-mode`: compiler + detect; assisted (`client.legacy()`). - `command-options`: `commandOptions()`, `isolated`, `executeIsolated`; compiler + detect; assisted (`withCommandOptions`, `createClientPool`). - `scan-iterators`: detect only; **rewrite** to a nested loop, declined when the body `break`s/`continue`s the loop. - `keep-alive`: compiler + detect; **rewrite** `keepAlive: N` → `keepAlive: true, keepAliveInitialDelay: N`. - `scan-cursor`: string cursors; compiler + detect (`cursor !== 0` never ends in JS); assisted. - `number-replies`: boolean → number replies; compiler; assisted. - `renamed-exports`: 17 enums → constants, `SearchOptions` → `FtSearchOptions`; **rewrite**: import under the old local name, declined when the file uses a member whose name or value changed (compared member by member in the declarations) or uses the name as a type. - `deep-imports`: `@redis/*/dist` internals; compiler only; assisted. - `graph-removed`: compiler + detect; assisted, a decision for the owner. - `quit-disconnect` (deprecated): **rewrite** to `close()`/`destroy()`, only where the result is unused. - `set-expiration` (deprecated): `EX`/`PX`/… → `expiration`; assisted. Not in the guide; from the declarations. - Notes: `search-dialect-2` (finding at query calls without `DIALECT`), `pipeline-discards-unwritten`, `node-18` (decisions). **5 → 6** - `client-type`: `RedisClientType` is invariant in RESP and clients default to RESP3, so a client typed `RedisClientType` / `ReturnType<typeof createClient>` breaks. **Not in the guide**; seen in all four real upgrades that reached 6; assisted. - `unstable-resp3`: compiler + detect; **rewrite**: delete the option. - `sentinel-node-command-options`: compiler; assisted. - Notes: `client-defaults` (finding + decision at each inline-options client that leaves RESP, command timeout, keep-alive delay or maintNotifications to the 6 default; the instructions pin the 5 defaults per client, as the ioredis pack does), `node-20`, `plain-object-replies`. ## Ground-truth repositories Drafted with `pnpm packs:truth` (each repository's own compiler and merged upgrade), reviewed by hand, and **committed in ffb4f90 before the first score (71a2172 before the rebases on main)**. | Repository | Hop | Pinned | Upgrade | Sites | |---|---|---|---|---| | deanmoses/tacocat-gallery-sam (`app`) | 4.6.12 → 5.10.0 | `34e1176` | PR #83 | 4 | | swoofer/mcp-coordinator | 5.12.1 → 6.2.0 | `8b337f0` | PR #251 | 2 | | deanmoses/tacocat-gallery-sam (`app`) | 5.12.1 → 6.2.1 | `30724e2` | PR #133 | 2 | | UNN-Devotek/FCM-Fallout-Chat-Mod (`backend`) | 4.7.1 → 6.0.0 | `2801904` | PR #125 | 9 | | tearingItUp786/blog | 4.7.0 → 6.0.0 | `bb88f3a` | #223 + `092ea34` | 5 | **One change after the first score:** FalkorDB/falkordb-ts (4.7.1 → 5.10.0, 20 sites, the richest 4 → 5 upgrade found) was in the committed truth, but the first run could not score it at all. Its `package.json` declares `workspaces: ["./packages/*"]` matching no directory, and `check` refuses the repository. It is removed (437e129), and none of its sites was ever scored. Also screened out: stamp (needs Node ≥ 24.15), semaphore-chat (lockfile outside the project directory), about 30 bump-only upgrades. Details in RESEARCH.md. ## `uptide pack test` output ``` uptide pack test redis · verified, recorded candidate fixtures 12 cases, 27 sites · pass ground truth deanmoses/tacocat-gallery-sam@34e11760f6e7 redis 4.6.12 → 5.10.0 predicted 4, expected 4, false positives 0, false negatives 0 swoofer/mcp-coordinator@8b337f0a081d redis 5.12.1 → 6.2.0 predicted 3, expected 2, false positives 1, false negatives 0 deanmoses/tacocat-gallery-sam@30724e21efd6 redis 5.12.1 → 6.2.1 predicted 2, expected 2, false positives 0, false negatives 0 UNN-Devotek/FCM-Fallout-Chat-Mod@280190489fac redis 4.7.1 → 6.0.0 predicted 5, expected 9, false positives 0, false negatives 4 tearingItUp786/blog@bb88f3afc40d redis 4.7.0 → 6.0.0 predicted 1, expected 5, false positives 1, false negatives 5 rule precision recall tp fp fn client-defaults 0% 0% 0 4 2 client-type 100% 33% 2 0 4 generic 50% 100% 2 2 0 renamed-exports 100% 100% 2 0 0 scan-cursor 100% 30% 3 0 7 overall (sites) 87% 59% 13 2 9 behavior notes (sites) 50% 50% 2 2 2 breaking (sites) 100% 61% 11 0 7 false positives (2) swoofer/mcp-coordinator@8b337f0a081d src/infra/redis.ts:50 client-defaults (deprecated) tearingItUp786/blog@bb88f3afc40d app/utils/redis.server.ts:15 client-defaults (deprecated) ✗ verification.json does not match this run: `uptide pack test redis --write` records it FAIL redis ``` **Why the record says candidate.** Today's gate (≥ 2 repos, 0 breaking false positives) computes `verified`, and the stricter UPT-48 gate is not on main yet. The evidence is thin exactly where it matters most. The core 4 → 5 API changes (`command-options`, `legacy-mode`, `scan-iterators`, `number-replies`, `keep-alive`) have **no real site** in the ground truth. The 4 → 5 rule set rests on one 4 → 5 repository (4 sites) plus the scan-cursor sites of the two 4 → 6 repositories. Three rewrites (`scan-iterators`, `keep-alive`, `unstable-resp3`) never met a real site, and under UPT-48 they would run as assisted. So I have not run `--write`, and CI's Pack test will flag the record as stale until this is decided: rebase on UPT-48 and re-score, or find a scorable 4 → 5 upgrade with those changes. **The 7 breaking false negatives are all outside the rules' reach:** - blog (5): its files reach the client through a local module and a local `RedisScanClient` type, so `check` reports no compile error there. - FCM and tacocat 5 → 6 (4 client-type): the repositories' own `tsc` reports TS2322, `check` doesn't. Those lines already had a different TS2322 at the installed version, so they were most likely subtracted as pre-existing. Not confirmed in the engine. **Behavior:** the two `client-defaults` false positives are clients whose repositories kept the RESP3 default without pinning it. ## Engine follow-ups found here (not in this PR) - `check` refuses a repository whose `workspaces` patterns match nothing (FalkorDB). - Compile errors in files that use the client without importing the package go unreported (blog). - Pre-existing errors seem to be subtracted by location, which hides a new error at the same line (FCM, tacocat). ## Checklist - [x] Every rule and note comes from a source listed in `meta.sources`. `client-type` and `set-expiration` come from the published declarations, which are listed. - [x] Every rule that rewrites or detects has a fixture case, with a site it must leave alone (`keep`) - [x] Ground truth: public repositories at the commit before their upgrade; expected findings read from their own upgrade commit and the compiler, never copied from Uptide's output - [x] At least two public repositories for `verified`; otherwise the pack ships as a candidate. Five scored; kept as a candidate on purpose, see above. - [ ] `pnpm uptide pack test redis` passes and its output is pasted above; `verification.json` written with `--write`. Intentionally not written yet. - [x] `pnpm lint && pnpm typecheck` pass; `vitest run src/packs` in core (259 tests) passes. The full `pnpm test` was not run locally. - [x] Changeset added - [x] No private code, names or paths anywhere: fixtures are synthetic, ground truth is public 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Lucas Braum <lucasbraum@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
lucasbraum
pushed a commit
that referenced
this pull request
Oct 15, 2026
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## uptide@0.13.0 ### Minor Changes - [#134](#134) [`b795705`](b795705) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Packs can name alternate leads: packages a repository declares instead of the pack's own (`react-router-dom` for `react-router`). `check` and `fix` asked for either name use the pack; where only the alternate is declared, the lead is analyzed at the version installed through it, pnpm stores included, and `fix` declares it beside the alternate at the target before upgrading both. - [#149](#149) [`3303118`](3303118) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Add a candidate express 4 to 5 migration pack. Route strings that path-to-regexp 8 rejects or reads differently are found on calls of an Express app or router (including lists registered in a loop); `req.params` values widened to `string | string[]`, `req.param`, `res.sendfile`, `express.static.mime` and `express.query` are claimed from the compiler; `res.redirect(url, status)`, `app.param(fn)`, `res.redirect('back')` and `res.send(<number>)` are found in the source; runtime defaults (async errors, `req.body`, the query parser, `urlencoded`, dotfiles, Node 18) go to decisions. No rule rewrites. With the pack in use, `check` no longer adds its text-matched express 5 runtime findings. - [#148](#148) [`5143c46`](5143c46) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Add a candidate joi 17 to 18 migration pack. `new ValidationError(message, {}, original)`, which stops compiling for code written against joi 17.0–17.6, is rewritten to pass `[]`; the Node.js 20 requirement and the top-level domains joi 18 no longer accepts in `email()`, `domain()` and `uri()` are reported as decisions. Scored against two public repositories, one of them a negative control. - [#131](#131) [`413d3da`](413d3da) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Add a verified lucide-react 0.x to 1.x migration pack. Brand icons removed in 1.x are found at their imports and every use, with a guide whose replacement choice goes to the owner; icon names dropped during 0.x are renamed at the import or member; deep `dist/` imports, untyped icon lookups, the `absoluteStrokeWidth` and `IconNode` deprecations, and the `aria-hidden` default for repositories before 0.489.0 are reported. Lucide's lockstep packages move together and the consumer's build is required. Verified against three public repositories, one of them a negative control. - [#132](#132) [`107c107`](107c107) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Add a verified React Router 6 → 7 migration pack, which also runs for applications that declare only `react-router-dom`. It moves `react-router-dom` imports to `react-router`. It reports, with a guide for the agent, the rest of the upgrade: the removed `future` prop and router option, `react-router-dom/server`, lowercase `formMethod` comparisons and `json`/`defer` calls (found by syntax where the compiler is blind), the other removed exports, and `generatePath` calls that pass `null` for a required parameter. The six v6 future flags that become defaults are reported as decisions where the repository's router setup has not enabled them. React Router 7 → 8 is a separate upgrade. - [#142](#142) [`9c57934`](9c57934) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `check` and `fix` plan what the target version requires: a package the target, or anything moving with it, declares as a peer or dependency at a range the installed copy rejects moves to the lowest release of its major that satisfies it (openai 7 moves zod 3.24.2 to 3.25.0), shown apart from release-group companions ("moved because openai 7.4.0 requires zod ^3.25 || ^4.0"). A move only a new major makes is never made: it is reported as a blocker with the exact range in `check`, the PR body and `pack test`. The npm install scope guard no longer lets a declared root dependency change because a moved package peers on it. - [#128](#128) [`06f11ab`](06f11ab) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Add verified Sentry JavaScript SDK 10 to 11 migration packs for `@sentry/nextjs` and `@sentry/browser`, with the SDK's lockstep release group, a required build for Next.js, and inventory-scoped decisions for data collection, span streaming, sampling and platform minimums. Mechanical rules move `withSentryConfig` to `@sentry/nextjs/config` and the removed build options under `webpack`, remove `enableLogs`/`enableMetrics: true`, and rename exports, integration names and `httpIntegration` options; privacy, sampling, profiling and OpenTelemetry choices stay assisted and are reported for the owner. Fix `uptide pack new` dropping registry entries long enough to be wrapped. - [#155](#155) [`922c5ab`](922c5ab) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide pack test` applies a stricter verified gate: at least five expected breaking sites across the public ground-truth repositories, besides two repositories and no breaking false positive. Upgrades the compiler cannot see are gated on their behavior notes instead. Mechanical rules no real site exercises are recorded as unexercised, and `check` and `fix` run their sites as assisted. `@sentry/browser` and `@sentry/react` become candidates (fewer than five real breaking sites). A pack can lower the minimum only with a written reason in its RESEARCH.md. The README packs table shows each pack's breaking-site count. The output shows the gate's class, site counts and every reason a pack is a candidate. - [#147](#147) [`0b88170`](0b88170) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `check` and `fix` remove a deprecated `@types` stub instead of upgrading to it. When the release the upgrade would move `@types/<pkg>` to is deprecated on npm, and `<pkg>`'s target declares its own types (`types`, `typings` or a `types` export), the plan removes it: "Removed: @types/ioredis, ioredis 5 ships its own types", in `check` and in the PR body. `check` now diffs such a package against its own target types instead of failing on a stub with no declarations, and `fix` removes the entry in the same install, with the install scope guard allowing exactly that removal. - [#150](#150) [`4c3212b`](4c3212b) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `check typescript` judges a TypeScript 5 to 6 upgrade with both compilers: every project of the workspace is compiled with the installed TypeScript and with the target's, and what is new at the target is a finding; the bundled compiler never stands in, and a target without a JavaScript API (TypeScript 7) is reported as not compiled. A candidate typescript 5 to 6 pack reports the tsconfig options 6.0 deprecates where they are set, the rootDir default, module-keyword namespaces and import assertions. A peer that rejects the target (typescript-eslint on TypeScript 6) is now reported as a peer conflict even when it is not grouped with the package. - [#137](#137) [`21894d7`](21894d7) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Add `verify.build` to uptide.yml: the package.json script a migration pack's required build runs. Unset, Uptide runs `build:ci`, then `build-ci`, then `build`. When a build fails before the framework's build starts because it needs an environment variable or a database (`prisma db push` without `DATABASE_URL`), the pull request and the report say so ("build needs DATABASE_URL; set verify.build") instead of a bare "build failed". `uptide init` writes the setting as a comment. ### Patch Changes - [#141](#141) [`9debb6d`](9debb6d) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `check` no longer reports a pre-existing type error as a breaking site when the upgrade only changes its wording, such as a type the message prints in full. An error at the same place with a different reason is still reported. - [#129](#129) [`e796e4c`](e796e4c) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Add a candidate ioredis 5 to 6 migration pack: client constructions that inherit the RESP3, reconnect and keep-alive defaults ioredis 6 changed, compiler-blind sites (`mode === 'subscriber'`, commands in a `connect` listener with the offline queue off, AUTH recognized by command name), dependents that bundle ioredis 5, instrumentation that stops patching ioredis 6, and Node 20. It is scored against five public upgrades and stays a candidate: none of them has a breaking site to verify it with. - [#139](#139) [`9ba005a`](9ba005a) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Allow local check and fix smoke tests of registered candidate packs with --allow-candidate, with an explicit unverified banner. Reject candidate opt-in in the Action and with publication flags, and prevent stored candidate runs from being published. Label pack-claimed assisted sites by rule id even with --no-llm. - [#157](#157) [`f3af6a1`](f3af6a1) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `check` no longer reports a blocker for a direct dependency whose plain dependency range rejects the target (`@azure/ai-projects` on `openai ^6`, verdaccio pinning express 4): the package manager gives it its own nested copy, so it is listed as staying, with the new major that would accept the target when there is one. `pack test` counts a `with` move that `check` leaves and explains (a blocker, a new-major stay or a peer conflict) as explained, and one of a package that does not depend on the lead as the repository's choice; neither fails the run. - [#130](#130) [`5ced450`](5ced450) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Add a candidate openai 4 to 7 migration pack: named path parameters, `del` → `delete`, promoted beta chat helpers, renamed runner events, removed entry points and request-option overloads are rewritten where unambiguous; tool-call unions, `httpAgent`, `fileFromPath` and the other removals get agent guides; `APIError.headers`, Web `Response` bodies and double-encoded path ids are detected even in code typed `any`; Node 22, zod, fetch and client-subclass decisions carry repository evidence. It stays a candidate (the dependency is still checked and fixed as generic) until two engine gaps its ground truth exposed are closed. - [#136](#136) [`578a042`](578a042) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Fold implicit-any errors (TS7006, TS7031, TS7019) inside an option the target removed into the pack finding on that option, as downstream evidence: the callbacks lost the types the option gave them, and the same edit resolves them. They no longer stand as separate breaking sites. - [#143](#143) [`883e587`](883e587) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide pack test` scores each expected site at the severity of the rule or behavior note it belongs to, so a missed note that only needs a look, or a decision, no longer counts as a missed break. Behavior-note sites get their own precision and recall in the output and in `verification.json` (`behavior`, with the counts behind each rate), false negatives show their severity, and a rate with nothing to measure prints `n/a` instead of 100%, in `pack test` and in the README's packs table. - [#154](#154) [`7aa3d58`](7aa3d58) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide pack test` fails a pack's `verification.json` only when it claims more than the run measures (recorded verified, measured candidate). A pack recorded as candidate on purpose passes, with `recorded candidate by choice; measures verified`, and a record whose numbers differ gets a note instead of a failure. A `generic` ground-truth site now counts as a true positive for the rule that claims it. - [#152](#152) [`ca31aee`](ca31aee) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide pack test` takes several packs: `uptide pack test zod ai`. - [#159](#159) [`1e157fd`](1e157fd) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide pack test --json` times each ground-truth repository: `timing` gives the fetch, install and check wall time and the process's peak memory. - [#144](#144) [`0881185`](0881185) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Ground truth can name a site for an upgrade that only adds lines inside an existing call, such as an option added to a constructor: the site is the call's start line, so a pack finding there scores as a true positive instead of a false positive. `pnpm packs:truth` drafts these anchors from insert-only hunks for review, and docs/packs.md and CONTRIBUTING describe the rule. - [#156](#156) [`9039e3c`](9039e3c) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Public ground truth now exercises three rules that ran as assisted: zod `error-params`, stripe `api-version-unpinned` and next `stable-cache-exports`. Their rewrites run mechanically again. All three packs stay verified. - [#138](#138) [`0db5640`](0db5640) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Respect repository tsconfig file selection, recursive project references, and workspace project options when checking dependency upgrades. Compiler diagnostics outside every TypeScript project no longer produce breaking findings; pack rules and build-configuration findings retain their own scope. Repositories without a tsconfig retain importer-based compilation, with that fallback stated in the report. Anchor missing members supplied by dependency augmentations at their setup import, retaining affected calls as evidence instead of counting each call as a separate breaking edit. - [#151](#151) [`9ddaaff`](9ddaaff) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Add a candidate redis (node-redis) 4 to 5 and 5 to 6 migration pack, one rule set per major: an upgrade gets the rules of the majors it crosses, so 4 to 6 gets both. 4 to 5 covers `legacyMode`, `commandOptions` and isolation, scan iterators that yield batches, string SCAN cursors, boolean replies that became numbers, enums that became constants, `keepAlive`, `@redis/graph` and internal `dist` imports; 5 to 6 covers the RESP3 client type, `unstableResp3`, and the client defaults 6 changes (RESP3, command timeout, keep-alive delay, maintenance notifications) with a decision per client. Scored against five public upgrades; it stays a candidate until the stricter verified gate lands, since the core 4 to 5 API changes have no real site yet. - [#135](#135) [`42a3b54`](42a3b54) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Move only packages released with the upgraded one. A pack's declared release group now bounds the lead's npm scope, and without one a same-scope package moves only when a pin, a peer or a release at the target relates it to the group. Packages of the scope that are versioned on their own (`@sentry/vite-plugin` 5.x, `@sentry/cli`, the old `@sentry/utils` 8.x) are reported as "stays at X" in `check`, the pull request and `pack test` instead of making `fix` refuse the whole upgrade. Registers the verified `@sentry/react` 10 to 11 pack. - [#146](#146) [`09b8682`](09b8682) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Report Stripe clients pinned to different API versions at the top of `check` and of the PR body, as a decision for the owner: every `new Stripe(...)` in TypeScript and JavaScript sources, scripts included, with the version it speaks after the upgrade (an unpinned client speaks the SDK default). The PR's decisions list, for each client behind the target, the changelog entries it would cross for the resources it calls. No pin is rewritten. - [#140](#140) [`bc428d1`](bc428d1) Thanks [@lucasbraum](https://github.com/lucasbraum)! - Keep migration PR bodies within their size budget while preserving summaries, representative diffs, decisions, review warnings and verification ahead of changelog and transitive detail. Keep every evidenced Stripe change visible and uncapped; group and cap only other changes to used resources, link the full changelog, and identify omitted text in the retained report. - [#145](#145) [`3c8902d`](3c8902d) Thanks [@lucasbraum](https://github.com/lucasbraum)! - The zod pack migrates `ZodError#errors` to `.issues` by rule: reads (`error.errors[0].message`, `error?.errors`) and destructuring (`const { errors } = error` becomes `const { issues: errors } = error`) where the type checker resolves the receiver to `ZodError`, and reads off an untyped catch binding of a zod parse. Writes are left to the agent. - [#127](#127) [`c4ead4f`](c4ead4f) Thanks [@lucasbraum](https://github.com/lucasbraum)! - zod: `check` now reports `ZodError#formErrors` and `#errors` read off an untyped catch binding of a zod `parse`/`parseAsync` (`catch (error: any) { return error.formErrors.fieldErrors }`), which zod 4 removed and the compiler cannot see. Files that only import a schema from the repository's own module (`@/lib/zod`) are covered too. `fix` rewrites `<err>.formErrors.fieldErrors` and `<err>.formErrors.formErrors` to `z.flattenError(<err>)`, importing `z` where it is missing; every other read goes to the agent. Co-authored-by: uptide-release[bot] <338861287+uptide-release[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pack
ioredis>=5 <6→>=6 <7(queue entryioredis, nowblocked). Candidate, not verified. No breaking false positive on five public merged upgrades. But none of them has a compiler diagnostic or a breaking site in application code; the only breaking expected sites are two lines of test data, which the pack doesn't predict. So the gate (≥1 predicted breaking finding) can't be met honestly. Evidence:packages/core/src/packs/ioredis/RESEARCH.md.Rebased on main
3c8902d(2026-10-15). The only conflicts wereregistry.ts(both entries kept, alphabetical) and the generated README table (regenerated). The rebase also picks up #145's re-recorded react-routerverification.json; the Packs workflow had been red on main since #143 without it.After the rebase, #143 and #144 changed scoring and the ground-truth rule:
pnpm packs:truthon all five repositories: still no compiler diagnostic.$commentand in RESEARCH.md.Sources
All read 2026-10-14 and pinned:
e859962)v6.0.0ioredis5.11.1/6.0.0 and@ioredis/commands1.10.0/2.0.0 tarballs, diffedv6 removes no command (373 → 382 methods) and no public export. Reply shapes and types are unchanged for default (
"legacy") clients, and so are event names andexec()types. Breaking changes: Node ≥20, RESP3 by default, plus newretryStrategy/keepAlivedefaults.Rules and behavior notes
Assisted rules (no rewrite):
bundled-ioredis: TS2322/2345/2739-41 namingRedis<"legacy">not assignable, i.e. a dependent that installs its own ioredis 5. The guide gives options (a)–(d), so the agent's choice lands in Decisions for you. Never a cast.duplicate-signature: TS2416/2430 on'duplicate'overrides.Finding notes (syntactic, binding-aware):
client-defaults(deprecated, also a decision): constructions that leaveprotocol/retryStrategy/keepAliveat the changed defaults. Tests are skipped.subscriber-mode(breaking):mode === 'subscriber'.offline-queue-connect(breaking): aconnectlistener sending a command whenenableOfflineQueue: false.handshake-auth(breaking): AUTH recognized by command name. v6 sends credentials insideHELLO 3 AUTH.Decision notes (with
applies()evidence):node-20bundled-copies: bullmq <6, bull, @hocuspocus/extension-redis, @chat-adapter/state-ioredis, cache-manager-ioredis-yetinstrumentation: OTelinstrumentation-ioredis<0.70 /auto-instrumentations-node<0.80stale-types:@types/ioredisstring-prototype: target exactly 6.0.0also-reviewNot declared:
requiresBuild: no case of a build failing where tsc passes.Ground-truth repositories
c59ed09c6838d5(#703)client-defaults(protocol: 2 pins, 2 of them insertion anchors)7391001a466e0dclient-defaults(1 anchor)83681edc475390(#15)client-defaults(v5 retryStrategy, 1 anchor), 2handshake-auth14bd05b80135d2(#3409)8ba3f0dcdecc03(#235)Screened out: elie222/inbox-zero#3821. Its TS2322 (two copies via bullmq 5) is a
packs:truthlinking artifact; a real scripts-off install compiles it. Its upgrade also moved bullmq 5 → 6.uptide pack testoutputThe dokunc "blocked" line is the engine's own report (#142) of the bundled hocuspocus copy that
bundled-copiesdescribes; the repository upgraded anyway, keeping both copies.Remaining false positives:
:101/:92against truth:98/:89: the anchor is at the first line of the rewritten run.client.ts:13: a client deliberately left on RESP3.Remaining false negatives:
embedding-cache.ts:134: the module name is hidden from static scans.:98/:89and open-foundry:136: anchor offsets.log.test.ts:20/28:handshake-auth, test data.Engine gaps still open (RESEARCH.md):
verified.@types/ioredis).withcan't model a required dependent major.packs:truthlinks per workspace, and--directoryfails with a root lockfile.Gaps 2 and 3 of the original list were resolved by #143 and #144.
Checklist
meta.sourceskeep). The two compiler-claim rules are unit-tested instead, because amessage:marker can't contain".verified; otherwise the pack ships as a candidate (5 repos, still a candidate: no predicted breaking site)pnpm uptide pack test ioredispasses and its output is pasted above;verification.jsonwritten with--writepnpm lint && pnpm typecheck && pnpm testpass: core 1727 passed / 32 network-skipped, CLI 454 passed; lint at baseline;docs:packs --checkcurrent. All 10 CI checks pass on30579da, Pack test included.No third-party repository was pushed to, commented on or modified. Every install ran with scripts off, and no bare
npxwas used.🤖 Generated with Claude Code
https://claude.ai/code/session_017BYqQRCpXk6jz9vcCnx21H