Repository navigation
feat(packs): vitest 4 → 5 candidate pack - #57
Closed
lucasbraum wants to merge 1 commit into
Closed
lucasbraum wants to merge 1 commit into
lucasbraum wants to merge 1 commit into
Conversation
A candidate pack for Vitest 4 → 5, scored against votingworks/vxsuite and vitorvasc/opentelemetry-ecosystem-explorer at the commit before their own upgrade. Rules: matcher-types, jest-dom-matchers, bench-api, removed-entrypoints (rewritten for the four with a drop-in path), removed-entrypoints-manual, sequential-removed; behavior notes for what the compiler cannot see, and coverage-patterns found from source. One false positive among breaking findings keeps it a candidate. The README packs table is regenerated, and one line of the documentation list is joined so the README stays within its 150-line test. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YNCujWu9MXTm48xoYVKHJC Signed-off-by: Claude <noreply@anthropic.com>
lucasbraum
added a commit
that referenced
this pull request
Oct 10, 2026
…n a repository augmentation (#60) ## What and why **1. Shared root through a setup or declaration file.** In the vitest pack test (#57, ground truth votingworks/vxsuite), `check` reported a breaking finding at `libs/image-utils/src/jest_pdf_snapshot.ts:42` (TS2551, `toMatchImageSnapshot` missing on `Assertion`) beside the declaration at `libs/image-utils/src/vitest_setup.ts:10` the pack already reports. The tracer did not link them: #51 handles parameters only, and this is a global augmentation. Two things turned out to be true on inspection: - The failing member is not a parameter and not declared in the setup file either: `toMatchImageSnapshot` comes from `jest-image-snapshot`'s own types, which declare it on the global `jest.Matchers`. The setup file augments that same interface (`declare global { namespace jest { interface Matchers<R> { toMatchImage…; toMatchPdfSnapshot… } } }`) and registers all three matchers with `expect.extend`. Vitest 5 stops reading the global namespace, so every matcher on it has to move to `Matchers<R, T>` of "vitest" together, and the repository's augmentation is where that edit goes. The cause tracer (`adapters/typescript/cause.ts`, `augmentationCause`) now handles missing-member diagnostics (TS2339, TS2551): the member is resolved in the baseline program, where it still exists, and - when its declaration is in a repository file, the diagnostic is anchored there: a `declare global` block at the block, a `declare module "x"` augmentation at the interface inside it, any other repository declaration at itself; - when its declaration is a package's (or another fixture's) on an interface the repository augments, it is anchored at the repository's augmentation of that interface, this workspace's first, with the package named in the reason; - one site is enough, unlike a parameter: the edit is never at the call. The anchor is also the diagnostic's `root`, so sites in other workspaces fold with it after the merge (#51's model, same report fields: `anchorOnly`, `downstream`, `root`, "N call sites"). It is tried after the compiler-option cause and before the parameter cause. On vxsuite the finding is now one site at `vitest_setup.ts:10`, with the `jest_pdf_snapshot.ts` call as evidence. **2. README line limit.** `readme.test.ts` capped the file at 150 lines and every pack row pushed against it. The test now excludes the generated block between `<!-- packs:start -->` and `<!-- packs:end -->` from the count. #57's two joined documentation lines are not on `main` (that change lives on #57's branch), so there is nothing to restore here; once this merges, #57 can drop that edit. No pack's rules, expectations or ground truth were edited. ## How it was verified Unit tests in `cause.test.ts` on a new fixture pair, `fixtures/deps/matchers-v1` (an assertion that reads the global `checks.Matchers`) and `matchers-v2` (its own `Matchers<R, T>`, the global no longer read), plus `matchers-extra-v1` (a package declaring a matcher on that global): - three call sites in two files, for a matcher the setup file declares, fold into one anchor at the setup file's `declare global` line, each site carrying the cause and the root; - a call site for a matcher another package declares on the interface the setup file augments anchors at that augmentation, naming the package; - a member the package itself dropped is left at the site. `pnpm lint`, `pnpm typecheck`, `pnpm docs:links`, `pnpm docs:packs --check`, the private-material guard, core (930 tests) and CLI (350 tests) pass. Pack tests, before (main at 45677eb) and after (this branch), breaking findings; vitest run in a scratch worktree with #57's pack copied in uncommitted: | pack | before: precision / recall, breaking FP | after: precision / recall, breaking FP | |---|---|---| | vitest (#57, candidate) | 94% / 50%, 1 (`jest_pdf_snapshot.ts:42`) | **100% / 50%, 0** | | zod | 100% / 71%, 0 | 100% / 71%, 0, identical | | stripe | 100% / 90%, 0 | 100% / 90%, 0, identical | | ai | 100% / 36%, 0 | 100% / 36%, 0, identical | | react | 100% / 69%, 0 | 100% / 69%, 0, identical | vitest per rule after: `matcher-types` 100% / 100% (3 of 3), was 75% / 100%. `pack test vitest` still exits non-zero on this branch only because #57's `verification.json` records 94% / 1 false positive and no longer matches; that file is #57's to rewrite with `--write`. Its 16 false negatives and the 18,900-odd `generic (unverified)` sites (not breaking) are unchanged. ## Checklist - [x] Tests for the change (a bug fix starts with a failing test) - [x] `pnpm lint && pnpm typecheck && pnpm test` pass - [x] README and docs updated if behavior changed (`docs/architecture.md`, `docs/concepts.md`) - [x] Changeset added if a user would notice (`pnpm changeset`): minor - [x] No new dependency, or it was agreed in an issue - [x] No private code, names or paths in fixtures, tests or docs - [x] The rules in CONTRIBUTING.md ("Rules that do not bend") still hold 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018u9b3Pp5H68DdYZGp5Qg1U --- _Generated by [Claude Code](https://claude.ai/code/session_018u9b3Pp5H68DdYZGp5Qg1U)_ --------- Signed-off-by: Lucas Braum <lucas.braum@deskfy.io> Co-authored-by: Lucas Braum <lucas.braum@deskfy.io> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Collaborator
Author
|
Superseded by #63 Generated by Claude Code |
lucasbraum
added a commit
that referenced
this pull request
Oct 10, 2026
Supersedes #57. ## Pack `vitest`, `>=4 <5` → `>=5 <6`. Status: **verified** (2 public repos, 100% precision on breaking findings, recall 50%). The only breaking false positive in #57 (`libs/image-utils/src/jest_pdf_snapshot.ts:42`) is gone since #60 folds missing-member diagnostics (TS2339/TS2551) into one finding at the repository's own declaration. Rules, expectations, ground truth and companions are unchanged from #57. Changes over #57: - The README edit that joined two lines is undone (the line limit no longer counts the packs table). - `verification.json`, `packs/queue.json` and the changeset say verified; the README packs table is regenerated (`pnpm docs:packs`; `--check` and `pnpm docs:links` pass). - Tests that used vitest as the example of a package without a pack (`packages/cli/src/cli.test.ts`, `packages/core/src/fix/isolate.test.ts`) use `lodash` instead, since vitest now has a verified pack. Only the example package changed. ## Sources - Migrating to Vitest 5.0: https://vitest.dev/guide/migration (read as `docs/guide/migration/index.md` of vitest-dev/vitest at 69e6369) - Vitest 5.0.0 release notes: https://github.com/vitest-dev/vitest/releases/tag/v5.0.0 - Benchmarking guide: https://vitest.dev/guide/benchmarking - Extending matchers: https://vitest.dev/guide/extending-matchers No official codemod exists for this major. ## Rules and behavior notes Rules: `matcher-types`, `jest-dom-matchers`, `bench-api`, `removed-entrypoints` (rewrites four entry points), `removed-entrypoints-manual`, `sequential-removed`. Behavior notes: `coverage-patterns` (also a finding), `clear-mocks-default`, `test-name-pattern`, `inline-projects-extend`, `hoisted-calls-top-level`, `mock-behavior`, `assertion-timing`, `pretty-format-output`, `fake-timers-temporal`, `artifact-locations`, `config-resolution`, `browser-mode`, `environment-requirements`, `node-api`. ## Ground-truth repositories - votingworks/vxsuite @ `5cb3c7d9c1abc891e7b12de57c0278790dd6160a`, upgrade commit bb04c5fb01aa9bcd285170d2277df04ad5d7e09c (vitest 4.1.11 → 5.0.2, with `@vitest/coverage-istanbul`): 30 expected findings. - vitorvasc/opentelemetry-ecosystem-explorer @ `46d5b37d00bb` (directory `ecosystem-explorer`), upgrade PR head 2b0497b35d0e49dbeeb0d6174ad09aed03d46697 (4.1.11 → 5.0.0): 2 expected findings. Dropped candidates are listed in #57 (siemens/element cannot install from a public registry; the others are version-only bumps). ## `uptide pack test` output ``` uptide pack test vitest · verified, recorded candidate (before --write; now recorded verified) fixtures 5 cases, 17 sites · pass ground truth votingworks/vxsuite@5cb3c7d9c1ab vitest 4.1.11 → 5.0.2 predicted 15333, expected 30, false positives 15319, false negatives 16 with @vitest/coverage-istanbul 4.1.11 → 5.0.2 vitorvasc/opentelemetry-ecosystem-explorer@46d5b37d00bb vitest 4.1.11 → 5.0.0 predicted 3621, expected 2, false positives 3619, false negatives 0 rule precision recall tp fp fn bench-api 15% 67% 2 11 1 coverage-patterns 100% 0% 0 0 13 generic 0% 100% 0 18927 0 jest-dom-matchers 100% 85% 11 0 2 matcher-types 100% 100% 3 0 0 overall (sites) 0% 50% 16 18938 16 breaking (sites) 100% 50% 16 0 16 PASS vitest: verified ``` (The ~18,900 `generic (unverified)` lines are omitted; the 11 `bench-api` false positives are unverified inferred sites, not breaking.) Existing packs, rerun, all unchanged: ``` zod verified overall 100% / 84%, breaking 100% / 71% (PASS) stripe verified overall 100% / 90%, breaking 100% / 90% (PASS) ai verified overall 100% / 77%, breaking 100% / 36% (PASS) react verified breaking 100% / 69% (89 sites, 0 false) (PASS) ``` Environment note: tldraw (react) pins Yarn Berry and the sandbox `yarn` is classic, so the react run used a corepack shim on PATH (sandbox only). With that shim on PATH, stripe's nextjs/saas-starter failed to install (`MODULE_NOT_FOUND`); stripe passes without the shim. Environment issues, not regressions. `pnpm lint`, `pnpm typecheck` and `pnpm test` pass. ## What is not covered - Pack `detect` runs only on files that import the package, so these expected sites are missed: 13 bare-directory `coverage.include/exclude` entries in configs that import a shared `defineConfig` wrapper, and jest-dom imports in two setup files that import nothing from `vitest`. Breaking recall is 50%. - `check` could not resolve imports in 69 of 71 vxsuite files (workspace packages are not built), so only a few of the 169 compiler diagnostics are visible there. - `bench-api` misses the describe wrapper at `render_ballot.bench.ts:16`. - Rules and notes with no evidence in either repo, resting on fixtures and the guide only: `removed-entrypoints*`, `sequential-removed`, nested `vi.mock`, and all behavior notes except `coverage-patterns`. - The two repos are one monorepo and one small app; a third that really migrated would make "verified" stronger. ## Tooling friction - `pnpm uptide check` from source fails inside its worker (`Cannot find module .../packages/core/src/errors.js`); `pnpm build` and the dist CLI work. `pack test` is unaffected. - `pnpm packs:truth` requires full 40-character SHAs and prints the new path with the old line number for moved files. - A corepack shim on PATH fixes Yarn Berry repos but breaks the pnpm install of nextjs/saas-starter in the same run; the packs had to be run with and without it. - vitest.dev is not reachable from the sandbox. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01NwSXH7Dy3r4KjJhwKYBuxT --- _Generated by [Claude Code](https://claude.ai/code/session_01NwSXH7Dy3r4KjJhwKYBuxT)_ --------- Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pack
vitest,>=4 <5→>=5 <6. Status: candidate (1 false positive among breaking findings; see below). Queue entry set tocandidate. No pack-request issue existed for it.Sources
docs/guide/migration/index.mdof vitest-dev/vitest at 69e6369)No official codemod exists for this major.
Rules and behavior notes
Rules:
matcher-types(breaking, detect + compiler claims, agent): matchers on the globaljest.Matchers, or on a one-parameterAssertion<T>/Matchers<R>. Site = thedeclare globalblock or the interface.jest-dom-matchers(breaking, detect, agent):import '@testing-library/jest-dom[/vitest]', whose types no longer reachexpect.bench-api(breaking, type diff, agent):benchand theBench*types removed.removed-entrypoints(breaking, detect, rewrites):vitest/coverageandvitest/reporters→vitest/node,vitest/environmentsandvitest/snapshot→vitest/runtime.removed-entrypoints-manual(breaking, detect, agent):vitest/runners,vitest/suite,vitest/mocker,vitest/internal/module-runner.sequential-removed(breaking, type diff, agent):test.sequential,describe.sequential,sequentialoption.Behavior notes (all "Decisions for you", some also test follow-ups):
coverage-patterns(also afinding, found from source: bare directory incoverage.include/exclude),clear-mocks-default,test-name-pattern,inline-projects-extend,hoisted-calls-top-level,mock-behavior,assertion-timing,pretty-format-output,fake-timers-temporal,artifact-locations,config-resolution,browser-mode,environment-requirements,node-api.Ground-truth repositories
5cb3c7d9c1abc891e7b12de57c0278790dd6160a, upgrade commit bb04c5fb01aa9bcd285170d2277df04ad5d7e09c (vitest 4.1.11 → 5.0.2, with@vitest/coverage-istanbul5.0.2): 30 expected findings (3 matcher-types, 11 jest-dom-matchers, 3 bench-api, 13 coverage-patterns).46d5b37d00bbf27f394e33cee64a0e206c31315f(directoryecosystem-explorer), upgrade PR head 2b0497b35d0e49dbeeb0d6174ad09aed03d46697 (4.1.11 → 5.0.0): 2 expected findings (jest-dom-matchers).Expected findings come from
pnpm packs:truth(their compiler plus the upgrade commit's changed lines), read by hand; the 169 compiler diagnostics under vxsuite'slibs/image-utils/src/vitest_setup.tsare evidence of that one declaration and are recorded once at it. Exclusions are listed in the$commentofground-truth.json.Considered and dropped: siemens/element (a real migration, but its
@simpl/*and@siemens-ux/*packages come from code.siemens.com, so it cannot install from a public registry); google/budoux, ckeditor/ckeditor5-vue, gopherium/gophenberg, justinlocsei/larkspur and abraham/nutmeg (version bumps with no source change, so no site).uptide pack testoutputNew pack (
pnpm uptide pack test vitest --write; the ~18,900generic (unverified)lines are omitted):Existing packs, rerun in this sandbox, all unchanged:
react note: the first run could not install tldraw (its
packageManageris yarn@4.7.0 and theyarnon PATH here is 1.22.22; the installer runs it directly). That is an environment issue, not a regression. With a corepack shim on PATH (corepack enable --install-directory <dir>), the rerun matched the recorded numbers.pnpm lint,pnpm typecheckandpnpm testpass.What is not covered
jest_pdf_snapshot.ts:42is a real new TS2551 (toMatchImageSnapshotmissing onAssertion), but it is a symptom of the declaration atvitest_setup.ts:10, which the pack reports.checkdid not trace it to that cause (a globaljest.Matchersaugmentation), so it stands as a second breaking site. The ground truth records one site at the cause, as docs/packs.md says, and I did not add the symptom to pass.detectruns only on files that import the package, so these expected sites are missed: 13 bare-directory coverage entries (configs that import a shareddefineConfigwrapper), and jest-dom imports in two setup files that import nothing fromvitest. Recall on breaking sites is 50%.checkcould not resolve imports in 69 of 71 vxsuite files (workspace packages are not built), so only a few of the 169 compiler diagnostics are visible there.bench-apiclaims 11 inferredunverifiedsites onBench*types in vxsuite that the upgrade did not touch (overall tally only, not breaking), and the describe wrapper atrender_ballot.bench.ts:16is missed.removed-entrypoints*,sequential-removed, nestedvi.mock, and all behavior notes exceptcoverage-patterns.verifiedto mean more than one monorepo: the other candidates found are version-only bumps.Tooling friction
pnpm uptide checkfrom source fails inside its worker (Cannot find module .../packages/core/src/errors.js);pnpm buildand the dist CLI work.pack testis unaffected.pnpm packs:truthrequires full 40-character SHAs, and for a file the upgrade moved it prints the new path with the old line number; the paths in the ground truth were read fromgit diff -M.🤖 Generated with Claude Code
https://claude.ai/code/session_01YNCujWu9MXTm48xoYVKHJC
Generated by Claude Code