[sentry-junior]

Makes vitest-evals work on Vitest 5. getsentry/toolkit uses Vitest 5, and its eval suite can't load the reporter from the vitest-evals 0.4.0 release it depends on, because that release imports vitest/reporters, which Vitest 5 removed.

The reporter on main already imports from vitest/node, and Vitest 5 still exports DefaultReporter and VerboseReporter from there, so it runs. What broke were the types and the test fixtures.

Warning

Breaking change for Vitest 4 consumers. The vitest-evals peer dependency changes from vitest >=4 <5 to vitest >=5 <6. One declare module "vitest" block can't merge with both the Vitest 4 Assertion<T> and the Vitest 5 Assertion<R, T>, so the typed matchers no longer support Vitest 4. Ship this as a minor bump (0.18.0), not a patch.

Changes

• Matcher types: Vitest 5 declares Assertion<R extends void | Promise<void> = void, T = unknown>. The augmentations in index.ts (toSatisfyJudge) and legacy.ts (toEval) used Assertion<T = any>, so tsc failed with TS2428. Both now use the Vitest 5 signature.
• Peer ranges: vitest-evals requires vitest >=5 <6, as the warning above describes. @vitest-evals/github-reporter doesn't augment Vitest types, so it accepts >=4 <6.
• Workspace: vitest and @vitest/coverage-v8 move to ^5.1.0, and the lockfile is regenerated with pnpm 10.33.0.
• Reporter tests: Vitest 5's base reporter reads state from the context and calls benchmarks() on each test case, so the fake context and TestCase now provide both. The verbose fallback test now expects file:12, because Vitest 5 prints the line number without the column.

Verification

• pnpm typecheck, pnpm lint, pnpm build, and pnpm release:check pass.
• pnpm test:ci: 408 tests pass and 1 fails. The failure is in report-ui, "returns 500 when asset reads fail". It fails the same way on main with Vitest 4 in my sandbox, where the file is still readable after chmod 000, so this change didn't cause it.
• Consumer check: I packed this branch and installed it into getsentry/toolkit main, which runs Vitest 5.0.5. With its imports switched from vitest-evals to vitest-evals/legacy, vitest run --reporter=vitest-evals/reporter loaded the reporter. All 27 files and 121 evals ran and wrote eval-results.json, and tsc passed for the eval package. The evals themselves failed only because the sandbox has no provider API keys.

Follow-up

After 0.18.0 is released, toolkit can move from vitest-evals 0.4.0 to it and keep Vitest 5. That change will replace getsentry/toolkit#1973's current approach, which pins Vitest 4 for the eval package.

via @dcramer.

--

View Session [Sentry]

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
vitest-evals-docs	Ready	Preview	Nov 5, 2026 3:50am UTC

[dcramer]

i assume its too copmlicated to support both versions?

[sentry-junior]

For the runtime, no. The reporter and matchers run the same on Vitest 4 and 5. For the types, mostly yes.

Vitest 5 changed Assertion<T> to Assertion<R, T>. TS merges an interface augmentation only when the type parameters match exactly, and none of the interfaces we could augment instead has the same shape in both versions:

• Matchers changed from <T> to <R, T>.
• CustomMatcher gained an <R> parameter.
• The global jest.Matchers is gone in v5.

So one declare module "vitest" block can't serve both versions.

I tested this branch's build in a Vitest 4.1.12 project:

• With skipLibCheck: false, it fails with TS2428 in vitest-evals/dist/index.d.ts and legacy.d.ts.
• With skipLibCheck: true, it compiles, and toSatisfyJudge and toEval resolve. But the merged interface probably no longer passes the received value's type through to the judge. That part is inferred; I didn't fully verify it.

Real dual support would mean shipping a second augmentation for v4, for example a vitest-evals/vitest4 types entry. That adds more public surface than a 0.x stopgap is worth. Widening the peer to >=4 <6 would quietly give v4 users looser types.

So we're keeping this PR v5-only and shipping it as 0.18.0.