Fix e2e tests on Node 24 and re-enable them in CI - #118
Merged
Conversation
…s the config natively, where PlaywrightTestConfig has no runtime export Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe
…npm 11 Playwright 1.45.3 predates Node 24: its ESM loader thread deadlocks before any browser launches, so every e2e test hung forever. 1.62.0 runs fine. Turbo 2.9.4 could not parse the patchedDependencies format written by pnpm 11 in pnpm-lock.yaml (bare hash string); 2.10.7 parses it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe
The test step had been commented out since the pnpm catalogs migration (#104). Note: only 3 demos have linux baselines; the others will fail with 'snapshot doesn't exist' until their baselines are generated in the CI container. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe
The repo's Actions cache contains only setup-node pnpm-store entries — zero turbo artifacts — so every CI run rebuilds all 160 demos. v1.x of the action predates GitHub's cache service v2 migration (legacy backend shut down early 2025) and has been silently failing to save since. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe
gh workflow run CI --ref <branch> -f update_snapshots=true then download the 'snapshots' artifact and commit it. Baselines must be generated on the CI runner (amd64 + CI Chromium build) to match what the test step compares against. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe
The github-pages environment protection already rejects non-main deploys (the regen dispatch run failed on this); make it explicit and stop wasting the build on regen runs. Also drop the stale aquarium linux baseline (Chromium 127): its drift under Chromium 151 sits right at the 5% threshold, so update-snapshots 'changed' mode won't reliably rewrite it — a missing baseline always is. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe
--force because a cached test task restores its cached outputs — i.e. the very stale baselines being regenerated (bitten by this: a regen run went FULL TURBO and resurrected a deleted baseline). =all because mode 'changed' keeps baselines whose drift sits under the diff threshold. e2e-test now forwards the mode value to playwright. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe
…ner) Generated by the update_snapshots dispatch (run 30529310256). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe
Regenerate locally on demand: pnpm --filter @demo/aquarium test -- --update-snapshots Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
pnpm testhas been broken locally since the Node 24 bump (16e8b16), and silently disabled in CI since the pnpm catalogs migration (#104) — the CI test step was commented out, so nothing caught it. On top of that, the CI turbo cache had been silently dead for months.Two distinct e2e failures on Node 24:
playwright.config.tsnatively with type stripping (bypassing Playwright's Babel transform), andPlaywrightTestConfigis a type-only export →SyntaxErrorat import.MessagePortprocessing, no browser ever launches) — the run hangs forever.Changes
import typeinpackages/e2e/playwright.config.ts@playwright/test1.45.3 → 1.62.0 (Node 24 support) + CI image →playwright:v1.62.0-jammyturbo2.9.4 → 2.10.7 (2.9.4 couldn't parse thepatchedDependenciesformat pnpm 11 writes in the lockfile)caching-for-turbov1.5 → v2.5.1 — v1.x predates GitHub's cache service v2 migration and had been silently failing to save: the repo had zero turbo cache entries, every CI run rebuilt all 160 demos. Verified fixed: a rerun goesCached: 160/161in ~30s (FULL TURbo).workflow_dispatchinputupdate_snapshotsto regenerate baselines on the runner:gh workflow run CI --ref <branch> -f update_snapshots=true, then commit the run'ssnapshotsartifact. Usesturbo --force(a cached test task would restore the stale baselines being regenerated) and--update-snapshots=all. Regen dispatches skip build/deploy; deploy is now also pinned torefs/heads/main(the github-pages environment protection already rejected non-main deploys).Verified
pnpm --filter @demo/aquarium test→ 1 passed.🤖 Generated with Claude Code
https://claude.ai/code/session_01NVkETanfY1gFZwmAS1peWe