⚙️ Run only impacted Playwright tests in PR checks - #71
Conversation
Co-authored-by: GPT-5.6 Sol <codex@openai.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b0f0c5e7ad
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
This PR changes the CI contract so PR checks run only the Playwright tests impacted by a PR (or merge-group) diff, while keeping a stable “Required PR checks” aggregate status and preserving full-suite behavior for manual runs and fail-safe cases.
Changes:
- Add a Playwright impact selector (
scripts/select-playwright-tests.js) plus Node tests that validate selection modes and enforce a tagging contract across the Playwright suite. - Tag Playwright specs/tests with consistent
@spec:*plus@game:*/@area:*tags, and derive load-test coverage directly fromgames-metadata.json. - Update the PR checks workflow and repo/agent docs to reflect the new “impact-selected” Playwright gate (including skipping browser jobs for documentation-only changes).
Reviewed changes
Copilot reviewed 29 out of 29 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
tests/tile-matching-sol.spec.ts |
Adds @spec + @game tags to support impact selection. |
tests/tile-matching-opus-5.spec.ts |
Adds @spec + @game tags to support impact selection. |
tests/threejs-pipeline.spec.ts |
Tags Three.js/site pipeline tests with @spec + @area/@game for scoped selection. |
tests/tetris-luna.spec.ts |
Adds @spec + @game tags to support impact selection. |
tests/sudoku-regressions.spec.ts |
Adds suite-level @spec tag and per-test @game tags for scoped selection. |
tests/smoke.spec.ts |
Tags smoke suite with @spec + @area. |
tests/ratings-feedback.spec.ts |
Tags ratings suite with @spec + @area. |
tests/perihelion-opus-5.spec.ts |
Adds @spec + @game tags to support impact selection. |
tests/outrun-racer-sol.spec.ts |
Adds @spec + @game tags to support impact selection. |
tests/outrun-racer-opus-5.spec.ts |
Adds @spec + @game tags to support impact selection. |
tests/models.spec.ts |
Tags models suite with @spec + @area. |
tests/model-identity.spec.ts |
Tags model-identity suite with @spec + @area. |
tests/mobile-layout.spec.ts |
Adds suite @spec tag and per-test @game tags for scoped selection. |
tests/mini-golf-sol.spec.ts |
Adds @spec + @game tags to support impact selection. |
tests/maze-3d-sol.spec.ts |
Adds @spec + @game tags to support impact selection. |
tests/marble-madness-sol.spec.ts |
Adds @spec + @game tags to support impact selection. |
tests/kart-racing-sol.spec.ts |
Adds @spec + @game tags to support impact selection. |
tests/kart-racing-opus-5.spec.ts |
Adds @spec + @game tags to support impact selection. |
tests/games-load.spec.ts |
Switches load coverage to metadata-derived getGames() and tracks known runtime-error baselines. |
tests/community-failure-verdicts.spec.ts |
Tags suite with @spec + @area for scoped selection. |
tests/clockwork-caper.spec.ts |
Adds suite @spec tag and per-test @game tags for scoped selection. |
tests-node/select-playwright-tests.test.js |
Adds Node tests validating selection modes and enforcing the tagging contract via Playwright’s --list JSON output. |
scripts/select-playwright-tests.js |
Implements diff-driven impact selection with scoped/full/none modes and GitHub Actions output wiring. |
README.md |
Updates contributor guidance to describe the change-scoped Playwright PR gate and new validation command. |
package.json |
Adds test:ci-selection to run the selector/contract tests. |
AGENTS.md |
Updates repository invariants to reflect “impact-selected Playwright” in CI. |
.github/workflows/pr-checks.yml |
Adds the selection job and gates Playwright shards/reporting based on the selected mode. |
.agents/skills/verify-changes/SKILL.md |
Updates agent verification guidance to match the new CI contract. |
.agents/skills/address-pr-review-comments/SKILL.md |
Updates review-comment workflow guidance to reference impact-selected Playwright coverage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: GPT-5.6 Sol <codex@openai.com>
Co-authored-by: GPT-5.6 Sol <codex@openai.com>
Summary
games-metadata.jsonRequired PR checkscontext while keeping manual runs completeValidation
npm run test:ci-selection— 15 passed, including the collected-suite tag auditnpm run lint— passednpm run build— passed (146 static pages)Existing runtime baseline
Deriving load tests from metadata exposed a pre-existing parse error in
outrun-racer/gpt-5-4-mini, which the previous hard-coded matrix omitted. This PR does not edit that model-owned game. The load test records the exact existing error so any additional regression or future fix forces the baseline to be reviewed.