Skip to content

⚙️ Run only impacted Playwright tests in PR checks - #71

Merged
sasler merged 3 commits into
mainfrom
chore/change-scoped-pr-tests
Jul 30, 2026
Merged

⚙️ Run only impacted Playwright tests in PR checks#71
sasler merged 3 commits into
mainfrom
chore/change-scoped-pr-tests

Conversation

@sasler

@sasler sasler commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Summary

  • compute Playwright impact from the complete PR or merge-group diff, with exact game, shared-area, changed-spec, documentation-only, and full-fallback modes
  • tag every browser test and derive the load-test matrix directly from games-metadata.json
  • preserve two-shard reports and the stable Required PR checks context while keeping manual runs complete
  • update repository and agent guidance to match the new CI contract

Validation

  • npm run test:ci-selection — 15 passed, including the collected-suite tag audit
  • npm run lint — passed
  • npm run build — passed (146 static pages)
  • scoped one-test probe — populated shard passed in 10.3s; empty shard passed
  • empty scoped shard with blob reporter — passed and emitted a mergeable report
  • complete local Playwright evidence — 261 unchanged non-load cases passed; the metadata-derived load matrix passed all 101 cases on its focused rerun

Existing runtime baseline

Deriving load tests from metadata exposed a pre-existing parse error in outrun-racer/gpt-5-4-mini, which the previous hard-coded matrix omitted. This PR does not edit that model-owned game. The load test records the exact existing error so any additional regression or future fix forces the baseline to be reviewed.

Co-authored-by: GPT-5.6 Sol <codex@openai.com>
@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
brainrot Ready Ready Preview Jul 30, 2026 6:30pm

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b0f0c5e7ad

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/select-playwright-tests.js
Comment thread scripts/select-playwright-tests.js

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR changes the CI contract so PR checks run only the Playwright tests impacted by a PR (or merge-group) diff, while keeping a stable “Required PR checks” aggregate status and preserving full-suite behavior for manual runs and fail-safe cases.

Changes:

  • Add a Playwright impact selector (scripts/select-playwright-tests.js) plus Node tests that validate selection modes and enforce a tagging contract across the Playwright suite.
  • Tag Playwright specs/tests with consistent @spec:* plus @game:* / @area:* tags, and derive load-test coverage directly from games-metadata.json.
  • Update the PR checks workflow and repo/agent docs to reflect the new “impact-selected” Playwright gate (including skipping browser jobs for documentation-only changes).

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/tile-matching-sol.spec.ts Adds @spec + @game tags to support impact selection.
tests/tile-matching-opus-5.spec.ts Adds @spec + @game tags to support impact selection.
tests/threejs-pipeline.spec.ts Tags Three.js/site pipeline tests with @spec + @area/@game for scoped selection.
tests/tetris-luna.spec.ts Adds @spec + @game tags to support impact selection.
tests/sudoku-regressions.spec.ts Adds suite-level @spec tag and per-test @game tags for scoped selection.
tests/smoke.spec.ts Tags smoke suite with @spec + @area.
tests/ratings-feedback.spec.ts Tags ratings suite with @spec + @area.
tests/perihelion-opus-5.spec.ts Adds @spec + @game tags to support impact selection.
tests/outrun-racer-sol.spec.ts Adds @spec + @game tags to support impact selection.
tests/outrun-racer-opus-5.spec.ts Adds @spec + @game tags to support impact selection.
tests/models.spec.ts Tags models suite with @spec + @area.
tests/model-identity.spec.ts Tags model-identity suite with @spec + @area.
tests/mobile-layout.spec.ts Adds suite @spec tag and per-test @game tags for scoped selection.
tests/mini-golf-sol.spec.ts Adds @spec + @game tags to support impact selection.
tests/maze-3d-sol.spec.ts Adds @spec + @game tags to support impact selection.
tests/marble-madness-sol.spec.ts Adds @spec + @game tags to support impact selection.
tests/kart-racing-sol.spec.ts Adds @spec + @game tags to support impact selection.
tests/kart-racing-opus-5.spec.ts Adds @spec + @game tags to support impact selection.
tests/games-load.spec.ts Switches load coverage to metadata-derived getGames() and tracks known runtime-error baselines.
tests/community-failure-verdicts.spec.ts Tags suite with @spec + @area for scoped selection.
tests/clockwork-caper.spec.ts Adds suite @spec tag and per-test @game tags for scoped selection.
tests-node/select-playwright-tests.test.js Adds Node tests validating selection modes and enforcing the tagging contract via Playwright’s --list JSON output.
scripts/select-playwright-tests.js Implements diff-driven impact selection with scoped/full/none modes and GitHub Actions output wiring.
README.md Updates contributor guidance to describe the change-scoped Playwright PR gate and new validation command.
package.json Adds test:ci-selection to run the selector/contract tests.
AGENTS.md Updates repository invariants to reflect “impact-selected Playwright” in CI.
.github/workflows/pr-checks.yml Adds the selection job and gates Playwright shards/reporting based on the selected mode.
.agents/skills/verify-changes/SKILL.md Updates agent verification guidance to match the new CI contract.
.agents/skills/address-pr-review-comments/SKILL.md Updates review-comment workflow guidance to reference impact-selected Playwright coverage.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

sasler and others added 2 commits July 30, 2026 20:25
Co-authored-by: GPT-5.6 Sol <codex@openai.com>
Co-authored-by: GPT-5.6 Sol <codex@openai.com>
@sasler
sasler merged commit 8bdee3e into main Jul 30, 2026
8 checks passed
@sasler
sasler deleted the chore/change-scoped-pr-tests branch July 30, 2026 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants