Playwright E2E Test Template

AI-driven end-to-end test generation using Playwright and @playwright/cli. Clone this template, point it at your app, and let the AI agent generate tests.

Works with Claude Code and GitHub Copilot.

Prerequisites

Before cloning, make sure you have:

Requirement	Version	Why
Node.js	20+ (see `.nvmrc`)	Playwright 1.52+ requires it; npm scripts use modern features
Git	any recent	standard tooling
Bash	system bash (macOS/Linux) or Git for Windows / WSL2 (Windows)	Slash commands (`/discover`, `/generate`, `/heal`, `/promote`, `/validate`, `/stability`) dispatch shell snippets through your agent's bash environment. Pre-commit hook also requires `sh`. Git for Windows installs Git Bash and adds it to PATH automatically. The npm scripts themselves run in any shell.
Google Chrome	installed on your machine	`playwright.config.ts` uses `channel: 'chrome'` (real Chrome binary, not bundled Chromium — closer-to-prod rendering and avoids `HeadlessChrome` UA substring that some WAFs flag)
An AI coding assistant	one of: Claude Code (CLI or VS Code extension) or GitHub Copilot Chat (VS Code extension)	Where you type the `/discover`, `/generate`, etc. slash commands. Both work identically with this template — pick whichever your team uses.
VS Code (recommended)	1.109+	The Agent Sessions view (Jan 2026+) lets you run Claude Code, Copilot, and Codex from one place. Not strictly required — Claude Code CLI works standalone.

Optional: a Rally instance if you want /story TC12345 to pull artifacts directly via the Rally MCP server (otherwise paste the story text).

Platform support

Works on macOS, Linux, and Windows. All harness scripts are Node.js (no sed/awk/mktemp in skill bodies — enforced by npm run check:skill-bodies in pre-commit and CI). Line endings are pinned to LF via .gitattributes so Windows checkouts don't mangle shell scripts or leave \r in parsed .env values. The harness-smoke CI job runs typecheck, invariant checks, locator audit, and skill-body lint on ubuntu-latest, macos-latest, and windows-latest on every PR, so cross-platform regressions are caught before merge.

Windows specifics. The npm scripts (npm test, npm run typecheck, npm run audit:locators, etc.) run in any shell — cmd.exe, PowerShell, or Git Bash. Slash commands are different: they execute shell snippets through your AI agent's bash environment, and several skill bodies use bash idioms ($VAR expansion, for/case loops, $(date ...) command substitution, heredocs). For /discover, /generate, /heal, /promote, /validate, and /stability to work on Windows you need Git Bash on PATH (installed by Git for Windows and added to PATH by default) or WSL2. The pre-commit hook also requires sh. Run bash --version to confirm; if it resolves, you're set.

Quick Start

There are two ways to set up the harness:

Method	Best for
npm install (recommended)	SDETs adding E2E tests to an existing project
Fork & clone	Teams maintaining their own harness fork, contributing upstream

The init command:

Copies all harness files (specs, fixtures, helpers, scripts, docs) to your project
Merges dependencies and scripts into your existing package.json
Creates necessary directories (e2e/tests/generated/, e2e/tests/stable/, etc.)
Preserves your existing files (use --update for smart upgrades or --force to overwrite all)

Preview before installing:

npx qe-harness init --dry-run

Upgrade to a newer version (recommended):

npm update qe-harness
npx qe-harness init --update    # updates harness files, preserves your specs/tests/helpers
npm install                            # install any new dependencies

The --update flag updates harness infrastructure (scripts, docs, base fixtures) while preserving your team's work (specs, tests, helpers, locators, patterns, app.fixture.ts).

For detailed upgrade modes and file ownership documentation, see NPM-PUBLISHING.md.

Post-install setup (Option B — fork & clone)

Using Option A (npm)? You're already set up after the steps above — skip to Your first run.

# 1. Install project dependencies
npm install

# 2. Configure environment — pick whichever fits your workflow.
#    (npm run setup installs the browsers + playwright-cli and syncs skills automatically.)
#
#   a) Interactive script (recommended for first-time setup):
#        npm run setup            # full walkthrough (6 steps)
#        npm run setup -- --help  # see all options
#        npm run setup:auth       # re-capture auth state only
#      Asks BASE_URL, AUTH_REQUIRED, and (if auth=yes) TEST_USERNAME /
#      TEST_PASSWORD. Also checks Node, installs browsers, syncs skills.
#      Safe to re-run at any time.
#      Full walkthrough: harness-docs/DEEP-DIVE.md → §3 "First-Run Experience"
#
#   b) Let the agent bootstrap it:
#        Just run /generate all — if .env is missing, the agent will ask
#        for the required values in chat and write .env before generating.
#        (Agents cannot drive npm run setup's interactive TTY prompts.)
#
#   c) Manual:
#        cp .env.example .env   # then edit by hand

# 3. (Optional) Verify your environment
npm run doctor

Where do I type `/discover`, `/generate`, etc.?

Slash commands run inside your AI assistant's chat — not in a shell. Where exactly depends on which tool you use:

Claude Code (CLI or VS Code extension): type /discover directly in the chat input.
GitHub Copilot Chat (VS Code): open the Agent dropdown in Copilot Chat, select qe-engineer, then type your prompt (e.g. discover). Copilot custom agents do NOT appear in @ mention autocomplete — the dropdown is the only entry point.

Your first run

# 4. Let the AI explore your app and write Gherkin specs
/discover
#   AI opens a browser, visits BASE_URL, navigates one level deep,
#   and writes .feature files to e2e/specs/. Review them before step 6.

# 5. Generate Playwright tests from those specs
/generate all
#   Spawns up to 5 spec-worker subagents in parallel. Each opens its own
#   browser session, verifies every locator against the live DOM, and writes
#   a .spec.ts file to e2e/tests/generated/.

# 6. Validate, heal, promote
/validate          # run the generated tests, report pass/fail
/heal              # fix any failures (asks your approval for assertion changes)
/promote all       # move passing tests to e2e/tests/stable/ (staged for git)

Dogfooding against an example app (maintainer-only)

Want to dogfood the harness end-to-end before pointing it at a real app? An overlay system lives under examples/ — scaffolding one and installing it symlinks app-specific specs/tests into the active e2e/ tree without ever modifying the universal harness on main.

The harness ships without any preloaded examples. Forks inherit a pristine examples/ directory containing only this README. To dogfood, scaffold an overlay first, then install it:

npm run example:scaffold <your-app>  # create empty examples/<your-app>/ skeleton
npm run example:install <your-app>   # symlink examples/<your-app>/ overlay into e2e/

Full options for scaffolding (base-url, auth, dry-run, force) are in examples/README.md § Scaffolding a new overlay.

After install, follow the For SDETs walkthrough below as-is — specs you create land in examples/<your-app>/e2e-overlay/specs/ and tests land in examples/<your-app>/e2e-overlay/tests/stable/ automatically. Nothing pollutes e2e/specs/ or e2e/tests/stable/ on main. Before committing example work, run npm run example:sync — it moves any new files agents wrote into e2e/helpers/, e2e/locators/, e2e/patterns/, or e2e/fixtures/ into the overlay (universal harness files are left alone). The agent skills (/discover, /generate, /heal) run an overlay sanity check at the top of every invocation and refuse if the install state is inconsistent — see MAINTAINER-DOGFOODING.md for recovery. When you're done:

npm run example:uninstall   # restore pristine harness, your .env is preserved
npm run example:status      # check what's installed (or "none")

Forks of this repo can ignore examples/ entirely — rm -rf examples/ is safe; nothing in the harness depends on it. Full details: examples/README.md.

Maintaining the harness itself? If you're modifying skills, scripts, fixtures, or CI hooks and want to dogfood your changes against an example without polluting main, read harness-docs/MAINTAINER-DOGFOODING.md — it covers the pre-commit quarantine checklist, recovery cookbook for common pollution patterns, and how to commit harness vs. example changes separately.

For SDETs: I have a test idea — now what?

You don't need to know Playwright, TypeScript, or how the harness works internally. Follow this path, type these prompts into your AI chat, and end up with a tested commit pushed to a feature branch for review.

Step 0 — create a feature branch (one-time per test batch; harness refuses to push to main/master/develop):

git checkout -b sdet/cart-checkout-$(date +%Y%m%d)

Step 1 — hand your test case to the AI. Pick whichever input shape you have. No Rally required — plaintext works fine.

You have	Paste this into chat
A paragraph/bullets of test intent	`/story` (hit enter, then paste your test scenarios in the next message)
A detailed test case (steps + expected results, e.g. "TC-0421")	`/story` (paste the whole case — ID, preconditions, steps, expected, all of it)
A Rally ticket	`/story TC12345` (Rally via MCP), or paste the ticket body
A markdown/text file	`/story path/to/my-story.md`
A folder of test-case files (batch)	Drop files in `e2e/test-cases/`, then `/story e2e/test-cases/` — the AI processes every file as a batch, presents one combined review, and writes one `.feature` per test case. Recommended for teams with 5+ cases at a time.
Nothing yet — want the AI to find test-worthy pages	`/discover`

Tip: the e2e/test-cases/ folder is the canonical home for human-written test cases. Ships with a README.md and two _EXAMPLE_ files showing accepted formats. Delete the examples when you start writing your own cases — they're templates, not active inputs.

My test case isn't in any of those formats — now what?

Test cases come from everywhere: Excel grids, Word docs, PDFs, TCM exports (TestRail, Zephyr, Xray, Qase), wiki pages, Slack threads. The harness doesn't need a parser for each format — it just needs the text. Use this table:

Source format	One-step conversion	Then
Excel / Google Sheets grid	Select the test-case rows → Copy (Cmd/Ctrl-C)	`/story` then paste
Word / Google Doc	Select the test-case text → Copy	`/story` then paste
PDF	Select text from the page → Copy (if PDF is image-only, OCR it first — the harness can't read images)	`/story` then paste
TCM web UI (TestRail, Zephyr, Xray, Qase)	View the test case → Copy text from the page	`/story` then paste
TCM CSV / TSV export	Save the file locally	`/story path/to/export.csv` — the AI parses columns
TCM HTML export	Save the file locally	`/story path/to/export.html` — the AI strips tags
Slack / Teams message	Copy the message body	`/story` then paste
Handwritten / whiteboard photo	Transcribe the key parts to text first (the harness doesn't OCR)	`/story` then paste

The AI figures out the structure — columns, numbered steps, "Given/When/Then", "Preconditions/Steps/Expected Results", bullet lists, freeform prose. What it cares about is content, not layout.

Example — a detailed test case pasted from Excel or a TCM (columns collapse to rows, AI parses):

/story

TC-0421: Premium user sees ad-free home feed

Preconditions:
  - User has premium tier enabled
  - Feature flag premium_ad_removal is ON

Steps:
  1. Log in as premium.test@example.com
  2. Navigate to /feed
  3. Scroll through the first 20 post cards

Expected:
  - Zero ad-slot elements rendered in the feed
  - Premium badge visible next to username in header
  - No "upgrade to premium" CTAs in the sidebar

Negative:
  - Non-premium user on the same page DOES see ad slots

The AI turns this into a Gherkin .feature file with one @alpha scenario per happy-path expected result and one @gamma scenario for the negative check. It writes # Test Case: TC-0421 at the top for traceability — anyone reading the generated test later can trace it back to your source case.

The AI converts your text into Gherkin scenarios, presents them for your review, and saves approved ones to e2e/specs/. You approve/edit/reject each scenario — nothing is written without your consent.

Simpler example — plain bullets also work (no formality required):

/story

The checkout page should:
- Show order summary with item count and total
- Validate credit card number format
- Show error for expired cards
- Redirect to confirmation on success
- Handle payment API timeout gracefully

Step 2 — turn specs into runnable tests:

/generate all

The AI opens browsers, explores your live app, writes .spec.ts files for every spec, runs them, and self-heals basic locator issues. End result: a pass/fail summary per test file.

Step 3 — fix failures. The AI classifies every failure for you first:

/heal

LOCATOR failures (selector stale) → auto-fixed, no approval needed.
ASSERTION / SPEC DRIFT failures (app actually changed) → AI asks you before changing anything.
ENVIRONMENT failures (network, timeouts) → flagged for investigation, not fixed.

After /heal completes, re-run /validate to confirm the fixes held.

Step 4 — push the full suite to remote:

/promote all --push

This runs every quality gate (validation ledger, stability grade, fragility score, credential scan, locator audit), moves passing tests to stable/, commits them with an informative message, rebases on the remote branch, and pushes. No --force, no --no-verify, no pushing to main — if anything breaks, it stops and tells you the exact next command.

If you leave off --push, the harness only stages files for you to commit manually.

Step 5 — open a PR. You get a green CI run on the smoke job (@alpha tests) within ~5–10 minutes; if it fails, run /heal stable and push again. CI handles everything else.

What if I get stuck?

Symptom	What to type
Skill output confuses you	Ask your AI: "what should I do next?" — every skill report ends with a suggested next command
A test is `FIXME` and you don't know why	`/heal <filename>` — it will classify and explain
Something says "permission prompt"	You're probably running on a fresh clone — approve once, skill will remember
`git commit` fails before it even starts	Pre-commit hook caught an agent-shim drift; run `npm run sync:agents` and re-commit
`/promote --push` says "refusing to push to main"	You forgot Step 0 — create a feature branch
Anything else	`npm run doctor` — diagnoses 6 common environmental issues

That's it. You don't need to know where e2e/tests/generated/ vs e2e/tests/stable/ live, what a spec-worker is, or how the trace artifact bridges Phase 2 and Phase 3. The harness handles it.

If you want to understand why the harness works this way: harness-docs/DEEP-DIVE.md. Optional.

Workflow

You have two spec sources — pick whichever fits your context. Most teams end up using both:

  ( /story  OR  /discover )  →  👤 review  →  /generate all  →  /validate  →  /heal  →  /promote all --push

(On a feature branch. /promote all without --push stages locally only — see "For SDETs" above.)

/story produces specs from user stories — business rules, acceptance criteria, compliance cases. Use when you have a Rally artifact, a written user story, or a list of test scenarios.
/discover produces specs from structural exploration of the live app — navigation, forms, a11y patterns. Use when you want structural coverage your stories don't mention.
Run both when you want both. Story-first gives business context; discovery fills structural gaps. The two produce separate spec files in e2e/specs/; review each set before /generate.

Expect the first run to feel slower than later runs. The harness gets faster the more you use it on the same project — /heal approvals populate heal-log.md, /discover writes page-specific patterns, and CI fills test-history/. Subsequent /generate runs reuse that accumulated knowledge and skip re-exploration. See DEEP-DIVE.md → Why the first run feels slower than steady state.

Step	What happens
`/story`	Generates Gherkin specs from test intent you paste into chat (most common), OR a Rally artifact (`/story TC12345`), OR a file (`/story story.md`). Presents for human review before saving. See DEEP-DIVE → "How does human input get converted to detailed Gherkin?" for worked examples.
`/discover`	AI explores your live app via `playwright-cli`, generates Gherkin specs for structural coverage (navigation, forms, a11y). Writes page-specific patterns to `e2e/patterns/`.
`/generate all`	Spawns up to 5 subagents in parallel — each opens its own browser session, explores, writes a `.spec.ts` test. After all workers complete, auto-promotes shared locators (3+ files) to the registry.
`/validate`	Runs all generated tests, reports pass/fail per file. Read-only — does not fix anything.
`/heal`	Diagnoses failures via `playwright-cli` snapshots. Auto-fixes locator failures; asks approval for assertion and spec-drift changes.
`/promote all --push`	Runs the full quality-gate suite → moves passing tests from `generated/` to `stable/` → commits → rebases on remote → pushes to your current feature branch. Refuses `main`/`master`/`develop`. Drop `--push` to only stage locally. See DEEP-DIVE → `/promote --push` end-to-end.

Skills

Skill	Description	Examples
`/discover`	Explore app, generate specs, update AGENTS.md	`/discover`, `/discover /pricing /about`, `/discover --refresh all`
`/generate`	Generate tests from specs (parallel subagents)	`/generate all`, `/generate homepage-*`, `/generate --tag=@alpha`
`/heal`	Fix failing tests	`/heal my-feature.spec.ts`, `/heal all`
`/promote`	Move generated tests to stable	`/promote all`, `/promote my-feature.spec.ts`
`/validate`	Run tests and report status	`/validate`, `/validate stable`
`/specs`	List specs with status and tags	`/specs`, `/specs --status=pending`, `/specs --tag=@beta`
`/story`	Generate specs from user stories or test scenarios (Rally via MCP, file, or describe directly). Add `--suggest-gaps` to propose additional edge/error scenarios from page patterns + heal-log (per-suggestion review gate).	`/story TC12345`, `/story --suggest-gaps`, `/story` (then describe scenarios)
`/stability`	Analyze test history for flap rates, break rates, fragile locators	`/stability`, `/stability my-feature.spec.ts`, `/stability --threshold=20`
`/promote-locator`	Extract inline locator used in 3+ test files into the locator registry	`/promote-locator homepage NAV_LOCATORS.servicesLink`
`/locator-audit`	Whole-corpus scan: auto-promote duplicates past threshold, flag dead registry entries, split oversized registry files	`/locator-audit`, `/locator-audit --threshold=4`, `/locator-audit --report-only`
`/helper-audit`	Whole-corpus scan: auto-extract duplicated interaction sequences (≥3 Playwright steps, ≥2 occurrences) into `e2e/helpers/`. Sequences containing `expect(...)` are never auto-extracted — assertions stay in tests.	`/helper-audit`, `/helper-audit --apply`, `/helper-audit --apply --auto`
`/fixture-audit`	Whole-corpus scan: surfaces duplicated `beforeEach`/`beforeAll` bodies and repeated setup prefixes that may warrant fixture extraction. Report-only — fixture authorship is a human decision and never auto-applied (lifecycle and scope require judgment).	`/fixture-audit`

Supporting infrastructure (not slash commands, called by skills and spec-workers):

Script	What it does	When it runs
`scripts/trace-io.js`	Reads/writes per-spec exploration traces (`e2e/traces/<spec>.json`)	Spec-worker Phase 2 writes, Phase 3 reads
`scripts/harness-record.js`	Appends per-spec + per-batch generation records to `test-history/harness-runs.jsonl`	End of every `/generate` invocation
`scripts/harness-report.js`	Summarizes harness records: phase breakdown, reliability, reuse, top expensive specs	`npm run harness:report` or on-demand
`scripts/heal/lib/memory.js`	TF-IDF semantic index over `heal-log.md` — retrieves similar past fixes by failure context	`/heal` Stage 3.5, spec-worker Phase 1
`scripts/classify-steps.js`	Pre-dispatch step classifier — resolves unambiguous steps deterministically via Playwright, skipping LLM round-trips	Spec-worker Phase 1.5 (optional)
`scripts/check-invariants.js`	AST-based quality-gate checker: expect-presence, spec-comments, file-size, trace-coverage, test-step-coverage	`npm run check:invariants`, spec-worker quality gates
`scripts/dashboard.js`	Self-contained HTML dashboard aggregating everything above	`npm run dashboard`

Project Structure

e2e/
  specs/           Gherkin specs (generated by /discover or /story)
  fixtures/        Auth + page fixtures; app.fixture.ts for team-owned extensions (see harness-docs/FIXTURES.md)
  helpers/         Shared plain functions: UI sequences, assertions, mocks
  helpers/INDEX.md One-line description per helper (read before loading any helper)
  locators/        Locator registry — high-churn shared locators only (nav, CTAs, headings)
  locators/INDEX.md One-line per registry file (read before loading any registry file)
  patterns/        App-specific UI patterns written by /discover — one file per page group
  tests/
    generated/     AI-written tests (gitignored)
    stable/        Promoted tests (version-controlled, CI)
test-history/      Per-spec run records written by CI — feeds /stability
examples/          Example app overlays for dogfooding (forks may delete) — see examples/README.md
scripts/           CI utilities (record-history.js)
.github/
  .claude/skills/  Shared skills (story, discover, generate, heal, promote, promote-locator, locator-audit, validate, specs, stability)
  agents/          Copilot agent definitions
  workflows/       CI pipeline
.claude/
  agents/          Claude Code agent definitions
  skills/          playwright-cli command reference
playwright.config.ts
AGENTS.md          Test generation workflow (owns: phases, trace, quality gates)
AGENTS-REFERENCE.md Workflow appendix — loaded on demand
STANDARDS.md       Coding standards (owns: locators, assertions, file limits)
.gitignore         Git ignore patterns

Commands

Setup & diagnostics

Command	Description
`npm run setup`	Interactive first-time setup — prompts for `BASE_URL`, `AUTH_REQUIRED`, credentials; installs browsers; runs `sync:skills` to populate `.claude/skills/`
`npm run setup:auth`	Re-run just the auth-state capture step (when saved session expires)
`npm run doctor`	Diagnostic — verifies Node version, browsers, `.env`, `playwright-cli` install

Running tests

Command	Description
`npm test`	Run all tests (generated + stable)
`npm run test:generated`	Run only generated tests (quarantine zone)
`npm run test:stable`	Run only stable tests (what CI runs)
`npm run test:headed`	Run with visible browser
`npm run test:generated:headed` / `npm run test:stable:headed`	Same, scoped
`npm run test:debug`	Run with interactive CLI debugger
`npm run report`	Open the HTML report from the last run

Quality gates

Command	Description
`npm run typecheck`	TypeScript compilation check (Gate #1)
`npm run check:invariants`	AST-based quality gates on all generated + stable tests (Gates #3, #5, #6, #8, #13)
`npm run check:invariants:generated` / `:stable`	Same, scoped
`npm run audit:locators`	Report promotion candidates, dead registry entries, under-threshold entries (human-readable)
`npm run audit:locators:json`	Same, JSON (consumed by `/locator-audit` skill)
`npm run audit:locators:ci`	Non-zero exit if anything needs attention — CI-enforced (Gate #12)
`npm run scan:credentials:stable`	Block hardcoded `TEST_USERNAME`/`TEST_PASSWORD` on auth-success paths in stable tests (Gate #9, CI-enforced)

Observability & maintenance

Command	Description
`npm run dashboard`	Generate SDET dashboard (per-test health, promotion readiness, recent runs, heal hotspots) — writes `test-results/dashboard.html`
`npm run dashboard:open`	Generate and open the dashboard in the default browser
`npm run harness:report`	Summarize `test-history/harness-runs.jsonl` — phase breakdown, reliability, reuse, top expensive specs
`npm run sync:agents`	Sync canonical agent bodies (`.github/agent-bodies/*.body.md`) into Claude Code + Copilot shims
`npm run sync:agents:check`	CI-enforced — fails if any shim has drifted from its canonical body
`npm run sync:skills`	Populate `.claude/skills/` from the canonical `.github/.claude/skills/` tree. Symlinks on macOS/Linux; directory copies on Windows. Run this whenever you edit a skill file — or if Claude Code says it can't find a skill after a fresh clone. `npm run setup` invokes this automatically on first run.
`npm run sync:skills:check`	CI-enforced — fails if the `.claude/skills/` mirror has drifted. Also runs in the pre-commit hook to catch stale Windows copies before they reach a PR.
`npm run classify:steps`	Pre-dispatch step classifier (see `scripts/classify-steps.js` for the input schema)
`npm run spec-hash <file>` / `spec-hash:read <file>`	Compute / read the 8-char spec hash that drives skip-on-unchanged logic

Architecture

This template works in both Claude Code and GitHub Copilot. Skills are shared; agent definitions are platform-specific due to incompatible schemas.

Ownership map — who owns what. Three files, three distinct scopes. No document claims jurisdiction over more than one of these, so there's no question about where to look:

Scope	File	Owns
Workflow	AGENTS.md	How agents run — phases, trace artifact, quality gates, manifest, scenario-to-test mapping. Auto-loaded every run; operational detail split to AGENTS-REFERENCE.md.
Code standards	STANDARDS.md	What generated `.spec.ts` must look like — locators, assertions, strict mode, file limits, forbidden patterns.
Design rationale	harness-docs/DEEP-DIVE.md	Why decisions were made — architecture, trade-offs, historical context. Not a how-to; a why-to.

Supporting documents:

Document	What it covers
HARNESS.md	Navigational — what the harness is, file ownership, platform setup, maintenance.
FORK-AND-SYNC.md	Forking, cloning, syncing forks with upstream, file categories (core vs. test artifacts), contributor roles
ARCHITECTURE_OVERVIEW.md	One-page flow diagram
OPERATIONS.md	Day-to-day operator commands; CI failure → local heal flow; scale strategies at 3000 tests
POM-ANALYSIS.md	Page Object Model rationale (why this harness doesn't use POM)
e2e/traces/SCHEMA.md	Exploration trace schema (Phase 2 → Phase 3 bridge)
test-history/HARNESS-SCHEMA.md	Harness run-record schema (observability layer)

Configuration

What	Where
Base URL	`.env` → `BASE_URL=https://your-app.com` (copy from `.env.example`)
Auth (if needed)	`.env` → `AUTH_REQUIRED=true`, `TEST_USERNAME`, `TEST_PASSWORD`. The `auth.setup.ts` project logs in and saves session state; all tests start authenticated. If `AUTH_REQUIRED=false` (default), auth is skipped. To customize the login flow for your app, edit the `loginProfile()` body in `e2e/tests/auth.setup.ts` — see harness-docs/HARNESS.md → Manual auth setup.
Multi-profile auth (RBAC, per-region, per-tenant, per-locale)	Add suffixed pairs to `.env` — `TEST_USERNAME_<NAME>` / `TEST_PASSWORD_<NAME>` (e.g. `TEST_USERNAME_ADMIN`, `TEST_USERNAME_CA`). For many profiles, use `npm run setup -- --profiles=<file.json>` to bulk-import (file must live outside the repo). Setup writes `playwright/.auth/{name}.json` per profile. Bind a spec to a profile with `@profile:<name>` at the Feature line, or use the `loginAs(name)` fixture for cross-profile scenarios in one test. Limit which profiles run setup with `AUTH_PROFILES=ca,tx`. To run the whole suite once per profile (e.g. all 50 US states), set `MULTI_PROFILE_PROJECTS=true` — each profile becomes its own Playwright project. 5-minute walkthrough: harness-docs/MULTI-PROFILE-QUICKSTART.md. Full reference: harness-docs/FIXTURES.md → Multi-profile credentials.
Bot-detection workaround (optional)	Set `WARMUP=true` to run `warmup.setup.ts` once before tests — visits `BASE_URL`, dismisses cookies, saves state to `playwright/.auth/warmup.json`. Every test then loads that state so requests look like a returning visitor. Enable only when: tests against the target intermittently return 406/forbidden on first request (some WAFs fingerprint empty cookie jars as bots). Do not enable for: apps you own with IPs allowlisted, or staging/pre-prod tiers with WAF relaxed — it's unnecessary overhead. Ignored when `AUTH_REQUIRED=true` (auth state already contains cookies). Does not help with IP-based rate limiting — for that, cap workers or allowlist the CI egress IP.
Rally (optional)	`.mcp.json` (Claude Code) and `.vscode/mcp.json` (Copilot) pre-configure the Rally MCP server. Your API key is prompted on first use and stored securely — never written to disk. See Rally Integration for how to create an API key and verify the connection.
CI	Set `BASE_URL` as a repo variable. If auth is needed, set `TEST_USERNAME` and `TEST_PASSWORD` as encrypted secrets (and any `TEST_USERNAME_<NAME>` / `TEST_PASSWORD_<NAME>` pairs for multi-profile). For sharded multi-profile runs, combine `AUTH_PROFILES` per matrix job with `--project=<name>` selection.
Browser	`playwright.config.ts` (default: Chromium via `channel: 'chrome'` — real Chrome binary for closer-to-prod rendering and to avoid `HeadlessChrome` UA substring that some WAFs flag)

Environment Gate: If .env is missing or incomplete when you run a skill (/discover, /generate, etc.), the agent will stop, ask you for the required values, and create .env before proceeding. If /discover detects a login wall during exploration, it will prompt you to enable auth and provide credentials. See HARNESS.md → Environment Gate for details.

Rally Integration (optional)

The /story TC12345 skill can pull user stories, defects, and test cases directly from Rally. It uses the Rally MCP server — a Model Context Protocol server hosted by Broadcom that exposes Rally artifacts as structured tool calls.

Step 1 — Create a Rally API key

Log in to your Rally instance (e.g. https://rally1.rallydev.com).
Click your profile avatar (top-right) → My Profile.
In the left sidebar, click API Keys (or navigate to My Profile → Security).
Click Create New API Key.
Give the key a description (e.g. qe-harness) and click Create.
Copy the key immediately — Rally only shows it once.

If you don't see the API Keys option, your workspace administrator may need to enable it. Ask your Rally admin to go to Workspace → Workspace Settings → Allow API Key Creation.

Step 2 — Configure the MCP server for GitHub Copilot

The workspace ships with a pre-configured .vscode/mcp.json. VS Code Copilot reads this file automatically and registers the Rally MCP server.

No manual editing is required. When you run /story TC12345 for the first time, Copilot will display a secure prompt asking for your Rally API key. The key is stored by VS Code's secret storage and is never written to disk.

If you want to inspect or update the configuration:

// .vscode/mcp.json
{
  "servers": {
    "rally": {
      "url": "https://mcp.rallydev.com/mcp",
      "headers": {
        "Authorization": "Bearer ${input:rally-api-key}"
      }
    }
  },
  "inputs": [
    {
      "id": "rally-api-key",
      "type": "promptString",
      "description": "Rally API Key for MCP server authentication",
      "password": true
    }
  ]
}

Step 3 — Configure the MCP server for Claude Code

Claude Code reads .mcp.json (project root), which is already pre-configured with the same Rally server definition. The first time the mcp__rally__* tools are invoked, Claude Code will prompt you for your API key.

Verifying the connection

Run /story TC12345 (replace with a real Rally artifact ID in your workspace). If the connection is working, the agent will print the artifact name and description before generating a spec. If it prints "paste the story text", see the troubleshooting entry in the table below.

CI / non-interactive environments

In CI, the MCP server is not used — the /story skill only runs interactively. The Rally API key should not be added to CI secrets. If you need to pull Rally artifacts in a CI pipeline, use the REST API script directly:

RALLY_API_KEY=<your-key> node scripts/get-rally-test-case.js --artifact TC12345

Set RALLY_API_KEY as an encrypted secret in your GitHub repository settings (Settings → Secrets and variables → Actions).

Troubleshooting

Symptom	Likely cause	Fix
`playwright-cli: command not found`	Not installed globally	`npm install -g @playwright/cli@latest && playwright-cli install --skills`
Skill prompts to approve `Bash(node:*)` every run	Running an old fork where skill frontmatter is incomplete	`git pull` — the shipped `allowed-tools` covers `node`, `cat`, `git`, etc.
`AUTH_REQUIRED=true` but tests redirect to `/login`	`auth.setup.ts` selectors don't match your app	Edit the `loginProfile()` body in `e2e/tests/auth.setup.ts` (the three `CUSTOMIZE` markers) — update `page.goto('/login')`, the `getByLabel` patterns, and the post-login `waitForURL`
`/generate` spawns 0 workers	No spec files exist	Run `/discover` or `/story` first to create specs in `e2e/specs/`
Test fails with "Target closed" / browser crash	Stale `playwright-cli` session	`playwright-cli close-all`
`/validate` marks tests as `STALE`	Spec was edited but test wasn't regenerated	`/generate <spec-name>` — or `/generate all --force` for a full refresh
CI fails on `sync:agents:check`	You edited a `.md` shim directly instead of the `.body.md`	Edit `.github/agent-bodies/{agent}.body.md` instead, then `npm run sync:agents`
Intermittent 406/forbidden on first request	WAF fingerprints empty cookie jars	Set `WARMUP=true` in `.env` (see Configuration)
Tests pass locally, fail in CI	Usually a timing or auth-state issue	Download the `test-results` artifact from the failed run: `gh run download <id> --name test-results` → then `/heal stable`
(Copilot) `@qe-engineer discover` does nothing	Copilot custom agents are invisible to `@` mention autocomplete	Open the Agent dropdown in Copilot Chat and select `qe-engineer` there — the dropdown is the only entry point. Then type `discover` (without the `@` or leading slash)
`/story TC12345` prints "paste the story text"	Rally MCP (`mcp__rally__*`) not available in the current host or not authenticated	Copy the Rally artifact body (name + description + AC) and paste it into the chat, then re-run `/story`. Everything after the fetch is identical regardless of input source
(Copilot) `/story --then-generate` saves the spec but no tests appear	The `Skill(...)` tool used for chaining is Claude-Code-only	`/story` will print the manual follow-up (`/generate {filename}.feature` or `/generate all`). Run that next
(Copilot) `playwright-cli` calls take ~10–15s each	VSCode sandbox approval prompt on every call	Approve `playwright-cli:*` once at the session level when the first prompt appears; see playwright-cli skill → Environment considerations
(Windows) Slash command fails with `'$BASE_URL' is not recognized`, `'for' was unexpected`, or similar	Skill body uses bash idioms; agent shelled out to cmd.exe / PowerShell instead of Git Bash	Confirm Git Bash is on PATH (`bash --version` should resolve). Reinstall Git for Windows if missing — it adds Git Bash to PATH by default.
(Windows) Skill complains it can't find a file under `.claude/skills/` after a fresh clone	Tracked symlinks materialize as text stubs when `core.symlinks=false`	`npm run sync:skills` repopulates the mirror with directory copies on Windows. `npm run setup` runs this automatically on first run.
(Windows) Pre-commit hook errors with `sh: not found`	Git wasn't installed with Git Bash, or `sh.exe` isn't on PATH	Reinstall Git for Windows with the default options, or skip hooks for one commit with `git commit --no-verify` (then fix PATH before pushing — CI runs the same checks).

More error patterns and their fixes: e2e/ERROR_INDEX.md (30+ entries covering locator failures, navigation timeouts, strict-mode violations, auth errors).

Still stuck? DEEP-DIVE.md explains the rationale behind most design decisions ("why is it built this way?"). For operational steps — "what command do I run now?" — use AGENTS.md (workflow) or harness-docs/OPERATIONS.md (day-to-day + CI failure flow).

Navigation Menu

Repository files navigation

Playwright E2E Test Template

Prerequisites

Platform support

Quick Start

Post-install setup (Option B — fork & clone)

Where do I type `/discover`, `/generate`, etc.?

Your first run

Dogfooding against an example app (maintainer-only)

For SDETs: I have a test idea — now what?

My test case isn't in any of those formats — now what?

What if I get stuck?

Workflow

Skills

Project Structure

Commands

Setup & diagnostics

Running tests

Quality gates

Observability & maintenance

Architecture

Configuration

Rally Integration (optional)

Step 1 — Create a Rally API key

Step 2 — Configure the MCP server for GitHub Copilot

Step 3 — Configure the MCP server for Claude Code

Verifying the connection

CI / non-interactive environments

Troubleshooting

About

Releases

Packages

Used by

Contributors

Languages

Name		Name	Last commit message	Last commit date
.claude		.claude
.github		.github
.husky		.husky
.vscode		.vscode
bin		bin
e2e		e2e
examples		examples
harness-docs		harness-docs
scripts		scripts
test-history		test-history
.env.example		.env.example
.gitattributes		.gitattributes
.gitignore		.gitignore
.mcp.json		.mcp.json
.npmignore		.npmignore
.npmrc.jfrog		.npmrc.jfrog
.nvmrc		.nvmrc
AGENTS-REFERENCE.md		AGENTS-REFERENCE.md
AGENTS.md		AGENTS.md
CLAUDE.md		CLAUDE.md
README.md		README.md
STANDARDS.md		STANDARDS.md
gitignore		gitignore
package.json		package.json
playwright.config.ts		playwright.config.ts
spec-lint.config.json		spec-lint.config.json
tsconfig.json		tsconfig.json

Latest commit

History

Folders and files

Repository files navigation

Playwright E2E Test Template

Prerequisites

Platform support

Quick Start

Post-install setup (Option B — fork & clone)

Where do I type /discover, /generate, etc.?

Your first run

Dogfooding against an example app (maintainer-only)

For SDETs: I have a test idea — now what?

My test case isn't in any of those formats — now what?

What if I get stuck?

Workflow

Skills

Project Structure

Commands

Setup & diagnostics

Running tests

Quality gates

Observability & maintenance

Architecture

Configuration

Rally Integration (optional)

Step 1 — Create a Rally API key

Step 2 — Configure the MCP server for GitHub Copilot

Step 3 — Configure the MCP server for Claude Code

Verifying the connection

CI / non-interactive environments

Troubleshooting

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

Where do I type `/discover`, `/generate`, etc.?