Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [Unreleased]

### Added

- Consumer bootstrap section in `templates/README.md` (templates → `tools/` → `npm run validate` / `score`); README enforcement map aligned with `validate-suite.sh` and `score-tests.js`.

## [3.1.1] - 2026-05-24

### Fixed
Expand Down
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,40 @@ See `patterns/anti-patterns.md` for the full list.

---

## Enforcement map

Not every row in `patterns/anti-patterns.md` is automated. Two CLI tools split responsibility; semantic gaps (test intent, data quality, network/auth strategy) stay in code review. See [`ARCH.MD` § Enforcement pipeline](ARCH.MD#enforcement-pipeline) for the diagram.

| Anti-pattern | `validate-suite.sh` | `score-tests.js` | Review / ESLint only |
|--------------|---------------------|------------------|----------------------|
| `waitForTimeout()` | Error | −10 | `eslint-plugin-playwright` |
| `waitForLoadState('networkidle')` | Error | −10 | ESLint |
| `expect(await x.isVisible())` | Error | −5 | ESLint (`prefer-web-first-assertions`) |
| `expect(await x.textContent())` | Error | — | ESLint |
| `.toBeTruthy()` / `.toBeFalsy()` | Error (truthy only) | −5 (both) | — |
| `test.only()` | Error | −3 | ESLint + `forbidOnly` on CI |
| `page.pause()` | Error | −5 | ESLint |
| `if (await …)` branching | Warning | −3 | ESLint (`no-conditional-in-test`) |
| `{ force: true }` | Warning | −2 | ESLint (warn) |
| CSS class/id in `.locator('…')` | Warning (class in string) | −2 each (cap −10) | — |
| `nth-child()` / `nth-of-type()` | Warning | −5 | — |
| `xpath=` selector | Warning | −5 | — |
| `page.$()` (deprecated) | Warning | — | ESLint |
| `page.click('text=…')` | Warning | — | — |
| `setTimeout()` in tests | Warning | — | — |
| Spec imports `@playwright/test` not `./fixtures` | — | −5 | `patterns/fixtures.md` |
| Missing `[TC-XXX]` in test title | — | −3 each (cap −10) | — |
| No `expect()` / fewer expects than tests | — | −15 / −5 | ESLint (`expect-expect`) |
| Missing `@P0`–`@P3` priority tag | — | −5 | — |
| Missing `@smoke` / `@regression` / `@critical` / `@a11y` | — | −5 | — |
| File > 400 lines | — | −5 | `patterns/test-structure.md` |
| POM holds test assertions | — | — | `patterns/page-object-model.md` |
| Auth, network mocking, CI sharding, visual baselines | — | — | respective `patterns/*.md` |

**Grep spot-check (2026-05-24):** rows above match `check_error` / `check_warning` in `tools/validate-suite.sh` and the rubric comment + `scoreFile()` in `tools/score-tests.js`. Warnings in validate-suite do not fail CI unless you pass `--strict`.

---

## Quality Gates

Before tests are "done":
Expand Down
69 changes: 69 additions & 0 deletions templates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,77 @@ Copy these into a fresh test project. Order matters.

---

## Consumer bootstrap

Greenfield repos can vendor playwrighter and pass the quality gates without reading all 23 pattern files first. Copy in this order, then run the npm scripts wired in `templates/package.json`.

### 1. Keep a playwrighter checkout (patterns + skill)

Clone or submodule the [playwrighter](https://github.com/weijia-89/playwrighter) repo somewhere on disk. You need it for:

- **`skill/SKILL.md`** — agent mandatory workflow (read patterns → use templates → validate before done)
- **`patterns/`** — consulted per task, not copied wholesale
- **`tools/`** — copied into your project in step 3

Wire your AI agent to the skill (symlink or copy `skill/SKILL.md`; see root `README.md` § Integration). Agents should follow the [Mandatory Workflow](../skill/SKILL.md#mandatory-workflow): read relevant `patterns/` files, start from these templates, then run validate + score.

### 2. Copy templates into your project root

From your playwrighter checkout:

```bash
PLAYWRIGHTER=/path/to/playwrighter
PROJECT=your-project

mkdir -p "$PROJECT/tests/specs" "$PROJECT/tests/pages"
cp "$PLAYWRIGHTER/templates/package.json" "$PROJECT/"
cp -r "$PLAYWRIGHTER/templates/scripts" "$PROJECT/"
cp "$PLAYWRIGHTER/templates/playwright.config.ts" "$PROJECT/"
cp -r "$PLAYWRIGHTER/templates/pages/"* "$PROJECT/tests/pages/"
cp "$PLAYWRIGHTER/templates/fixtures.ts" "$PROJECT/tests/"
cp "$PLAYWRIGHTER/templates/auth.setup.ts" "$PROJECT/tests/"
cp "$PLAYWRIGHTER/templates/test-template.ts" "$PROJECT/tests/specs/example.spec.ts"
```

Adjust paths if your suite lives under a different folder; update `playwright.config.ts` `testDir` and the `./tests` paths in `package.json` scripts to match.

### 3. Copy `tools/` (validate + score)

```bash
cp -r "$PLAYWRIGHTER/tools" "$PROJECT/tools"
chmod +x "$PROJECT/tools/validate-suite.sh"
```

`scripts/check-tools.js` (from step 2) fails fast with copy instructions if `tools/` is missing.

### 4. Install, configure env, gate before merge

```bash
cd "$PROJECT"
npm install
npx playwright install chromium --with-deps

cat > .env << 'EOF'
BASE_URL=http://localhost:3000
TEST_USER_EMAIL=user@example.com
TEST_USER_PASSWORD=TestPass123!
EOF
echo -e ".env\nplaywright/.auth/" >> .gitignore

npm run validate # check-tools → validate-suite → lint → typecheck
npm run score # check-tools → score-tests (threshold 80)
```

Both scripts resolve `./tools/` relative to your project root — do not edit those paths unless you relocate `tools/`.

**CI:** run `npm run validate` then `npm run score` on every PR (same order as [dogfood workflow](../.github/workflows/dogfood-northwind-qa.yml) against northwind-qa).

---

## Order of Operations

See [Consumer bootstrap](#consumer-bootstrap) for the full copy-paste flow. Summary:

1. **`package.json`**, install dependencies first
2. **Copy `tools/`** from the playwrighter repo into your project root (`validate-suite.sh`, `score-tests.js`)
3. **`playwright.config.ts`**, Playwright config (references `playwright/.auth/`)
Expand Down
Loading