Open Source Playwright Reporter

Deterministic Playwright failure triage: CLI diagnosis, root-cause grouping, and a shareable debugging report.

What You Get

SAMPLE REPORT: https://sentinelqa.com/share/run/permanent-demo-playwright-report

After a failed Playwright run, Sentinel prints:

• a deterministic CLI diagnosis (1–3 root causes)
• how many tests are affected per root cause
• what to inspect first (usually trace)
• a report link (hosted by default, local in offline mode)

Requirements

• Node.js 18+
• @playwright/test 1.40+ (newer is recommended)

Install

npm install -D @sentinelqa/playwright-reporter

Setup (Recommended)

Wrap your playwright.config.ts:

import { defineConfig } from "@playwright/test";
import { withSentinel } from "@sentinelqa/playwright-reporter";

export default withSentinel(
  defineConfig({
    reporter: [["line"]],
    outputDir: "test-results",
  }),
  {
    project: "my-app",
  },
);

Run tests normally:

npx playwright test

How It Works (High Level)

Sentinel is a Playwright reporter that:

1. Ensures a Playwright JSON report exists (adds a JSON reporter if needed).
2. Collects the artifacts Playwright already produces (trace.zip, screenshots, video, logs, report.json).
3. Builds a deterministic failure summary:
   • groups repeated failures into 1–3 canonical root causes
   • extracts normalized evidence (where, blocker, target state, expected/received)
4. Publishes a report:
   • default: a hosted share link
   • offline: a local HTML report folder (no upload)

This project is intentionally heuristic-driven (not “AI guesses”) for root-cause grouping.

Modes

Sentinel behavior is controlled by SENTINEL_MODE.

Hosted Mode (Default)

Do nothing. Sentinel uploads and prints a share link when failures happen.

Environment:

• SENTINEL_MODE unset (or SENTINEL_MODE=hosted)

Workspace Mode (Private History)

If you have a workspace token:

SENTINEL_TOKEN=your_project_ingest_token npx playwright test

Environment:

• SENTINEL_MODE unset (or SENTINEL_MODE=hosted)
• SENTINEL_TOKEN=...

Offline Mode (No Uploads)

To keep everything local:

SENTINEL_MODE=offline npx playwright test

Offline mode is strict:

• uploads are skipped
• a local report is generated (default ./sentinel-report/index.html)
• the CLI prints a local file:// link (or a relative path)

If a hosted upload fails, Sentinel falls back to generating the local report automatically.

Local Workspace Uploads (When Not In CI)

If you set SENTINEL_TOKEN locally, Sentinel will not upload by default (to avoid accidental data egress). To allow a local upload to your workspace, set:

SENTINEL_UPLOAD_LOCAL=1 SENTINEL_TOKEN=your_project_ingest_token npx playwright test

This is useful for quickly generating a private run history entry from your laptop.

Implicit Local Public Upload (No Token, Not In CI)

If you are not in CI and you did not set SENTINEL_TOKEN, Sentinel can still upload in public mode by enabling:

SENTINEL_UPLOAD_LOCAL=1 npx playwright test

Share Links Are Optional (Open Source / Self-Hosted)

• Hosted mode prints a share URL on failures.
• Offline mode skips uploads and produces only a local report.

Pick the behavior explicitly with SENTINEL_MODE (and SENTINEL_UPLOAD_LOCAL for local uploads).

Serving Local Reports

Local reports are static HTML + copied artifacts. You can open index.html directly, or run a tiny local server:

cd sentinel-report
npx --yes serve -p 4173 .

Then open http://localhost:4173.

Secrets Masking

Sentinel masks common secret patterns before data is shared. This includes:

• environment-variable looking strings
• common credential/token patterns
• internal URLs/hosts (where possible)

If you find something that should be masked but isn’t, file an issue with a minimal reproducible example (redact the secret).

Optional: Richer Failure Evidence

withSentinel() is enough for most setups. If you want richer UI evidence for actionability/timeouts, you can attach the failure capture fixture:

// tests/test.ts
import { test as base, expect } from "@playwright/test";
import { attachSentinelFailureCapture } from "@sentinelqa/playwright-reporter/fixtures";

export const test = attachSentinelFailureCapture(base);
export { expect };

Then import from that file in your specs:

import { test, expect } from "./test";

Configuration Options

withSentinel(config, {
  project: "my-app",
  playwrightJsonPath: "playwright-report/report.json",
  playwrightReportDir: "playwright-report",
  testResultsDir: "test-results",
  artifactDirs: ["tmp/extra-artifacts"],
  verbose: true,
});

Troubleshooting

“POST /api/runs failed …”

• Check SENTINEL_MODE (offline skips uploads).
• If using a workspace token, confirm SENTINEL_TOKEN is correct.
• If a server schema changed, upgrade the reporter/uploader and/or run DB migrations for self-hosted deployments.

“Local report generation failed …”

Make sure Playwright produced:

• playwright-report/report.json
• test-results/ (or your configured outputDir)

Then rerun with SENTINEL_MODE=offline to force local generation.

License

See LICENSE.