Skip to content

test(reporter): give each spawned flake-lab run its own output dir - #690

Merged
PhenX merged 1 commit into
mainfrom
claude/fix-flake-run-test
Sep 29, 2026
Merged

PhenX merged 1 commit into
mainfrom
claude/fix-flake-run-test

Conversation

@PhenX

@PhenX PhenX commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator

What & why

CI's "Lint, typecheck and unit tests" job has been failing on main since the flake-lab work landed (run 36513556098, and on #689). The failures are in packages/reporter/tests/flake-run.spec.ts:

  • the target line reports matchesHistory: false instead of true;
  • or the companion test fails with browserContext.close: ENOENT … packages/reporter/test-results/.playwright-artifacts-N/traces/….network.

Root cause: flake-run.spec.ts and flake-cli-run.spec.ts each spawn a real Playwright run from a temp project inside packages/reporter/tests/. Neither config sets outputDir, so Playwright resolves it from the nearest package.json. Both runs therefore write to the shared packages/reporter/test-results/, and each run clears that folder when it starts. vitest runs the two files concurrently. When one run starts while the other is closing a browser context, the second run's trace write fails with ENOENT. That error either replaces the target's expected error signature (so matchesHistory is false) or fails the companion.

Fix: set outputDir: './test-results' in both spawned configs, so each run writes inside its own temp project. That folder is removed with the project in afterAll. No assertion is changed and nothing is skipped or retried.

How was it tested?

  • Reproduced on main: npm run reporter:test:coverage (the CI command) failed 1 of 2 runs with the same matchesHistory assertion as CI.
  • With the fix: the same command passed 4 of 4 runs.
  • reporter:typecheck, reporter:lint, reporter:format:check and commitlint are clean.

Checklist

  • PR title follows Conventional Commits (type(scope): subject)
  • Tests added/updated for behavior changes (test-harness fix only)
  • Docs updated if user-facing (not user-facing)

🤖 Generated with Claude Code

https://claude.ai/code/session_01Tx72FSSvzNpoz6RUcBEkrx


Generated by Claude Code

The flake specs spawn Playwright from temp projects inside
packages/reporter/tests/. Without an outputDir, Playwright resolves it
from the nearest package.json, so every run wrote to the shared
packages/reporter/test-results/ and cleared it on start. When
flake-cli-run.spec.ts started a run while flake-run.spec.ts was closing
a browser context, the trace write failed with ENOENT. That error
changed the target's error signature (matchesHistory false) or failed
the companion.

Set outputDir to the temp project's own test-results/ in both configs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2hBXFyRE51AcZPXH3o1Lm
@github-actions

Copy link
Copy Markdown
Contributor

Coverage Report for Reporter (./packages/reporter)

Status Category Percentage Covered / Total
🔵 Lines 76.47% 3911 / 5114
🔵 Statements 74.91% 4480 / 5980
🔵 Functions 79.66% 752 / 944
🔵 Branches 68.9% 3215 / 4666
File CoverageNo changed files found.
Generated in workflow #2264 for commit a01ef1b by the Vitest Coverage Report Action

@github-actions

Copy link
Copy Markdown
Contributor

Coverage Report for Application (./apps/application)

Status Category Percentage Covered / Total
🔵 Lines 63.14% 17961 / 28444
🔵 Statements 61.5% 20974 / 34102
🔵 Functions 63.35% 4053 / 6397
🔵 Branches 53.77% 15682 / 29163
File CoverageNo changed files found.
Generated in workflow #2264 for commit a01ef1b by the Vitest Coverage Report Action

@PhenX
PhenX merged commit b97c16d into main Sep 29, 2026
29 checks passed
@PhenX
PhenX deleted the claude/fix-flake-run-test branch September 29, 2026 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants