Skip to content

TESTING.md

Latest commit

 

History

History
252 lines (177 loc) · 9.35 KB

TESTING.md

File metadata and controls

252 lines (177 loc) · 9.35 KB
Raw

Doom workspace — testing guide

Last updated: 2026-06-17

Canonical testing documentation for the HyperCrab2000 Doom stack under IdeaProjects/doom/. Each sub-repo has a repo-local copy or summary that links here.

Repositories

Repo Role Test doc
doom-wad-lab Browser app, WebGL renderer, parity harness docs/TESTING.md
doom-wad-core WAD parse, GZSTATE export (@hypercrab2000/doom-wad-core) docs/TESTING.md
doom-gzengine-core Game engine WASM scaffold, GZTICK docs/TESTING.md
gzdoom-project GZDoom fork — -dumpgzstate, future -dumpgztick docs/PARITY-TOOLS.md

Philosophy

  1. Every bug becomes a regression test — reproduce, fix, widen the suite.
  2. GZDoom is ground truth — Node/TS/WASM outputs compare against fork dumps, not against each other in isolation.
  3. Layered gates — unit → golden → state (GZSTATE) → draw-state → frame pixels → tick (GZTICK, planned).
  4. Parallel by default — ~130 unit files and heavy IWAD corpora run across CPU cores; see Parallelization.

Prerequisites

Always (unit smoke)

  • Node 20+
  • npm ci in each repo you touch

IWAD parity (local)

Place commercial IWADs (not in git):

doom-wad-lab/public/wads/DOOM.WAD
doom-wad-lab/public/wads/DOOM2.WAD

GZSTATE corpus (68 maps)

Generate once after GZDoom exporter or parser changes:

cd doom-wad-lab
npm run corpus:parity:all

Artifacts land in artifacts/gzrender-v2/corpus/{DOOM,DOOM2}/<MAP>/.

GZDoom binary

Build from gzdoom-project/ (see PARITY-TOOLS.md). Dump scripts in doom-wad-lab/tools/gzrender-v2/ invoke it.

Quick commands (developer)

Run from repo roots after npm ci.

doom-wad-lab

Goal Command Typical wall time*
Full unit suite (~130 files) npm run test:unit 1–3 min
All CPU cores npm run test:unit:fast 1–2 min
GZSTATE 68 maps npm run test:corpus ~20s
Modular stages @ spawn npm run test:modular ~3s
Vanilla BSP invariants npx vitest run --project unit src/wad/renderer/bsp/vanilla/vanillaBspParity.test.ts ~78s
E1M1 frame diff GZFRAME_PARITY_REQUIRED=1 npm run test:frame varies
Mod stack GZSTATE npm run test:mod-parity varies
Integration (browser canvas) npm run test:integration 1–3 min
Coverage gate (≥90%) npm run test:coverage 2–4 min
Production build npm run build ~30s

*Local M-series Mac, 8–16 cores; CI runners differ.

doom-wad-core

npm test          # unit + fixture parity
npm run build

doom-gzengine-core

npm test          # GZTICK codec + types (scaffold)
npm run build

Full stack smoke (parity developer)

cd doom-wad-core && npm run build && npm test
cd ../doom-wad-lab && npm run test:corpus && npm run test:modular

Parallelization

Heavy suites were optimized 2026-06-17. Three mechanisms work together:

1. Vitest file pool (doom-wad-lab/vitest.config.ts)

Setting Unit project Integration project
pool threads forks
maxWorkers cpus - 1 (override: VITEST_MAX_WORKERS) min(4, maxWorkers)
fileParallelism true true
maxConcurrency min(16, cpus) for it.concurrent default
testTimeout 60s 180s
hookTimeout 120s —

Why threads for unit? CPU-heavy BSP sector-probe batches exceeded fork RPC onTaskUpdate timeouts (~60s). threads keeps long map batches stable while still parallelizing ~130 test files.

Why forks for integration? Puppeteer / canvas isolation; cap workers at 4 to avoid browser contention.

# Use every core for unit files
VITEST_MAX_WORKERS=100% npm run test:unit
# or
npm run test:unit:fast

2. In-test parallelism (doom-wad-lab/test/parallelMap.ts)

  • parallelMap(items, worker, concurrency?) — bounded async map; default concurrency min(16, cpus).
  • batchItems(items, size) — splits IWAD map lists so no single Vitest case runs past worker RPC limits.
  • Override in-test workers: VITEST_IN_TEST_PARALLEL=8.

Used by:

  • vanillaBspParity.test.ts — 68 maps in batches of 4, parallel sector probes per batch.
  • corpus.parity.test.ts — parallel map export vs GZDoom fixtures per IWAD.

3. Snapshot cache (spawnStageSnapshotHarness.ts)

captureSpawnModularFrameSnapshot() memoizes Classic GL and WASM federated snapshots per (wad, map, backend).

modularStageParity.test.ts:

  • beforeAll — preload all IWAD maps, capture Classic once, warm WASM cache.
  • describe.concurrent — 11 per-stage BSP hash tests read cache (no 68×11 redundant captures).

Before: redundant captures made test:modular very slow. After: ~3s.

Test layers (GZRender-V2)

Layer 1  Unit          Parsers, GZSTATE codec, raster, geometry helpers
Layer 2  Golden        Fixture byte stability
Layer 3  State         GZSTATE Node vs GZDoom (per map)
Layer 4  Draw-state    BSP visible set, modular stages, vanilla invariants
Layer 5  Frame         PNG diff vs GZDoom reference (E1M1 gate)
Layer 6  Event         Scripted input timelines (planned)
Layer 7  Tick          GZTICK engine vs GZDoom (doom-gzengine-core, planned)
Layer 8  WASM smoke    Browser WebGL2 federated path

Closed gates (2026-06-17)

Gate Maps Command Status
GZSTATE load 68 npm run test:corpus Closed
Modular BSP @ spawn 68 × 11 stages npm run test:modular Closed*
Vanilla BSP invariants 68 + 9000+ sector probes vanillaBspParity.test.ts Closed

*Valid while WASM federated delegates to Classic drawScene (GAP-0003).

Open gates

Gate Command Tracker
E1M1 frame pixels npm run test:frame GAP-0001, GAP-0002
Independent WASM draw manual + future test GAP-0003
Mod stack runtime npm run test:mod-parity, mod:parity GAP-0005
Game engine tick doom-gzengine-core GAP-0006

See parity-gap-tracker.md.

CI vs local

doom-wad-lab GitHub Actions (ci.yml)

Runs on every PR and main push:

  1. npm ci
  2. SoundFont download (music paths)
  3. npm run test:unit
  4. npm run test:coverage (≥90% on scoped code)
  5. npm run test:integration (IWAD tests skip without WADs)
  6. npm run build
  7. Smoke: vite preview + Puppeteer console capture

Not in default CI yet: test:corpus, test:modular, vanilla BSP corpus — require IWADs + generated artifacts. Run locally before merging parity work.

Recommended pre-merge (parity PRs)

cd doom-wad-lab
npm run corpus:parity:all    # if exporter changed
npm run test:corpus
npm run test:modular
npx vitest run --project unit src/wad/renderer/bsp/vanilla/vanillaBspParity.test.ts
npm run build

Troubleshooting

Symptom Cause Fix
Missing IWAD / corpus skip No public/wads/*.WAD Add IWADs locally; CI skips by design
Missing corpus summary Artifacts not generated npm run corpus:parity:all
[vitest-worker]: Timeout calling "onTaskUpdate" Long test in fork pool Ensure unit project uses threads (see vitest.config.ts); reduce batch size in vanillaBspParity.test.ts
Test timed out in 5000ms Default timeout on concurrent suite Unit project sets 60s; upgrade Vitest config if missing
Modular test slow Cache cleared / cold start Second run should be ~3s; check clearModularSnapshotCache not called mid-suite
Integration Puppeteer flake Too many parallel browsers Integration capped at 4 fork workers

Regression policy

When fixing a parity mismatch:

  1. Add or update a failing test that reproduces it.
  2. Fix the code.
  3. Confirm the test passes.
  4. Run the nearest broader suite (see table above).
  5. Update parity-gap-tracker.md and test-matrix.md.

Stop-the-line: parser regressions, unintentional renderer drift, GZSTATE format change without version bump, golden fixture change without explanation.

Related documents

Document Location
WAD Lab full test reference doom-wad-lab/docs/TESTING.md
GZRender test matrix doom-wad-lab/docs/gzrender-v2/test-matrix.md
Corpus runner spec doom-wad-lab/docs/gzrender-v2/corpus-testing.md
Testing rules doom-wad-lab/docs/gzrender-v2/testing-rules.md
CI pipeline doom-wad-lab/docs/ci.md
Game engine vs renderer doom-wad-lab/docs/gzrender-v2/game-engine-vs-renderer.md
GZDoom dump tools gzdoom-project/docs/PARITY-TOOLS.md