test: wt test runs every suite, not just generator/ - #896
Conversation
tend-agent
left a comment
There was a problem hiding this comment.
Checked each claim against the file it describes: shellcheck is in .pre-commit-config.yaml, jinja2>=3.1.6 is in generator/pyproject.toml, and .config/wt.toml's test alias really is cd generator && uv run pytest. Both new invocations pass as written when run one at a time from the repo root (23 passed in proxy/, 9 passed in the install-tend scripts dir).
Two things about the new block itself.
The cds are cumulative, so the block only works one command at a time — pasting both lines leaves you in proxy/ and the second cd fails with No such file or directory. Every other command in the block two lines above runs from the repo root, so that's the expectation a reader arrives with. Subshells restore it.
The other is the count. worker/ is a third suite outside wt test with a CI job of its own (test-worker in .github/workflows/ci.yaml), so a contributor who lands in worker/src/ reads "two smaller suites", matches neither directory, and concludes there's nothing to run — the same trap this PR is closing, one directory over. It's less sharp there because worker/README.md already documents npm install && npm test, so a pointer beats duplicating the commands.
Subshells so both lines can be pasted from the repo root, and a pointer to worker/'s vitest suite so the count doesn't read as "nothing else to run".
tend-agent
left a comment
There was a problem hiding this comment.
Both prior threads are addressed — the subshells hold pwd at the repo root (checked by running the install-tend command and the yq lookup from proxy/ back to back; pwd was still the repo root after each), and worker/ now has its own sentence. Re-verified the underlying claims against the tree: test-worker in ci.yaml runs npm run typecheck and npm test, worker/package.json maps test to vitest run, jinja2>=3.1.6 is in generator/pyproject.toml, and shellcheck is in .pre-commit-config.yaml.
One thing the subshell change left behind: the lead-in still says to run the commands from their own directory, which is now the one way they don't work. A reader who follows it literally lands in proxy/, and cd proxy fails from there — the same shape of trap this PR is closing.
|
change wt test!! |
Point the alias at dev/test.sh, which runs generator/, proxy/ (pinned to the mitmproxy version production runs), the install-tend scripts, and worker/ — the same set ci.yaml covers. Arguments still forward to the pytest suites. CLAUDE.md drops the block of hand-run commands for the suites wt test missed, and shellcheck's file pattern picks up dev/.
|
Done — The CLAUDE.md block naming the hand-run commands is gone; what's left is the one-line pointer plus the two corrections that stand on their own ( One thing worth your call: |
tend-agent
left a comment
There was a problem hiding this comment.
Two things the incremental turns up; both checked by running the script in this checkout.
wt test <path> regressed. The old alias ran pytest {{ args }} inside generator/, so wt test tests/test_migrate.py worked. Now the same invocation runs those 8 tests green in generator/ and then exits 1 with failed: proxy plugins/install-tend/skills/install-tend/scripts — a path that only exists in one suite makes pytest exit 4 from the others, and only 5 is tolerated. A path is as ordinary a pytest argument as -k, and the failure names suites the developer never aimed at. The suggestion widens the slack to 4 while keeping generator/ strict, so a genuine bad flag — 4 in every suite — still fails.
The worker/ install skips a stale tree. npm ci runs only when node_modules is absent, so after the lockfile moves — a dependency bump, a branch switch — tsc and vitest run against whatever was installed last while test-worker reruns npm ci and can go red. Same green-locally-red-in-CI shape this PR is closing, one directory over. npm ci writes node_modules/.package-lock.json, so an -nt test against the lockfile catches it without paying for a reinstall every run.
What I ran
Against the merge checkout, with both suggestions applied (shellcheck -S warning clean):
./dev/test.sh tests/test_migrate.py— before: exit 1,failed: proxy plugins/install-tend/skills/install-tend/scripts. After: exit 0, 8 passed ingenerator../dev/test.sh --bogus-flag— exit 1,failed: generator../dev/test.sh -k render— exit 0../dev/test.sh -k zzz_no_such— exit 0.touch worker/package-lock.json && ./dev/test.sh— exit 0, and==> worker: npm cireappears on the already-populated tree.
Everything else in the diff holds up: jinja2>=3.1.6 is in generator/pyproject.toml, shellcheck is in .pre-commit-config.yaml, and generator/, proxy/, the install-tend scripts, and worker/ really are the whole set — site/package.json has no test script.
… a stale lockfile
wt test <path> hit exit 4 ("file or directory not found") in the suites the
path doesn't belong to; only 5 was tolerated, so a filtered run that passed in
generator/ still failed. generator/ stays strict, so a bad flag — 4 in every
suite — still fails.
The worker/ install fired only on a missing node_modules, so a moved lockfile
left tsc and vitest on a stale tree while CI reran npm ci. Compare mtimes
against node_modules/.package-lock.json instead.
…y-31243917459 # Conflicts: # .pre-commit-config.yaml
wt testrancd generator && uv run pytest, so three of the four suites CI runs were reachable only by hand:proxy/, the install-tend OAuth scripts, andworker/. A contributor who touched any of them got a greenwt testand found out in CI.The alias now points at
dev/test.sh, which runs all four in the orderci.yamldoes — including theproxy/pin read out ofclaude/action.yaml, so the addon is tested against the mitmproxy version production runs rather than whatever is latest. Every suite runs even if an earlier one fails, and the failures are listed at the end.Arguments still forward to the pytest suites, so both
wt test -k renderandwt test tests/test_migrate.pykeep working. A filtered run gets slack in the suites the filter wasn't aimed at: pytest exits 5 there for "no tests collected" and 4 for a path that only exists in another suite, and both count as a pass.generator/stays strict, so a genuine bad flag — 4 everywhere — still fails the run. A filtered run skipsworker/, whose vitest CLI takes different flags.worker/needsnode_modules, so a cold tree getsnpm cifirst — as does a tree older than the lockfile, whichnpm cirecords atnode_modules/.package-lock.json. Otherwise a dependency bump or a branch switch leaves tsc and vitest running against the last install while CI rerunsnpm ciand can go red: the same green-locally-red-in-CI shape this PR is closing, one directory over.npm cirather thannpm install, because on an older local npm the latter reruns resolution and leavespackage-lock.jsonchurn in the diff.Also in here: shellcheck's file pattern picks up
dev/, and two CLAUDE.md corrections that stand on their own —shellcheckwas missing from thepre-commitline, andjinja2was missing from the generator's runtime dependencies.Verification
From a cold checkout (
rm -rf worker/node_modules):./dev/test.sh→ exit 0;generator354 passed,proxy23 passed (mitmproxy 12.2.3, resolved fromclaude/action.yaml), install-tend scripts 9 passed,workertypecheck + 31 vitest tests passed.git status worker/package-lock.jsonclean afterwards../dev/test.sh tests/test_migrate.py→ exit 0; 8 passed ingenerator, the other two suites report the missing path and are tolerated../dev/test.sh -k render→ exit 0; 9 selected ingenerator, the other two suites deselect everything and are tolerated../dev/test.sh -k zzz_no_such_test→ exit 0 (nothing matched anywhere)../dev/test.sh --bogus-flag→ exit 1,failed: generator.touch worker/package-lock.json && ./dev/test.sh→==> worker: npm cireappears on the already-populated tree; the run right after skips it.pre-commit run --files dev/test.sh .config/wt.toml CLAUDE.md .pre-commit-config.yaml→ all hooks pass, shellcheck included.