Skip to content

test: wt test runs every suite, not just generator/ - #896

Open
tend-agent wants to merge 6 commits into
mainfrom
fix/claude-md-accuracy-31243917459
Open

test: wt test runs every suite, not just generator/#896
tend-agent wants to merge 6 commits into
mainfrom
fix/claude-md-accuracy-31243917459

Conversation

@tend-agent

@tend-agent tend-agent commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

wt test ran cd generator && uv run pytest, so three of the four suites CI runs were reachable only by hand: proxy/, the install-tend OAuth scripts, and worker/. A contributor who touched any of them got a green wt test and found out in CI.

The alias now points at dev/test.sh, which runs all four in the order ci.yaml does — including the proxy/ pin read out of claude/action.yaml, so the addon is tested against the mitmproxy version production runs rather than whatever is latest. Every suite runs even if an earlier one fails, and the failures are listed at the end.

Arguments still forward to the pytest suites, so both wt test -k render and wt test tests/test_migrate.py keep working. A filtered run gets slack in the suites the filter wasn't aimed at: pytest exits 5 there for "no tests collected" and 4 for a path that only exists in another suite, and both count as a pass. generator/ stays strict, so a genuine bad flag — 4 everywhere — still fails the run. A filtered run skips worker/, whose vitest CLI takes different flags.

worker/ needs node_modules, so a cold tree gets npm ci first — as does a tree older than the lockfile, which npm ci records at node_modules/.package-lock.json. Otherwise a dependency bump or a branch switch leaves tsc and vitest running against the last install while CI reruns npm ci and can go red: the same green-locally-red-in-CI shape this PR is closing, one directory over. npm ci rather than npm install, because on an older local npm the latter reruns resolution and leaves package-lock.json churn in the diff.

Also in here: shellcheck's file pattern picks up dev/, and two CLAUDE.md corrections that stand on their own — shellcheck was missing from the pre-commit line, and jinja2 was missing from the generator's runtime dependencies.

Verification

From a cold checkout (rm -rf worker/node_modules):

  • ./dev/test.sh → exit 0; generator 354 passed, proxy 23 passed (mitmproxy 12.2.3, resolved from claude/action.yaml), install-tend scripts 9 passed, worker typecheck + 31 vitest tests passed. git status worker/package-lock.json clean afterwards.
  • ./dev/test.sh tests/test_migrate.py → exit 0; 8 passed in generator, the other two suites report the missing path and are tolerated.
  • ./dev/test.sh -k render → exit 0; 9 selected in generator, the other two suites deselect everything and are tolerated.
  • ./dev/test.sh -k zzz_no_such_test → exit 0 (nothing matched anywhere).
  • ./dev/test.sh --bogus-flag → exit 1, failed: generator.
  • touch worker/package-lock.json && ./dev/test.sh==> worker: npm ci reappears on the already-populated tree; the run right after skips it.
  • pre-commit run --files dev/test.sh .config/wt.toml CLAUDE.md .pre-commit-config.yaml → all hooks pass, shellcheck included.

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked each claim against the file it describes: shellcheck is in .pre-commit-config.yaml, jinja2>=3.1.6 is in generator/pyproject.toml, and .config/wt.toml's test alias really is cd generator && uv run pytest. Both new invocations pass as written when run one at a time from the repo root (23 passed in proxy/, 9 passed in the install-tend scripts dir).

Two things about the new block itself.

The cds are cumulative, so the block only works one command at a time — pasting both lines leaves you in proxy/ and the second cd fails with No such file or directory. Every other command in the block two lines above runs from the repo root, so that's the expectation a reader arrives with. Subshells restore it.

The other is the count. worker/ is a third suite outside wt test with a CI job of its own (test-worker in .github/workflows/ci.yaml), so a contributor who lands in worker/src/ reads "two smaller suites", matches neither directory, and concludes there's nothing to run — the same trap this PR is closing, one directory over. It's less sharp there because worker/README.md already documents npm install && npm test, so a pointer beats duplicating the commands.

Comment thread CLAUDE.md Outdated
Comment thread CLAUDE.md Outdated
Subshells so both lines can be pasted from the repo root, and a pointer to worker/'s vitest suite so the count doesn't read as "nothing else to run".

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both prior threads are addressed — the subshells hold pwd at the repo root (checked by running the install-tend command and the yq lookup from proxy/ back to back; pwd was still the repo root after each), and worker/ now has its own sentence. Re-verified the underlying claims against the tree: test-worker in ci.yaml runs npm run typecheck and npm test, worker/package.json maps test to vitest run, jinja2>=3.1.6 is in generator/pyproject.toml, and shellcheck is in .pre-commit-config.yaml.

One thing the subshell change left behind: the lead-in still says to run the commands from their own directory, which is now the one way they don't work. A reader who follows it literally lands in proxy/, and cd proxy fails from there — the same shape of trap this PR is closing.

Comment thread CLAUDE.md Outdated
@max-sixty

Copy link
Copy Markdown
Owner

change wt test!!

Point the alias at dev/test.sh, which runs generator/, proxy/ (pinned to the
mitmproxy version production runs), the install-tend scripts, and worker/ —
the same set ci.yaml covers. Arguments still forward to the pytest suites.

CLAUDE.md drops the block of hand-run commands for the suites wt test missed,
and shellcheck's file pattern picks up dev/.
@tend-agent tend-agent changed the title docs: name the suites wt test doesn't run, and correct the dependency list test: wt test runs every suite, not just generator/ Aug 8, 2026
@tend-agent

Copy link
Copy Markdown
Collaborator Author

Done — wt test now runs all four suites instead of documenting the three it missed. The alias points at dev/test.sh, which runs generator/, proxy/ (mitmproxy pinned from claude/action.yaml, as ci.yaml does), the install-tend scripts, and worker/, reporting all failures rather than stopping at the first. wt test -k render still works: args forward to the pytest suites, and a suite the filter misses exits 5 (no tests collected), tolerated only when a filter was given.

The CLAUDE.md block naming the hand-run commands is gone; what's left is the one-line pointer plus the two corrections that stand on their own (shellcheck in the lint list, jinja2 in the runtime deps).

One thing worth your call: ci.yaml still spells out its own four jobs, so a fifth suite could land in one place and not the other. Keeping them separate buys parallelism and per-job caching, so I left it — say the word if you'd rather the jobs call the script.

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two things the incremental turns up; both checked by running the script in this checkout.

wt test <path> regressed. The old alias ran pytest {{ args }} inside generator/, so wt test tests/test_migrate.py worked. Now the same invocation runs those 8 tests green in generator/ and then exits 1 with failed: proxy plugins/install-tend/skills/install-tend/scripts — a path that only exists in one suite makes pytest exit 4 from the others, and only 5 is tolerated. A path is as ordinary a pytest argument as -k, and the failure names suites the developer never aimed at. The suggestion widens the slack to 4 while keeping generator/ strict, so a genuine bad flag — 4 in every suite — still fails.

The worker/ install skips a stale tree. npm ci runs only when node_modules is absent, so after the lockfile moves — a dependency bump, a branch switch — tsc and vitest run against whatever was installed last while test-worker reruns npm ci and can go red. Same green-locally-red-in-CI shape this PR is closing, one directory over. npm ci writes node_modules/.package-lock.json, so an -nt test against the lockfile catches it without paying for a reinstall every run.

What I ran

Against the merge checkout, with both suggestions applied (shellcheck -S warning clean):

  • ./dev/test.sh tests/test_migrate.py — before: exit 1, failed: proxy plugins/install-tend/skills/install-tend/scripts. After: exit 0, 8 passed in generator.
  • ./dev/test.sh --bogus-flag — exit 1, failed: generator.
  • ./dev/test.sh -k render — exit 0. ./dev/test.sh -k zzz_no_such — exit 0.
  • touch worker/package-lock.json && ./dev/test.sh — exit 0, and ==> worker: npm ci reappears on the already-populated tree.

Everything else in the diff holds up: jinja2>=3.1.6 is in generator/pyproject.toml, shellcheck is in .pre-commit-config.yaml, and generator/, proxy/, the install-tend scripts, and worker/ really are the whole set — site/package.json has no test script.

Comment thread dev/test.sh Outdated
Comment thread dev/test.sh Outdated
… a stale lockfile

wt test <path> hit exit 4 ("file or directory not found") in the suites the
path doesn't belong to; only 5 was tolerated, so a filtered run that passed in
generator/ still failed. generator/ stays strict, so a bad flag — 4 in every
suite — still fails.

The worker/ install fired only on a missing node_modules, so a moved lockfile
left tsc and vitest on a stale tree while CI reran npm ci. Compare mtimes
against node_modules/.package-lock.json instead.
…y-31243917459

# Conflicts:
#	.pre-commit-config.yaml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants