test: broaden quickstart --concurrency witnesses - #202
Open
ethanj wants to merge 1 commit into
Open
Conversation
Four additions to the JSON-stdout oracle, all pinning coordinates the defect report named but the original test left open: - the invalid value is parameterized over eight/0/2.5/-1 - one case per rejection branch of !Number.isInteger(n) || n <= 0; - every case pins exit 0 via expectCLIExit (runCLI never throws, so a nonzero-exit regression with an intact envelope previously passed); - a human-mode witness pins that the warning stays visible without --json (channel-agnostic by design: it survives the channel fix and fails only if the diagnostic is dropped); - the fixture env blanks LLMWIKI_COMPILE_CONCURRENCY and points LLMWIKI_CLAUDE_SETTINGS_PATH at a nonexistent file, so ambient developer configuration cannot perturb the run. Mutation-tested both directions: restoring the stdout write fails the four channel witnesses; deleting the diagnostic fails all five. Full suite green (5033 passed), tsc clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Broaden the quickstart --concurrency witnesses (stacked on #201)
Test-only additions to
test/quickstart-json-stdout.test.ts, pinning the coordinates thedefect report named but the original oracle left open:
eight/0/2.5/-1— one case perrejection branch of
!Number.isInteger(n) || n <= 0.expectCLIExit(result, 0)in every case.runCLInever throws, sopreviously a nonzero-exit regression with an intact envelope would pass; the defect was
precisely "exit 0 + unparseable stream", so both coordinates are pinned.
--json— exit 0 and the warning visible.Channel-agnostic by design: it survives the channel fix and fails only if the diagnostic is
dropped entirely.
LLMWIKI_COMPILE_CONCURRENCYand pointsLLMWIKI_CLAUDE_SETTINGS_PATHat a nonexistent file, so ambient developer configurationcannot perturb the subprocess.
Mutation-tested both directions: restoring the stdout write turns the four channel witnesses
red (human-mode stays green, as designed); deleting the diagnostic turns all five red.
Full suite green (5033 passed),
tsc --noEmitclean, fallow clean.Based on
fix/quickstart-json-stdout(#201) — merging order stays with #201.