fix(test): fail loudly when BATS_LOAD names a file that is not there - #179
Open
fentas wants to merge 1 commit into
Open
fix(test): fail loudly when BATS_LOAD names a file that is not there#179fentas wants to merge 1 commit into
fentas wants to merge 1 commit into
Conversation
`load_source` treated a missing file as "skip, silently" regardless of how it was chosen. For the DEFAULT path that is right — the `.sh` beside a `.bats` is a convention, and suites legitimately have none. For an EXPLICIT `BATS_LOAD` it is not: the caller named a file, and returning 0 sources NOTHING, so the suite runs against whatever the environment already provides. That is how `BATS_LOAD=argsh.min.sh` from the wrong directory reports on a bundle it never read. Measured on this tree: a misdirected BATS_LOAD turned 370 passing tests into 136 failures, every one of them `command not found` with no mention of BATS_LOAD anywhere — and any test file that happens not to touch the source still PASSED. After this change the same run stops at one failure that names the cause and the cwd. Unchanged where it should be: the default run still reports its usual 2 failures, and BATS_LOAD=argsh.min.sh still loads the bundle and reports its usual 3. Only the unhonourable case moved. No automated regression test: exercising this needs bats-inside-bats against the helper that bats itself loads, which is more machinery than the guard. Verified by before/after measurement instead, quoted above. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the Bats test helper to treat a missing BATS_LOAD target as a hard error, while keeping the existing “silent skip” behavior for the default adjacent .sh convention. This prevents tests from accidentally running against whatever happens to already be in the environment when an explicit source file was intended.
Changes:
- Add an explicit guard: if
BATS_LOADis set and the resolved file is missing, print a clear error (includingcwd) and return non-zero. - Preserve the existing behavior for the default source path (no
BATS_LOAD): missing file returns success and remains silent.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+39
to
+41
| # EXPLICIT (BATS_LOAD set): the caller named a file. Returning 0 sources | ||
| # NOTHING and runs the suite against whatever the environment already | ||
| # provides, so `BATS_LOAD=argsh.min.sh` from the wrong directory reports on a |
Comment on lines
+42
to
+45
| # bundle it never read. Measured: a misdirected BATS_LOAD turns 370 passing | ||
| # tests into 136 failures that say `command not found` and never mention | ||
| # BATS_LOAD — and any test file that does not happen to touch the source | ||
| # still PASSES. An explicit request that cannot be honoured is an error. |
This was referenced Aug 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
load_sourcetreated a missing file as "skip, silently" regardless of how it was chosen.For the default path that is right — the
.shbeside a.batsis a convention, and suites legitimately have none.For an explicit
BATS_LOADit is not: the caller named a file, andreturn 0sources nothing, so the suite runs against whatever the environment already provides. That is howBATS_LOAD=argsh.min.shfrom the wrong directory reports on a bundle it never read.Measured
BATS_LOAD=/nonexistentcommand not found, none mentioning BATS_LOADBATS_LOAD=argsh.min.shWorse than the noise: under a misdirected
BATS_LOAD, any test file that happens not to touch the source still passed.Notes
lintjob covers it.argsh.min.sh/ CI item in a downstream backlog. Separately worth knowing: onmain,test,minify(including its "Test minified" step) andlintall pass — the only failing job iscoverage(argsh coverage builtinexits 1 with no output), and it has been red since 2026-05-10.🤖 Generated with Claude Code