test: fix two macOS bash-3.2 suite failures + silence a spurious ERR trap - #134
Merged
Conversation
Three fixes from one investigation into the maintainer's macOS run, which
aborted at 913 checks with 2 failures. All three are invisible to CI (Ubuntu,
bash 5, no `sandy` on PATH).
1. §68 — the test suite was EXECUTING sandy. The assert script lives in a
double-quoted `python3 -c "..."` string, and a Python comment inside it
read:
# Regression: the proxy of a `sandy`-named workspace ...
Bash expands backticks in a double-quoted string regardless of Python
comment syntax, so the suite ran the real `sandy` command and spliced its
multi-line output into the script:
# Regression: the proxy of a [sandy] Building...
[sandy] Sandbox image up to date.-named workspace must join to ...
The second line is not a comment -> SyntaxError: invalid decimal literal ->
both §68 checks fail. On CI there is no `sandy` on PATH, so the expansion
is empty, the line stays a comment, and it passes. Beyond the false
failure, a test suite silently invoking the tool under test is its own bug
(that 'Sandbox image up to date' is a real image check).
Fixed by switching the block to a QUOTED heredoc (`python3 - "$_json"
<<'DFPY'`), which kills the whole expansion class rather than just this
backtick; `python3 - ARG` keeps sys.argv[1] == the JSON, so the assertions
are byte-identical. Verified both directions on Linux with a fake `sandy`
on PATH: old block FAILS (reproducing macOS), new block PASSES.
Same bug class as the Dockerfile.proxy backtick regression already guarded
by §49. Scanned all 35 `python3 -c "..."` blocks — this was the only one.
2. §83 — nested `source <(sed ...)` inside `$(...)`, which bash 3.2 does not
reliably support: the source yields nothing, both calls exit 127, and the
ERR trap aborts the entire run, so every section after §83 never executed.
run-tests.sh already documents this trap twice in its own comments and
avoids it elsewhere. Switched to the suite's extract-then-eval idiom.
3. sandy:1043 — `_ac="$(docker exec ... )" || _ac=""` handles the status,
but with `set -E` the introspection ERR trap is inherited by the
substitution subshell and bash 3.2 fires it anyway, so every macOS
--print-state with an unreachable tmux probe printed 'introspection handler
failed ... Report this at <issues url>' for entirely normal behavior. Moved
the guard INSIDE the substitution (`|| true`), silencing it on both bash
versions. Emitted values are unchanged (verified: 2 and null).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
rappdw
force-pushed
the
fix/test-s83-macos-bash32
branch
from
August 11, 2026 21:15
040e4b1 to
f05a782
Compare
source <(...) inside $(...))
rappdw
added a commit
that referenced
this pull request
Aug 12, 2026
…136) Base-image modernization plus two additive features and a LAN-allowlist fix. Every pinned toolchain was on an unsupported or maintenance-only release: - Debian 12 bookworm left regular security support 2026-07-12 -> trixie (13), which brings system Python 3.11 -> 3.13. (#133) - Go 1.24 left its two-release support window entirely, and the pin was a Mar-2025 patch untouched for 16 months -> 1.26, with the newest patch now resolved at build time instead of frozen. The egress proxy's documented monthly --pull refresh had been silently no-op since ~Feb 2026 (no further 1.24.x pushes existed), so it was refreshing Debian but not the Go stdlib implementing its TLS/HTTP/CONNECT I/O; pinning a supported minor restores it. Node 22 (Maintenance LTS) -> 24 (Active LTS). (#131) - SANDY_EFFORT pins Claude Code reasoning effort and records it in the session marker, so a run's effort is provable rather than inferred. (#115) - SANDY_SESSION_NONCE lets an operator pin the attestation nonce so a harness can prove a run is the one it launched; env-only, so a committed workspace config cannot set it. (#118) - SANDY_ALLOW_LAN_HOSTS no longer reports success when iptables rejected the rule — a silently missing hole is the worst direction to be wrong in. (#119) Additive minor per the CLAUDE.md semver rule: new keys, no retiering or renames, introspection schema_version stays 1. SANDY_SANDBOX_MIN_COMPAT stays 0.7.10 — the Python bump moves where pip --user packages live, but the sandbox still works, so this is not a compat-floor event and no sandbox needs recreating. Four user-visible upgrade consequences are documented at the top of RELEASE_NOTES.md: the first launch rebuilds every image; persistent pip --user packages become invisible to 3.13 (sandy now detects and reports the stale tree); native Node addons may need npm rebuild; and binaries built inside sandy now link glibc 2.41. Also lands the test-suite work: three macOS-only failures CI structurally cannot see (#134) — including §68 executing the real sandy binary via a backtick inside a double-quoted python3 -c string — and per-section timing, section selection, a fast-model pin, and an image warm-up preflight for the integration suite (#135). Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three fixes from one investigation into the maintainer's macOS run, which aborted at 913 checks with 2 failures. All three are invisible to CI (Ubuntu, bash 5, no
sandyonPATH).1. §68 — the test suite was executing
sandyThe assert script lives in a double-quoted
python3 -c "…"string, and a Python comment inside it read:Bash expands backticks in a double-quoted string regardless of Python comment syntax — so the suite ran the real
sandycommand and spliced its multi-line output into the script:The second line isn't a comment →
SyntaxError: invalid decimal literal→ both §68 checks fail. On CI there's nosandyonPATH, the expansion is empty, the line stays a comment, and it passes.Beyond the false failure: a test suite silently invoking the tool under test is its own bug. That "Sandbox image up to date" is a real image check running mid-suite.
Fix: switch the block to a quoted heredoc (
python3 - "$_json" <<'DFPY'), killing the whole expansion class rather than just this one backtick.python3 - ARGkeepssys.argv[1] == the JSON, so the assertions are byte-identical.Proof (on Linux, with a fake
sandyonPATH): old block FAILS — reproducing the macOS failure exactly — new block PASSES, both modes.Same class as the
Dockerfile.proxybacktick regression already guarded by §49. I scanned all 35python3 -c "…"blocks; this was the only one.2. §83 — nested
source <(…)aborted the entire run_hd_out="$(bash -c 'source <(sed -n "..." "$1"); ...' -- "$_S83")"bash 3.2 doesn't reliably support process substitution nested in command substitution: the
sourceyields nothing, both calls exit 127, the ERR trap fires, and every section after §83 never executes.run-tests.shalready documents this trap twice in its own comments and avoids it in the_sandy_translate_args/build_*extractions. Switched to that same extract-then-eval idiom.3.
sandy:1043— spurious "report this bug" on every macOS--print-state_ac="$(docker exec … )" || _ac=""handles the status, but withset -Ethe introspection ERR trap is inherited by the substitution subshell and bash 3.2 fires it anyway. Every macOS--print-statewith an unreachable tmux probe printed:…for entirely normal behavior (a container mid-startup, or a dead session). Moved the guard inside the substitution (
|| true), which silences it on both bash versions. Emitted values unchanged — verifiedattached_clientsstill2andnullrespectively.How this was found
§68 passed standalone on the maintainer's Mac but failed in the full suite, so I instrumented
_df_assertto capture cwd/env/PATH/exit-code/stderr and the Python error (whichcheck()normally discards into/dev/null). The.pyerrfile named the cause outright. That instrumentation has been removed;grep -c TEMP-DIAGNOSTIC→ 0.Possible follow-up (not included)
A guard asserting no expanding
python3 -c "…"block contains a backtick or$(. This is now the second occurrence of this bug class in the repo. I left it out because a robust detector has to distinguish blocks nested inside single-quotedbash -cwrappers, and a fragile guard is worse than none — but it's worth considering.🤖 Generated with Claude Code