Skip to content

test: fix two macOS bash-3.2 suite failures + silence a spurious ERR trap - #134

Merged
rappdw merged 1 commit into
mainfrom
fix/test-s83-macos-bash32
Aug 11, 2026
Merged

test: fix two macOS bash-3.2 suite failures + silence a spurious ERR trap#134
rappdw merged 1 commit into
mainfrom
fix/test-s83-macos-bash32

Conversation

@rappdw

@rappdw rappdw commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Three fixes from one investigation into the maintainer's macOS run, which aborted at 913 checks with 2 failures. All three are invisible to CI (Ubuntu, bash 5, no sandy on PATH).

1. §68 — the test suite was executing sandy

The assert script lives in a double-quoted python3 -c "…" string, and a Python comment inside it read:

# Regression: the proxy of a `sandy`-named workspace must join to sandy-92aa9f98,

Bash expands backticks in a double-quoted string regardless of Python comment syntax — so the suite ran the real sandy command and spliced its multi-line output into the script:

# Regression: the proxy of a [sandy] Building...
[sandy] Sandbox image up to date.-named workspace must join to sandy-92aa9f98,

The second line isn't a comment → SyntaxError: invalid decimal literal → both §68 checks fail. On CI there's no sandy on PATH, the expansion is empty, the line stays a comment, and it passes.

Beyond the false failure: a test suite silently invoking the tool under test is its own bug. That "Sandbox image up to date" is a real image check running mid-suite.

Fix: switch the block to a quoted heredoc (python3 - "$_json" <<'DFPY'), killing the whole expansion class rather than just this one backtick. python3 - ARG keeps sys.argv[1] == the JSON, so the assertions are byte-identical.

Proof (on Linux, with a fake sandy on PATH): old block FAILS — reproducing the macOS failure exactly — new block PASSES, both modes.

Same class as the Dockerfile.proxy backtick regression already guarded by §49. I scanned all 35 python3 -c "…" blocks; this was the only one.

2. §83 — nested source <(…) aborted the entire run

_hd_out="$(bash -c 'source <(sed -n "..." "$1"); ...' -- "$_S83")"

bash 3.2 doesn't reliably support process substitution nested in command substitution: the source yields nothing, both calls exit 127, the ERR trap fires, and every section after §83 never executes.

run-tests.sh already documents this trap twice in its own comments and avoids it in the _sandy_translate_args/build_* extractions. Switched to that same extract-then-eval idiom.

3. sandy:1043 — spurious "report this bug" on every macOS --print-state

_ac="$(docker exec … )" || _ac="" handles the status, but with set -E the introspection ERR trap is inherited by the substitution subshell and bash 3.2 fires it anyway. Every macOS --print-state with an unreachable tmux probe printed:

[sandy] ERROR: introspection handler failed (exit 1)
[sandy]        Report this at https://github.com/rappdw/sandy/issues

…for entirely normal behavior (a container mid-startup, or a dead session). Moved the guard inside the substitution (|| true), which silences it on both bash versions. Emitted values unchanged — verified attached_clients still 2 and null respectively.

How this was found

§68 passed standalone on the maintainer's Mac but failed in the full suite, so I instrumented _df_assert to capture cwd/env/PATH/exit-code/stderr and the Python error (which check() normally discards into /dev/null). The .pyerr file named the cause outright. That instrumentation has been removed; grep -c TEMP-DIAGNOSTIC → 0.

Possible follow-up (not included)

A guard asserting no expanding python3 -c "…" block contains a backtick or $(. This is now the second occurrence of this bug class in the repo. I left it out because a robust detector has to distinguish blocks nested inside single-quoted bash -c wrappers, and a fragile guard is worse than none — but it's worth considering.

🤖 Generated with Claude Code

Three fixes from one investigation into the maintainer's macOS run, which
aborted at 913 checks with 2 failures. All three are invisible to CI (Ubuntu,
bash 5, no `sandy` on PATH).

1. §68 — the test suite was EXECUTING sandy. The assert script lives in a
   double-quoted `python3 -c "..."` string, and a Python comment inside it
   read:

       # Regression: the proxy of a `sandy`-named workspace ...

   Bash expands backticks in a double-quoted string regardless of Python
   comment syntax, so the suite ran the real `sandy` command and spliced its
   multi-line output into the script:

       # Regression: the proxy of a [sandy] Building...
       [sandy] Sandbox image up to date.-named workspace must join to ...

   The second line is not a comment -> SyntaxError: invalid decimal literal ->
   both §68 checks fail. On CI there is no `sandy` on PATH, so the expansion
   is empty, the line stays a comment, and it passes. Beyond the false
   failure, a test suite silently invoking the tool under test is its own bug
   (that 'Sandbox image up to date' is a real image check).

   Fixed by switching the block to a QUOTED heredoc (`python3 - "$_json"
   <<'DFPY'`), which kills the whole expansion class rather than just this
   backtick; `python3 - ARG` keeps sys.argv[1] == the JSON, so the assertions
   are byte-identical. Verified both directions on Linux with a fake `sandy`
   on PATH: old block FAILS (reproducing macOS), new block PASSES.

   Same bug class as the Dockerfile.proxy backtick regression already guarded
   by §49. Scanned all 35 `python3 -c "..."` blocks — this was the only one.

2. §83 — nested `source <(sed ...)` inside `$(...)`, which bash 3.2 does not
   reliably support: the source yields nothing, both calls exit 127, and the
   ERR trap aborts the entire run, so every section after §83 never executed.
   run-tests.sh already documents this trap twice in its own comments and
   avoids it elsewhere. Switched to the suite's extract-then-eval idiom.

3. sandy:1043 — `_ac="$(docker exec ... )" || _ac=""` handles the status,
   but with `set -E` the introspection ERR trap is inherited by the
   substitution subshell and bash 3.2 fires it anyway, so every macOS
   --print-state with an unreachable tmux probe printed 'introspection handler
   failed ... Report this at <issues url>' for entirely normal behavior. Moved
   the guard INSIDE the substitution (`|| true`), silencing it on both bash
   versions. Emitted values are unchanged (verified: 2 and null).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@rappdw
rappdw force-pushed the fix/test-s83-macos-bash32 branch from 040e4b1 to f05a782 Compare August 11, 2026 21:15
@rappdw rappdw changed the title test(§83): fix macOS bash-3.2 suite abort (nested source <(...) inside $(...)) test: fix two macOS bash-3.2 suite failures + silence a spurious ERR trap Aug 11, 2026
@rappdw
rappdw merged commit d8b685c into main Aug 11, 2026
2 checks passed
rappdw added a commit that referenced this pull request Aug 12, 2026
…136)

Base-image modernization plus two additive features and a LAN-allowlist fix.
Every pinned toolchain was on an unsupported or maintenance-only release:

- Debian 12 bookworm left regular security support 2026-07-12 -> trixie (13),
  which brings system Python 3.11 -> 3.13. (#133)
- Go 1.24 left its two-release support window entirely, and the pin was a
  Mar-2025 patch untouched for 16 months -> 1.26, with the newest patch now
  resolved at build time instead of frozen. The egress proxy's documented
  monthly --pull refresh had been silently no-op since ~Feb 2026 (no further
  1.24.x pushes existed), so it was refreshing Debian but not the Go stdlib
  implementing its TLS/HTTP/CONNECT I/O; pinning a supported minor restores
  it. Node 22 (Maintenance LTS) -> 24 (Active LTS). (#131)
- SANDY_EFFORT pins Claude Code reasoning effort and records it in the session
  marker, so a run's effort is provable rather than inferred. (#115)
- SANDY_SESSION_NONCE lets an operator pin the attestation nonce so a harness
  can prove a run is the one it launched; env-only, so a committed workspace
  config cannot set it. (#118)
- SANDY_ALLOW_LAN_HOSTS no longer reports success when iptables rejected the
  rule — a silently missing hole is the worst direction to be wrong in. (#119)

Additive minor per the CLAUDE.md semver rule: new keys, no retiering or
renames, introspection schema_version stays 1. SANDY_SANDBOX_MIN_COMPAT stays
0.7.10 — the Python bump moves where pip --user packages live, but the sandbox
still works, so this is not a compat-floor event and no sandbox needs
recreating.

Four user-visible upgrade consequences are documented at the top of
RELEASE_NOTES.md: the first launch rebuilds every image; persistent pip --user
packages become invisible to 3.13 (sandy now detects and reports the stale
tree); native Node addons may need npm rebuild; and binaries built inside sandy
now link glibc 2.41.

Also lands the test-suite work: three macOS-only failures CI structurally
cannot see (#134) — including §68 executing the real sandy binary via a
backtick inside a double-quoted python3 -c string — and per-section timing,
section selection, a fast-model pin, and an image warm-up preflight for the
integration suite (#135).

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant