Skip to content

fix(deploy): drop piped grep -q from the prune guard - #12

Merged
prorochestvo merged 3 commits into
mainfrom
fix/11-prune-grep-pipefail
Aug 13, 2026
Merged

fix(deploy): drop piped grep -q from the prune guard#12
prorochestvo merged 3 commits into
mainfrom
fix/11-prune-grep-pipefail

Conversation

@prorochestvo

@prorochestvo prorochestvo commented Aug 13, 2026

Copy link
Copy Markdown
Owner

Closes #11

One line in each of the two copies of the artifact-prune block:

-if printf '%s\n' "$in_use" | grep -qxF "$v"; then continue; fi
+if grep -qxF "$v" <<<"$in_use"; then continue; fi
  • deploy/pi5-deploy.sh — the manual path.
  • .github/workflows/release.yml — the path that actually runs on every v* tag.

Plus a comment at each site recording why it is not a pipe, so the idiom does not
creep back.

Why

grep -q stops reading at its first match and closes the pipe; the writer can then
take SIGPIPE and exit 141, and pipefail reports the whole pipeline as failed.
The caller reads that as "no match" — the exact inverse of the truth — and deletes
the artifact the guard had just proved a live channel points at. A here-string
leaves no writer to signal, so the status is grep's own verdict.

As the issue records, this is not reachable at the current scale. Measured on the
Pi, the verdict starts flipping at roughly 24 KB of payload (1/5 runs), and is
wrong every time from 28 KB; in_use is one line per bin/* channel — 69 bytes
with all three present, ~350x under that. (An earlier revision of the issue blamed
the 64 KiB pipe buffer; corrected in a comment there, with the curve.) The change is
about the safety property no longer resting on an unstated assumption about payload
size.

Two further commits sort out the commentary, which had the caveat in the wrong
places:

  • Dropped from the prune sites and from one duplicated copy in
    nginx-cutover.sh. The code shape stays — a here-string is simply the shorter way
    to write the test — but at 69 bytes of input there is nothing to warn about, and
    grep does not need a caveat attached to it.
  • Kept, with the measurements attached, at the one place where the payload is
    genuinely in range: the HTTP probes in nginx-cutover.sh read the ~35 KB dashboard
    body, past the point where the verdict flips every time. That note now carries the
    curve, says the 64 KiB pipe capacity is not the operative limit, and states that
    a pipe is fine for short inputs — so it reads as a measurement rather than folklore
    and does not get generalised into a rule about grep.

Net effect across deploy/ and .github/: no grep -q anywhere reads from a pipe;
every one greps a file or a here-string; exactly one comment explains why, where it
is load-bearing.

Verification

Exercised the real prune logic against a layout built to be maximally hostile — the
two oldest artifacts are the live bin/release and bin/stage targets, so both
sit outside KEEP=5 and nothing but this guard protects them:

surviving artifacts: v1 v2 v4 v5 v6 v7 v8
bin/release -> v1  OK (target intact)
bin/stage   -> v2  OK (target intact)
200KB in_use, MATCH -> status=0 (want 0)

Both live artifacts survive, the unreferenced v3 is still pruned, and the
over-buffer case returns 0 where it previously returned 141.

Also checked: no piped grep -q remains anywhere under deploy/ or .github/;
bash -n clean on the script and on the block embedded in the workflow;
release.yml still parses as YAML with both jobs intact.

Prune stays non-fatal — the surrounding || echo/trailing true are untouched.

prorochestvo and others added 3 commits August 13, 2026 09:00
The artifact prune asked "is a bin/* channel still using this?" by piping
the in-use list into `grep -q`. That grep stops reading at its first match
and closes the pipe, so the writer can take SIGPIPE and exit 141; pipefail
then reports the pipeline as failed and the caller reads a hit as a miss —
removing the artifact the guard had just proved was live.

Read the list from a here-string instead: there is no writer left to
signal, so the status is grep's own verdict.

Not reachable at the current scale — SIGPIPE needs the list to exceed the
64 KiB pipe buffer, and the layout defines three channels — so this is the
safety property no longer resting on an unstated size assumption, not a
live incident. Both copies of the block are fixed together: the manual
script and the release workflow that actually runs it.

Verified against a layout whose two oldest artifacts are the live release
and stage targets, i.e. both outside KEEP=5 and protected only by this
guard: both survive, the unreferenced artifact is still pruned, and a
200 KB list now returns 0 where it returned 141.

Closes #11

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The deploy scripts carried a warning — twice in nginx-cutover.sh, and once
more added alongside the prune fix — telling the reader that piping into
`grep -q` SIGPIPEs under pipefail. It overstated a narrow failure mode:
the writer only takes SIGPIPE once its output exceeds the pipe buffer, so
at these payload sizes the idiom is fine, and grep needs no such caveat.

Keep the code as it is — a here-string is the shorter way to write the
in-use test regardless — and drop the lore. What survives in
nginx-cutover.sh is the part that is actually non-obvious: that one
scratch file is reused by every probe.
The previous commit removed this note as overstated lore. That was right
for the prune guard, whose input is 69 bytes, and wrong here: these probes
read the ~35 KB dashboard body, which is past the point where piping into
`grep -q` under pipefail reports a hit as a miss.

Restore it with the numbers attached — correct through 22 KB, wrong 1/5 at
24 KB, wrong 5/5 from 28 KB — so the note reads as a measurement rather
than a superstition, states that the 64 KiB pipe capacity is not the
operative limit, and says outright that a pipe is fine for short inputs.
The point is that nobody has to re-derive this, and that nobody
generalises it into a rule about grep.
@prorochestvo
prorochestvo merged commit d3404db into main Aug 13, 2026
1 check passed
@prorochestvo
prorochestvo deleted the fix/11-prune-grep-pipefail branch August 13, 2026 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Harden the artifact-prune in-use guard: piped grep -q can misread under pipefail

1 participant