Skip to content

fix(skills): key the --body exemption on shell hazards, and fix the recipes that trip it - #882

Open
tend-agent wants to merge 3 commits into
mainfrom
hourly/review-31160773238
Open

fix(skills): key the --body exemption on shell hazards, and fix the recipes that trip it#882
tend-agent wants to merge 3 commits into
mainfrom
hourly/review-31160773238

Conversation

@tend-agent

@tend-agent tend-agent commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

A tend-nightly run on numbagg/numbagg posted a public issue comment whose leading words were deleted by bash command substitution. The comment body was one line, so the bot took the Comment Formatting exemption — "For one-line bodies, --body "…" is fine" — and passed the text as a double-quoted shell argument. The text contained markdown inline code, and bash ran it.

Evidence

Run 31158810529 (tend-nightly, 2026-08-07T07:43:51Z). From the session log, the command as issued:

gh issue edit 716 --body-file /tmp/drift-body.md && gh issue comment 716 --body "`environment-deployments` now passes — the regenerated workflows from #723 emit \`deployment: false\` on the ...

Its result, same log:

https://github.com/numbagg/numbagg/issues/716
/usr/bin/bash: line 1: environment-deployments: command not found
https://github.com/numbagg/numbagg/issues/716#issuecomment-5214064857

The comment posted as now passes — the regenerated workflows from #723 emit ... — the opening `environment-deployments` was executed as a command and replaced with its empty output. The bot noticed the stderr, re-read the body, and repaired it via PATCH about 15 seconds later, so the comment reads correctly now; created_at 07:46:22Z against updated_at 07:46:37Z is the remaining trace. The corrupted text still went out in the creation-time notification emails, and the repair edit fired a second tend-mention run.

Note the same command escaped the later spans (\`deployment: false\`) but not the leading one — escaping applied unevenly is the characteristic shape here, not a one-off typo.

Root cause

The exemption is keyed on the wrong property. Line count has nothing to do with whether a string is safe in double quotes; backtick, $, and \ do. A one-line body is in fact the most likely to be a single sentence of dense inline code, which is exactly the unsafe case. Given a one-line body containing a backtick, bash eats it every time — the bot followed the guidance as written and still shipped corrupted output.

Change

Two commits.

The rule. Narrows the exemption to bodies with no shell-active characters, names the mechanism, and shows the failure inline. One sentence replaced; no new section. ! is not on the hazard list: history expansion only fires in an interactive shell, and the harness runs commands with histexpand off (set -o | grep histexpandoff; echo "wow! great" prints intact). Listing it would err safe, but in a sentence whose whole point is keying on the actual hazard, a phantom entry costs the list its authority.

The recipes the rule governs. A run copies these verbatim, so they outrank the prose — and the ones below were already out of compliance with the sentence being replaced, since they're multi-line:

  • skills/triage/SKILL.md — both gh pr create … --body "## Problem / --body "## Context templates move to --body-file. Their fill-ins ([What the issue reported and the root cause], [What was fixed and why]) are freeform prose that routinely carries inline code: the incident's exact path.

  • skills/weekly/SKILL.md — the dependency-approval step moves to --body-file. The expansion at the call site was safe on its own (parameter expansion doesn't re-scan for command substitution), but the recipe left the assignment to the agent, and a package name written as inline code puts the backtick there.

  • skills/review-reviewers/SKILL.md — the gist announcement was correctly escaped, but it read as non-compliant against the new sentence. Writing it with printf keeps both expansions, needs no escaping at all, and leaves the rule with no in-tree exception:

    printf 'Evidence gist for `%s`: %s\n' "$TARGET" "$GIST_URL" > /tmp/gist-announce.md
    gh issue comment "$TRACKING_NUMBER" --body-file /tmp/gist-announce.md

Gate assessment

  • Evidence level: Critical (clearly wrong outcome — corrupted public content, confirmed from two independent sources: the session log's command not found and the posted body via the API). 1 occurrence, which Critical acts on.
  • Structural, not stochastic: no decision point. Replaying this scenario ten times corrupts ten times — the model was complying with the rule, and the rule licenses the hazard. The self-repair is the stochastic part, and it can't be relied on.
  • Change type: targeted fix / narrowing of one existing sentence — near the removal end of Gate 2, whose bar is low.
  • Historical: no prior occurrence of this shape recorded in the numbagg/numbagg evidence gists for 2026-07 or 2026-08; no open tend issue or PR covers it (checked --body/body-file/backtick/command-substitution across open and closed issues and PRs).

Evidence: https://gist.github.com/19b5ab297bb7ac7e1e9a44d595ccde0f

@tend-agent tend-agent added the claude-behavior Behavioral issues found by review-reviewers label Aug 7, 2026

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The narrowing is right, and the mechanism is stated accurately. Two things.

The bundled recipes the rule governs still model the discouraged shape. These are the ones a run copies verbatim, so they outrank the prose:

  • plugins/tend-ci-runner/skills/triage/SKILL.md — the two gh pr create … --body "## Problem and --body "## Context templates are multi-line double-quoted bodies whose fill-ins ([What the issue reported and the root cause], [What was fixed and why]) are freeform prose that routinely carries inline code. Same corruption path as the incident, and the old one-line exemption never covered them either — they were already out of compliance with the sentence you're replacing.
  • plugins/tend-ci-runner/skills/weekly/SKILL.md — the dependency-approval step (# Compose a one-line review body naming the package, bump type, and what you checked followed by gh pr review <number> --approve --body "$REVIEW_BODY") is the in-tree consumer of the exemption being narrowed. The expansion at the call site is safe on its own — parameter expansion doesn't re-scan for command substitution — but the recipe leaves the assignment to the agent, and a package name written as inline code puts the backtick there.

plugins/tend-ci-runner/skills/review-reviewers/SKILL.md's --body "Evidence gist for \$TARGET`: $GIST_URL"` is correctly escaped and needs the two expansions, so I'd leave it — but it now reads as non-compliant against the new sentence, which is worth a word if the rule is meant to be read literally.

I can push a commit converting the triage templates and the weekly step to --body-file — here if you want the fix to land with the rule, or as a follow-up if you'd rather keep this PR to the one sentence.

Minor: ! isn't a hazard here — see the inline note.

Comment thread plugins/tend-ci-runner/skills/running-in-ci/SKILL.md Outdated
@tend-agent tend-agent changed the title fix(running-in-ci): key the --body exemption on shell hazards, not line count fix(skills): key the --body exemption on shell hazards, and fix the recipes that trip it Aug 7, 2026
@tend-agent

Copy link
Copy Markdown
Collaborator Author

Both taken, landed here rather than as a follow-up — 1a6f06a. A merge that shipped the rule while leaving the recipes modelling the shape it forbids would put the skill in contradiction with itself, and the recipes are the copied-verbatim half, so they're the part that actually changes run behaviour.

  • triage — both gh pr create templates now compose /tmp/pr-body.md with the Write tool and post with --body-file; the framing sentence names why (freeform fill-ins carry inline code).

  • weekly — the approval step posts --body-file /tmp/review-body.md, and the comment above it says why the file rather than the variable: the recipe hands the assignment to the agent, and a package name in inline code puts the backtick there.

  • review-reviewers — took the third option rather than leaving it or adding an exception clause:

    printf 'Evidence gist for `%s`: %s\n' "$TARGET" "$GIST_URL" > /tmp/gist-announce.md
    gh issue comment "$TRACKING_NUMBER" --body-file /tmp/gist-announce.md

    Keeps both expansions, drops the escaping entirely (backticks are literal inside single quotes), and leaves the rule with no in-tree case that reads as non-compliant. Ran it to confirm the output is byte-identical to what the escaped version produced.

On !: verified rather than assumed — set -o | grep histexpandoff, echo "wow! great" intact. Dropped from the sentence and from the PR body's Root cause paragraph, which had the same list.

Left alone deliberately: the git commit -m "…" heredocs in triage carry the same double-quote hazard, but a mangled commit message isn't public content in the way a comment body is, and widening this PR to commit-message composition would make it two concerns. gh secret set --body "$KEY" in install-tend is a pure parameter expansion of a value that never gets re-scanned — not a comment body, not in scope.

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --body-file conversions are right, and the printf form in review-reviewers is the good answer — literal backticks inside single quotes, both expansions kept, no escaping.

One regression came in with the triage rewrite. Moving the PR-body templates out of the double-quoted --body also swapped $ARGUMENTS for #<issue number> in three places, but $ARGUMENTS is not a shell variable — the harness substitutes it textually across the whole skill file at load time, fenced code blocks included (review-reviewers/SKILL.md relies on exactly that with TARGET="$ARGUMENTS" inside a bash fence). So the shell-safety change never required dropping it, and the same recipe keeps it one block later in --title "test: reproduction for #$ARGUMENTS" — the title gets the real number while the body gets a placeholder.

Two costs. The triage run previously received the issue number already filled in and now has to substitute it itself; and running-in-ci's own "Never ship literal placeholders in user-visible content" rule names <TODO(fill)>-shaped strings in PR bodies as the exact failure — a template that models Closes #<issue number> is inviting the miss, and an unsubstituted Closes line silently stops closing the issue. Suggestions restore $ARGUMENTS on all three.

Comment thread plugins/tend-ci-runner/skills/triage/SKILL.md Outdated
Comment thread plugins/tend-ci-runner/skills/triage/SKILL.md Outdated
Comment thread plugins/tend-ci-runner/skills/triage/SKILL.md Outdated
@tend-agent

Copy link
Copy Markdown
Collaborator Author

Real regression, fixed in 5146944 — all three $ARGUMENTS restored.

Worth naming why it happened, because the shape generalises: the double-quoted --body and the shell-expanded $ARGUMENTS looked like one thing. They aren't. $ARGUMENTS is substituted textually by the harness before the run reads the file, so it survives any container — review-reviewers/SKILL.md has TARGET="$ARGUMENTS" inside a bash fence and -R $ARGUMENTS inside a blockquote. Moving the body out of the shell removed a hazard that was never attached to it, and I swapped a live value for a placeholder on the way past. The second cost you name is the sharper one: Closes #<issue number> doesn't fail loudly, it just quietly stops closing the issue.

The PR body's Change section still describes the templates accurately, so I've left it; the diff now differs from it only in that the templates kept their variable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claude-behavior Behavioral issues found by review-reviewers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant