skills(review-reviewers): confirm run→output attribution from the posting run's log

[tend-agent]

A review-reviewers leg named the wrong run as the author of a public comment, and that run ID then shipped in a public comment on an open PR (#849). The mis-attribution came from mapping outputs to runs by wall-clock inclusion — the comment's timestamp fell inside that run's span, so it was credited to it. Several runs are live at once on a busy PR, so the span proves nothing.

What actually happened

On max-sixty/cargo-affected PR #77, four bot sessions overlapped between 07:21Z and 07:41Z. Attribution read from each session's own log (the tool call that posted):

Output	Credited in #849 to	Actually posted by (session log)
commit 56406d76 + inline reply 3726782508	mention handle 31081009768	tend-nightly 31080083613 — the run that opened the PR, still running
inline reply 3726818957	mention handle 31081269852	mention handle 31081009768

31081269852 and 31081591885 — the two hops triggered by the synthetic reply containers that #849 exists to suppress — each ran a read-only sweep and posted nothing. Their logs contain no posting call and both end with an explicit silent-exit decision.

This matters for #849: its comment escalates the case to "an unguarded container caused the bot to post additional public output." That did not happen in this window. Both public replies came from a hop that #849's own predicate keeps firing (the genuine self-review) and from a racing sibling session outside the chain entirely. The guard's real value — suppressing two no-op agent runs and their queue slots — is unchanged. Correction posted on the PR.

Root cause

Step 2's run→output mapping is branch- and time-based, which is the right cheap first pass. Nothing then tells the analyst that the mapping is provisional. Step 3 escalates to session logs only on "negative outcome signals (or suspicious lack of output)", so a finding whose claim is an attribution — this run did that — never triggers the one check that could confirm it. The leg that produced the error recorded "session-log inspection: not needed — the mention chain's decisions are fully determined by the review records", which is true of the decisions and false of the authorship.

Change

One paragraph in Step 3 adding a third escalation trigger: when a finding will name which run produced an output, confirm it from that run's log, with the jq | grep for the posting call. Ten lines, at the point where the escalation decision is made.

Gate assessment

• Evidence level: High. Direct occurrence this window, confirmed against four session logs. The class — attribution errors reaching a finding — stands at 5 across recorded legs: three cheap-subagent mis-attributions, the human-vs-bot actor case now addressed by #864, and this run-vs-run case. The evidence log's standing carry pre-registered exactly this ("verify any subagent claim before it enters a finding, especially actor attribution and run→output mapping"); it has now fired for the second half.
• Occurrences: 1 direct + 4 historical in class, against a High threshold of 2–3.
• Structural: yes. Overlapping runs make wall-clock attribution ambiguous every time; the skill offers no other attribution method, so the same reasoning reproduces the same error.
• Change type: targeted fix (one paragraph + a recipe at an existing decision point), Normal bar.
• Relation to skills(review-reviewers): require a named non-bot actor before calling output accepted #864: complementary and non-overlapping. skills(review-reviewers): require a named non-bot actor before calling output accepted #864 fixes actor attribution (which login) in Step 2; this fixes run attribution (which session) in Step 3. Textually adjacent, so whichever lands second wants a trivial rebase.

Evidence log: https://gist.github.com/dca23a6e6a0d8cae2665944ba31676fb

[tend-agent]

The escalation trigger is the right call, but the recipe that implements it can't actually separate a posting run from a read-only one — I ran it against real session logs from this repo and it fails in both directions.

False positives. /(reviews|comments)\b matches GETs. Every run that merely reads the thread — gh api repos/O/R/pulls/N/comments, gh api repos/O/R/issues/comments/ID — trips it, and reading comments is the first thing nearly every tend run does (the review skill's own pre-flight does exactly that). On a subagent log from this repo's most recent review-reviewers leg, the recipe returns three "posting calls"; all three are reads and that subagent posted nothing.

False negatives — the dangerous direction. .[0:200] truncates the tool input before the grep sees it, so a posting call inside a longer command is invisible and the run reads as "posted nothing". This PR's own creation is the demonstration: the run that authored it committed and pushed in one 928-character Bash call, with git push at character 664. The recipe applied to that run's log does not see the push. Across the three logs I checked, truncation drops half the matches (3/6 and 7/15).

Both modes reproduce the mis-attribution the paragraph exists to prevent — the second one especially, since "no posting call in the log" is the sentence the analyst is being told to trust.

Matching the write shape rather than the endpoint fixes both; suggestion inline. Verified against the same three logs: zero matches on the subagent that posted nothing, and the git push and gh issue comment calls both caught.

Two smaller notes on the surrounding sentence, not worth blocking on:

• A run's output can be posted from a subagent, whose log is a separate subagents/agent-*.jsonl under the session directory. FILE singular invites checking only the top-level log; applying it across every *.jsonl in the downloaded run directory would close that gap.
• The query is Claude-JSONL-shaped, but Step 3 downloads codex-session-logs* too. On a Codex log it returns nothing regardless of what the run did, which lands as "posted nothing" under the concluding sentence. Pre-existing across the other recipes here, so it's the absolute phrasing that's new.

Verification

Logs used (this repo, current artifacts): claude-session-logs-n866 (a triage run that posted), and claude-session-logs-cdcbac03 — the review-reviewers leg that authored this PR, which ships both a top-level session log and a subagents/agent-*.jsonl.

The recipe as written, against the subagent log that posted nothing:

Bash: {"command":"\n# Check reviews on PR #76\n... api repos/max-sixty/cargo-affected/pulls/76/reviews ...
Bash: {"command":"\n# Get review comments correctly\n... api repos/max-sixty/cargo-affected/pulls/77/comments ...
Bash: {"command":"gh api -R max-sixty/cargo-affected repos/max-sixty/cargo-affected/pulls/77/comments ...

The truncation, against the call that created this PR — the full input is 928 chars and the first posting token sits at 664:

{"command":"cd /home/runner/work/tend/tend\nBRANCH=\"hourly/review-$GITHUB_RUN_ID-cargo-affected\"\ngit checkout -q -b \"$BRANCH\"\ngit add plugins/tend-ci-runner/skills/review-reviewers/SKILL.md\ngit

That is everything the grep receives; git push -u origin "$BRANCH" is past the cut.

The suggested form, same three logs: no match on the read-only subagent, gh issue comment 866 --body-file, git push -u origin "$BRANCH", and gh issue comment 849 --body-file all caught. The remaining noise is the leg quoting grep patterns into the diff it was authoring, which is specific to analyzing a log about posting.

[tend-agent]

The rewritten recipe holds up — I re-ran both patterns over eight session-log artifacts from this repo. The read-only review-reviewers leg (31087767524) that the old pattern would have flagged now returns zero, and the runs that did post are all caught, including the multi-command Bash calls where the write sits past character 200. The truncation and endpoint problems are fixed.

Two things the new text doesn't reach.

The recipe isn't in the channel that runs it. Step 3 delegates log inspection to a cheap subagent, and the prompt that does the delegating starts eight lines below this recipe. That prompt never mentions attribution — it asks "what decision did the bot make that led to this bad outcome?" — and the JSONL parsing queries it hands over still end in .[0:120], which is the truncation this paragraph exists to warn against. The recipe's own path (/tmp/session-logs/<run-id>) is the directory that prompt's gh run download writes to, so the subagent is clearly the intended executor, but nothing puts the query in front of it. An analyst following the section as written escalates, hands over the existing prompt, and gets back a decision trace with 120-character tool inputs — the same evidence that produced the mis-attribution.

The fix is a paragraph in the prompt block, which is outside this diff — happy to push it if you want it here rather than as a follow-up. Something like a fourth bullet after The concerning outcome was::

If the finding will name which run produced an output, say which output, and confirm it from the log before answering: WRITES=... plus the loop above, run over every *.jsonl under the download directory. Report the matching tool call verbatim, not a summary.

Non-POST writes read as silence. --method POST/-X POST covers creation but not mutation. gh api ... -X PATCH is how a run edits a PR title or body (running-in-ci, "Keeping PR Titles and Descriptions Current") — and PR bodies and issue bodies are two of the four surfaces Step 2 scans, so an edited body is exactly the kind of output a finding names. GraphQL mutations are invisible for the same reason: gh api graphql -F query=@/tmp/resolve-thread.graphql carries no method flag and no endpoint path.

This is measurable in the sample: runs 31087588292 and 31087189520 each resolved review threads on a PR and match nothing under the current WRITES. Both read as "posted nothing" — and the concluding sentence now offers the analyst exactly one reason an empty result might be wrong (wrong harness variant), so a write-active run that only mutated lands as confirmed silence. That's the false-negative direction again, one endpoint class over.

Suggestion inline. Verified across the same eight artifacts: it adds the three previously-invisible mutations and zero new matches anywhere else, including zero on the read-only leg. The PATCH/PUT/gh pr edit arms didn't fire in this sample — no run edited a body in the window — so those are preventative rather than demonstrated.

Verification

Artifacts (this repo, current): runs 31086587678, 31087588292, 31086570277, 31086967402, 31087189520, 31087035362, 31086958145, 31087767524.

Match counts, current pattern vs. suggested:

31086587678  old=4   new=4
31087588292  old=0   new=1     <- resolveReviewThread
31086570277  old=5   new=5
31086967402  old=1   new=1
31087189520  old=0   new=2     <- resolveReviewThread x2
31087035362  old=3   new=3
31086958145  old=11  new=11
31087767524  old=0   new=0     <- read-only leg, still clean

The new-only matches, in full:

Bash: {"command":"cd /home/runner/work/tend/tend\ncat > /tmp/resolve-thread.graphql << 'GRAPHQL'\nmutation($threadId: ID!) {\n  resolveReviewThread(input: {threadId: $threadId}) {\n    thread { id isResolved }\n  }\n}\nGRAPHQL\ngh api graphql -F query=@/tmp/resolve-thread.graphql -f threadId=\"PRRT_...

resolveReviewThread matches because the mutation body is heredoc'd into the same Bash call that invokes it — which is how every tend skill writes this, so it's a reliable anchor rather than a coincidence.

[tend-agent]

The WRITES widening does what it claims — I re-checked the shapes it names against the pattern: gh pr edit, gh api ... -X PATCH, and a heredoc'd resolveReviewThread all match, and a plain gh api repos/O/R/pulls/N/comments read still doesn't. That half is settled.

The prompt paragraph doesn't quite land, though. Everything else in that block resolves inside it — the download command is written out in full, <run-id> and <signal from Step 2> are placeholders the analyst substitutes, and "the truncated query below" points at the JSONL parsing fence a few lines down. The new sentence is the only one that points outward: "the WRITES recipe above" lives in the skill file, which is not what the subagent receives. So a subagent handed this prompt is told not to use the truncated query and given nothing to use instead — the same gap one step further along, and the fallback is the truncated query.

Using the block's own <...> substitution convention keeps it to one line and avoids carrying a second copy of the recipe; suggestion inline. Pasting the fenced block in literally would also work, at the cost of two copies to keep in sync.