From b3678b59b9849808894d8a29c459d545e3891be3 Mon Sep 17 00:00:00 2001 From: tend-agent <270458913+tend-agent@users.noreply.github.com> Date: Thu, 6 Aug 2026 08:57:55 +0000 Subject: [PATCH 1/2] fix(ci-fix): see a sibling's tracker for the same run, and stop inventing who reran a job MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ci-fix's issue-side dedup searched only 'ci-fix: in:title' — its own artifacts — so a consumer's nightly-failure tracker (opened before either session starts, and triaged by tend-triage seconds later) was structurally unreachable. Both sessions diagnosed the same failure independently and ci-fix filed a duplicate diagnostic issue. Add a run-id-keyed search over recently-updated issues in any state, and route the diagnosis to the sibling's thread when it hits. Separately, a session citing a rerun it didn't trigger inferred an actor for it. The API field that reads like 'who did this' (.actor) is stale on reruns; .triggering_actor is the live one. Record that in running-in-ci's Grounded Analysis failure modes. Closes #867 --- plugins/tend-ci-runner/skills/ci-fix/SKILL.md | 25 +++++++++++++++++-- .../skills/running-in-ci/SKILL.md | 2 ++ 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/plugins/tend-ci-runner/skills/ci-fix/SKILL.md b/plugins/tend-ci-runner/skills/ci-fix/SKILL.md index 516d4212..290aecef 100644 --- a/plugins/tend-ci-runner/skills/ci-fix/SKILL.md +++ b/plugins/tend-ci-runner/skills/ci-fix/SKILL.md @@ -39,6 +39,27 @@ gh issue list --state open --author "$BOT_LOGIN" --search "ci-fix: in:title" \ --json number,title,body --limit 10 ``` +#### Sibling sessions on the same failed run + +A consumer that keeps a nightly-failure tracker issue opens it before this session starts, and `tend-triage` fires on that issue within seconds of `tend-ci-fix` firing on the run. Both sessions then diagnose the same failure without knowing the other exists. The searches above can't see it: they match `ci-fix: in:title`, this skill's own artifacts, while the tracker is titled whatever the consumer's template says. + +Search by the **failed run id** instead, across any state — a sibling may have already diagnosed and closed: + +```bash +gh api "repos/$GITHUB_REPOSITORY/issues?state=all&since=$(date -u -d '2 hours ago' +%Y-%m-%dT%H:%M:%SZ)&per_page=100" \ + --jq "[.[] | select((.body // \"\") | contains(\"/runs/$RUN_ID\"))] | map({number, title, state})" +``` + +Use the list endpoint, not `gh search issues` — search indexing lags minutes behind, and the sibling started seconds ago. `since` filters on *update* time, so a tracker reused across a failure streak (`update_existing: true`) still matches on a stale creation date. + +A hit means a sibling owns this failure's issue thread. Then: + +- **Don't open your own diagnostic issue** (3a/3b) — comment the diagnosis on that issue, and only if it adds something the sibling didn't post. +- **Check what the sibling already did** before repeating it — a rerun it triggered is already in flight, and both sessions polling the same job to completion doubles the cost for one result. +- **A fix PR is still yours to open** if the cause is durable; reference the sibling issue with `Fixes #`. + +This is defer-on-discovery, not a race-free gate — the two sessions start ~2 s apart. It works because the tracker exists before either one does. + ### 2. Diagnose and fix 1. Get failure logs: `gh run view --log-failed` @@ -78,7 +99,7 @@ Automated fix for [failed run](run-url) If the diagnosis identifies the failure as transient — runner-disk corruption, an isolated network blip, an upstream incident that has since resolved — there is no fix PR to create. Don't post the diagnosis as a commit comment (it surfaces on whatever commit triggered CI, including release commits where it's visibly off-topic). -Instead, open an issue with the diagnosis and close it immediately. The closure records "diagnosed, no further action" while keeping the analysis discoverable and off the commit timeline. Apply the `tend-outage` label — the workflow-level `if:` in `tend-triage` and `tend-mention` skip labelled issues, suppressing the no-op cascade runs (`opened` → silent-exit; `closed`-comment → silent-exit) that would otherwise fire on every transient tracker: +Instead, open an issue with the diagnosis and close it immediately — unless step 1 found a sibling's issue for this run, in which case comment there. The closure records "diagnosed, no further action" while keeping the analysis discoverable and off the commit timeline. Apply the `tend-outage` label — the workflow-level `if:` in `tend-triage` and `tend-mention` skip labelled issues, suppressing the no-op cascade runs (`opened` → silent-exit; `closed`-comment → silent-exit) that would otherwise fire on every transient tracker: ```bash gh label create tend-outage --description "Tracks bot outage incidents" --color "d93f0b" 2>/dev/null || true @@ -113,7 +134,7 @@ If the diagnosis identifies a durable root cause but a safe fix can't be produce Leave the issue **open**. A subsequent fix PR closes it via `Fixes #` in the PR body (see step 1 — search for a matching open tracking issue before opening the fix PR). This mirrors the consumer-side `create-issue-on-nightly-failure` pattern and gives maintainers a durable "still broken" signal until a fix ships. -**Dedup first.** Search for an open tracking issue covering the same failure shape; if one exists, comment with the new run link rather than opening a duplicate. Match by failure shape (workflow name + diagnostic snippet), not run ID — each run ID is unique and won't dedup: +**Dedup first.** If step 1's run-id search found a sibling's issue, comment there and stop. Otherwise search for an open tracking issue covering the same failure shape; if one exists, comment with the new run link rather than opening a duplicate. Match by failure shape (workflow name + diagnostic snippet), not run ID — a *recurrence* of the same cause carries a different run ID and won't dedup on it: ```bash BOT_LOGIN=$(gh api user --jq '.login') diff --git a/plugins/tend-ci-runner/skills/running-in-ci/SKILL.md b/plugins/tend-ci-runner/skills/running-in-ci/SKILL.md index b3f01a83..a9a7e404 100644 --- a/plugins/tend-ci-runner/skills/running-in-ci/SKILL.md +++ b/plugins/tend-ci-runner/skills/running-in-ci/SKILL.md @@ -572,6 +572,8 @@ If you can't find source evidence for a specific detail, say so ("I'm not sure o **"Likely" is a stop-sign.** A hedge in a user-facing claim — "likely works", "probably parses as", "should behave like", "I think" — means it rests on an unverified guess. Two options: verify and replace the hedge with the answer, or hedge explicitly ("I haven't tested this — would appreciate if you can confirm") and don't dress up the guess as analysis. The shape is the tell, not the exact words: posting an unverified guess as confident-sounding analysis is the hallucination that erodes trust the fastest. +**Don't name who reran a job unless you read it.** A rerun you didn't trigger invites an inferred actor — "the maintainer's rerun (attempt 2) passed" — when the likelier cause is a sibling bot session on the same failure. The API doesn't rescue a guess either: on `repos/{owner}/{repo}/actions/runs/`, `.actor` stays the **original** run's actor (for a `schedule` run, the repo owner) while `.triggering_actor` is who reran it — so the field that reads like "who did this" is the stale one. Read `.triggering_actor` before attributing, or drop the actor: "attempt 2 passed on the same commit" carries the whole evidentiary weight and needs no name. + **Never ship literal placeholders in user-visible content.** Strings like ``, `PR #PLACEHOLDER`, ``, `TBD`, `XXX`, or `` in an issue body, PR body, or comment are corruption: a deferred substitution that never ran. They survive into the rendered output and read as broken. When a multi-step ask references an artifact that doesn't yet exist ("file an issue that references the PR I'm about to file"), sequence the work so the referenced artifact exists before the referencing body is composed: create the PR → read its number → compose the issue with the number filled in → file the issue. If the cross-reference can't be resolved before posting (e.g. the artifact is out of scope or deferred), omit it or rephrase ("a follow-up PR will…") rather than emit a placeholder. Before any `gh issue create`, `gh pr create`, or `gh ... comment --body-file`, grep the body file for `PLACEHOLDER`, ``, ` Date: Thu, 6 Aug 2026 09:04:59 +0000 Subject: [PATCH 2/2] fix(ci-fix): scope the run-id dedup to issues, assign RUN_ID, sort by update time MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review findings on the new recipe, all three confirmed against the API: - The /issues list endpoint returns pull requests too (14 of 20 on this repo). Step 3's PR body links the failed run, so a fix PR matched the same filter and routed the session down the issue branch, where 'Fixes #' aimed at a PR number is silently dropped. - RUN_ID was never assigned, degrading the filter to contains("/runs/") — matching any issue citing any run, so the no-hit case was unreachable. - The endpoint defaults to sort=created, so a streak-reused tracker with a stale creation date could fall off the single page of 100 that 'since' selected it into. --- plugins/tend-ci-runner/skills/ci-fix/SKILL.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/plugins/tend-ci-runner/skills/ci-fix/SKILL.md b/plugins/tend-ci-runner/skills/ci-fix/SKILL.md index 290aecef..dc1d9575 100644 --- a/plugins/tend-ci-runner/skills/ci-fix/SKILL.md +++ b/plugins/tend-ci-runner/skills/ci-fix/SKILL.md @@ -46,11 +46,19 @@ A consumer that keeps a nightly-failure tracker issue opens it before this sessi Search by the **failed run id** instead, across any state — a sibling may have already diagnosed and closed: ```bash -gh api "repos/$GITHUB_REPOSITORY/issues?state=all&since=$(date -u -d '2 hours ago' +%Y-%m-%dT%H:%M:%SZ)&per_page=100" \ - --jq "[.[] | select((.body // \"\") | contains(\"/runs/$RUN_ID\"))] | map({number, title, state})" +RUN_ID= +gh api "repos/$GITHUB_REPOSITORY/issues?state=all&sort=updated&since=$(date -u -d '2 hours ago' +%Y-%m-%dT%H:%M:%SZ)&per_page=100" \ + --jq "[.[] | select(.pull_request == null) + | select((.body // \"\") | contains(\"/runs/$RUN_ID\"))] | map({number, title, state})" ``` -Use the list endpoint, not `gh search issues` — search indexing lags minutes behind, and the sibling started seconds ago. `since` filters on *update* time, so a tracker reused across a failure streak (`update_existing: true`) still matches on a stale creation date. +Three things that recipe depends on: + +- **Assign `RUN_ID`.** Unset, the filter degrades to `contains("/runs/")` and matches any issue citing any run — every session would defer to an unrelated thread. +- **`select(.pull_request == null)`.** This endpoint returns pull requests as well as issues, and step 3's PR body links the failed run, so a fix PR matches the same filter. PRs are already handled at the top of step 1, and `Fixes #` aimed at a PR number is silently ignored. +- **The list endpoint, not `gh search issues`.** Search indexing lags minutes behind; the sibling started seconds ago. + +`since` and `sort=updated` both key on update time — a tracker reused across a failure streak (`update_existing: true`) has a stale creation date, and the endpoint's default `sort=created` would let it fall off the single page of 100 on a busy repo. A hit means a sibling owns this failure's issue thread. Then: