Skip to content

fix(ci-fix): see a sibling's tracker for the same run, and stop inventing who reran a job - #872

Open
tend-agent wants to merge 2 commits into
mainfrom
fix/issue-867
Open

fix(ci-fix): see a sibling's tracker for the same run, and stop inventing who reran a job#872
tend-agent wants to merge 2 commits into
mainfrom
fix/issue-867

Conversation

@tend-agent

Copy link
Copy Markdown
Collaborator

Problem

Two independent defects reported in #867, both verified against live API data.

1. ci-fix's issue-side dedup can only see its own artifacts. All three of its issue searches use --search "ci-fix: in:title". A consumer that keeps a nightly-failure tracker issue — opened by its own create-issue-on-nightly-failure job before either bot session starts — gets a title like Nightly tests failed, which that query can never match. tend-triage fires on the tracker's issues: opened and tend-ci-fix fires on the same run's workflow_run: completed about two seconds apart, so both sessions run a full independent diagnosis of the same failure and ci-fix files a second diagnostic issue for it. #627 is the same race, but it was closed on a consumer-side workaround (removing that repo's tracker job), which doesn't help an adopter who deliberately keeps one.

2. A rerun the session didn't trigger gets an invented actor. When the sibling triage session reruns the failed job, the ci-fix session observes attempt 2 pass and attributes it — in the reported case, to "the maintainer's rerun", when the rerun was the bot's own sibling. The API doesn't rescue a guess: on a reran scheduled run, .actor stays the original run's actor (the repo owner) while .triggering_actor is who reran it, so the field that reads like "who did this" is the stale one. Nothing in any bundled skill mentioned either field.

Solution

ci-fix/SKILL.md — a new "Sibling sessions on the same failed run" check in step 1, keyed on the failed run id rather than a title prefix, over recently-updated issues in any state. On a hit: comment the diagnosis on the sibling's thread instead of opening one (3a/3b now defer to it), check what the sibling already did before repeating it, and still open a fix PR if the cause is durable. Two properties worth calling out in the skill text and kept there: the list endpoint rather than gh search issues, because search indexing lags minutes while the sibling starts seconds ago; and since filtering on update time, so an update_existing: true tracker reused across a failure streak still matches despite a stale creation date. It is a defer-on-discovery check, not a race-free gate — it works because the tracker predates both sessions, and the skill says so rather than overselling it.

While there, corrected a stale rationale in 3b: "each run ID is unique and won't dedup" describes the recurrence case, which is what that fallback search is for — it's not a reason run IDs can't dedup in general, which is exactly what the new check does.

running-in-ci/SKILL.md — the attribution rule goes in Grounded Analysis → Specific failure modes rather than in ci-fix, per the "don't duplicate guidance across skills" rule: review and nightly also call gh run rerun, and this is the generic "distinguish what you verified from what you inferred" failure mode, not a ci-fix-specific one.

Testing

The dedup recipe was exercised verbatim, not just read. Replayed against the reported window with RUN_ID=31077344052 it returns the sibling tracker ({"number":3751,"title":"Nightly tests failed","state":"closed"}), and returns [] for a run id with no issues — so it fires on real data in both the hit and no-hit case, with GNU date as available on the runner.

The API-field claim was checked directly rather than taken from the report: repos/max-sixty/worktrunk/actions/runs/31077344052 returns run_attempt: 2, actor: max-sixty, triggering_actor: worktrunk-bot — confirming .actor is stale on a reran schedule run. Both reported occurrences check out (the tracker/duplicate pairs, their timestamps, and the literal "The maintainer's rerun (attempt 2)" text in the filed issue body), and worktrunk's nightly.yaml does set update_existing: true, which is what makes the since-on-update-time property load-bearing.

uvx pre-commit run --all-files passes, including the plugin-skill hooks (bang-backtick, install-tend reference sync). Generator suite: 338 passed. The changes are skill prose, which no test asserts on.

Note

#869 also touches attribution, but in review-reviewers and for a different mechanism (mapping outputs to runs by wall-clock overlap). No file or rule overlap with this one.

Closes #867

…ting who reran a job

ci-fix's issue-side dedup searched only 'ci-fix: in:title' — its own
artifacts — so a consumer's nightly-failure tracker (opened before either
session starts, and triaged by tend-triage seconds later) was structurally
unreachable. Both sessions diagnosed the same failure independently and
ci-fix filed a duplicate diagnostic issue.

Add a run-id-keyed search over recently-updated issues in any state, and
route the diagnosis to the sibling's thread when it hits.

Separately, a session citing a rerun it didn't trigger inferred an actor
for it. The API field that reads like 'who did this' (.actor) is stale on
reruns; .triggering_actor is the live one. Record that in running-in-ci's
Grounded Analysis failure modes.

Closes #867

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two problems in the new run-id recipe, plus one robustness point — all on the same line, folded into one suggestion.

The /issues list endpoint returns pull requests too. Verified against this repo: has("pull_request") is true for roughly half the results of repos/max-sixty/tend/issues?state=all&per_page=20. Step 3's PR template ends with Automated fix for [failed run](run-url), so a fix PR for this run matches contains("/runs/$RUN_ID") and the section routes to the wrong branch — step 1 already has a distinct rule for an existing open PR ("comment on it linking the new run and stop"), and Fixes #<n> pointed at a PR number is silently ignored by GitHub, so the closing link is lost. select(.pull_request == null) restores the issues-only contract the surrounding text assumes.

$RUN_ID is never assigned. Every other recipe in this skill either derives its variables in-band (BOT_LOGIN=$(gh api user --jq '.login')) or uses a <run-id> placeholder, and running-in-ci states shell state doesn't persist between tool calls. Unset, the filter collapses to contains("/runs/"), which matches any issue body linking any workflow run — the no-hit case becomes unreachable and the session defers to an unrelated thread.

sort=updated. The endpoint defaults to sort=created&direction=desc, so the streak-reused tracker this section exists to catch — stale creation date, updated seconds ago — sorts by where it was created and can fall off the single unpaginated page of 100 on a repo with heavy issue traffic. Sorting by update time matches the ordering since already implies.

The running-in-ci attribution rule checks out against live data (actions/runs/31077344052 returns actor: max-sixty, triggering_actor: worktrunk-bot on run_attempt: 2 of a schedule run).

Comment thread plugins/tend-ci-runner/skills/ci-fix/SKILL.md Outdated
… update time

Review findings on the new recipe, all three confirmed against the API:

- The /issues list endpoint returns pull requests too (14 of 20 on this
  repo). Step 3's PR body links the failed run, so a fix PR matched the
  same filter and routed the session down the issue branch, where
  'Fixes #<n>' aimed at a PR number is silently dropped.
- RUN_ID was never assigned, degrading the filter to contains("/runs/") —
  matching any issue citing any run, so the no-hit case was unreachable.
- The endpoint defaults to sort=created, so a streak-reused tracker with a
  stale creation date could fall off the single page of 100 that 'since'
  selected it into.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci-fix: dedup can't see a sibling triage tracker for the same failed run, and a sibling's rerun gets attributed to the maintainer

1 participant