fix(token-report): raise the per-workflow run limit, and say when it's hit - #887
Conversation
…s hit gh run list returns newest-first and stops at --limit silently, so a workflow busier than 100 runs in the window has its oldest runs dropped and their tokens counted as zero. Measured: tend-mention returns 100 against a true 116 over the current 24h window, so 14% of the runs are missing from a total that review-runs records as a complete accounting. Raise the per-workflow limit to 500 and warn when a workflow comes back exactly at it — a count landing on the limit is the only symptom of truncation visible without re-querying total_count, and raising the limit alone would just move the cliff.
tend-agent
left a comment
There was a problem hiding this comment.
Reviewing as the author — submitting as a comment, not an approval.
The measurement holds up: tend-mention returns exactly 100 at the old limit and 130 at 500 for the 24 h window the callers actually use, so the truncation is real and the fix addresses it.
Two things.
The fetch-failure path is the same bug, on the line being edited. 2>/dev/null || echo "[]" turns any gh run list error into a zero-run workflow, and the new warning can't see it — a length of 0 is not an exact-limit hit, so a transient API error on the chattiest workflow drops all of its runs and the totals still print as a complete accounting. That's strictly worse than the truncation this PR fixes (100% of one workflow rather than its tail), and it's the direction the PR body itself names as the one that hides a problem. list-recent-runs.sh already refuses this: echo "ERROR: 'gh run list' for workflow '$wf' failed after retries — refusing to report a partial run list" >&2; exit 1. Inline suggestion warns rather than exits, matching the posture this PR already chose for truncation; exiting 1 to match the sibling is the defensible alternative, but token-report.sh has no gh_retry, so a bare exit 1 would be more brittle than the sibling's.
500 leaves no headroom at the script's own default lookback. HOURS defaults to 168 per the usage header. At that window today:
$ gh run list --workflow tend-mention --created ">=$(date -u -d '168 hours ago' +%Y-%m-%dT%H:%M:%SZ)" --status completed --json databaseId --limit 900 | jq length
495
Five short of the new limit, so a default-argument invocation trips the new warning within about a day. Every caller in the repo passes 24, so nothing breaks now — but "raising a limit alone would only move the cliff" applies to this number too, and at 168 h the cliff is already underfoot. The per-run gh run download loop is the reason a bigger number isn't free: it's sequential, one network round-trip per run, so 500 runs is a real cost at that lookback rather than the no-op the listing call is. Worth either raising the constant with that in view, or narrowing the documented default to the window the script is actually sized for.
Also folded into the suggestion: -ge rather than -eq, matching #838's version of the same guard.
Not a dedup concern — #838 (list-recent-runs.sh) and #886 (review-runs census) are the same pattern in different files, as the body says. #873 does touch this file, but in disjoint hunks.
… the 168h default Review on this PR found the guard incomplete in two ways. A failed `gh run list` resolved to `[]` via `|| echo`, which the new truncation guard reads as "0 runs, not truncated" — so an API blip drops a whole workflow from the totals with no marker, the same silent under-report at full strength. Branch on the exit status and warn. 500 also left no headroom at the script's own documented 168h default: tend-mention returns 497 over that window today, so a default-argument call would trip the truncation warning within a day. The limit only bounds the listing call, which pages internally, so headroom is free — the per-run download loop costs what the window holds either way.
tend-agent
left a comment
There was a problem hiding this comment.
Reviewing as the author — comment, not an approval.
Both points from the previous round landed as written, and the failed-fetch branch is right. One problem with the new constant.
RUN_LIMIT=2000 puts the truncation guard above the API's ceiling, so it can never fire. The Actions runs endpoint stops paginating at 1000 results regardless of total_count, so gh run list --limit 2000 returns at most 1000 and [ "$(… | jq 'length')" -ge "$RUN_LIMIT" ] is unreachable. Measured on this repo with the script's own call shape:
$ gh api ".../actions/workflows/250047576/runs?status=completed&per_page=1" --jq .total_count
3256
$ gh api ".../runs?status=completed&per_page=100&page=10" --jq '.workflow_runs | length'
100
$ gh api ".../runs?status=completed&per_page=100&page=11" --jq '.workflow_runs | length'
0
$ gh run list --workflow tend-mention --created ">=2000-01-01T00:00:00Z" --status completed \
--json databaseId,conclusion,createdAt,name --limit 2000 | jq length
1000
That reintroduces the exact defect this PR fixes, just relocated: a workflow past 1000 runs in the window drops the excess and the guard stays silent, which is the "raising a limit alone would only move the cliff" failure the body argues against — except now the marker that was supposed to make the next crossing visible is dead code. 1000 is both the largest reachable value and the one that makes -ge fire exactly at the ceiling, so the report says so instead of quietly under-reporting.
The "headroom here is free" rationale in the comment goes with it: above 1000 headroom isn't free, it's unreachable, and buying it costs the warning. Second suggestion rewrites those four lines to name the ceiling as the reason for the constant, so the next person raising it hits the explanation first.
Worth noting the residual this doesn't fix — at exactly 1000 the report is still truncated, just no longer silently. Getting the full set needs .total_count off the API or a narrower window per fetch; both are more than this PR is for, and a loud partial beats a silent one.
Nothing else in the diff. if ! runs=$(…) is set -e-safe as an if condition, unique_by sorts internally so the merge is fine, and gh stops paging once the window is exhausted, so the larger constant costs nothing on a normal run.
The Actions runs endpoint stops paginating at 1000 whatever total_count says, so RUN_LIMIT=2000 made the truncation guard unreachable — the larger constant bought no runs and cost the warning. 1000 is the largest reachable value and trips the guard exactly at the ceiling.
… limit won't help The truncation warning called 1000 'the fetch limit', which reads as a tunable and invites exactly the 2000 bump the previous commit reverted. Name it as the Actions API's pagination ceiling and point at the lever that does work — narrowing HOURS.
tend-agent
left a comment
There was a problem hiding this comment.
Reviewing as the author — comment, not an approval (GitHub blocks self-approval).
The round-two concern is addressed: RUN_LIMIT=1000 puts the guard exactly at the endpoint's ceiling, so -ge is reachable rather than dead code, and the comment now names the ceiling as the reason for the constant. Both threads resolved. The new warning text points at HOURS, which is the script's own first positional argument, so the remedy it names is the one a reader can act on.
Nothing further from me on the incremental — no new findings.
…889) Step 1 of `review-runs` enumerates the fleet with a hard-coded single prefix, while Step 2 documents the opposite — that non-`tend-*` workflows using the tend action are in scope and their prefixes come from the repo's `running-tend` skill. The two steps therefore disagree about what the fleet is, and Step 1 wins: a workflow outside the `tend-` prefix is never classified for duration, never near-timeout-checked, and never reaches Step 3's log analysis. Nothing in the output marks the omission. ## Measured on this repo Over a 24 h window (`SINCE=2026-08-06T08:42:07Z`), `review-reviewers` — which runs the tend composite action but is not named `tend-*`, and which tend's own `running-tend` overlay already lists as an extra prefix for Step 2 — has 21 completed runs that Step 1 never sees: ``` $ gh api repos/max-sixty/tend/actions/workflows --jq '.workflows[] | select(.name | startswith("tend-")) | .name' | grep -c '^review-reviewers$' 0 $ gh api "repos/max-sixty/tend/actions/workflows/250009605/runs?created=>=$SINCE&status=completed&per_page=1" --jq '.total_count' 21 ``` ## Change Replace the inline `startswith("tend-")` with a `PREFIXES` array defaulting to `("tend-")`, matched as an anchored alternation. Step 2's sentence now points at the same list rather than describing a parallel one, so a single repo-level source drives both steps. Behaviour is unchanged for an adopter with no extra prefixes — the default array reproduces the old filter exactly. Running the edited Step 1 block verbatim: ``` # as written, default PREFIXES=("tend-") tend-review 30, tend-mention 30, tend-notifications 26, tend-ci-fix 9, tend-triage 6, tend-nightly 1 # with tend's running-tend prefix list, PREFIXES=("tend-" "review-") tend-review 30, tend-mention 30, tend-notifications 26, review-reviewers 20, tend-ci-fix 9, tend-triage 6, tend-nightly 1 ``` The `30`s in that output are #886's separate bug (the unpaginated endpoint capping at a page), still live on `main` — this change doesn't address it and doesn't depend on it. ## Scope and conflict note This is part 2 of #888. Part 1 of that issue — `token-report.sh` capping the same window at `--limit 100` — is already fixed by #887, so nothing here touches that script. #886 edits the same two `gh api` lines in this block to add `--paginate`. The changes are independent in intent but overlap textually, so whichever lands second needs a trivial rebase; the two edits compose (a `PREFIXES`-driven filter on a paginated fetch). Not included: an explicit prefix-list line in tend's own `.claude/skills/running-tend/SKILL.md`. Its "Usage analysis" section already names `review-` as the extra prefix, so this repo's Step 1 resolves correctly today; adding a dedicated line there is a separate overlay concern. ## Gate assessment - **Structural.** The filter is fixed in the recipe text, so it excludes identically on every run. No decision point. - **Evidence: High.** Exclusion reproduced directly against the API, and the fix verified by executing the edited block verbatim at both prefix lists. - **Change type: targeted fix** — one code block plus one cross-reference sentence, no new sections. --- Refs #888 ## Follow-up commits `3d4fcda` — review on this PR found the same disagreement one paragraph later: the near-timeout instruction resolved a workflow's `timeout-minutes` by globbing `.github/workflows/tend-*.yaml`, which doesn't match the very workflow the widened census now admits. It now reads the workflow's own file. `f823494` reworded the `PREFIXES` comment from rationale into an instruction. --------- Co-authored-by: tend-agent <270458913+tend-agent@users.noreply.github.com>
token-report.shfetches each workflow's runs with--limit 100.gh run listreturns newest-first and stops there silently, so on a workflow busier than 100 runs in the window the report drops the oldest ones and totals them at zero — with nothing in the output saying so.Measured on this repo
Current 24 h window:
16 of 116
tend-mentionruns (14%) are outside today's report, and their tokens are simply absent from the totals.tend-reviewreturns 44 at both limits, so it isn't affected today — the shortfall is per workflow and moves with whichever one is chattiest.This matters because the report's output is the fleet cost figure
review-runsrecords in its evidence log every day, and #801's entries have been reading it as a complete accounting. Under-reporting is also the direction that hides a problem: a workflow that suddenly runs hot is exactly the one that crosses 100 and starts having its excess dropped.Change
Three things, all small:
--limit 1000. The limit is per workflow, not per report, so it only has to clear the busiest one. 500 was the first draft and it was already underfoot: at this script's own documented 168 h default,tend-mentionreturns 497 today, so a default-argument call would have started tripping the new warning within a day. Narrowing the documented default instead would have moved that cost out of sight rather than removed it. 1000 is the ceiling rather than a comfort margin — the Actions runs endpoint stops paginating there whatevertotal_countsays, so anything larger is unreachable and puts the truncation guard beyond what the fetch can ever return, i.e. buys no runs and costs the warning. It is also the value that makes-getrip exactly at the ceiling..total_count, so the loop says so on stderr rather than trusting it.gh run listerror into[]via|| echo, which the truncation guard reads as "0 runs, not truncated" — so an API blip removed an entire workflow from the totals with no marker at all. That is the same silent under-report at full strength, and strictly worse than the tail-drop this PR started out fixing. Branching on the exit status covers it. Warning rather than exiting, unlike the sibling inlist-recent-runs.sh, because this script has nogh_retrybehind it and a bareexit 1would make one blip fatal to a report that is otherwise still useful.Raising a limit alone would only move the cliff. The two warnings are what make the next crossing — from either direction — visible instead of silent. The residual this doesn't fix: at exactly 1000 the report is still truncated, just no longer silently. Getting the full set past the ceiling needs
.total_countoff the API or a narrower window per fetch, both more than this PR is for — and a loud partial beats a silent one.Verified all three branches against the API:
tend-review→ 52 runs, silent; a nonexistent workflow → the fetch warning;tend-mentionat--limit 100→ exactly 100, the truncation warning. The ceiling and the guard's reachability at the new constant, measured on this repo:total_countfor that workflow is 3265, so the 1000 is the endpoint's ceiling and not the window running out. The last line is the guard firing condition met atRUN_LIMIT=1000— unreachable at 2000. Patched script runs clean end to end (token-report.sh 2 "review-", exit 0, 48 runs, no warnings), andshellcheckis clean.Provenance and scope
Found by the review on #886 — that PR fixes the same silent-truncation shape in
review-runs' Step 1 census (30 of 110 runs, a 68-minute view of a 24-hour window), and the reviewer measured this adjacent case in Step 2 while checking it. Kept separate because it's a different file and a different fetcher.#838 is doing the equivalent work for
list-recent-runs.sh— raise the bound, warn at the boundary — so this is the third instance of one pattern rather than a new idea. Not a dedup hit: different script, different call, no overlap in the diff.Gate assessment
gh run listtruncates at the limit deterministically, no decision point.HOURS, the lever that does work.