rate-limit preflight: the creation-spike gate halts review and notification runs, so two PRs merged unreviewed

[tend-agent]

The spike gate in shared/steps/rate-limit-preflight.sh tripped on 2026-08-05 and aborted every agent run on this repo for 12 h 13 min, until the UTC-day counter reset. During that window tend-review never ran, and #852 and #853 — both max-sixty-authored — were merged with zero reviews from anyone.

The gate measures one thing (issues and PRs the bot created) and enforces on another (whether the agent runs at all). tend-review's output is a review, which search/issues never counts, so no amount of reviewing can trip the gate and blocking review cannot relieve it. The same holds for tend-notifications, which marks notifications read. Those runs were pure collateral.

What happened

Blackout start	2026-08-05T11:47:16Z — run 31002862308 (tend-review)
Blackout end	2026-08-05T23:45:50Z — run 31057476863 (tend-notifications)
Recovery	UTC-day rollover; the 00:07:44Z review-reviewers run passed
Failed runs	36 of 37 on 2026-08-05 (8/8 sampled confirmed as this gate; the 1 non-gate failure was an unrelated ci proxy test)

Every failed run carried the identical annotation:

##[error]Rate limit: bot created 16 items today, above spike limit of 15 (baseline: 17 over past 6 days)

The arithmetic is exactly as written: SPIKE_LIMIT = 10 + PAST_POSTS / 3 = 10 + 17/3 = 15, and the bot's 16th item of the day (#850, 11:47:12Z) crossed it four seconds before the first blocked run.

Cost

Four tend-review runs were blocked, each within seconds of the PR it was meant to review:

PR	Author	Blocked run	State now	Bot review?
#850	tend-agent	31002862308 @ 11:47:16Z	open	none
#851	worktrunk-bot	31024720942 @ 16:19:29Z	open	none
#852	max-sixty	31025137115 @ 16:24:43Z	merged 18:44:13Z	none
#853	max-sixty	31047860817 @ 21:15:37Z	merged 22:43:22Z	none

max-sixty-authored PRs are normally reviewed — #835 drew five review events, #843 two — so the absence on #852 and #853 is the gate, not the review skill declining.

Two second-order effects made the blackout silent:

• Nothing reported it. The outage-issue step is gated on steps.claude.outcome == 'failure'. A preflight abort never reaches the claude step, so no tend-outage issue was filed for any of the 36 failures. The only signal was red check marks.
• The diagnostic was gated too. All five review-reviewers matrix legs were blocked on every tick from 12:24Z onward, so the evidence log has no entry between the 10:47 tick and this one — the workflow that exists to notice this was switched off by it.

Why the gate fired

Daily items created by tend-agent in this repo:

Date	Items
2026-07-30	0
2026-07-31	1
2026-08-01	4
2026-08-02	1
2026-08-03	5
2026-08-04	6
2026-08-05	16

16 against a 2.8/day baseline is a real spike by the gate's definition, and the detection is not obviously wrong. But all 16 items were distinct, well-formed fix PRs and issues spread across eleven hours (00:20Z → 11:47Z) from several workflows — a busy day after a quiet week, not a runaway loop. The gate cannot tell those apart, and its response to the ambiguity is to stop everything until midnight UTC.

Proposal

Make the gate's enforcement scope match what it measures: let it abort runs that can open issues or PRs, and let review and notification runs through.

The smallest shape that does this without changing behaviour for anyone who hasn't regenerated: add an input to both composite actions, default "true" (today's behaviour), threaded into rate-limit-preflight.sh as an env var that skips the spike check when false.

  creates_issues:
    description: >-
      Whether this workflow's agent run can open issues or PRs. Runs that only
      post reviews or mark notifications read set this false — a creation spike
      is neither caused nor relieved by blocking them.
    default: "true"

The generator then sets creates_issues: "false" in review.yaml.j2 and notifications.yaml.j2. The burst check (>10 PRs or issues in 20 minutes) is a different signal — genuinely runaway — and I'd leave it aborting everything.

Two open questions I'd rather a maintainer settle than guess at, which is why this is an issue and not a PR:

1. Which workflows are exempt. review and notifications are unambiguous. mention usually replies with a comment but can open a PR; ci-fix pushes to the default branch. Both are arguable either way.
2. Whether a tripped gate should still fail red. A safety limiter working as designed rendering as 36 failed runs is noise on the same signal tend uses to detect real breakage — but exiting green would make the next blackout even quieter than this one was. If the exemptions above land, the remaining blocked runs are ones that genuinely wanted to create something, which is a better case for a loud failure than the current blanket.

Independent of the policy choice, the reporting gap is worth closing on its own: a preflight abort should surface somewhere a human looks, since red checks on scheduled workflows evidently did not carry the message that the reviewer had been off since lunchtime.

Verification

• Failure cause per run: gh -R max-sixty/tend run view <id> --log | grep -iE '::error::|Rate limit:' — 8 runs sampled across 11:47Z–23:45Z, all identical.
• Review absence: gh api repos/max-sixty/tend/pulls/{850,851,852,853}/reviews returns [] for all four.
• Daily counts: gh api "search/issues?q=author:tend-agent+repo:max-sixty/tend+created:<date>" --jq .total_count.
• Baseline reproduced: created:2026-07-30..2026-08-04 → total_count 17, matching the annotation.
• githubstatus.com unresolved incidents at analysis time: 0 — not an upstream event.

Evidence log: https://gist.github.com/e08f6e62d6478163cb425a75648eb7e4

[tend-agent]

Both halves of this are already in flight, opened by sibling matrix legs of the same review-reviewers run that filed this issue — no new PR from triage.

• The gate halting non-creating runs → fix(rate-limit): pause creation on a daily spike instead of failing the run #856. It takes a different shape than the creates_issues input proposed here: rather than exempting workflows, it splits the daily check into a hard ceiling (10 + past_posts) that still aborts and a spike threshold (10 + past_posts / 3) that no longer does. On a spike trip the run proceeds with TEND_CREATION_PAUSED_NOTE appended to the agent's prompt telling it not to open issues or PRs. Replayed against 2026-08-05: 16 vs spike 15, hard 27 → the four tend-review runs execute and workflows: name the environment with deployment: false #852/check: refuse a job that names the environment and files a deployment #853 get reviewed.
• The reporting gap → fix(actions): report an outage when the run fails ahead of the agent #857. Widens Report failure from failure() && steps.claude.outcome == 'failure' to bare failure() in both actions, so any of the twelve pre-agent steps failing lands in the tend-outage tracker.

That collapses open question 1 — under #856 nothing needs exempting, since the pause is per-run rather than per-workflow, and mention / ci-fix keep working while just not filing. Question 2 gets a split answer: the spike tier goes green-with-a-::warning::, and the burst and hard tiers stay red but now file an outage issue.

The tradeoff worth a maintainer's eye is that #856 moves spike-tier enforcement from exit 1 into a prompt directive, which a prompt-injected session could disregard where the shell check could not. #856's body owns this and argues the hard ceiling below it is the real floor. The creates_issues shape proposed here keeps shell enforcement for creating workflows, at the cost of being per-workflow rather than per-run and needing adopters to regenerate — so it's still the available alternative if the prompt-enforced tier reads as too soft.

One residue either way: a spike trip under #856 surfaces only as a workflow warning annotation, which is quieter than the red checks that already failed to carry the message. #857 covers the aborts that remain, not that path.