Skip to content

WIP gate counts Renovate and Dependabot as generated pull requests #398

Description

@s-hiraoku

The scheduled run of 2026-08-18T21:29:52Z selected issue #14 in
s-hiraoku/topcoat-sandbox, passed execution authorization, and was then
stopped by the WIP gate:

"skipped": [{
  "number": 14,
  "reason": "generated pull request WIP limit reached (organization 186/2, repository 0)"
}],
"queue": { "health": { "state": "degraded", "reasonCode": "repeated_gate" } }

repository 0 is correct — the target has no open pull requests at all. The
186 comes from unrelated repositories, and almost none of it is Kaizen's work.

Cause

isGeneratedPullRequest (src/orchestrator/wipLimit.ts:45-51) ends with an
author test:

return Boolean(author?.is_bot || author?.type?.toLowerCase() === 'bot' || author?.login?.endsWith('[bot]'));

type is author.__typename from the owner search
(src/github/client.ts:568, mapped at :764). GitHub reports Bot for every
GitHub App author, so Renovate and Dependabot both match.

The search scope makes this worse. searchOpenPullRequestsForOwner queries
is:pr is:open owner:<owner> (src/github/client.ts:661), where owner is
just the first path segment of the target repo
(src/orchestrator/run.ts:1117). For a personal account the "organization"
count is therefore every open pull request the user owns, anywhere.

Measured against the same query the code issues:

is:pr is:open owner:s-hiraoku   ->  199 open pull requests
page 1 (100):  87 authored by Bot, 13 by User
bot logins:    renovate 82, dependabot 5
generated by the current rule: 91 of 100
on kaizen//codex//claude/ branches: 4 of 100

So roughly 4% of what the gate counts is plausibly Kaizen's, and 96% is
dependency automation in repositories that have nothing to do with the target.

With wipLimit: 2, the gate is permanently closed. Every scheduled run will
report success with zero throughput, and the queue health degrades to
repeated_gate — an accurate signal pointing at a cause that is not real.

Why the branch and title rules are not enough on their own

The two rules above the author test are precise: kaizen/, codex/, claude/
branch prefixes and [scout], [monitor], kaizen: title prefixes. The author
test was presumably added to catch generated pull requests that carry neither —
but it cannot distinguish "a bot opened this" from "Kaizen opened this", and on
any account that runs Renovate the second meaning is swamped by the first.

Suggested direction

  • Drop the bare Bot author test, or qualify it. Matching on the specific
    app identity that Kaizen publishes under is checkable; __typename == "Bot"
    is not. If the intent was to catch pull requests opened through the broker,
    that identity is known at publication time and could be recorded.
  • Scope the count to what the limit is meant to protect. A per-repository
    limit needs a per-repository count. If a cross-repository limit is genuinely
    wanted, it should count Kaizen-registered repositories, not everything an
    owner happens to have.
  • Report the two numbers separately in the reason string. The current
    message already prints repository 0, which is what made this diagnosable;
    keeping that contrast is worth preserving in any fix.

Impact on external adoption

This is a greenfield-adoption defect of the same family as the release blockers
recorded in docs/first-external-run-2026-08-12.md. Dogfood repositories hid it
because the organization has no Renovate installation; the first target that
belongs to an account with ordinary dependency automation cannot run at all,
and the failure reports itself as success.

Environment

kaizen-loop main at e1c45bc, run through the root scheduled-publication
daemon with KAIZEN_HOME=~/.kaizen-sandbox/home. Evidence:
runs/2026-08-18T21-29-52Z/summary.json.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions