chore(harden): enforcement-claim checklist item; escalate docs-drift to rung P#5
Conversation
…to rung P Two reactive hardenings, one ledger row each, both surfaced by the gates or a bot. 1. unverified-enforcement-claim (new class) -> rung P Six citable instances across one spec, its plan and its diff where something was described as enforced, caught or guaranteed and no mechanism did it. Each was caught by a review gate; none by me. New docs/prompt-standards.md item 11 requires such a claim to name its mechanism and the author to verify the mechanism exists first. Mirrored into the scaffolded template. The class is minted rather than filed as docs-drift: drift is two artifacts disagreeing, this is one claim untrue on its own terms even with nothing to contradict it. Different fix, so different grep. How the count was settled is the best argument for the item. It was asserted as six without checking, "corrected" to five from memory, and only settled at six by reading the commits — two unverified claims in the course of hardening against unverified claims. 2. docs-drift -> rung P (escalated from the 2026-07-18 `1 prose` row) A Gate-B fix changed behaviour the approved spec specified and the spec was never updated, so the two disagreed until a PR bot found it. The prior rung was an AGENTS.md rule about manifest claims — too narrow to reach this. The rule now lives in the Gate-B instructions themselves: a fix that changes specified behaviour is not complete until the spec is updated in the same commit. No mechanical rung: deciding whether a fix altered behaviour a spec pins down needs both artifacts read, and the crude proxies fire on every change no spec covers. BOT MATRIX Greptile posted summary-only findings on PR #1 and two valid inline comments on PR #4. Its column now reads `inconsistent` with both observations dated, rather than being reclassified from one more data point. That exposed a routing bug: `inconsistent` was not consumable by a command that knew only line-findings-vs-summary. The Wait-for list is now the routing authority in all four copies — the command's Steps 0 and 2, the matrix, AGENTS.md's tree comment, and the scaffolded template — with the table descriptive. The scaffold no longer derives Wait-for from a column that stopped holding yes/no. Also corrected: a fabricated "four days apart" (the observations are hours apart on the same day, timestamps cited), and "both post a check" (the API returns only quality and CodeRabbit for that head, so Greptile's completion is watched via its comment instead). Gate B: five passes, final clean on both reviews. Battery green — shellcheck x4, hook suite, 61 invariant assertions, invariant scan, plugin validate --strict.
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
Warning Review limit reached
Next review available in: 22 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThe changes update workflow guidance for enforcement claims, PR-review bot routing, Gate-B specification synchronization, checklist numbering, and historical hardening records. ChangesWorkflow Hardening
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR adds two new hardening entries —
Confidence Score: 4/5Safe to merge — no executable code changed, all docs and prompt files are internally consistent and cross-referenced. All ten changed files are documentation and prompt text with no executable code. The Wait-for routing change is applied consistently across five locations. The only issue is a prose structure problem in docs/prompt-standards.md where the actionable remedy sentence for item 11 is appended to the anecdote paragraph without a blank line. docs/prompt-standards.md — item 11's remedy instruction is buried at the end of the anecdote paragraph; the scaffold version in workflow-init.md is cleaner. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Step 0: Read docs/pr-review-bots.md] --> B{Wait for list present?}
B -- No --> C[gh pr view --json comments,reviews\nProcess only what is visibly there]
B -- Yes --> D[Step 2: Route on Wait for list]
D --> E[Wait for every listed bot]
E --> F[Read BOTH channels per bot\ninline comments + PR summary body]
F --> G{Table column reads inconsistent?}
G -- Yes --> H[Observation only — not a routing instruction\nStill read both channels]
G -- No --> I[Follow bot-specific completion signal\nnot always a status check]
H --> J[Step 3: Process claims]
I --> J
C --> J
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[Step 0: Read docs/pr-review-bots.md] --> B{Wait for list present?}
B -- No --> C[gh pr view --json comments,reviews\nProcess only what is visibly there]
B -- Yes --> D[Step 2: Route on Wait for list]
D --> E[Wait for every listed bot]
E --> F[Read BOTH channels per bot\ninline comments + PR summary body]
F --> G{Table column reads inconsistent?}
G -- Yes --> H[Observation only — not a routing instruction\nStill read both channels]
G -- No --> I[Follow bot-specific completion signal\nnot always a status check]
H --> J[Step 3: Process claims]
I --> J
C --> J
Prompt To Fix All With AIFix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
docs/prompt-standards.md:81-85
**Actionable instruction buried in anecdote paragraph**
The remedy sentence — "When the mechanism turns out not to exist, say what actually happens instead" — is appended mid-run onto the anecdote paragraph that concludes "which is the strongest available argument for this item." A reader skimming item 11 for what to *do* has to read through the historical anecdote to reach it. In `workflow-init.md`, the same instruction is cleanly the final line of the item's own prose, not embedded in the anecdote. A blank line before the remedy sentence here would separate the justification from the rule.
Reviews (1): Last reviewed commit: "chore(harden): enforcement-claim checkli..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| Worth knowing how that count was arrived at: it was first asserted as six without | ||
| checking, then "corrected" to five from memory, and only settled at six by reading | ||
| the commits. Both wrong numbers were themselves unverified claims — which is the | ||
| strongest available argument for this item. When the mechanism turns out not to exist, say what actually happens | ||
| instead ("this is a rule the agent keeps; nothing counts for it"). |
There was a problem hiding this comment.
The remedy sentence — "When the mechanism turns out not to exist, say what actually happens instead" — is appended mid-run onto the anecdote paragraph that concludes "which is the strongest available argument for this item." A reader skimming item 11 for what to do has to read through the historical anecdote to reach it. In workflow-init.md, the same instruction is cleanly the final line of the item's own prose, not embedded in the anecdote. A blank line before the remedy sentence here would separate the justification from the rule.
Prompt To Fix With AI
This is a comment left during a code review.
Path: docs/prompt-standards.md
Line: 81-85
Comment:
**Actionable instruction buried in anecdote paragraph**
The remedy sentence — "When the mechanism turns out not to exist, say what actually happens instead" — is appended mid-run onto the anecdote paragraph that concludes "which is the strongest available argument for this item." A reader skimming item 11 for what to *do* has to read through the historical anecdote to reach it. In `workflow-init.md`, the same instruction is cleanly the final line of the item's own prose, not embedded in the anecdote. A blank line before the remedy sentence here would separate the justification from the rule.
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Accepted — fixed in 1b9dc38.
Fair, and the asymmetry you spotted is the tell: the scaffolded copy in workflow-init.md already had the remedy as its own closing line, because I wrote that one from scratch. The repo copy got the remedy appended mid-line onto the anecdote in a later edit, and nobody re-read the paragraph afterwards.
The remedy is now its own paragraph ahead of the anecdote, so item 11 reads rule-then-justification like every other item.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
docs/hardening-taxonomy.md (1)
40-43: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy liftMove this stack-neutral class to the base taxonomy.
These lines acknowledge that
unverified-enforcement-claimbelongs in theharden-findingbase list, while this file is reserved for project-specific vocabulary. Keeping it here leaves the taxonomy internally inconsistent and makes recurrence detection repository-local. Add it to the base taxonomy, or make the class genuinely project-specific before merging.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/hardening-taxonomy.md` around lines 40 - 43, Move the unverified-enforcement-claim class from the project taxonomy in docs/hardening-taxonomy.md into the harden-finding base taxonomy, preserving its existing definition and references. Do not leave it listed as a project-specific class unless you revise it into genuinely project-specific vocabulary.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/superpowers/plans/2026-07-18-finding-triage-agent.md`:
- Around line 3-7: Remove the blank line between the historical note’s
blockquote lines so the entire note remains one contiguous blockquote and
satisfies markdownlint MD028.
In `@plugins/dev-workflow/commands/workflow-init.md`:
- Around line 560-563: Update the authoritative bot-waiting instructions in the
workflow-init template to use each bot’s configured completion signal from the
Notes column, rather than requiring a posted finding. Preserve the requirement
to wait for every listed bot, including zero-finding completions, and read both
channels for findings after each bot signals completion.
---
Nitpick comments:
In `@docs/hardening-taxonomy.md`:
- Around line 40-43: Move the unverified-enforcement-claim class from the
project taxonomy in docs/hardening-taxonomy.md into the harden-finding base
taxonomy, preserving its existing definition and references. Do not leave it
listed as a project-specific class unless you revise it into genuinely
project-specific vocabulary.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 7ecfb63f-5920-430a-a4e0-d7461364f102
📒 Files selected for processing (10)
AGENTS.mdCLAUDE.mddocs/hardening-log.mddocs/hardening-taxonomy.mddocs/pr-review-bots.mddocs/prompt-standards.mddocs/superpowers/plans/2026-07-18-finding-triage-agent.mddocs/superpowers/specs/2026-07-18-subagent-definitions-design.mdplugins/dev-workflow/commands/process-pr-review.mdplugins/dev-workflow/commands/workflow-init.md
Processing PR #5's comments. Step 0 fired — the PR touches AGENTS.md, CLAUDE.md and two commands — so subagent triage was skipped and these were validated here. First live use of that rule, working as designed. CodeRabbit (major) — the scaffolded template said processing starts once each bot "has posted, even with zero comments". A bot that finds nothing can complete via its status check without posting anything, so that instruction hangs forever on a clean review. It is also the exact hang docs/pr-review-bots.md exists to prevent, reintroduced by my own wording while fixing the routing model. Now waits on the per-bot completion signal recorded in Notes, and says why a post is not it. Greptile (P2) — item 11's actionable remedy was appended mid-line to the anecdote paragraph, so a reader skimming for the rule had to read the story first. The remedy is now its own paragraph ahead of the anecdote, matching how the scaffolded copy already reads. CodeRabbit (minor) — MD028: the plan's historical note sat directly above the pre-existing agentic-workers blockquote with a bare blank line between, which renderers may merge. Joined with a quoted separator. Checked the spec for the same pattern; its blockquote is followed by prose, so it does not apply. Battery green. Gate B skipped per CLAUDE.md §5 (trivial): three prose edits, no change to any contract.
…ic category Folding in the third and fourth data points, and correcting what the earlier rows claimed. The verification revised the premise. My earlier queries filtered issue comments on the login `greptile-apps`, but they are authored by `greptile-apps[bot]`, so they returned zero and looked like absence. Across #1, #2, #4 and #5 the summary comment was posted EVERY time; it is the inline comments that varied — present on #2, #4 and #5, absent only on #1. So the row now reads "summary always; inline usually", not "inconsistent findings channel". The login trap is written down, since it is what produced the wrong reading. The completion signal is the genuinely unreliable part: `gh pr checks` displayed a Greptile entry for #4 and #5, while the check-runs and statuses APIs return none for any observed head. Two tools, two answers, so neither is proof. That forced a third routing category. Greptile is a real findings source with no signal you can block on — blocking hangs the loop, dropping it loses findings, and the file exists to prevent exactly those two failures. Routing is now: wait for (block), process opportunistically (never block), ignore. Propagated to process-pr-review Steps 0 and 2 and the scaffolded template in the same commit, rather than leaving copies disagreeing — which is the drift this repo hardened against two commits ago. Docs-only; no executable changed. Battery green.
1 participant
Two reactive hardenings — one ledger row each, both surfaced by the gates or a bot. Plus the bot-matrix correction those findings forced.
1 ·
unverified-enforcement-claim(new class) → rung PSix citable instances across one spec, its plan and its diff where something was described as enforced, caught or guaranteed and no mechanism did it:
todos.md's own documented false-✓ pathse30568d)agents/is_prompt_pathdoes notEvery one was caught by a review gate. None by me. New
docs/prompt-standards.mditem 11 requires an enforcement claim to name its mechanism, and the author to verify that mechanism exists before writing the sentence. Mirrored into the scaffolded template.Minted as its own class rather than filed under
docs-drift: drift is two artifacts disagreeing; this is one claim untrue on its own terms even with nothing to contradict it. Different fix, so it needs a different grep.How the count was settled is the strongest argument for the item. It was asserted as six without checking, "corrected" to five from memory when challenged, and only settled at six by reading the commits. Two unverified claims, made while hardening against unverified claims. That's recorded in the checklist item itself.
2 ·
docs-drift→ rung P, escalated from the existing1 proserowA Gate-B fix changed behaviour the approved spec specified — reordered a precedence rule, made a row terminal, added a fourth terminal to
Done— and the spec was never updated. The two disagreed until a PR bot caught it after merge-readiness.The prior rung was an
AGENTS.mdrule about manifest claims: too narrow to reach spec-vs-implementation drift. The new rule lives where the recurring slice lives — the Gate-B instructions themselves: a fix that changes specified behaviour isn't complete until the spec is updated in the same commit, and the re-review covers both.No mechanical rung, and I argued this rather than forcing one: deciding whether a fix altered behaviour a spec pins down requires reading both artifacts. The crude proxies — "any command change must touch a spec" — would fire on every change no spec covers. Codex agreed after independent analysis.
Bot matrix
Greptile posted summary-only findings on PR #1 and two valid inline comments on PR #4. Its column now reads
inconsistentwith both observations dated, rather than being reclassified from one more data point — reclassifying on thin evidence is what put the wrong value there originally.That exposed a routing bug:
inconsistentwasn't consumable by a command that knew only line-findings-vs-summary. The Wait for list is now the routing authority across all four copies — the command's Steps 0 and 2, the matrix,AGENTS.md's tree comment, and the scaffolded template — with the table descriptive. The scaffold no longer derives Wait-for from a column that stopped holding yes/no.Two claims in that file were also wrong and are corrected: a fabricated "four days apart" (the observations are hours apart on the same day; timestamps now cited), and "both post a check" (the check-runs/statuses API returns only
qualityandCodeRabbitfor that head, so Greptile's completion is watched via its comment instead).Review
Gate B ran 5 passes, final clean on both spec and quality reviews. Codex independently verified each of the six historical instances against the commits, the 4/1/1 provenance split, the PR timestamps, and the API evidence.
Battery green: shellcheck ×4, hook suite, 61 invariant assertions, invariant scan,
plugin validate --strict. No executable changed — this is all prompts and docs.Summary by CodeRabbit