Skip to content

docs: replace the guessed blast radius of the prefix bug with measured behaviour - #118

Merged
douglasmun merged 1 commit into
mainfrom
docs-staging-verification
Aug 17, 2026
Merged

docs: replace the guessed blast radius of the prefix bug with measured behaviour#118
douglasmun merged 1 commit into
mainfrom
docs-staging-verification

Conversation

@douglasmun

Copy link
Copy Markdown
Owner

Follow-up to #116/#117. I verified both runbook findings against live AWS in ap-southeast-1 — dry-run changeset only, nothing deployed, nothing left behind.

The EventBridge finding is worse than I wrote

Pitfall #62 hedged: "best case ResourceNotFoundException, worst case they inspect the unrelated production service." That was reasoning, not measurement. There is no best case:

$ aws events describe-rule --name cdr-s3-object-created --query State
ENABLED

$ aws events list-targets-by-rule --rule cdr-s3-object-created --query 'Targets[].Arn'
arn:aws:lambda:ap-southeast-1:667467573925:function:cdr-lambda

The rule exists, is ENABLED, and targets the live askkaifbot cdr-lambda. So an operator deployed at cdr-staging, following the old runbook, does not get an error telling them they are looking at the wrong thing — they get a green light from somebody else's production and conclude their own pipeline is correctly wired.

A command that errors is self-correcting. One that confidently answers about the wrong resource is not. Rule recorded: when estimating a doc bug's blast radius, run the wrong command — do not reason about what it would probably do.

MaxMemoryUsed, with controls in both directions

Check Result
list-metrics --metric-name MaxMemoryUsed 0 — no such metric
Old command, runbook's own 10-min window null, exit status 0
Same shape against Duration (negative control) 2 metrics — so the form was never the problem
Replacement Logs Insights query max_mem = 130 MB, status Complete

Dry-run changeset

All 19 resources + Add — no Modify, Delete or Replacement. The changeset reports PhysicalResourceId: None for Adds, so it cannot prove non-collision by itself; I resolved the names from the changeset template and checked all 14 against the account — every one free. Changeset and its REVIEW_IN_PROGRESS stack shell deleted; live cdr-lambda LastModified unchanged at 2026-07-05T06:19:32.

The collision evidence is now surfaced in the runbook's own warning, where an operator actually reads it, not only in the pitfalls file.

Note

check_runbook_prefix.py — added in #117 — flagged the first draft of this very commit, because I wrote describe-rule --name cdr-s3-object-created in the new warning. Correct call by its own rule. I reworded rather than adding an exemption, so the guard stays strict.

454 tests pass; all 8 guards pass.

Scope: documentation only. No code, no deployment. The pipeline is still verified only as far as changeset validation — no Lambda ran.

🤖 Generated with Claude Code

…d behaviour

Verified both runbook findings against live AWS in ap-southeast-1 (2026-08-17),
via a dry-run changeset only — nothing was deployed.

The EventBridge finding is worse than pitfall #62 originally guessed. That
write-up hedged "best case ResourceNotFoundException, worst case they inspect
the unrelated service". There is no best case: describe-rule on the default
`cdr-s3-object-created` returns ENABLED and list-targets-by-rule shows it wired
to the live askkaifbot cdr-lambda. An operator deployed at cdr-staging and
following the old runbook gets a green light from somebody else's production.
A command that errors is self-correcting; one that confidently answers about
the wrong resource is not. Recorded, with the rule: run the wrong command,
do not reason about what it would probably do.

MaxMemoryUsed confirmed with controls in both directions: list-metrics returns
0, the old command returns null with exit status 0, and the identical shape
against Duration returns 2 — so the query form was never the problem. The
replacement Logs Insights query returns max_mem = 130 MB against the real log
group.

Surfaced the collision evidence in the runbook's own warning, where an operator
reads it, rather than only in the pitfalls file.

Dry-run result (not committed, recorded here): all 19 resources "+ Add", no
Modify/Delete/Replacement; all 14 cdr-staging-* names confirmed free; changeset
and its REVIEW_IN_PROGRESS stack shell deleted; live cdr-lambda LastModified
unchanged at 2026-07-05T06:19:32.

454 tests pass; all 8 CI guards pass — including check_runbook_prefix.py, which
flagged the first draft of this very commit and made me reword it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@douglasmun
douglasmun merged commit b3f8d59 into main Aug 17, 2026
4 checks passed
@douglasmun
douglasmun deleted the docs-staging-verification branch August 17, 2026 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant