Skip to content

ci: guard that runbook commands do not hardcode a prefixed resource name - #117

Merged
douglasmun merged 1 commit into
mainfrom
add-runbook-prefix-guard
Aug 17, 2026
Merged

ci: guard that runbook commands do not hardcode a prefixed resource name#117
douglasmun merged 1 commit into
mainfrom
add-runbook-prefix-guard

Conversation

@douglasmun

Copy link
Copy Markdown
Owner

Closes the last item from the Codex review — the one I deferred out of #116 because it needed a real false-positive story rather than a grep.

Why this is a security guard, not a docs nit

Lambda, SNS topic, DLQ, IAM role/policy, EventBridge rule and alarm names all derive from ResourcePrefix. The runbook tells operators not to deploy at the cdr default, because a live askkaifbot service already owns those names in ap-southeast-1. So a command hardcoding cdr-<suffix> is wrong for precisely the readers who followed the runbook's own advice — best case ResourceNotFoundException mid-incident, worst case they inspect an unrelated production service and draw conclusions from it.

That shipped twice for the EventBridge rule (fixed in #116). This makes the class non-recurring.

What it checks

Suffixes are derived from src/template.yaml and terraform/main.tf at runtime — 13 of them — never hand-listed. That is pitfall #61's lesson applied one commit later: the #60 cap audit enumerated from memory, missed one, and shipped a fail-open. A resource added to either IaC file is covered without touching this script.

A cdr-<suffix> is flagged only where it is passed to a command:

--rule cdr-s3-object-created                     an argument value
--dimensions Name=FunctionName,Value=cdr-lambda  a Key=Value argument
/aws/lambda/cdr-lambda                           a log-group path

Prose, parameter tables and S3 tag names (cdr-status=sanitised) are deliberately not flagged. The runbook must be able to say "do not leave ResourcePrefix at the cdr default" and show cdr-staging-source-<alias> as a suggested value — a guard that flags its own advice gets silenced rather than fixed.

The exemption

Section 8's ResourceExistenceCheck block investigates the foreign cdr-lambda the reader did not deploy. There the literal is correct and $PREFIX-lambda would be actively wrong. No syntactic rule tells that apart from a bug, so the doc declares it inline:

<!-- prefix-literal-ok: this whole block investigates the FOREIGN cdr-lambda … -->

A marker with no reason fails. A marker inside a fenced block fails — an HTML comment there is not a comment; it renders literally and lands mid-command when the command is line-continued (I wrote it that way first, which is how I found it). Every exemption is printed in the summary, so a growing pile is visible rather than quietly normal.

Negative controls — two caught real guard bugs

A guard never shown failing is an assumption (#57). Five controls run:

Control Expect Result
Pre-fix runbook (4dc275a^) FAIL ✅ catches both real bugs
Exemption with empty reason FAIL
Marker inside a fence FAIL
Newly injected Value=cdr-lambda FAIL (after fix)
Current runbook PASS ✅ canary clean

Two bugs this caught in my own guard:

  1. The first regex matched only a name immediately after --opt, so an injected --dimensions Name=FunctionName,Value=cdr-lambdathe form the runbook's own CloudWatch commands use — passed clean. Now matches the whole argument run.
  2. The harness initially counted grep -c 'hardcodes', which also matched the help text, so a failing control looked like it fired. A guard shown failing by a broken harness is worse than one never shown failing, because it looks like evidence.

Verification

🤖 Generated with Claude Code

Lambda, SNS topic, DLQ, IAM role/policy, EventBridge rule and alarm names all
derive from ResourcePrefix, and the runbook tells operators NOT to deploy at
the `cdr` default — a live askkaifbot service already owns those names in
ap-southeast-1. A command hardcoding `cdr-<suffix>` therefore sends the reader
at a resource they do not have, or at somebody else's production, during an
incident. That shipped twice for the EventBridge rule (fixed in #116).

scripts/check_runbook_prefix.py derives the prefixed-name suffixes from
src/template.yaml and terraform/main.tf at runtime (13 of them) rather than
hand-listing them — pitfall #61's lesson applied — and flags a `cdr-<suffix>`
only where it is passed to a command. Prose, parameter tables and S3 tag names
are deliberately untouched: the runbook must be able to advise against the
`cdr` default, and a guard that flags its own advice gets silenced, not fixed.

Section 8 legitimately inspects the FOREIGN cdr-lambda, where $PREFIX would be
wrong; that declares itself with a reasoned `<!-- prefix-literal-ok: … -->`
marker. An empty reason fails, and so does a marker inside a fenced block — an
HTML comment there renders literally and splits a line-continued command.

Five negative controls run, two of which caught real guard bugs: the first
regex missed `--dimensions Name=FunctionName,Value=cdr-lambda` (the form the
runbook's own CloudWatch commands use), and the harness initially counted a
help-text line, making a failing control look like it fired. Recorded as
pitfall #62.

454 tests pass; all 8 CI guards pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@douglasmun
douglasmun merged commit e513045 into main Aug 17, 2026
4 checks passed
@douglasmun
douglasmun deleted the add-runbook-prefix-guard branch August 17, 2026 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant