feat: 30-runbook catalog, in-plugin sample rules, WebhookHost UX + cross-team receiver-name fix#37
Merged
Merged
Conversation
Adds 10 runbooks (20 -> 30): a new security category (unexpected-container-image, apiserver-auth-failure-spike, privileged-container-started, interactive-shell-in-container, rbac-privilege-escalation, security-tooling-down) plus SRE gap-fillers (cpu-throttling-high, image-pull-backoff, pods-unschedulable, postgres-connections-near-max). Each is wired with a scaffoldSets entry, add/remove/validate autocomplete, help text, and inline Quick Diagnostics. Adds embedded docs ALERT_CATALOG.md (docs alerts) and ALERT_REQUIREMENTS.md (docs requirements). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
README (20->30 runbooks, +security category), SLASH_COMMANDS (security group set + alerts/requirements docs topics + seven-set count), QUICKSTART (security set), CONFIGURATION (security label profile + docs requirements pointer). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sweep of README + docs (DEVELOPMENT smoke-test steps, KUBERNETES route comment, CONFIGURATION, SLASH_COMMANDS worked examples) missed in the first pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
For each slug in scaffoldSets, asserts renderReceiverYAML produces config Alertmanager's own parser accepts, with Quick Diagnostics injected and no unresolved plugin placeholders. Closes the gap where a malformed runbook only surfaced at /alertmanager add time. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tier-1 pure-Go guards (run in make test, no infra): (1) scaffoldSets <-> runbook files match both ways (drift guard); (2) every Quick-Diagnostics <placeholder> is an allowlisted label or a known literal (catches typo'd labels that render literally); (3) every /alertmanager docs topic resolves to an embedded file; (4) every scaffold category is surfaced in help + autocomplete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Badges pointed at the old christopherfickess fork (broken) -> mattermost-community. Add 'make test' to the build block + a DEVELOPMENT.md link, and a 'Docs & help' section pointing to /alertmanager docs topics (alerts/requirements/...), the runbook INDEX, and the setup/ops docs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
golangci-lint (modernize) flagged strings.Split in a range; match the repo convention (SplitSeq, as in runbook_diagnostics.go). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 'What an alert looks like in chat' block nested ```bash/```promql fences inside a 3-backtick fence, so markdown closed the outer block early and the Quick Diagnostics rendered outside it. Bump the outer fence to 4 backticks so the inner fences render literally and the full example (incl. inline diagnostics) shows correctly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tion Bring samples/prometheus-rules.yaml up to the full catalog: add the 11 rules missing since the security category + SRE gap-fill runbooks landed (4 compute: cpu-throttling-high, container-oom, image-pull-backoff, pods-unschedulable; 1 database: postgres-connections-near-max; 6 new security: unexpected-container-image, apiserver-auth-failure-spike, privileged-container-started, interactive-shell-in-container, rbac-privilege-escalation, security-tooling-down). 31 rules cover 30 runbooks — ContainerOOMKilled reuses the high-memory-usage runbook (same remediation). Add a validate-rules job to test.yml running `promtool check rules` on the shipped sample on every PR, so a broken PromQL expr or malformed YAML in the file users copy can never merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ttings Users shouldn't have to clone the repo to get the Prometheus rules — the plugin only owns the Alertmanager side, but their Prometheus has to emit the `runbook:`-labelled alerts. So surface the shipped sample rules from inside the plugin: - render-docs now renders samples/prometheus-rules.yaml into a browsable help page (public/help/prometheus-rules.html) AND copies the raw file alongside it for download. Single source of truth: the page is built live from the yaml, so it can't drift from what users download. - New `/alertmanager rules` command DMs both links (view + download) and explains the runbook-label join contract. Registered in help text and autocomplete. - System Console header gets a "Sample Prometheus rules" link. System Console cleanup (was a wall of text, now scannable): - All 5 settings' help_text cut to one sentence each; the removed detail (k8s DNS example, openssl command) already lives in the placeholders. - Header and footer trimmed; footer command table shortened. Stale-count fixes (catalog is 30 runbooks / 31 rules, not 20): - plugin.json footer, CONTRIBUTING.md, render-docs runbook index (+ the missing security category and new compute/database runbooks), and three Go comments (admin_inventory.go, cmd_query.go). CHANGELOG.md left as-is (historical release notes). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two discoverability fixes for the URLs users have to get right: am-url autocomplete hint — the `/alertmanager add` URL argument now spells out the container-networking gotcha at type time: a containerized Mattermost reaches Alertmanager via http://host.docker.internal:9093 (not localhost, which is the MM container itself), or a service name on a shared Docker network. Previously just "API base URL (no trailing slash)", which let people enter localhost and hit a confusing connection error later. WebhookHost preset dropdown — new System Console `WebhookHostPreset` dropdown offers known-good hosts for common setups (Docker Desktop, default-namespace K8s) so admins don't have to know the magic host.docker.internal string. Kept the free-text WebhookHost field for custom URLs (non-default namespace, bare-metal FQDN). resolveWebhookHost collapses the two into one effective value with custom-wins precedence, so existing installs (text field only) behave identically — the dropdown defaults to None. Resolution happens at config-load time into the existing configuration.WebhookHost, so no consumer changed. Validation now runs on the resolved value. Added TestResolveWebhookHost pinning the precedence. Mattermost plugin settings can't do a native select-or-type combo and this plugin is server-only (no webapp), so a two-field dropdown+text is the achievable shape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The single dense hint line was hard to read in the autocomplete dropdown. Replace it with a static list of the three real patterns, each with its own per-item hover: Docker Desktop (host.docker.internal), Kubernetes (service DNS), and a custom URL template. Non-enforcing, so users can still type any URL - matches the existing target/remove arg pattern. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ions
Receiver names were <slug>--<channel>, but channel names are only unique
per team (town-square exists in every team). Two same-named channels in
different teams produced identical receiver names, so:
- `/alertmanager add` in the second team silently skipped every receiver
as "already exists" (dedup matched on channel name alone), creating
nothing and misrouting that team's alerts to the first team's webhook;
- list / remove / export / validate (all via configsForCurrentChannel)
matched on channel name too, so they'd show — and `remove all` could
delete — another team's receivers.
Fix (forward-only; no migration, nobody has data yet):
- receiverNameForChannel now takes team: <slug>--<team>-<channel>.
Alertmanager requires globally-unique receiver names, so team must be
in the name, not just the stored fields.
- add-dedup, configsForCurrentChannel, and resolveReceiverName all
team-scope now (configsForCurrentChannel resolves team via
channel.TeamId -> GetTeam). That one function backs 13 command sites.
- name length cap raised 64 -> 190 to fit <slug>--<team>-<channel>
(~164 worst case with 64-char team/channel slugs); AM/YAML don't care.
- the runbook-slug boundary (first `--`) is still unambiguous;
uniqueness of the team-channel tail is enforced by the existing
duplicate-name rejection, not the separator.
Updated CONFIGURATION.md / SLASH_COMMANDS.md and the name tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the stub with project-specific guidance capturing the invariants that break silently: catalog count sync (30 runbooks / 31 rules) + drift guards, tests required in all CI jobs, the three promtool tiers, security coverage (Grype SARIF gate, the open GO-2026-5932 finding, version pins), the team-qualified receiver-name contract, the runbook-label join key, single-source rules file, and auto-update discipline for mirrored counts and docs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Route-tester form cleanups on the inventory page: - Severity is now shown only in end-to-end mode (hidden + disabled for simulate and webhook-test). In simulate the routes key on the `runbook` label, so a severity-only sim just fell through to the default route and confused people; webhook-test ignores severity entirely. Server drops the dedicated severity outside end-to-end; a `change` listener on the mode dropdown toggles the field. Admins who genuinely have severity-based routes still simulate them via the extra-labels box (placeholder now hints `severity=critical`). - Added a Team scope dropdown. Channel names repeat across teams (town-square is in every team), so filtering the tester by channel name alone targeted the wrong team's receivers for webhook-test/end-to-end. filterConfigsByChannel now takes team; the channel dropdown cascades to the selected team's channels (teamChannels map + JS, mirroring the existing type/value cascade). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
christopherfickess
changed the title
rahb3rt
approved these changes
Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Started as "expand the alert catalog to 30 runbooks + a security category" and grew into a broader hardening pass on the catalog, the Prometheus-rules story, System Console UX, and a real cross-team bug. Grouped below; each group is independently reviewable.
1. Catalog: 20 → 30 runbooks (+ security category)
scaffoldSets, help text, autocomplete, docs, and the runbook index all updated to match. All stale "20 runbook/receiver" references corrected to 30 acrossplugin.json, docs,CONTRIBUTING.md, and code comments (CHANGELOG left as historical).2. Sample Prometheus rules: complete set + CI validation
samples/prometheus-rules.yamlbrought to the full 31 rules covering all 30 runbooks (ContainerOOMKilledreuses thehigh-memory-usagerunbook → one extra rule than runbooks).validate-rulesCI job intest.ymlrunspromtool check ruleson the shipped sample on every PR (pinnedprom/prometheus:v3.5.0), so a malformed rule file that users copy can't merge.3. Ship the rules in-plugin (no repo clone needed)
build/render-docsnow renderssamples/prometheus-rules.yamlinto a browsable help page and copies the raw file for download — single source of truth, the page is built live from the yaml so it can't drift./alertmanager rulescommand surfaces both links (view + download) and explains therunbook:-label join contract. System Console header gains a "Sample Prometheus rules" link.4. System Console cleanup + WebhookHost UX
WebhookHostPresetdropdown (Docker Desktop / default-namespace K8s / none) alongside the free-textWebhookHost.resolveWebhookHostcollapses them (custom text wins) into the existingconfiguration.WebhookHost, so consumers are unchanged and existing installs behave identically. Covered byTestResolveWebhookHost./alertmanager addam-url argument now offers three hover-able autocomplete suggestions (Docker host / Kubernetes / custom), surfacing the container-networking gotcha (host.docker.internal, notlocalhost) at type time.5. Bug fix: cross-team receiver-name collision
Receiver names were
<slug>--<channel>, but channel names are unique only per team (town-squareis in every team). Two same-named channels in different teams produced identical receiver names, so/alertmanager addin the second team silently skipped everything as "already exists" and misrouted its alerts to the first team's webhook;list/remove/export(viaconfigsForCurrentChannel) mixed teams too.<slug>--<team>-<channel>(Alertmanager requires globally-unique receiver names, so team must be in the name).configsForCurrentChannel, the add-dedup, andresolveReceiverNameare all team-scoped now. Name length cap raised 64 → 190 to fit the longer form.6. Admin route-tester (inventory page) polish
runbook. Advanced users still simulate severity via the extra-labels box.filterConfigsByChannelis team-aware and the Channel dropdown cascades to the selected team's channels.7. Contributor guidance
.claude/CLAUDE.mdfrom a stub into a real agent guide: catalog-count sync rule + drift guards, tests-in-all-jobs, the three promtool tiers, security coverage (Grype SARIF gate, the intentionally-openGO-2026-5932finding, version-pin policy), the invariants (team-qualified names,configsForCurrentChannelchokepoint, therunbook:join key), and auto-update discipline.Testing
go test -race ./...,golangci-lint(v2.12.2),promtool check rules,make render-docs,plugin.jsonJSON validity — all green (Test / Lint / Build pass; Security/Grype carries the known open finding).TestResolveWebhookHost, team-qualified name cases inTestReceiverNameForChannel, plus the existing catalog-consistency drift guards.Known limitations / follow-ups
promtool test rules) live in a separate local test bed, not this repo's CI — only syntax is gated here./alertmanager validateslash-command autocomplete still shows--severityin all modes — Mattermost positional autocomplete can't hide it (only the admin form can).🤖 Generated with Claude Code