LLM-powered root-cause analysis for monitoring alerts. Incoming alerts from Alertmanager (Kubernetes) or CheckMK trigger automated diagnostic collection, which is sent to Claude for analysis. The resulting root-cause assessment is delivered to operators via ntfy.
Instead of staring at a 3 AM "DiskPressure" alert and manually running ten kubectl / ssh commands, you get a short markdown summary on your phone: likely cause, blast radius, suggested remediation — derived from real metrics, events, pod logs, and (for CheckMK hosts) live SSH diagnostics.
Alert fires → Webhook → Gather diagnostics → Claude / LLM API → ntfy notification
Two independent analyzers share a common library but run as separate binaries:
| Analyzer | Alert Source | Diagnostics Gathered |
|---|---|---|
| k8s-analyzer | Alertmanager webhook | Prometheus metrics, K8s events, pod status, pod logs + agentic kubectl_exec / promql_query loop |
| checkmk-analyzer | CheckMK notification script | CheckMK REST API (host/service state), agentic SSH diagnostics |
Both deduplicate repeat alerts (configurable cooldown) and process work concurrently (5 workers, queue depth 20). All diagnostic output is passed through a secret-redaction filter before leaving the analyzer.
Several features ship disabled by default and only activate when explicitly configured:
| Feature | Default | Enable via |
|---|---|---|
| Alert history (recurrence detection, prior-analysis context) | Off | HISTORY_ENABLED=true + writable volume |
| Severity-based model routing / agent-round budgets | Off | CLAUDE_MODEL_<SEVERITY>, MAX_AGENT_ROUNDS_<SEVERITY> |
| Group cooldown | Off | GROUP_COOLDOWN_SECONDS |
| Storm mode (degrade to static analysis under alert floods) | Off | STORM_MODE_THRESHOLD |
| Circuit breaker (stop calling a failing Claude API) | Off | CIRCUIT_BREAKER_THRESHOLD |
Out of the box you get the core pipeline: webhook → diagnostics → analysis → ntfy, with per-alert cooldown deduplication and prompt caching. See docs/configuration.md for the full reference and docs/cost-and-storm-protection.md for rollout guidance on the cost/storm features.
The fastest way to try the checkmk-analyzer. For Kubernetes, see docs/install-k8s.md.
You need:
- An Anthropic API key or compatible token
- An ntfy server
- SSH key +
known_hostsfor the monitored hosts (unprivileged user) - A CheckMK automation user
mkdir -p ./ssh
cp /path/to/id_ed25519 ./ssh/id_ed25519
cp /path/to/known_hosts ./ssh/known_hosts
chmod 600 ./ssh/id_ed25519
docker run -d --name checkmk-analyzer \
--restart unless-stopped --read-only --cap-drop ALL --user 65534:65534 \
-p 127.0.0.1:8080:8080 -p 127.0.0.1:9101:9101 \
-v "$(pwd)/ssh:/ssh:ro" \
-e WEBHOOK_SECRET="change-me" \
-e ANTHROPIC_API_KEY="sk-ant-..." \
-e CHECKMK_API_URL="https://checkmk.example.com/mysite/check_mk/api/1.0/" \
-e CHECKMK_API_USER="automation" \
-e CHECKMK_API_SECRET="..." \
-e NTFY_PUBLISH_URL="https://ntfy.example.com" \
-e NTFY_PUBLISH_TOPIC="checkmk-analysis" \
ghcr.io/madic-creates/claude-alert-checkmk-analyzer:latest
curl -sf http://127.0.0.1:8080/healthThen install the CheckMK notification script and create a notification rule — full steps in docs/install-checkmk.md.
Pre-built images are published to GHCR on every push to main:
ghcr.io/madic-creates/claude-alert-kubernetes-analyzer:latest
ghcr.io/madic-creates/claude-alert-checkmk-analyzer:latest
| Image | Base | Size |
|---|---|---|
claude-alert-kubernetes-analyzer |
scratch |
~60 MB |
claude-alert-checkmk-analyzer |
alpine:3.23 |
~20 MB (includes openssh-client) |
Install
docs/install-k8s.md— Kubernetes deployment (Kustomize manifests, Alertmanager wiring, RBAC)docs/install-checkmk.md— CheckMK deployment (Docker / Compose / Kubernetes, notification script, rules, optionalai_contexthost attribute)
Operations
docs/configuration.md— full env-var reference (shared, k8s, checkmk, LLM provider, storm robustness)docs/observability.md— API endpoints, Prometheus metrics, scrape config, loggingdocs/hardening.md— runtime hardening, agentic-loop guardrails, RBAC and SSH detailsdocs/cost-and-storm-protection.md— operator guide for prompt caching, severity-based routing, token-cost dashboards, storm-mode and circuit-breaker rollout
Development & maintenance
docs/development.md— build, test, project layout, key patterns, CI/CDdocs/cost-and-storm-protection-internals.md— architecture and component reference for the cost/storm featuresdocs/pre-commit.md— pre-commit hook configurationdocs/renovate.md— dependency update automationdocs/cleanup-ghcr.md— GHCR tag retention
Licensed under the Apache License, Version 2.0. See NOTICE for attribution of bundled third-party software.