Skip to content

ops: SLOs, CloudWatch dashboard, alert routing #38

Description

@jonmatum

Priority: P0 — launch blocker
Parent: #34

Context

Nine CloudWatch alarms exist today, all pointing to the ssb-{env}-daily-digest SNS topic. There is no dashboard, no published SLO, and no severity-based routing. A Bedrock throttle at 03:00 and a stale-node surfacing event both go to the same place.

Requirements

SLOs (publish in docs/ops/slo.md)

  • API Gateway availability (e.g. 99.5 % monthly)
  • POST /capture p95 latency (e.g. ≤ 1.5 s)
  • Enrich success rate (e.g. ≥ 99 % over 24h)
  • Error budget + burn-rate alarms

Dashboard

Alert routing

  • Two SNS topics: ssb-{env}-alerts-critical and ssb-{env}-alerts-info
  • Critical topic: API 5xx burst, DDB throttle, any DLQ depth > 0, Bedrock cost alarm, WAF block spike
  • Info topic: daily digest, surfacing results
  • Decide integration target (email vs PagerDuty vs Slack webhook) and document in docs/ops/oncall.md

Acceptance criteria

  • SLO doc published
  • Dashboard reachable and linked from docs
  • Two alert topics exist with correct subscriptions; manual alarm test triggers the critical path

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions