Repository navigation
feat(fleet-report): per-site error budgets from drosera's SLO queries - #290
lentago-claude-runner[bot] wants to merge 1 commit into
Conversation
Adds a "Site error budgets — last 30 days" section to fleet-report.md for the three SLO sites in drosera ADR-0008: target, 30-day availability, budget used, budget left in minutes, and burn-rule stretches in the window. The PromQL is drosera's SLO board and burn rules verbatim, sent to the stack's grafanacloud-prom data source through Grafana's data-source proxy. The token is an optional new repo secret, FLEET_REPORTS_GRAFANA_TOKEN (Viewer service account). Unset, the section is one "not configured" line and the run stays green; a failing token renders an error note and a workflow warning. ci/validate.py gains a `budget` check pinning the expressions, the burn-stretch counter and the no-token path. Refs lentago/drosera#199 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review (haiku, claude-review.yml)Review of PR #290: per-site error budgets in the fleet report
Not verified in this review: |
Closes lentago/drosera#199
What it adds
A Site error budgets — last 30 days section in
fleet-reports/fleet-report.md, between Activity and the Code census. The weeklyfleet-reports.ymlrun produces it. There is one row for each SLO site in drosera ADR-0008 (lentago.dev, icecreamtofightwith.com, pondviewlane.com), with:The epics and incidents reports are untouched.
Data path
dashboards/sites-slo-error-budget.json(attainment, budget remaining, budget minutes), as instant queries at the generation time;terraform/alerts.tf("Site SLO burn rate"), as 30-day range queries over the long and short windows. The generator then applies the rules' ownA > t ∧ B > tper step.https://lentago.grafana.net/api/datasources/proxy/uid/grafanacloud-prom/api/v1/{query,query_range}. That is the stack's Prometheus-compatible API, reached through Grafana's data-source proxy, on the samegrafanacloud-promdata source the alert rules name.fordelays are not applied, so the column counts stretches where the condition held, not notifications sent. The report states this under the table. Grafana's alert state-history API would be exact, but it needs per-rule UIDs and its Cloud backend could not be checked from here, so I left it out.FLEET_REPORTS_GRAFANA_TOKENsecret is unset…, and the run stays green. A token that is set but fails (401, unreachable, rejected query) renders Unavailable this run: HTTP 401… plus a::warning::in the run, and the rest of the report publishes as normal. Error text never includes the request, so the token cannot reach the CI log.budgetcheck inci/validate.py. It pins the expressions verbatim against drosera's text for lentago.dev, tests the burn-stretch counter (including gt-not-ge), and tests the one-line no-token path. I confirmed it can fail by breaking each of the three in turn; each run went red, then green again once restored.Secret the maintainer must create
FLEET_REPORTS_GRAFANA_TOKENis a Grafana service-account token (glsa_…) under an account with the built-in Viewer role. Viewer can query data sources and change nothing. This follows ADR-0011 rule 4 and drosera'sservice_accounts.tfpattern: Terraform holds the slot, and the value is minted by hand. The full steps are in the header of.github/workflows/fleet-reports.yml:grafana_service_account "fleet_reports_viewer"(name = "fleet-reports-viewer",role = "Viewer") toterraform/service_accounts.tf, plus its row interraform/README.md§ Service accounts. Merge so CI applies it. That drosera change is not in this PR.gh secret set FLEET_REPORTS_GRAFANA_TOKEN --repo lentago/.githuband paste the value at the prompt. If you set an expiry, record it in the credential register (Credential register: every fleet credential's store, scope, issuer and expiry (names only) #243).workflow_dispatch) and check that the refresh PR shows the table, not the note (ADR-0011 rule 5).The secret goes only to the "Regenerate reports" step.
Testing
python3 ci/validate.py: all checks pass.🤖 Generated with Claude Code