Skip to content

status page: curated incident record drives bars, uptime and Past Incidents; retire the dead gateway probe - #7

Merged
SilenNaihin merged 1 commit into
mainfrom
fix/curated-incident-record
Sep 7, 2026
Merged

SilenNaihin merged 1 commit into
mainfrom
fix/curated-incident-record

Conversation

@SilenNaihin

Copy link
Copy Markdown
Contributor

Why

The status page read "weird": a permanently red Gateway (authenticated) row at 30% uptime, an API incident that showed ~1,445 minutes for a blip, an 85.67% weekly figure, and bars that were only ever red or green.

Root causes: (1) Upptime derives downtime from how long each incident issue stayed open, not from the checks, so an issue that lingered a day counted as a day down; (2) the authenticated probe key was revoked on 2026-09-05 during a hand rotation that never updated the repo secret, so that row sent a dead key and reported a false down for two days; (3) no maxResponseTime was set, so the degraded (yellow) state was unreachable.

Changes

  • Curated incident record assets/status-ui/incidents.json: 15 incidents reconstructed from the gateway ledger (completions and gateway-owned failures per 10-minute bucket), production deploy runs, and the day's incident notes, each with the real UTC start/end and a short technical description. down (not serving) counts against uptime; degraded (elevated errors/latency) renders yellow and does not. Live checker issues opened after liveIssuesSince merge in automatically; annulledIssues [4, 5] are the two monitoring artifacts.
  • uptime-bars.js now renders the 90-day bars, the legend uptime, and the dated Past Incidents section (component · Outage/Degraded · HH:MM–HH:MM UTC · minutes, plus description) from that merged record.
  • .upptimerc.yml: maxResponseTime: 5000 on every site (yellow is reachable); Gateway (authenticated) disabled until a key is minted under the status-monitor org (a key from any other org must never be used here); the generator's issue-derived uptime figure and its period selector are hidden so the page carries one uptime number.
  • README: how the record works, how to add or annul an incident, the probe-key rotation runbook, and why issue-derived minutes can exceed the real outage.

Resulting 90-day uptime: API 99.83%, Web 99.78%, Docs 100% (real downtime: API 33 min, Web 41 min).

Follow-up (after merge)

Remove the status label from issues #4 and #5 and close #4 so Upptime's own numbers stop counting them; dispatch Static Site CI so the new overlay deploys.

🤖 Generated with Claude Code

…s; retire the dead gateway probe

The page derived downtime from how long each incident ISSUE stayed open, which
turned a 15-minute API blip whose issue lingered a day into ~1,440 minutes down
(85.67% week) and kept a revoked-probe-key 401 on the books as a 30% 'gateway'.

- assets/status-ui/incidents.json: 15 incidents reconstructed from the gateway
  ledger (per-10-minute completions / gateway-owned failures), production
  deploy runs and the day's incident notes, with the real UTC start/end and a
  short technical description each. 'down' counts against uptime; 'degraded'
  is shown yellow and not counted. Live checker issues after liveIssuesSince
  merge in; annulledIssues [4, 5] are the two monitoring artifacts.
- uptime-bars.js renders the 90-day bars, legend uptime and the dated Past
  Incidents (component · state · HH:MM–HH:MM UTC · minutes + description)
  from that merged record.
- .upptimerc.yml: Gateway (authenticated) disabled until a status-monitor key
  exists; the generator's issue-derived uptime figure and period selector are
  hidden so the page carries one uptime number.
- README documents the record and how to add or annul an incident.

Resulting 90-day uptime: API 99.83%, Web 99.78%, Docs 100%.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@SilenNaihin
SilenNaihin merged commit b1741e5 into main Sep 7, 2026
0 of 2 checks passed
@SilenNaihin
SilenNaihin deleted the fix/curated-incident-record branch September 7, 2026 04:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🛑 Gateway (authenticated) is down

1 participant