Skip to content

feat: event monitoring setup wizard + pipeline test tooling - #30

Merged
AhmadHammad21 merged 9 commits into
mainfrom
feat/event-monitoring-setup-wizard
May 12, 2026
Merged

AhmadHammad21 merged 9 commits into
mainfrom
feat/event-monitoring-setup-wizard

Conversation

@AhmadHammad21

Copy link
Copy Markdown
Owner

Summary

  • 4-step setup wizard on /init — account creation → AWS config → permission check → infrastructure creation (SQS + 9 EventBridge rules + aggregate CloudWatch alarm)
  • Settings → AWS Configuration — new Event Monitoring section with Create/Teardown infrastructure buttons and status indicator
  • Monitoring page — Send test event button (admin only) for quick pipeline validation
  • Backend — two new endpoints: DELETE /api/init/infra (teardown) and POST /api/init/test-event (synthetic SQS push)
  • Aggregate CloudWatch alarm (opendevops-lambda-errors-aggregate) created automatically during setup — monitors all Lambdas account-wide, no per-function alarms needed
  • Polling → /monitoring fix — _persist_and_notify now calls add_alert() so polling investigations appear on the Monitoring page
  • Event consumer fix — service=unknown bug fixed, now reads from services_affected[0]
  • LiteLLM log spam fix — suppresses "Provider List" info messages on startup
  • Test script — scripts/test_pipeline.py (boto3-based, Windows-compatible) with --lambda-only flag for testing real alarm path vs direct SQS push
  • Docs — docs/monitoring.md rewritten to cover both detection modes end-to-end; docs/slack_and_polling.md updated with comparison table

Test plan

Settings — Infrastructure (test these first, infra is live)

  • Settings → AWS Configuration → Teardown deletes SQS queue, 9 EventBridge rules, and opendevops-lambda-errors-aggregate alarm in AWS console
  • After teardown, UI shows "Disabled" badge and "Create Infrastructure" button
  • Create Infrastructure button re-provisions everything and flips back to "Enabled"

Monitoring page

  • Send test event button (admin) calls POST /api/init/test-event, shows toast, new card appears on /monitoring within ~90s
  • Non-admin (user role) does not see the Send test event button
  • Error toast shown when button is clicked but infra not set up (teardown first)

/init wizard (requires clean state — reset data/agent.db or use fresh env)

  • Step 1 — admin account creation works, advances to Step 2
  • Step 2 — region selector + optional SNS ARN, advances to Step 3
  • Step 3 — permission check grid renders green ✓ / red ✗ badges per service; Required badge shown for cloudwatch/lambda/sqs/events
  • Step 4 — "Create Infrastructure" creates SQS + rules + alarm, shows success summary
  • Step 4 — "Skip for now" shows disclaimer listing 4 missing features; navigates to app
  • Step indicator shows checkmarks for completed steps
  • Navigating to /init when already initialized redirects away

Polling

  • POLL_INTERVAL_MINUTES=1 + Lambda errors → investigation appears on /monitoring within ~2 min
  • POLL_INTERVAL_MINUTES=0 (default) — poller never starts, no log lines about poller

Event-driven (direct SQS push — fastest)

  • uv run python scripts/test_pipeline.py --function <name> → card on /monitoring in ~90s

Event-driven (real alarm path)

  • uv run python scripts/test_pipeline.py --function <name> --lambda-only → alarm trips in ~2 min → card appears on /monitoring

Auth edge case

  • Wizard works with JWT_SECRET unset (auth disabled mode)

- Rewrites InitPage as a 4-step wizard: account creation, AWS region/SNS
  config, IAM permission check, and infrastructure creation with skip option
- Adds Event Monitoring section to Settings AWS tab with Enabled/Disabled
  badge, Create Infrastructure and Teardown buttons, and live queue/rule count
- Adds Send test event button (admin only) to Monitoring page header; sends
  a synthetic Lambda alarm to SQS to verify the end-to-end pipeline
- Adds DELETE /api/init/infra (teardown) and POST /api/init/test-event
  endpoints to the init router
- Updates docs/ui.md to document the wizard, event monitoring section,
  and test event button
Automatically creates opendevops-lambda-errors-aggregate alarm (no
dimensions, >= 1 error/min) when infrastructure is provisioned. This
alarm triggers the existing EventBridge alarm-state rule on any Lambda
error in the account, making the event-driven pipeline work without
any per-function CloudWatch alarm setup.

Teardown removes the alarm alongside the SQS queue and EventBridge rules.
- Rewrites docs/monitoring.md to cover both detection modes with
  comparison table, full flows, latency explanation, and SQS vs SNS
  disambiguation
- Adds --lambda-only flag to test_pipeline.py for real alarm path testing
- Removes bash test_pipeline.sh (replaced by Python script)
- Suppresses LiteLLM 'Provider List' info spam in app startup
@AhmadHammad21
AhmadHammad21 merged commit f3a5772 into main May 12, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant