Skip to content

Bug: Monitoring page Services section shows "unknown" instead of the real service name #34

Description

@AhmadHammad21

What the user sees

The Services panel on /monitoring shows a tile labelled unknown instead of the actual service name (e.g., Lambda/my-function, ECS/my-cluster).


Root cause

Service names displayed on the monitoring page come from update_service(service, ...) in monitor_store.py. The service string is resolved in two places after an investigation completes:

Event consumer (event_consumer._deliver)

services_affected = result.get("services_affected", [])
fallback = services_affected[0] if services_affected else event.get("source", "unknown")
service = result.get("service_name", result.get("service", fallback))

Resolution order:

  1. result["service_name"] — not a field in submit_investigation; never set by the agent
  2. result["service"] — not a field in submit_investigation; never set by the agent
  3. result["services_affected"][0] — depends on the agent filling this correctly
  4. event.get("source", "unknown") — falls back to the raw EventBridge source string ("aws.cloudwatch", "aws.lambda")

Poller (poller._persist_and_notify)

services_affected = result.get("services_affected", [])
service = services_affected[0] if services_affected else "unknown"

The poller hardcodes "unknown" when services_affected is empty — no fallback to anything context-aware.

Why services_affected is often empty

submit_investigation defines services_affected: list[str] but the docstring gives no format guidance. The agent sometimes submits an empty list (particularly when the recursion limit is hit and the exception path fires), or submits generic strings like ["Lambda"] instead of ["Lambda/my-function"].

When the investigation fails and hits the exception path in _run_investigation, investigation_result is set to:

{
    "_status": "failed",
    "root_cause_summary": f"Investigation failed: {e}",
    "confidence": "LOW",
    "mitigation_steps": [...],
}

There is no services_affected key at all → services_affected = []service = "unknown".


Affected paths

Path Fallback when services_affected empty Result
Event consumer (success) event.get("source") "aws.cloudwatch" — not great but not "unknown"
Event consumer (failed investigation) event.get("source") "aws.cloudwatch"
Poller (success) hardcoded "unknown" "unknown"
Poller (failed investigation) hardcoded "unknown" "unknown"

Proposed fixes

Fix 1 — Better fallback in the poller (minimal change)

Extract the service name from the trigger key or prompt instead of hardcoding "unknown":

# In _check_alarms
service_hint = name   # alarm name, e.g. "HighErrorRate-payment-processor"

# In _check_lambda_errors  
service_hint = f"Lambda/{name}"   # e.g. "Lambda/payment-processor"

# Pass service_hint into _persist_and_notify and use it as fallback:
service = services_affected[0] if services_affected else service_hint

Fix 2 — Better fallback in the event consumer for failed investigations

Add a service_hint derived from the event detail before the investigation runs, and pass it into _deliver as an explicit fallback:

# Before running the investigation, extract the best available service label:
alarm_name = detail.get("alarmName", "")
fn_name = detail.get("functionName", "")
cluster = detail.get("clusterArn", "").split("/")[-1]
service_hint = alarm_name or fn_name or cluster or event.get("source", "unknown")

Fix 3 — Add service_name: str to submit_investigation (schema change)

Add a required service_name field to submit_investigation so the agent is forced to provide one, and use it directly in _deliver/_persist_and_notify:

def submit_investigation(
    service_name: str,   # primary affected service, e.g. "Lambda/payment-processor"
    ...

This is a breaking change to the tool schema — all three invocation paths (chat, event consumer, poller) would benefit, and it removes ambiguity for the agent.

Fix 4 — Improve submit_investigation docstring guidance

At minimum, add explicit format guidance so the agent populates services_affected with specific resource names:

services_affected: list of affected resource names in "ServiceType/ResourceName" format,
e.g. ["Lambda/payment-processor", "SQS/payment-queue"]. Never leave empty.

Relevant files

File Location of bug
src/agent/poller.py _persist_and_notify — hardcoded "unknown" fallback
src/agent/event_consumer.py _deliver — first two resolution steps check non-existent keys
src/tools/final_answer.py submit_investigation — no format guidance, no required service name field
src/agent/monitor_store.py update_service — receives whatever string is passed in

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions