Extend Agent Trace Workbench

CHANGELOG.md

@@ -4,6 +4,21 @@ All notable changes to Agent Trace Workbench appear in this file.
 
 The version format follows a release cycle. A release adds one coherent capability to the workbench.
 
+## 1.11.0 - 2026-08-04
+
+### Added
+
+- Ordered failed-span summaries for runs in the dashboard day drill-down.
+- Error summary fields in the day drill-down API and CLI JSON output.
+- Failed-span count and messages in the day CSV export.
+- Deterministic tests for storage, API, dashboard, CLI, and CSV summaries.
+
+### Changed
+
+- Version numbers moved to 1.11.0.
+- Day cards show the failure count and first failure message.
+- The day drill-down reuses the error timeline failure rule.
+
 ## 1.10.0 - 2026-08-04
 
 ### Added

README.md

@@ -30,6 +30,8 @@ Release 1.9 adds a run-level error timeline to the run detail page. It marks eac
 
 Release 1.10 adds a span detail panel to the error timeline. Click a marker or an event row to open the full span record. Read the same record from the API or the CLI.
 
+Release 1.11 adds ordered failed-span summaries to each day drill-down card and day export.
+
 ## Value
 
 Agent debugging needs evidence at tool boundaries.
@@ -72,7 +74,7 @@ SQLite runs in WAL mode with a busy timeout. Readers keep a committed snapshot.
 
 - `models.py` defines the portable trace contract.
 - `handlers.py` loads local handler config and applies side-effect guards.
-- `storage.py` owns the SQLite schema, WAL coordination, idempotent ingestion, and local annotations. It computes the review list, bulk labels, and the library report. It computes the failure trend, status breakdown, agent overlay, and run error timeline. It lists the runs for one day and enforces the retention cutoff. It returns the full record for one span. A cleanup log records each scheduled sweep.
+- `storage.py` owns the SQLite schema, WAL coordination, idempotent ingestion, and local annotations. It computes the review list, bulk labels, and the library report. It computes the failure trend, status breakdown, agent overlay, and run error timeline. It lists one day with ordered error summaries and enforces the retention cutoff. It returns the full record for one span. A cleanup log records each scheduled sweep.
 - `ingestion.py` watches JSON files and returns stable schema error reports.
 - `otlp.py` converts the OTLP JSON encoding to and from the trace contract.
 - `replay.py` runs guarded local handlers and records mismatches.
@@ -489,6 +491,8 @@ curl.exe "http://127.0.0.1:8000/api/trend/2026-07-31"
 
 The response lists the runs for that day.
 
+Each run includes an ordered `error_summary` list. The list contains failed span IDs, names, kinds, statuses, and messages.
+
 ```json
 {
   "day": "2026-07-31",
@@ -504,7 +508,25 @@ The response lists the runs for that day.
       "run_id": "run-candidate-001",
       "agent_name": "catalog-assistant",
       "status": "error",
-      "tool_count": 3
+      "tool_count": 3,
+      "error_summary": [
+        {
+          "span_id": "span-agent-101",
+          "sequence": 0,
+          "name": "agent.run",
+          "kind": "agent",
+          "status": "error",
+          "message": "agent.run ended with status error"
+        },
+        {
+          "span_id": "span-tool-103",
+          "sequence": 3,
+          "name": "reserve_inventory",
+          "kind": "tool",
+          "status": "error",
+          "message": "reservation window expired"
+        }
+      ]
     }
   ]
 }
@@ -524,11 +546,11 @@ Download the day runs as CSV.
 curl.exe -o runs-2026-07-31.csv "http://127.0.0.1:8000/api/trend/2026-07-31?format=csv"
 ```
 
-The file lists one row per run. The day cell repeats the drill target.
+The file lists one row per run. It adds the failed-span count and messages.
 
 ```text
-day,run_id,agent_name,status,tool_count,duration_ms,source_dir,label
-2026-07-31,run-baseline-001,catalog-assistant,ok,2,220.0,fixtures,
+day,run_id,agent_name,status,error_count,error_summary,tool_count,duration_ms,source_dir,label
+2026-07-31,run-baseline-001,catalog-assistant,ok,0,,2,220.0,fixtures,
 ```
 
 Use the CLI for scripts.
@@ -540,6 +562,8 @@ python -m agent_trace_workbench.cli trend --day 2026-07-31 --format csv
 
 The day panel offers the same CSV download. A day outside the active window is ignored. The dashboard draws no panel for it.
 
+The dashboard card shows the failure count and first message. Open the run for the full error timeline.
+
 ## Status breakdown
 
 The dashboard shows which run statuses shape each trend day.
@@ -1473,7 +1497,7 @@ python scripts/check_requirements.py
 python -m compileall agent_trace_workbench tests
 ```
 
-Current verification passes 357 tests, Ruff lint, dependency checks, and Python compilation. CI installs from `requirements-lock.txt` and runs these checks on Python 3.11, 3.12, and 3.13 for every push and pull request.
+Current verification passes 360 tests, Ruff lint, dependency checks, and Python compilation. CI installs from `requirements-lock.txt` and runs these checks on Python 3.11, 3.12, and 3.13 for every push and pull request.
 
 ## Limitations
 
@@ -1553,6 +1577,10 @@ The span detail offsets match the error timeline. They count from the recorded r
 
 The span detail panel loads over the API. It needs a running server to fetch a record.
 
+Day cards show the first failure message. The API and CSV include every failed span.
+
+Error summaries use the same status and tool-outcome rule as the error timeline.
+
 The cleanup history records policy and counts. It does not store the deleted traces.
 
 The report retention line counts runs under the current policy. It uses `older_than_days` from the request or the 30-day default.
@@ -1610,13 +1638,14 @@ The span exporter sends each workbench span as it ends. It does not batch spans.
 - Release 1.8 complete: add an agent comparison overlay to the failure trend.
 - Release 1.9 complete: add a run-level error timeline to the run detail page.
 - Release 1.10 complete: add a span detail panel to the run-level error timeline.
-- Release 1.11: add a run error summary to the dashboard trend drill-down.
+- Release 1.11 complete: add ordered failed-span summaries to day cards, API output, CLI output, and CSV export.
+- Next: choose the next bounded evidence-review slice.
 
 ## Repository map
 
 `fixtures/` contains meaningful baseline, candidate, and second-agent traces. It also contains a handler config and demo scripts.
 
-`tests/` contains deterministic tests for the core. It covers coordination, guards, search, annotations, OTLP, and export. It covers review, reports, retention cleanup, scheduled cleanup, and the server scheduler. It covers the failure trend, including the agent filter, window selector, day drill-down, status breakdown, overlay, and the run error timeline. It covers the span detail panel on the error timeline.
+`tests/` contains deterministic tests for the core. It covers coordination, guards, search, annotations, OTLP, and export. It covers review, reports, retention cleanup, scheduled cleanup, and the server scheduler. It covers the failure trend, including the agent filter, window selector, day drill-down, run error summaries, status breakdown, overlay, and the run error timeline. It covers the span detail panel on the error timeline.
 
 `static/` and `templates/` contain the presentation layer.
 

agent_trace_workbench/__init__.py

@@ -1,3 +1,3 @@
 """Agent Trace Workbench package."""
 
-__version__ = "1.10.0"
+__version__ = "1.11.0"

agent_trace_workbench/export.py

@@ -85,6 +85,8 @@
     "run_id",
     "agent_name",
     "status",
+    "error_count",
+    "error_summary",
     "tool_count",
     "duration_ms",
     "source_dir",
@@ -402,6 +404,11 @@ def day_runs_to_csv(day: str, runs: list[dict[str, Any]], agent_name: str = "")
                     "run_id": run.get("run_id", ""),
                     "agent_name": run.get("agent_name", ""),
                     "status": run.get("status", ""),
+                    "error_count": len(run.get("error_summary", [])),
+                    "error_summary": "; ".join(
+                        item.get("message", "")
+                        for item in run.get("error_summary", [])
+                    ),
                     "tool_count": _number(run.get("tool_count")),
                     "duration_ms": _number(run.get("duration_ms")),
                     "source_dir": run.get("source_dir", ""),

agent_trace_workbench/storage.py

@@ -874,7 +874,8 @@ def runs_on_day(
         The drill-down view uses this method. A reviewer clicks a day on
         the trend chart and sees the runs that started that day. Pass
         agent_name to keep the day view in sync with the trend filter.
-        The day must use the YYYY-MM-DD format.
+        Each summary includes ordered failed-span details. The day must
+        use the YYYY-MM-DD format.
         """
 
         try:
@@ -892,7 +893,7 @@ def runs_on_day(
         with traced_operation("storage.runs_on_day", {"trend.day": day}):
             with self._connect() as connection:
                 rows = connection.execute(query, params).fetchall()
-                return _summarize_runs(connection, rows)
+                return _summarize_runs(connection, rows, include_error_summary=True)
 
     def get_run(
         self,
@@ -971,7 +972,7 @@ def error_timeline(self, run_id: str) -> dict[str, Any] | None:
         origin = ensure_utc(datetime.fromisoformat(run["started_at"]))
         events: list[dict[str, Any]] = []
         for row in rows:
-            if row["status"] != "error" and row["outcome"] != "failure":
+            if not _is_failed_span(row):
                 continue
             start_offset_ms = _offset_ms(
                 origin, datetime.fromisoformat(row["start_time"])
@@ -1241,6 +1242,25 @@ def _error_message(row: sqlite3.Row) -> str:
     return f"{row['name']} ended with status error"
 
 
+def _is_failed_span(row: sqlite3.Row) -> bool:
+    """Return whether a stored span belongs in a run error summary."""
+
+    return row["status"] == "error" or row["outcome"] == "failure"
+
+
+def _error_summary_item(row: sqlite3.Row) -> dict[str, Any]:
+    """Return the stable, compact failure record used by day drill-downs."""
+
+    return {
+        "span_id": row["span_id"],
+        "sequence": row["sequence_index"],
+        "name": row["name"],
+        "kind": row["kind"],
+        "status": row["status"],
+        "message": _error_message(row),
+    }
+
+
 def _retention_ids(
     connection: sqlite3.Connection,
     cutoff: datetime,
@@ -1280,6 +1300,8 @@ def _folder_name(source_dir: str) -> str:
 def _summarize_runs(
     connection: sqlite3.Connection,
     rows: list[sqlite3.Row],
+    *,
+    include_error_summary: bool = False,
 ) -> list[dict[str, Any]]:
     summaries = [_run_row(row) for row in rows]
     for summary in summaries:
@@ -1288,6 +1310,27 @@ def _summarize_runs(
             (summary["run_id"],),
         ).fetchone()
         summary["tool_count"] = tool_row["count"]
+    if include_error_summary:
+        errors_by_run: dict[str, list[dict[str, Any]]] = {
+            summary["run_id"]: [] for summary in summaries
+        }
+        if errors_by_run:
+            run_ids = list(errors_by_run)
+            placeholders = ", ".join("?" * len(run_ids))
+            error_rows = connection.execute(
+                f"""
+                SELECT run_id, span_id, sequence_index, name, kind, status, outcome, error
+                FROM spans
+                WHERE run_id IN ({placeholders})
+                  AND (status = 'error' OR outcome = 'failure')
+                ORDER BY run_id, sequence_index IS NULL, sequence_index, start_time, span_id
+                """,
+                run_ids,
+            ).fetchall()
+            for row in error_rows:
+                errors_by_run[row["run_id"]].append(_error_summary_item(row))
+        for summary in summaries:
+            summary["error_summary"] = errors_by_run[summary["run_id"]]
     return summaries
 
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "agent-trace-workbench"
-version = "1.10.0"
+version = "1.11.0"
 description = "A local workbench for recording, replaying, comparing, and inspecting agent traces."
 readme = "README.md"
 requires-python = ">=3.11"

static/styles.css

@@ -84,6 +84,10 @@ select { min-height: 44px; padding: 0 12px; font-family: inherit; }
 .run-card h3 { margin-top: 29px; color: var(--navy); font: 500 21px Georgia, serif; }
 .run-id { margin: 5px 0 20px; overflow: hidden; color: var(--muted); font: 11px ui-monospace, SFMono-Regular, Consolas, monospace; text-overflow: ellipsis; white-space: nowrap; }
 .run-card-bottom { padding-top: 12px; border-top: 1px solid var(--line); color: var(--muted); font-size: 11px; }
+.run-card-error { display: flex; flex-direction: column; gap: 4px; margin-top: 12px; padding: 10px 11px; border-left: 3px solid var(--coral); background: rgba(217,95,79,.06); color: var(--coral); font-size: 11px; line-height: 1.35; }
+.run-card-error-label { font: 800 9px ui-monospace, SFMono-Regular, Consolas, monospace; letter-spacing: .08em; text-transform: uppercase; }
+.run-card-error-message { overflow: hidden; color: var(--navy); text-overflow: ellipsis; white-space: nowrap; }
+.run-card-error-more { color: var(--muted); font: 10px ui-monospace, SFMono-Regular, Consolas, monospace; }
 .empty-state { padding: 65px 25px; border: 1px dashed #cbd3d8; text-align: center; background: rgba(255,255,255,.5); }
 .empty-mark { display: block; margin-bottom: 17px; color: var(--teal); font: 500 31px Georgia, serif; }
 .empty-state h3 { margin-bottom: 9px; color: var(--navy); font: 500 23px Georgia, serif; }

templates/dashboard.html

@@ -146,6 +146,13 @@ <h1>See what your agent did.<br><em>Replay why it did it.</em></h1>
             <h3>{{ run.agent_name }}{% if run.label %} <span class="badge badge-label">{{ run.label }}</span>{% endif %}</h3>
             <p class="run-id">{{ run.run_id }}</p>
             <div class="run-card-bottom"><span>{{ run.tool_count }} tool calls</span><span>{{ run.source_dir }}</span></div>
+            {% if run.error_summary %}
+            <div class="run-card-error">
+                <span class="run-card-error-label">{{ run.error_summary|length }} failed span{% if run.error_summary|length != 1 %}s{% endif %}</span>
+                <span class="run-card-error-message">{{ run.error_summary[0].message }}</span>
+                {% if run.error_summary|length > 1 %}<span class="run-card-error-more">+ {{ run.error_summary|length - 1 }} more</span>{% endif %}
+            </div>
+            {% endif %}
         </a>
         {% endfor %}
     </div>

tests/test_csv_export.py

@@ -471,6 +471,9 @@ def test_day_runs_csv_renders_one_row_per_run(tmp_path, baseline, candidate):
     assert {row["run_id"] for row in rows} == {baseline.run_id, candidate.run_id}
     assert {row["status"] for row in rows} == {"ok", "error"}
     assert {row["tool_count"] for row in rows} == {"2", "3"}
+    candidate_row = next(row for row in rows if row["run_id"] == candidate.run_id)
+    assert candidate_row["error_count"] == "2"
+    assert "reservation window expired" in candidate_row["error_summary"]
 
 
 def test_day_runs_csv_carries_agent_filter(tmp_path, baseline, support):

tests/test_trend.py

@@ -302,6 +302,27 @@ def test_runs_on_day_lists_runs_for_one_day(tmp_path, baseline, candidate):
     assert day_runs[0]["status"] == "ok"
 
 
+def test_runs_on_day_includes_ordered_error_summary(tmp_path, candidate):
+    store = TraceStore(tmp_path / "trend.db")
+    store.ingest(candidate, "candidate.json")
+    _set_started(store, candidate.run_id, 2)
+
+    error_summary = store.runs_on_day(_day(2))[0]["error_summary"]
+
+    assert [item["span_id"] for item in error_summary] == [
+        "span-agent-101",
+        "span-tool-103",
+    ]
+    assert [item["name"] for item in error_summary] == [
+        "agent.run",
+        "reserve_inventory",
+    ]
+    assert [item["message"] for item in error_summary] == [
+        "agent.run ended with status error",
+        "reservation window expired",
+    ]
+
+
 def test_runs_on_day_sorts_newest_first(tmp_path, baseline, candidate):
     store = TraceStore(tmp_path / "trend.db")
     store.ingest(baseline, "baseline.json")
@@ -356,6 +377,18 @@ def test_api_trend_day_returns_runs(tmp_path, baseline, candidate):
     assert body["day"] == _day(2)
     assert body["agent"] == ""
     assert [run["run_id"] for run in body["runs"]] == [baseline.run_id]
+    assert body["runs"][0]["error_summary"] == []
+
+
+def test_api_trend_day_returns_error_summary(tmp_path, candidate):
+    client = TestClient(create_app(tmp_path / "api.db"))
+    client.post("/api/traces", json=candidate.as_jsonable())
+    store = TraceStore(tmp_path / "api.db")
+    _set_started(store, candidate.run_id, 2)
+
+    body = client.get(f"/api/trend/{_day(2)}").json()
+
+    assert body["runs"][0]["error_summary"][1]["message"] == "reservation window expired"
 
 
 def test_api_trend_day_filters_by_agent(tmp_path, baseline, candidate, support):
@@ -398,7 +431,10 @@ def test_api_trend_day_csv_returns_attachment(tmp_path, baseline, candidate):
     )
     assert (
         response.text.splitlines()[0]
-        == "day,run_id,agent_name,status,tool_count,duration_ms,source_dir,label"
+        == (
+            "day,run_id,agent_name,status,error_count,error_summary,"
+            "tool_count,duration_ms,source_dir,label"
+        )
     )
 
 
@@ -442,6 +478,9 @@ def test_dashboard_day_drill_down_lists_runs(tmp_path, baseline, candidate):
     assert "DAY DRILL-DOWN" in page
     assert f"Runs on {_day(2)}" in page
     assert baseline.run_id in page
+    assert "2 failed spans" in page
+    assert "agent.run ended with status error" in page
+    assert "+ 1 more" in page
     assert f"/api/trend/{_day(2)}?" in page
 
 
@@ -497,6 +536,21 @@ def test_cli_trend_day_lists_runs(tmp_path, baseline, candidate, monkeypatch, ca
     assert [run["run_id"] for run in body["runs"]] == [baseline.run_id]
 
 
+def test_cli_trend_day_includes_error_summary(tmp_path, candidate, monkeypatch, capsys):
+    store = TraceStore(tmp_path / "cli.db")
+    store.ingest(candidate, "candidate.json")
+    _set_started(store, candidate.run_id, 2)
+
+    monkeypatch.setattr(
+        "sys.argv",
+        ["atw", "--db", str(tmp_path / "cli.db"), "trend", "--day", _day(2)],
+    )
+    main()
+
+    body = json.loads(capsys.readouterr().out)
+    assert body["runs"][0]["error_summary"][0]["name"] == "agent.run"
+
+
 def test_cli_trend_day_rejects_bad_format(tmp_path, monkeypatch, capsys):
     monkeypatch.setattr(
         "sys.argv",