Extend Agent Trace Workbench

.github/workflows/ci.yml

@@ -21,9 +21,9 @@ jobs:
         python-version: ["3.11", "3.12", "3.13"]
     steps:
       - name: Check out source
-        uses: actions/checkout@v4
+        uses: actions/checkout@v7
       - name: Set up Python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v7
         with:
           python-version: ${{ matrix.python-version }}
           cache: pip

CHANGELOG.md

@@ -4,6 +4,29 @@ All notable changes to Agent Trace Workbench appear in this file.
 
 The version format follows a release cycle. A release adds one coherent capability to the workbench.
 
+## 1.8.0 - 2026-08-04
+
+### Added
+
+- An agent comparison overlay on the dashboard failure trend.
+- A second failure line that draws one agent beside the primary series.
+- A compare select on the trend filter that lists the recorded agents except the primary one.
+- A legend that shows the failure rate of each drawn series.
+- `GET /api/trend/overlay` route that returns both trend series for scripts.
+- `GET /api/trend/overlay?format=csv` route that returns the series as a CSV attachment.
+- A `series` column in the overlay CSV that marks each row as primary or compare.
+- `atw trend --compare <name>` command that prints both failure trend series.
+- `atw trend --compare <name> --format csv` command that prints the series as CSV.
+- Dashboard JSON and CSV download links that keep the compare agent and window.
+- Deterministic tests for the overlay, the API routes, the CLI options, the CSV export, and the dashboard panel.
+
+### Changed
+
+- Version numbers moved to 1.8.0.
+- The dashboard trend chart can now draw two failure lines on one time axis.
+- The architecture now includes an agent comparison overlay beside the failure trend.
+- CI now uses the latest GitHub Actions checkout and setup-python actions.
+
 ## 1.7.0 - 2026-08-04
 
 ### Added

README.md

@@ -24,6 +24,8 @@ Release 1.6 adds a trend window selector and a per-day drill-down on the dashboa
 
 Release 1.7 adds a status breakdown beside the daily failure line. Each trend day shows a stacked bar of run status counts. Read the same counts from the API or the CLI.
 
+Release 1.8 adds an agent comparison overlay to the failure trend. Choose a second agent on the dashboard and the chart draws its failure line beside the primary series. Read both series from the API, the CLI, or a CSV file.
+
 ## Value
 
 Agent debugging needs evidence at tool boundaries.
@@ -66,7 +68,7 @@ SQLite runs in WAL mode with a busy timeout. Readers keep a committed snapshot.
 
 - `models.py` defines the portable trace contract.
 - `handlers.py` loads local handler config and applies side-effect guards.
-- `storage.py` owns the SQLite schema, WAL coordination, idempotent ingestion, and local annotations. It also computes the review list, applies bulk labels, builds the library report, computes the daily failure trend and the status breakdown, lists the runs for one day, and enforces the retention cutoff for cleanup. A cleanup log records each scheduled sweep.
+- `storage.py` owns the SQLite schema, WAL coordination, idempotent ingestion, and local annotations. It also computes the review list, applies bulk labels, builds the library report, computes the daily failure trend, the status breakdown, and the agent comparison overlay, lists the runs for one day, and enforces the retention cutoff for cleanup. A cleanup log records each scheduled sweep.
 - `ingestion.py` watches JSON files and returns stable schema error reports.
 - `otlp.py` converts the OTLP JSON encoding to and from the trace contract.
 - `replay.py` runs guarded local handlers and records mismatches.
@@ -468,6 +470,75 @@ python -m agent_trace_workbench.cli trend --statuses --agent catalog-assistant -
 
 The panel JSON and CSV links keep the active agent and window.
 
+## Agent comparison overlay
+
+Compare one agent's failure line with another on the dashboard.
+
+Choose a second agent in the compare select above the chart. The chart draws a dashed line for that agent beside the primary line. The legend shows the failure rate of each series.
+
+The compare select lists every recorded agent except the primary one. It appears only when the library records two or more agents.
+
+Read both series over the API.
+
+```powershell
+curl.exe "http://127.0.0.1:8000/api/trend/overlay?agent=catalog-assistant&compare=support-assistant"
+```
+
+The response returns the primary series and the compare series on one window.
+
+```json
+{
+  "days": 14,
+  "primary_agent": "catalog-assistant",
+  "compare_agent": "support-assistant",
+  "primary": [
+    {
+      "day": "2026-07-31",
+      "runs": 2,
+      "failures": 1,
+      "failure_rate": 0.5
+    }
+  ],
+  "compare": [
+    {
+      "day": "2026-07-31",
+      "runs": 1,
+      "failures": 0,
+      "failure_rate": 0.0
+    }
+  ]
+}
+```
+
+Omit `agent` to compare the whole library with one agent.
+
+```powershell
+curl.exe "http://127.0.0.1:8000/api/trend/overlay?compare=support-assistant"
+```
+
+Download the overlay as a CSV document.
+
+```powershell
+curl.exe -o overlay.csv "http://127.0.0.1:8000/api/trend/overlay?compare=support-assistant&format=csv"
+```
+
+The file lists one row per day per series. A `series` column marks each row as primary or compare.
+
+```text
+day,series,agent_name,runs,failures,failure_rate
+2026-07-31,primary,catalog-assistant,2,1,0.5
+2026-07-31,compare,support-assistant,1,0,0.0
+```
+
+Use the CLI for scripts.
+
+```powershell
+python -m agent_trace_workbench.cli trend --agent catalog-assistant --compare support-assistant
+python -m agent_trace_workbench.cli trend --compare support-assistant --format csv
+```
+
+The dashboard panel links to both downloads. The links keep the active agents and window. The compare agent must differ from the primary agent.
+
 ## Saved comparisons
 
 Save a comparison for later review.
@@ -1263,7 +1334,7 @@ curl.exe -X POST http://127.0.0.1:8000/api/traces `
 
 ## Test status
 
-The test suite covers the core flows. It covers storage, ingestion, replay, comparison, search, annotations, bulk labels, export, review, reports, retention cleanup, and scheduled cleanup. It covers the CLI, the API, collector export, the server scheduler, and the dashboard failure trend, including the agent filter, the window selector, the day drill-down, the status breakdown, and the CSV exports.
+The test suite covers the core flows. It covers storage, ingestion, replay, comparison, search, annotations, bulk labels, export, review, reports, retention cleanup, and scheduled cleanup. It covers the CLI, the API, collector export, the server scheduler, and the dashboard failure trend, including the agent filter, the window selector, the day drill-down, the status breakdown, the agent comparison overlay, and the CSV exports.
 
 Run the checks with these commands.
 
@@ -1274,7 +1345,7 @@ python scripts/check_requirements.py
 python -m compileall agent_trace_workbench tests
 ```
 
-Current verification passes 308 tests, Ruff lint, dependency checks, and Python compilation. CI installs from `requirements-lock.txt` and runs these checks on Python 3.11, 3.12, and 3.13 for every push and pull request.
+Current verification passes 329 tests, Ruff lint, dependency checks, and Python compilation. CI installs from `requirements-lock.txt` and runs these checks on Python 3.11, 3.12, and 3.13 for every push and pull request.
 
 ## Limitations
 
@@ -1324,6 +1395,16 @@ The trend CSV repeats the active agent in every row. The all-agents view leaves
 
 The trend export lists one row per day. It does not add a window total row.
 
+The compare overlay matches each agent name exactly. An unknown name draws a flat line at zero.
+
+The compare overlay shares the primary trend window. It does not add a third series.
+
+The overlay CSV repeats the agent name in every row. The all-agents view leaves the primary cell empty.
+
+The overlay CSV lists both series in one file. Plot tools filter rows by the series column.
+
+The day drill-down stays bound to the primary series. It does not drill into the compare line.
+
 The day drill-down groups runs by the UTC calendar day they started. It ignores a day outside the active trend window.
 
 The day CSV repeats the active agent in every row. The all-agents view leaves that cell empty.
@@ -1382,13 +1463,14 @@ The span exporter sends each workbench span as it ends. It does not batch spans.
 - Release 1.5 complete: add a CSV export for the failure trend and an agent-level trend filter on the dashboard.
 - Release 1.6 complete: add a trend window selector and a per-day drill-down on the dashboard chart.
 - Release 1.7 complete: add a status breakdown beside the daily failure line on the dashboard.
-- Release 1.8: add an agent comparison overlay to the failure trend.
+- Release 1.8 complete: add an agent comparison overlay to the failure trend.
+- Release 1.9: add a run-level error timeline to the run detail page.
 
 ## Repository map
 
 `fixtures/` contains meaningful baseline, candidate, and second-agent traces. It also contains a handler config and demo scripts.
 
-`tests/` contains deterministic tests for the core. It covers coordination, guards, search, annotations, OTLP, export, review, reports, retention cleanup, scheduled cleanup, the server scheduler, and the failure trend, including the agent filter, the window selector, the day drill-down, the status breakdown, and the CSV exports.
+`tests/` contains deterministic tests for the core. It covers coordination, guards, search, annotations, OTLP, export, review, reports, retention cleanup, scheduled cleanup, the server scheduler, and the failure trend, including the agent filter, the window selector, the day drill-down, the status breakdown, the agent comparison overlay, and the CSV exports.
 
 `static/` and `templates/` contain the presentation layer.
 

agent_trace_workbench/__init__.py

@@ -1,3 +1,3 @@
 """Agent Trace Workbench package."""
 
-__version__ = "1.7.0"
+__version__ = "1.8.0"

agent_trace_workbench/cli.py

@@ -18,6 +18,7 @@
     report_to_csv,
     run_tools_to_csv,
     status_trend_to_csv,
+    trend_overlay_to_csv,
     trend_to_csv,
 )
 from .handlers import ReplayPolicy, load_handler_config
@@ -155,6 +156,11 @@ def build_parser() -> argparse.ArgumentParser:
         default=None,
         help="Restrict the trend to one agent name",
     )
+    trend.add_argument(
+        "--compare",
+        default=None,
+        help="Draw a second failure line for one agent comparison",
+    )
     trend.add_argument(
         "--format",
         choices=["json", "csv"],
@@ -424,6 +430,16 @@ def main() -> None:
                 )
         elif args.days < 1 or args.days > 90:
             raise SystemExit("--days must be between 1 and 90")
+        elif args.compare:
+            if args.compare == (args.agent or ""):
+                raise SystemExit("--compare must differ from --agent")
+            overlay = store.failure_trend_overlay(
+                args.days, agent_name=args.agent, compare_agent=args.compare
+            )
+            if args.format == "csv":
+                print(trend_overlay_to_csv(overlay), end="")
+            else:
+                print(json.dumps(overlay, indent=2))
         elif args.statuses:
             buckets = store.status_trend(args.days, agent_name=args.agent)
             if args.format == "csv":

agent_trace_workbench/export.py

@@ -69,6 +69,15 @@
 
 _TREND_HEADERS = ["day", "agent_name", "runs", "failures", "failure_rate"]
 
+_TREND_OVERLAY_HEADERS = [
+    "day",
+    "series",
+    "agent_name",
+    "runs",
+    "failures",
+    "failure_rate",
+]
+
 _STATUS_TREND_HEADERS = ["day", "agent_name", "status", "runs"]
 
 _DAY_RUNS_HEADERS = [
@@ -266,6 +275,41 @@ def trend_to_csv(trend: list[dict[str, Any]], agent_name: str = "") -> str:
         return _to_csv(_TREND_HEADERS, rows)
 
 
+def trend_overlay_to_csv(overlay: dict[str, Any]) -> str:
+    """Render an agent comparison overlay as a CSV document.
+
+    The document lists one row per day per series. A series column marks
+    each row as the primary line or the compare line, so a spreadsheet
+    can plot both series from one file. Empty days stay in both series,
+    because the overlay shares one time axis.
+    """
+
+    primary = overlay["primary"]
+    compare = overlay["compare"]
+    attributes = {
+        "trend.days": len(primary),
+        "trend.compare": overlay.get("compare_agent", ""),
+    }
+    with traced_operation("export.trend_overlay_csv", attributes):
+        rows: list[dict[str, Any]] = []
+        for series, buckets, agent_name in (
+            ("primary", primary, overlay.get("primary_agent", "")),
+            ("compare", compare, overlay.get("compare_agent", "")),
+        ):
+            for bucket in buckets:
+                rows.append(
+                    {
+                        "day": bucket["day"],
+                        "series": series,
+                        "agent_name": agent_name,
+                        "runs": bucket["runs"],
+                        "failures": bucket["failures"],
+                        "failure_rate": _number(bucket.get("failure_rate")),
+                    }
+                )
+        return _to_csv(_TREND_OVERLAY_HEADERS, rows)
+
+
 def status_trend_to_csv(
     trend: list[dict[str, Any]], agent_name: str = ""
 ) -> str: