Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,26 @@ All notable changes to Agent Trace Workbench appear in this file.

The version format follows a release cycle. A release adds one coherent capability to the workbench.

## 1.7.0 - 2026-08-04

### Added

- A status breakdown beside the daily failure line on the dashboard.
- A stacked bar per trend day that shows the run status counts.
- A status legend with window totals for each recorded run status.
- `GET /api/trend/statuses` route that returns the daily status counts.
- `GET /api/trend/statuses?format=csv` route that returns the counts as a CSV attachment.
- `atw trend --statuses` command that prints the daily status counts.
- `atw trend --statuses --format csv` command that prints the counts as CSV.
- The dashboard status panel keeps the active agent and window in its export links.
- Deterministic tests for the status trend, the API routes, the CLI options, the CSV export, and the dashboard panel.

### Changed

- Version numbers moved to 1.7.0.
- The dashboard trend section now shows the status composition of each day.
- The architecture now includes a status breakdown beside the failure trend.

## 1.6.0 - 2026-08-04

### Added
Expand Down
79 changes: 74 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ Release 1.5 adds a CSV export for the failure trend and an agent-level trend fil

Release 1.6 adds a trend window selector and a per-day drill-down on the dashboard. Choose 7, 14, 30, or 90 day views. Click a day to see the runs that started that day.

Release 1.7 adds a status breakdown beside the daily failure line. Each trend day shows a stacked bar of run status counts. Read the same counts from the API or the CLI.

## Value

Agent debugging needs evidence at tool boundaries.
Expand Down Expand Up @@ -64,7 +66,7 @@ SQLite runs in WAL mode with a busy timeout. Readers keep a committed snapshot.

- `models.py` defines the portable trace contract.
- `handlers.py` loads local handler config and applies side-effect guards.
- `storage.py` owns the SQLite schema, WAL coordination, idempotent ingestion, and local annotations. It also computes the review list, applies bulk labels, builds the library report, computes the daily failure trend, lists the runs for one day, and enforces the retention cutoff for cleanup. A cleanup log records each scheduled sweep.
- `storage.py` owns the SQLite schema, WAL coordination, idempotent ingestion, and local annotations. It also computes the review list, applies bulk labels, builds the library report, computes the daily failure trend and the status breakdown, lists the runs for one day, and enforces the retention cutoff for cleanup. A cleanup log records each scheduled sweep.
- `ingestion.py` watches JSON files and returns stable schema error reports.
- `otlp.py` converts the OTLP JSON encoding to and from the trace contract.
- `replay.py` runs guarded local handlers and records mismatches.
Expand Down Expand Up @@ -408,6 +410,64 @@ python -m agent_trace_workbench.cli trend --day 2026-07-31 --format csv

The day panel offers the same CSV download. A day outside the active window is ignored. The dashboard draws no panel for it.

## Status breakdown

The dashboard shows which run statuses shape each trend day.

Each day draws one stacked bar beside the failure line. The bar shows how many runs ended ok and how many failed. A legend lists the window totals for each status.

Read the breakdown over the API.

```powershell
curl.exe "http://127.0.0.1:8000/api/trend/statuses?days=14"
```

The response lists one bucket per day. Each bucket maps status names to run counts.

```json
[
{
"day": "2026-07-31",
"runs": 2,
"statuses": {
"ok": 1,
"error": 1
}
}
]
```

Filter the breakdown to one agent.

```powershell
curl.exe "http://127.0.0.1:8000/api/trend/statuses?agent=catalog-assistant"
```

The dashboard panel keeps the active agent and window. It follows the same trend filter.

Download the breakdown as CSV.

```powershell
curl.exe -o status-trend.csv "http://127.0.0.1:8000/api/trend/statuses?format=csv"
```

The file lists one row per status on a day. Empty days produce no rows.

```text
day,agent_name,status,runs
2026-07-31,,ok,1
2026-07-31,,error,1
```

Use the CLI for scripts.

```powershell
python -m agent_trace_workbench.cli trend --statuses
python -m agent_trace_workbench.cli trend --statuses --agent catalog-assistant --format csv
```

The panel JSON and CSV links keep the active agent and window.

## Saved comparisons

Save a comparison for later review.
Expand Down Expand Up @@ -1203,7 +1263,7 @@ curl.exe -X POST http://127.0.0.1:8000/api/traces `

## Test status

The test suite covers the core flows. It covers storage, ingestion, replay, comparison, search, annotations, bulk labels, export, review, reports, retention cleanup, and scheduled cleanup. It covers the CLI, the API, collector export, the server scheduler, and the dashboard failure trend, including the agent filter, the window selector, the day drill-down, and the CSV exports.
The test suite covers the core flows. It covers storage, ingestion, replay, comparison, search, annotations, bulk labels, export, review, reports, retention cleanup, and scheduled cleanup. It covers the CLI, the API, collector export, the server scheduler, and the dashboard failure trend, including the agent filter, the window selector, the day drill-down, the status breakdown, and the CSV exports.

Run the checks with these commands.

Expand All @@ -1214,7 +1274,7 @@ python scripts/check_requirements.py
python -m compileall agent_trace_workbench tests
```

Current verification passes 289 tests, Ruff lint, dependency checks, and Python compilation. CI installs from `requirements-lock.txt` and runs these checks on Python 3.11, 3.12, and 3.13 for every push and pull request.
Current verification passes 308 tests, Ruff lint, dependency checks, and Python compilation. CI installs from `requirements-lock.txt` and runs these checks on Python 3.11, 3.12, and 3.13 for every push and pull request.

## Limitations

Expand Down Expand Up @@ -1250,6 +1310,14 @@ The failure trend groups by the calendar day a run started. It uses the UTC day.

The trend counts a run by its recorded status. A run with any error span counts as a failure.

The status breakdown groups by the same UTC calendar day as the trend.

The status breakdown counts runs by their recorded status. A run with an error span counts as an error run.

The status breakdown draws one stacked bar per day. The bar height scales to the busiest day in the window.

The status CSV lists one row per status present on a day. Empty days produce no rows.

The trend agent filter matches the exact recorded agent name.

The trend CSV repeats the active agent in every row. The all-agents view leaves that cell empty.
Expand Down Expand Up @@ -1313,13 +1381,14 @@ The span exporter sends each workbench span as it ends. It does not batch spans.
- Release 1.4 complete: add a server-side sweep scheduler and a failure trend line on the dashboard.
- Release 1.5 complete: add a CSV export for the failure trend and an agent-level trend filter on the dashboard.
- Release 1.6 complete: add a trend window selector and a per-day drill-down on the dashboard chart.
- Release 1.7: add a status breakdown beside the daily failure line on the dashboard.
- Release 1.7 complete: add a status breakdown beside the daily failure line on the dashboard.
- Release 1.8: add an agent comparison overlay to the failure trend.

## Repository map

`fixtures/` contains meaningful baseline, candidate, and second-agent traces. It also contains a handler config and demo scripts.

`tests/` contains deterministic tests for the core. It covers coordination, guards, search, annotations, OTLP, export, review, reports, retention cleanup, scheduled cleanup, the server scheduler, and the failure trend, including the agent filter, the window selector, the day drill-down, and the CSV exports.
`tests/` contains deterministic tests for the core. It covers coordination, guards, search, annotations, OTLP, export, review, reports, retention cleanup, scheduled cleanup, the server scheduler, and the failure trend, including the agent filter, the window selector, the day drill-down, the status breakdown, and the CSV exports.

`static/` and `templates/` contain the presentation layer.

Expand Down
2 changes: 1 addition & 1 deletion agent_trace_workbench/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""Agent Trace Workbench package."""

__version__ = "1.6.0"
__version__ = "1.7.0"
12 changes: 12 additions & 0 deletions agent_trace_workbench/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
day_runs_to_csv,
report_to_csv,
run_tools_to_csv,
status_trend_to_csv,
trend_to_csv,
)
from .handlers import ReplayPolicy, load_handler_config
Expand Down Expand Up @@ -170,6 +171,11 @@ def build_parser() -> argparse.ArgumentParser:
default=None,
help="List the runs that started on one YYYY-MM-DD day",
)
trend.add_argument(
"--statuses",
action="store_true",
help="Show the per-day run status breakdown",
)

annotate = subparsers.add_parser(
"annotate", help="Label a run and add local review notes"
Expand Down Expand Up @@ -418,6 +424,12 @@ def main() -> None:
)
elif args.days < 1 or args.days > 90:
raise SystemExit("--days must be between 1 and 90")
elif args.statuses:
buckets = store.status_trend(args.days, agent_name=args.agent)
if args.format == "csv":
print(status_trend_to_csv(buckets, agent_name=args.agent or ""), end="")
else:
print(json.dumps(buckets, indent=2))
else:
trend = store.failure_trend(args.days, agent_name=args.agent)
if args.format == "csv":
Expand Down
30 changes: 30 additions & 0 deletions agent_trace_workbench/export.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@

_TREND_HEADERS = ["day", "agent_name", "runs", "failures", "failure_rate"]

_STATUS_TREND_HEADERS = ["day", "agent_name", "status", "runs"]

_DAY_RUNS_HEADERS = [
"day",
"run_id",
Expand Down Expand Up @@ -264,6 +266,34 @@ def trend_to_csv(trend: list[dict[str, Any]], agent_name: str = "") -> str:
return _to_csv(_TREND_HEADERS, rows)


def status_trend_to_csv(
trend: list[dict[str, Any]], agent_name: str = ""
) -> str:
"""Render a daily run status breakdown as a CSV document.

The document lists one row per status present on a day. The
agent_name cell repeats the active filter, so a filtered file stays
self-describing. Empty days produce no rows, because they carry no
status counts.
"""

with traced_operation(
"export.status_trend_csv", {"trend.days": len(trend), "trend.agent": agent_name}
):
rows: list[dict[str, Any]] = []
for bucket in trend:
for status, count in sorted(bucket["statuses"].items()):
rows.append(
{
"day": bucket["day"],
"agent_name": agent_name,
"status": status,
"runs": count,
}
)
return _to_csv(_STATUS_TREND_HEADERS, rows)


def day_runs_to_csv(day: str, runs: list[dict[str, Any]], agent_name: str = "") -> str:
"""Render the runs that started on one day as a CSV document.

Expand Down
Loading
Loading