feat(results): separate unsupported vs failed, add run deltas, and surface failure tags

[wolfiesch]

Motivation

• Make triage faster by visually distinguishing explicit unsupported cases from functional failed test-cells in the HTML dashboard and surface concise failure tags for quick categorization.
• Provide run-to-run context by adding a compact “Delta Since Last Run” summary derived from results/*/history.jsonl so regressions/improvements are immediately visible in report/dashboard layers.

Description

• HTML dashboard: added detection for unsupported test-cases, new CSS class and symbol for unsupported cells, and adjusted per-test rendering to show unsupported vs fail visually. (src/excelbench/results/html_dashboard.py)
• Dashboard/report: wired history paths through render_dashboard() and added a Delta Since Last Run section that summarizes fidelity and perf changes computed from the two most recent history.jsonl entries. (src/excelbench/results/dashboard.py)
• Failure explainer: extended FailureExplanation with a concise tag field and propagated tags across classifier branches (unsupported/style/formula/etc.) for faster triage. (src/excelbench/results/failure_explainer.py)
• Resilience: added fallbacks so interactive Plotly scatter/radar code does not crash when plotly (or interactive scatter) is unavailable, rendering a readable placeholder instead. (src/excelbench/results/html_dashboard.py)
• Tests and docs: added/updated unit tests to cover new delta sections and failure tags, and refreshed docs with guidance about interpreting failed vs unsupported and the new delta summary. (tests in tests/test_results_html_dashboard.py, tests/test_dashboard.py, tests/test_failure_explainer.py; docs under docs-site/benchmark/ and docs-site/cli/report.mdx).

Testing

• Ran unit tests: uv run pytest -o addopts='' tests/test_results_html_dashboard.py tests/test_dashboard.py tests/test_failure_explainer.py and all tests passed (9 passed).
• Lint & type-check: uv run ruff check ... reported no issues and uv run mypy src/excelbench/results/dashboard.py src/excelbench/results/failure_explainer.py completed successfully.
• Tried to capture Playwright screenshots for the UI changes, but browser binaries were not installed in this environment and an npx playwright install step would be required to produce screenshots.

---

Codex Task

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dee42a4455

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Copilot]

Pull request overview

This PR improves results triage and run-to-run visibility by (1) visually distinguishing explicitly unsupported cases from functional failures in the HTML dashboard, (2) adding a compact “Delta Since Last Run” summary derived from history.jsonl, and (3) surfacing concise failure tags from the failure explainer.

Changes:

• HTML dashboard: add unsupported-case rendering and a “Delta Since Last Run” section, plus Plotly/interactive-scatter fallbacks.
• Markdown dashboard: thread history paths through render_dashboard() and emit a delta summary for fidelity + perf.
• Failure explainer/tests/docs: add a tag field to explanations, extend unit tests, and update docs to explain unsupported vs failed + deltas.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/excelbench/results/html_dashboard.py	Adds unsupported detection, delta section, and graceful fallbacks when Plotly/interactive scatter aren’t available.
src/excelbench/results/dashboard.py	Adds delta summary rendering and history parsing for the combined markdown dashboard.
src/excelbench/results/failure_explainer.py	Extends FailureExplanation with a tag field and propagates tags across classification paths.
tests/test_results_html_dashboard.py	Adds coverage for delta output and unsupported rendering in the HTML dashboard.
tests/test_dashboard.py	Adds coverage for the “Delta Since Last Run” section in the markdown dashboard.
tests/test_failure_explainer.py	Asserts newly introduced failure tags for style/formula classifications.
docs-site/cli/report.mdx	Documents the new “Delta Since Last Run” section emitted by report/dashboard layers.
docs-site/benchmark/scoring.mdx	Documents failed vs unsupported semantics and mentions the delta summary.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.