Problem
Users sometimes need to compare performance across hardware or software environments.
Currently, selecting All cohorts displays an independent set of charts for every cohort. This creates significant visual clutter and makes direct comparison difficult because:
- Cohorts are shown in separate charts.
- Axis ranges may differ.
- Colors do not consistently represent cohort identity.
- Users cannot deliberately choose which cohorts to compare.
- Incompatible benchmark environments may appear comparable.
Proposed solution
Add an explicit Compare cohorts mode.
Users should be able to select two cohorts, with an optional maximum of three. Each metric should then be displayed in a shared chart with one series per selected cohort.
Comparison should use the canonical cohort identity and readable labels introduced by the cohort-selector ticket.
Entry points
Compare mode should be accessible from:
- The main dashboard controls.
- The all-cohort overview.
- Optionally, the detailed single-cohort view.
When entering from a selected cohort, that cohort should be preselected.
Cohort selection
- Require at least two cohorts to render a comparison.
- Allow no more than three cohorts.
- Constrain available cohorts using the active Model, GPU, Source, date, and advanced filters.
- Show readable configuration summaries as primary labels.
- Keep hardware, software, and recipe IDs available as secondary details.
- Explain why a cohort is unavailable or incompatible when possible.
Compatibility rules
The implementation must define when two cohorts are suitable for direct comparison.
At minimum, compatibility should consider:
- Model or benchmark identity
- Benchmark schema/version
- Metric definitions and units
- Benchmark recipe
- Input shape or workload configuration
- Any schedule or execution property that affects comparability
Hardware and software profiles may differ because comparing those profiles is a primary use case.
Legacy and v2 data should not be combined without an explicit warning. If their metric definitions are incompatible, the comparison should be rejected.
Compatibility rules should be centralized rather than implemented separately in individual chart components.
Chart behavior
- Render one shared chart per metric.
- Render one series per selected cohort.
- Assign each cohort a stable color.
- Use the same cohort color across all metrics.
- Display a persistent legend with readable cohort labels.
- Use a common axis for every series within a metric.
- Include exact hardware, software, recipe, commit, source, and timestamp details in tooltips.
- Represent missing observations accurately.
- Do not interpolate across missing records in a misleading way.
- Clearly indicate metrics that are unavailable for one or more selected cohorts.
URL state
Encode Compare mode and the selected cohort keys in the URL so comparisons can be bookmarked and shared.
The dashboard should handle:
- Missing cohort keys
- Duplicate cohort keys
- Stale or unknown keys
- More than the maximum number of cohorts
- Cohorts excluded by another active filter
- A comparison URL containing fewer than two valid cohorts
Dependency
Blocked by the ticket that introduces canonical cohort identity, readable cohort labels, and the Benchmark cohort selector.
Preferably follows the all-cohort overview ticket so Compare mode can reuse its cohort-selection interaction.
Shared multi-series chart infrastructure may be developed earlier, but final integration should use the canonical cohort and overview contracts.
Wait till #1634 and #1635 lands
Acceptance criteria
Problem
Users sometimes need to compare performance across hardware or software environments.
Currently, selecting
All cohortsdisplays an independent set of charts for every cohort. This creates significant visual clutter and makes direct comparison difficult because:Proposed solution
Add an explicit Compare cohorts mode.
Users should be able to select two cohorts, with an optional maximum of three. Each metric should then be displayed in a shared chart with one series per selected cohort.
Comparison should use the canonical cohort identity and readable labels introduced by the cohort-selector ticket.
Entry points
Compare mode should be accessible from:
When entering from a selected cohort, that cohort should be preselected.
Cohort selection
Compatibility rules
The implementation must define when two cohorts are suitable for direct comparison.
At minimum, compatibility should consider:
Hardware and software profiles may differ because comparing those profiles is a primary use case.
Legacy and v2 data should not be combined without an explicit warning. If their metric definitions are incompatible, the comparison should be rejected.
Compatibility rules should be centralized rather than implemented separately in individual chart components.
Chart behavior
URL state
Encode Compare mode and the selected cohort keys in the URL so comparisons can be bookmarked and shared.
The dashboard should handle:
Dependency
Blocked by the ticket that introduces canonical cohort identity, readable cohort labels, and the Benchmark cohort selector.
Preferably follows the all-cohort overview ticket so Compare mode can reuse its cohort-selection interaction.
Shared multi-series chart infrastructure may be developed earlier, but final integration should use the canonical cohort and overview contracts.
Wait till #1634 and #1635 lands
Acceptance criteria