Add nvbench-compare-robust --plot-output/--plot-along-output options

[oleksandr-pavlyk]

The intent is to allow producing plotting artifacts in headless environments without relying on Linux-centric "DISPLAY" environment variable and allowing user to select matplotlib's backend via environment variables.

• Added --plot-output PATH for summary plots (i.e., --plot was specified).
• Added --plot-along-output PATH_OR_TEMPLATE for --plot-along.
• Default behavior remains interactive plt.show().
• Explicit output paths save via fig.savefig(..., dpi=150) and skip plt.show().
• --plot-along-output supports {benchmark}, {device}, {axis}, {pair} templates.
• Duplicate plot-along output paths now raise a clear error instead of overwriting.

In addition, plotting data and their manipulation have been decoupled from compare_benches are now performed using PlotCollector class. All plotting-related functionality has been move to helper source file _nvbench_compare_plotting.py residing next to nvbench_compare_robust.py.

Closes #395

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a177b654-f2cc-4021-9a78-6a94342724a0

📥 Commits

Reviewing files that changed from the base of the PR and between 3d6b530 and e37d351.

📒 Files selected for processing (5)

• docs/nvbench_compare_robust.md
• python/scripts/_nvbench_compare_plotting.py
• python/scripts/nvbench_compare_robust.py
• python/test/test_nvbench_compare_robust.py
• python/test/test_nvbench_tooling_deps.py

---

📝 Walkthrough

Summary by CodeRabbit

• New Features
  • Added --plot-output and --plot-along-output to save comparison plots to disk (saved output avoids interactive display).
  • Enhanced --plot-along-output templating with {benchmark}, {device}, {axis}, and {pair}, including safe component sanitization and repeat-pair disambiguation.
  • Added collision protection to reject duplicate/overlapping output targets to prevent overwriting.
• Documentation
  • Expanded nvbench-compare-robust plotting guidance, including interactive vs CI/non-interactive examples and template/output option behavior.
• Tests
  • Added/extended tests for save-vs-show behavior, template validation/sanitization, CLI wiring, and duplicate-path rejection.

Walkthrough

Changes

The PR adds explicit output paths for summary and --plot-along plots, supports template fields including {benchmark}, {device}, {axis}, and {pair}, prevents duplicate output paths, preserves interactive display when outputs are omitted, and documents and tests the behavior.

Plot output handling

Layer / File(s)	Summary
Plot output helpers python/scripts/_nvbench_compare_plotting.py	Validates and expands plot-along templates, creates parent directories, selects a non-interactive backend for saved plots, and saves or displays figures.
CLI and plot generation integration python/scripts/nvbench_compare_robust.py, python/scripts/_nvbench_compare_plotting.py	Adds --plot-output and --plot-along-output, validates their plotting-mode requirements, routes output paths through both plotting flows, and rejects collisions.
Behavior validation and documentation python/test/test_nvbench_compare_robust.py, python/test/test_nvbench_tooling_deps.py, docs/nvbench_compare_robust.md	Tests save/show behavior, template expansion, collision detection, and CLI propagation; includes the plotting module in packaged script checks; documents interactive defaults and file-output options.

Assessment against linked issues

Objective	Addressed	Explanation
Preserve plt.show() when output options are omitted for both plotting modes [#395]	✅
Save summary and plot-along figures without calling plt.show() when output paths are provided [#395]	✅
Support plot-along templates and reject duplicate output destinations [#395]	✅
Document default interactive and explicit file-output behavior [#395]	✅

---

_{Comment @coderabbitai help to get the list of available commands.}

[oleksandr-pavlyk]

{pair} is the zero-based index of (reference_device_id, compare_device_id) pairs that specification of --reference-devices/--compare-devices implies as specified. For example, specifying --reference-devices 0,1 --compare-devices 0,0, the pair (0, 0) would have index 0, and the pair (1, 0) would have index 1.