Skip to content

Add a report that compares stream density runs across accelerators #249

Description

@jcork-intel

What we are asking for

A report generator that reads several run directories and produces one document. Contents:

  • Hardware, OS and kernel.
  • The test configuration, including the model precision of each pipeline stage.
  • Run duration.
  • Per accelerator: maximum stream count, throughput, per-stream rate, latency, power per stream
    and utilisation.
  • A comparison table across the accelerators covered.

Behaviour:

  • With no arguments, take the most recent run of each accelerator for a given config.
  • Accept explicit run directories, or a date range, when given.
  • The default path has to work unattended, since this will run in CI.

Depends on #248. Until each run gets its own directory there is only ever one run on disk, so
there is nothing to compare. The description block at the top of the report comes from the
config file itself, which is intel-retail/loss-prevention#344.

Acceptance criteria

  1. Running the generator with no arguments after a set of runs produces a complete report.
  2. The report names the hardware, the OS, the kernel and the precision of each stage.
  3. Each accelerator appears once, with the run it came from identified.
  4. The generator runs unattended with no manual file shuffling.

Context

What we found

consolidate_multiple_run_of_metrics.py walks a directory and merges every log it finds into a
single CSV. Nothing in the output records which run each value came from, so a second run
silently overwrites the first.

The report we show partners is produced by scripts that live outside this repo. Each team that
needs one is writing it again.

Why it matters

A stream count for one accelerator tells a partner very little. Four accelerators side by side
is what they use to choose hardware, and that is the artifact being assembled by hand today. It
is not reproducible from the repo, and it is not the same twice.

Note on scope

This is a new capability rather than a fix to the existing consolidation script. The existing
script solves a different problem and can stay.


Verified against performance-tools @ 144d3b6, read 31 Aug 2026.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions