Skip to content

Add support for agentic (multi-step, stateful, tool-calling) evals #1038

Description

@abatilo

Summary

aiperf handles simple request-response evals but does not support what we'd call "agentic evals": multi-step, stateful evaluations where the model issues a tool call, the tool executes, and the resulting world-state is fed back to the model in a loop. As aiperf matures toward being a single destination for model testing, this is the one category that currently forces users out to an external tool.

Context

aiperf already covers the testing needs that matter today and is maturing quickly:

  • Performance testing — inference performance, acceptance length for speculative-decoding models.
  • Accuracy / intelligence evals — point aiperf at a hosted inference endpoint and get an accuracy measurement for a given eval. Multiple-choice accuracy evals (e.g., MMLU) are available today, with more in the repo moving through PR review.

These accuracy evals have been genuinely useful for Model Optimizer / quantization work: validating that a quantized model stays within an accuracy range of the original. The tool's momentum and responsiveness are a big part of why it's the default place to go.

The gap

Today's evals are essentially request-response: send a multiple-choice question, get a response, parse the last line, match against the question-answer pair. Simple and minimal.

The evals that increasingly matter are agentic: a multi-step, stateful loop where:

  1. The model receives a question and decides it needs a tool call.
  2. The tool call executes somewhere.
  3. The results — the changed state of the world — are represented back to the model.
  4. The model decides the next step and issues another tool call.
  5. The cycle repeats.

aiperf cannot run this multi-step stateful loop. Under the assumption that aiperf wants to be the one-stop shop for model testing, this is the area where it doesn't solve the problem, which forces use of an external tool.

Current workaround

Two parts:

  1. Run the agentic eval externally via Harbor. Harbor handles the multi-step tool-call loop aiperf can't. The choice of Harbor was essentially a coin flip — it solved the problem first. Comparable agent-eval suites exist (e.g., Inspect AI), and Nemo Gym is noted as a potential Dynamo-built equivalent. No strong preference among them.
  2. Translate traces back into aiperf. A small custom script converts Harbor's request-response traces into the mooncake trace format, which aiperf can ingest to do request-response replay against a hosted inference endpoint. This enables testing realistic multi-step / multi-turn conversations against inference. The reporter is willing to upstream or contribute this translator.

Possible directions

Floated as options, not prescriptions — the implementation is entirely the team's call:

  • Make Harbor an optional/extra Python dependency of aiperf (Harbor exposes a Python API), with optional pass-through to Harbor.
  • Build a first-party agentic-eval capability within Dynamo (roughly what Nemo Gym is positioned as).
  • Independently / additionally: accept the trace translator (Harbor → mooncake format) as an upstream contribution.

The team also floated simple Harbor ↔ aiperf interoperability so users wouldn't have to maintain a translation layer themselves; the reporter did not push back, but explicitly declined to prescribe any solution.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions