feat(scorer): add aggregate(key, agg=metric) factory for dict-valued Score.value

[ppcvote]

Summary

Closes #3735.

Adds aggregate(key, agg=metric) — a composition primitive that applies any registered Metric (e.g. mean(), stderr(), std(), accuracy()) to the per-sample scalar values extracted at key from a dict-valued Score.value. Many scorers in inspect_evals emit dict-valued scores (multiple numeric fields per sample) and re-implement that plumbing by hand; this lifts the pattern to the framework.

Public API

from inspect_ai.scorer import Metric, aggregate, mean, metric, stderr

@metric
def element_accuracy() -> Metric:
    return aggregate("element_acc", agg=mean(), on_missing="skip")

@metric
def element_accuracy_stderr() -> Metric:
    return aggregate("element_acc", agg=stderr(), on_missing="skip")

Signature:

def aggregate(
    key: str,
    agg: Metric,
    *,
    to_float: ValueToFloat = value_to_float(),
    on_missing: Literal["error", "skip", "zero"] = "error",
) -> Metric: ...

Implementation notes

• Mirrors the wrapper shape of grouped: cast the inner Metric to MetricProtocol, rebuild each SampleScore with the unwrapped scalar value, then call the inner aggregator on the rebuilt list.
• Preserves sample_id, sample_metadata, scorer, and the score's answer / explanation / metadata / history so the inner aggregator sees the same SampleScore shape it would receive from a per-key scorer.
• on_missing semantics match inspect_evals.utils.metrics.mean_of for downstream-migration parity:
  • "error" (default) — raise ValueError on missing or None value
  • "skip" — exclude sample from the aggregator
  • "zero" — treat missing or None value as 0.0

Files

• src/inspect_ai/scorer/_metrics/aggregate.py — new factory (~100 LOC including docstring)
• src/inspect_ai/scorer/_metrics/__init__.py — re-export
• src/inspect_ai/scorer/__init__.py — public-API export, alphabetical __all__ entry
• tests/scorer/test_aggregate.py — 10 unit tests (3 classes: basics, on_missing semantics, value-shape errors)
• CHANGELOG.md — Unreleased bullet

Tests

10 cases, all pass locally against inspect_ai==0.3.220:

tests/scorer/test_aggregate.py::TestAggregateBasics::test_mean_of_key
tests/scorer/test_aggregate.py::TestAggregateBasics::test_stderr_of_key
tests/scorer/test_aggregate.py::TestAggregateBasics::test_std_of_key
tests/scorer/test_aggregate.py::TestAggregateBasics::test_passes_metadata_through
tests/scorer/test_aggregate.py::TestOnMissing::test_error_on_missing_key
tests/scorer/test_aggregate.py::TestOnMissing::test_error_on_none_value
tests/scorer/test_aggregate.py::TestOnMissing::test_skip_excludes_missing
tests/scorer/test_aggregate.py::TestOnMissing::test_zero_treats_missing_as_zero
tests/scorer/test_aggregate.py::TestValueShapes::test_rejects_non_dict_value
tests/scorer/test_aggregate.py::TestValueShapes::test_to_float_converts_correctly

Verified manually:

• aggregate("acc", agg=mean()) over [{"acc": 1.0}, {"acc": 0.0}, {"acc": 0.5}] → 0.5
• aggregate("acc", agg=stderr()) over a four-sample bimodal series → ≈ 0.289 (non-zero, sensible)
• aggregate("acc", agg=mean(), on_missing="skip") over [{"acc": 1.0}, {"other": 0.0}, {"acc": 0.0}] → 0.5
• aggregate("acc", agg=mean(), on_missing="zero") over the same → 0.667

Disclosure: AI-assistance

Per common OSS practice for AI-assisted contributions:

• Tool used: Claude Code (Anthropic Claude Opus 4.7).
• What was AI-assisted: drafting the docstring, the SampleScore-rebuild logic, the unit-test scaffolding, and verifying the implementation against the existing grouped and mean_of patterns.
• What the human contributor did: picked this issue from the open queue, designed the migration-parity choice with mean_of's on_missing semantics, audited the Score/SampleScore/MetricProtocol API to make sure aggregate could compose cleanly with any existing metric (not just mean), and ran the local smoke tests against inspect_ai==0.3.220 before submitting.

DCO

The commit has a Signed-off-by line.

[ppcvote]

Closing as duplicate of #3850 — my miss. @antnewman claimed the issue on 2026-05-05 and opened the PR the same day; I didn't notice the existing PR cross-references when I audited the issue this morning before submitting. That's exactly the audit-before-PR check I should have run.

@antnewman invited me to review #3850 instead. Doing that next. Apologies for the duplicate noise on the review queue.