Skip to content

Add cross_model_verifier scorer (grader must differ from the model under evaluation)#4477

Open
blewis-maker wants to merge 1 commit into
UKGovernmentBEIS:mainfrom
blewis-maker:add-cross-model-verifier-scorer
Open

Add cross_model_verifier scorer (grader must differ from the model under evaluation)#4477
blewis-maker wants to merge 1 commit into
UKGovernmentBEIS:mainfrom
blewis-maker:add-cross-model-verifier-scorer

Conversation

@blewis-maker

Copy link
Copy Markdown

Draft PR

Title: Add cross_model_verifier scorer (grader must differ from the model under evaluation)


Summary

This adds a small model-graded scorer, cross_model_verifier, that behaves like
model_graded_qa but guarantees the grader (the verifier) is a different model
from the one that produced the answer (the actor). If the two resolve to the
same model it raises by default, so a model never silently grades its own output.

Motivation

model_graded_qa / model_graded_fact default to model_role="grader", but when
no grader role is bound they fall back to the model under evaluation. That makes it
easy to accidentally have a model grade its own output, which tends to be
self-favouring. For governed or third-party evaluations it's often a hard
requirement that the answering model and the grading model are not the same.

Rather than everyone re-implementing that check around model_graded_qa, this makes
actor≠verifier a first-class, enforced property with an audit trail in the score
metadata. (The idea comes from governed multi-agent evaluation, where the actor and
verifier are deliberately kept separate.)

What it does

  • Resolves the verifier with the same precedence as model_graded_qa
    (model > model_role > model under evaluation).
  • Compares the verifier's fully-qualified name against the actor's (state.model).
  • on_same_model controls the same-model case: "error" (default) raises a
    ValueError, "warn" logs and grades anyway, "allow" grades silently.
  • Records actor_model, verifier_model, and actor_verifier_distinct in the
    score metadata so the separation is visible in the eval log.
  • Reuses the existing grading template, grade parsing, and prompt-injection
    hardening from _model.py unchanged — no forked logic.

Usage

from inspect_ai import Task, eval
from inspect_ai.dataset import Sample
from inspect_ai.model import get_model
from inspect_ai.scorer import cross_model_verifier

task = Task(
    dataset=[Sample(input="What is the capital of France?", target="Paris")],
    scorer=cross_model_verifier(),
)

eval(
    task,
    model="openai/gpt-4o",
    model_roles={"grader": get_model("anthropic/claude-sonnet-4-latest")},
)

Tests

tests/scorer/test_cross_model.py covers distinct-model grading, the
same-model error / warn paths, the default-role fallback, a bound grader role,
and grade-parse-failure being unscored. All pass locally; ruff and mypy are
clean.

Open question for maintainers

Would you prefer this as a standalone scorer, or as an option on
model_graded_qa (e.g. require_distinct_grader=...)? Happy to refactor to
whichever fits the codebase better. I can also add a docs entry if you point me
at the right file.

…der evaluation)

model_graded_qa falls back to the model under evaluation when no grader role is
bound, so a model can grade its own output. cross_model_verifier enforces
actor != verifier: it resolves the verifier with the same precedence as
model_graded_qa, then raises (by default) if the verifier resolves to the same
model as the actor. Records actor_model / verifier_model / actor_verifier_distinct
in score metadata. Reuses _model.py grading/parsing/hardening unchanged.

Signed-off-by: Brandan Lewis <lewisgeospatial@gmail.com>
@blewis-maker
blewis-maker marked this pull request as ready for review July 12, 2026 01:53

@dragonstyle dragonstyle left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me that a better way to approach the issue is to require that the user provide the role rather than that the roles be distinct. We have this available in get_model(required=True), but the model graders we have built in don't expose this. I think it would be better to expose this behavior in the existing graders and allow tasks using those graders to require specific roles be defined, if they like. Perhaps model_role_required could be added to the graders and make its way down to the get_model call.

@dragonstyle dragonstyle self-assigned this Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants