fix(evals): ground judge verdicts, don't trust ungrounded met=true (#99) - #102
Merged
Conversation
…99) The LLM-judge path (llm_bool/goals_met/constraint_respected assert types + judge_run) let a criterion's met=true stand even when the judge cited no transcript evidence for it, and unconditionally promoted a fail→pass whenever the model self-labeled the failing criteria as "irrelevant" — neither self-label is independently verified. Together these let a topically-related-but-wrong agent reply (e.g. "I'm ready to hear more about your company..." instead of re-asking for the missing company name) slip through as a pass. - evals/types.py::parse_judgment_payload: when a criterion is reported met=True with blank/missing evidence, flip it to met=False and mark the whole judgment needs_human_review — an ungrounded "met" claim is not trustworthy. - evals/relevancy.py::apply_relevancy: keep the fail→pass promotion when all relevant criteria are met (usually correct), but flag needs_human_review instead of trusting the model's own relevant=False self-label blindly. Tests: added grounding/promotion coverage to tests/test_evals_judge.py. Full suite (uv run pytest -q) → 683 passed.
Same pre-existing #97 merge damage as on fix/turn-split-merge — this branch was cut from main before that fix landed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #99.
Problem
The LLM-judge path (
llm_bool,goals_met,constraint_respected, and whole-conversationjudge_run) had no independent check on the model's ownmet/relevantself-labels:met: truewith noevidencecited at all —parse_judgment_payloadaccepted it as-is.apply_relevancyunconditionally promoted afailverdict topasswhenever the model self-labeled the failing criteria asrelevant: false, with no verification that the self-label was correct.This is exactly how a topically-related-but-wrong agent reply slips through, e.g.:
A judge that reads the second reply as "on-topic enough" can mark the criterion met without ever quoting evidence, or excuse it as "not relevant to this criterion" — either path currently produces a silent pass.
Fix
evals/types.py::parse_judgment_payload: a criterion reportedmet=Truewith blank/missingevidenceis flipped tomet=False, and the judgment is markedneeds_human_review=True. An ungrounded "met" claim is not evidence of a met criterion.evals/relevancy.py::apply_relevancy: keep the fail→pass promotion when all relevant criteria are met (it's usually correct), but now also setsneeds_human_review=Trueinstead of trusting the model's ownrelevant=Falseself-label blindly.This doesn't require a schema/prompt change — it's a pure grounding check on the judge's existing output shape.
Tests
tests/test_evals_judge.py: addedtest_parse_judgment_ungrounded_met_criterion_is_not_trusted,test_parse_judgment_grounded_met_criterion_is_trusted,test_relevancy_promotion_flags_human_review.uv run pytest -q→ 683 passed.