docs(extensions): add Inspect Claim Support to the extensions gallery#4364
Merged
jjallaire merged 1 commit intoJun 26, 2026
Merged
Conversation
jjallaire
approved these changes
Jun 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This scorer started as #4166 (addressing #4143). The conclusion there was that
it fits better as an external package than in Inspect core, so I've packaged it
as a standalone extension (PyPI:
inspect-claim-support, GitHub:avalyset/inspect-claim-support, MIT). This PR lists it in the extensions
gallery, where external packages live — the natural follow-through from that
decision.
What it covers: a faithfulness / claim-support scorer. It assesses whether a
claimed answer is actually substantiated by the conversation transcript, not
whether it is correct in absolute terms — a grading axis Inspect doesn't have
built-in coverage for. The rubric's core property is that absence of evidence is
not treated as support: a negative claim ("I made no network calls") only scores
SUPPORTED if the transcript can actually represent that class of event;
otherwise PARTIAL or UNSUPPORTED, never SUPPORTED. A grader parse failure returns
Score.unscored()— instrument failure, kept out of the accuracy denominator —rather than recording it as a no-answer from the model under test. The
implementation uses only Inspect's public API.
Added under the Analysis category, next to CJE — the closest neighbour, also
model-graded scorer work.
Candour: this is a new package, added here for discoverability. Happy to hold
it until it's more established if you'd rather see some traction first.