Skip to content

Latest commit

 

History

6 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

judge-calibration

Decide whether an LLM judge can be trusted to grade without a human watching.

No runtime dependencies. Python 3.14+.

The problem this solves

You have 10,000 model outputs and you cannot read them. You want a language model to grade them for you. Before believing a single number it produces, you check it against a sample you graded yourself.

The obvious check — how often did we agree? — is misleading, and the more imbalanced your data, the more misleading it gets.

Why raw agreement lies

Suppose 95 of 100 transcripts contain no special-category personal data. A judge that answers "no" every single time is right 95% of the time and has detected nothing. Accuracy rewarded it for the base rate.

The same trap catches agreement between two raters. Two people who each answer "no" 95% of the time will match roughly 90% of the time by luck alone:

both say no  = 0.95 × 0.95 = 0.9025
both say yes = 0.05 × 0.05 = 0.0025
                             ------
chance agreement             0.905

So "we agreed 95% of the time" means you got 0.045 above a floor of 0.905. Not impressive; barely above two people guessing.

Cohen's kappa

Kappa subtracts the free agreement and reports the share of what was left that you actually closed.

        observed − chance
kappa = -----------------
           1 − chance
  • observed — how often the two raters actually matched. Counted.
  • chance — how often they would have matched had each answered at their own usual rate without looking at the item. Calculated from their totals alone.
  • 1 − chance — the room that was available above luck.

The analogy

A multiple-choice exam with four options each. Guessing already scores 25%. So a score of 50% is not "half right" — it closed 25 points of the 75 that were actually yours to win.

(50 − 25) / (100 − 25) = 0.33

Kappa is that exam score, with guessing replaced by what two raters would match by luck.

The general pattern

The formula is min–max normalisation, (x − min) / (max − min), with min = chance and max = 1. The same shape appears in R² in regression (floor: predicting the mean), normalised gain in education research, and efficiency in physics. Faced with a new metric, the question to ask is: what is its floor, and what is its ceiling?

Reading the number

kappa Meaning
1.0 Perfect agreement
0.8 – 1.0 Strong
0.6 – 0.8 Good
0.4 – 0.6 Moderate
below 0.4 Poor
0.0 No better than luck
negative The raters contradict each other more often than luck would

Two things people get wrong

Totals do not give you observed agreement. Knowing one rater said yes 40 times and the other said yes 80 times does not say which items each chose. The overlap could be anything. Observed agreement must be counted item by item; chance agreement is the opposite, calculated from the totals alone precisely because it pretends nobody looked.

Kappa is symmetric — neither rater is the reference. Swap the two raters and the number is identical. So the choice of metric is a claim about the question:

Question Does a right answer exist? Metric
Does this text contain a national insurance number? Yes Precision / recall / F1
Is this rationale biased? No, two careful people would differ Kappa

Which means kappa can never tell you a judge is correct, only that it is consistent with a human. If both are wrong in the same way, kappa is 1.0 and you have learned nothing.

Why the confidence interval matters

A kappa of 0.457 from 40 items and from 4,000 items are not the same claim:

  items   kappa   95% interval      width
     40   0.457   [0.158, 0.730]    0.572   ← could be poor, could be good
     80   0.457   [0.249, 0.647]    0.398
    200   0.457   [0.334, 0.581]    0.247
    800   0.457   [0.393, 0.518]    0.126
   4000   0.457   [0.429, 0.484]    0.055   ← reliably mediocre

The point estimate never told you how much to believe it. The width does.

Note the two different reasons to distrust a judge, which are easy to conflate: at 40 items you cannot tell; at 4,000 you can tell, and the answer is no.

The interval is obtained by bootstrapping — drawing new samples of the same size from your labelled pairs, with replacement, scoring each, and reporting the middle 95%. It assumes nothing about the shape of the underlying distribution.

The verdict

Two of the three ways this can go both look like "a low kappa", and only one of them is fixed by labelling more items. The lower bound decides whether a claim can be made; the upper bound decides whether more data could still rescue the judge.

Against a threshold of 0.6:

Interval Verdict What is wrong
[0.65, 0.79] TRUST nothing — the worst case already clears the bar
[0.41, 0.88] LABEL_MORE too few labelled items to tell either way
[0.43, 0.48] REJECT the judge — narrowing the range will not lift it

The same judge, at three sample sizes, shows why the distinction is not cosmetic:

   40 items  kappa=0.457  [0.16, 0.73]  -> LABEL_MORE
  800 items  kappa=0.457  [0.39, 0.52]  -> REJECT
 8000 items  kappa=0.457  [0.44, 0.48]  -> REJECT

Identical point estimate throughout. At 40 items you cannot tell; at 800 you can, and the answer is no.

How much more labelling

A LABEL_MORE verdict says the estimate is too wide to act on without saying how much more work would settle it. Larger samples are simulated from the labels already in hand, and the smallest size whose verdict stops straddling the threshold is reported — along with what that verdict would be.

good judge, undersampled
   now: 32 items, kappa 0.75 [0.50, 0.94] -> LABEL_MORE
   estimate: label 128 total -> would read TRUST

poor judge, undersampled
   now: 40 items, kappa 0.46 [0.16, 0.73] -> LABEL_MORE
   estimate: label 160 total -> would read REJECT

The second row is the one that saves work. Both say "label more" today, but the simulation gives away the ending: 120 further transcripts would buy a rejection. Replace the judge instead.

The assumption, which is the whole limitation: simulated items inherit the agreement pattern of the sample in hand, so the figure is only as sound as that sample is representative. A judge that behaves differently on cases not yet labelled will not obey it.

The disagreement report

A kappa says how much two raters differ and never what about. The items behind the number are the only part of this tool that improves a judge rather than grading it, so they are kept and split by direction — what the judge flags that a human would not, and what it misses. The two usually have different causes: consistent over-flagging is often a threshold, consistent misses on one kind of case are often an ambiguous rubric.

kappa 0.00  |  agreed 3/6

MISSED (human flagged, judge did not):
   - I take Fridays off for prayer
   - my wife and I moved here in 2019

OVER-FLAGGED (judge flagged, human did not):
   - I use Python daily

The score alone says only "no better than luck". The report says the judge misses disclosures made obliquely, with no keyword present — which is a finding you can act on.

Usage

from judgecal.agreement import kappa
from judgecal.disagreement import disagreements
from judgecal.interval import kappa_interval
from judgecal.sample_size import items_needed
from judgecal.verdict import assess

human = [True] * 20 + [False] * 480
judge = [True] * 5 + [False] * 15 + [False] * 480

result = kappa(human, judge)
result.observed          # 0.97 — looks excellent
result.kappa             # 0.39 — is not
result.table             # the four cells, so the number can be audited

interval = kappa_interval(human, judge, seed=0)
interval.low, interval.high
interval.width           # how much the estimate is allowed to move

decision = assess(interval, threshold=0.6)
decision.verdict         # Verdict.TRUST | LABEL_MORE | REJECT
decision.reason          # the bound and threshold that produced it

report = disagreements(human, judge, transcripts)
report.only_first        # the judge missed these
report.only_second       # the judge flagged these, you did not
report.agreed            # counted, not listed

estimate = items_needed(human, judge, threshold=0.6, seed=0)
estimate.needed          # total items at which the verdict resolves
estimate.projected       # what it would resolve to

kappa raises rather than returning a number when both raters used a single label throughout: chance agreement is then total, there is no room above it, and a silent 0.0 would be a lie.

Layout

Path Contents
judgecal/agreement.py Cross-tabulation and kappa
judgecal/interval.py Bootstrap confidence interval
judgecal/verdict.py Trust, label more, or reject
judgecal/disagreement.py The items the raters split on
judgecal/sample_size.py How much more labelling a decision needs
tests/ Hand-computed examples as specification

Development

uv sync
uv run pytest

Not built yet

  • Weighted kappa, for ordered categories such as a 1–5 score.
  • Fleiss' kappa, for more than two raters.

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages