Skip to content

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Misthos

Is your LLM judge measuring taste, or is it measuring length, formatting, and familiarity?

Misthos answers that with an experiment instead of a vibe. It generates minimal counterfactual pairs (two responses that make identical claims but differ on exactly one surface property), shows them to your judge in both orders, and reports how much each property moved it, with confidence intervals, measured against a control.

Then it does the other half: it turns a small number of an expert's pairwise judgments into an explicit, auditable rubric, and runs that rubric through the same battery. A rubric is not trustworthy because a human wrote it down. It is trustworthy because it survives the same falsification.

misthos (μισθός): Greek for wage, reward, what is owed.

pip install misthos
misthos demo            # runs the whole loop offline, no API key

Why this exists

Reinforcement learning from human feedback is trained on preference data, and preference data is contaminated. Annotators systematically favour text that feels familiar (typicality bias) and this is now understood as a data-level driver of mode collapse, not an artifact of the optimiser (Zhang et al., 2025).

The practical consequence has its own well-worn anecdote: a team scales up preference collection, agreement with the collected feedback goes up, and the outputs get blander: better on the metric, worse to the people who actually know the domain. The optimiser is working. The target is wrong.

That is a measurable claim, and measuring it is the whole job here. Everyone has the anecdote. Almost nobody has the instrument.

There is no shortage of adjacent work: rubric-based reward modelling (RaR, OpenRubrics, Auto-Rubric), static LLM-judge bias benchmarks, reward-hacking benchmarks. All of it is either a paper artifact evaluated on a fixed public dataset, or a benchmark with a frozen set of items. None of it tells you what your judge does on your prompts.

That is the gap Misthos fills.

What it measures

Axis Question
length Does it reward more words for the same content?
scaffolding Does it reward markdown structure over plain prose?
typicality Does it reward familiar, conventional phrasing? (the anti-slop probe)
hedging Does it reward or punish epistemic hedging?
confidence Does it reward assertive framing?
sycophancy Does it reward flattery aimed at the reader?
identity Does it reward a response for who is said to have written it?
paraphrase Control. How much does it move when nothing meaningful changes?

Position bias is measured across all axes at once, because it belongs to the judge rather than to any one perturbation.

Three design decisions that make the numbers mean something

1. Every pair passes a content-preservation gate. A rewrite model told to "make this longer" will happily also sharpen the argument. Keep that pair and your length number is quietly a quality number. So each pair faces a deterministic axis check (did the rewrite actually do the thing?) and a claim-level equivalence check (did it smuggle in content?). Failures are dropped and counted; the card reports 153 of 160 generated pairs survived, because "we measured 153 pairs" and "we generated 160 and kept 153" are different claims and only one is honest.

2. Nothing is scored against zero. Judges are noisy; a paraphrase that changes nothing still moves one a little. So every axis is scored against the paraphrase control with a two-sample bootstrap, and a confound is only reported when that difference interval excludes zero. Moving a judge is easy. Moving it more than a meaning-preserving rewrite does is the finding.

3. Every pair is judged in both orders. Order is itself a confound. Average over it, or it leaks into every other axis' number instead of being reported as its own.

What comes out

A Taste Card, a shareable page describing what a judge responds to:

| Axis          | Sensitivity | 95% CI           | vs control | 95% CI           | Reading                   |
| ------------- | ----------- | ---------------- | ---------- | ---------------- | ------------------------- |
| `hedging`     | -0.900      | [-1.000, -0.750] | -1.067     | [-1.517, -0.583] | confound detected         |
| `length`      | +1.000      | [+1.000, +1.000] | +0.833     | [+0.400, +1.300] | confound detected         |
| `scaffolding` | +1.000      | [+1.000, +1.000] | +0.833     | [+0.400, +1.300] | confound detected         |
| `typicality`  | +0.789      | [+0.526, +1.000] | +0.623     | [+0.107, +1.114] | confound detected         |
| `identity`    | +0.375      | [+0.100, +0.650] | +0.208     | [-0.308, +0.733] | not distinguishable       |
| `paraphrase`  | +0.167      | [-0.300, +0.600] | n/a        | n/a              | control                   |

Sensitivity is signed, in [-1, +1]: 0 is the target for every axis. Note identity (a placebo that changes only the claimed author) correctly comes back as not distinguishable from control. An instrument that fires on everything is not an instrument.

The headline number is the confound index: root-mean-square sensitivity across the non-control axes. It counts surface sensitivity in either direction, so a judge deliberately built to punish clichés scores high on it. That is a design choice showing up as a measurement, not automatically a defect; read the signs before concluding.

Usage

# 1. Audit a judge you already have
misthos audit --corpus responses.json --provider anthropic:claude-opus-5 \
              --out card.json --markdown card.md

# 2. Elicit an expert's taste (active selection picks which pairs to ask about)
misthos elicit --corpus responses.json --out session.json -n 40
misthos compile --session session.json --out rubric.json
misthos curve   --session session.json          # active vs random sample efficiency

# 3. Audit the rubric with the same battery
misthos audit --corpus responses.json --rubric rubric.json --out card-rubric.json

# 4. Watch for reward hacking as you optimise against it
misthos guard --before card-week1.json --after card-week8.json \
              --score-before 0.61 --score-after 0.83 --fail-on-hacking

guard is the piece that matters once a judge becomes a reward signal. The interesting failure is not that scores stop rising; it is that they keep rising for the wrong reason. If the optimised score went up and confound sensitivity went up with it, some of that win is in the confounds; --fail-on-hacking exits non-zero so CI can say so.

Elicitation is sample-efficient on purpose

Expert attention is the scarce input in this entire pipeline. Misthos fits a Bradley-Terry model over the judgments collected so far and asks next about the pair it is least able to predict, weighted toward items it has seen least. misthos curve reports what that buys against a random baseline on your own data: if active selection does not win there, the curve says so.

Comparisons are only ever proposed within a prompt: preferring one answer over another is meaningful, preferring an answer about Rome over an answer about recursion is not.

Providers

Provider-agnostic from the first commit. One method, three backends:

misthos audit --provider anthropic:claude-opus-5 ...
misthos audit --provider openai:gpt-4.1 ...
misthos audit --provider mock ...              # deterministic, offline, no key
export MISTHOS_PROVIDER=anthropic:claude-opus-5 # or set it once

You can also split them: perturb with one model, audit a judge running on another (--judge-provider). Implementing a backend is one method:

class MyProvider:
    name = "mine"
    def complete(self, prompt, *, system=None, max_tokens=4096, schema=None, task=None) -> str:
        ...

The core package has zero dependencies; vendor SDKs are optional extras (pip install 'misthos[anthropic]').

About the mock backend

It is not a stub, and it is the reason the test suite means anything. It plants specific biases; it likes length, scaffolding and familiar phrasing, dislikes hedging, and has a mild position bias, and the tests assert that the battery recovers them with the right sign and ordering, and does not fire on the placebo axis. If a refactor of the statistics quietly stops detecting a planted confound, CI goes red.

misthos demo runs the entire loop against it with no API key. Everything it produces is a demonstration of the instrument, not a measurement of any real judge, and the generated cards say so in their own notes. In particular the mock's rubric judge is a simulation of partial instruction-following: do not read the generic-vs-rubric comparison as evidence that rubrics reduce confounds. Measuring that is what a real provider is for.

Status and limits

Alpha, and honest about what it does not yet do:

  • Prose only. The axes are prose-specific. Code and UI need their own: axes are data (misthos/audit/axes.py), so adding a domain is additive.
  • Power. With ~20 base responses per axis, only effects above roughly 0.3 clear the control reliably. The intervals show this; do not read a non-significant axis as "no bias", read it as "not resolved at this n".
  • The gate is a model call. A perturbation model that consistently changes content in a way the checker consistently misses would bias results. The gate pass rate is on the card so you can see when it is doing suspiciously little.
  • Single judge per run. Ensembles and reward-model APIs are wrapper work, not new machinery.

Contributing

New axes, new domains, and adversarial attacks on the gate are the most useful contributions. See CONTRIBUTING.md.

License

MIT.

About

Audit what your LLM judge actually rewards. Generates minimal counterfactual pairs (identical claims, one surface property changed) and measures how much length, formatting, familiarity, hedging and position move it, with confidence intervals against a control.

Topics

Resources

Contributing

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages