Hi, I'm Louis, an MSc student at the Oxford Internet Institute working on the science of AI evaluation.
Benchmarks are the exams AI models sit. I study what those scores measure and when they predict performance beyond the test, drawing on psychometrics (the statistics of human testing), statistical modelling and economics. I am increasingly interested in what changes when the system being evaluated uses tools or works with people.
The work sits under four questions.
- Can the score be trusted?
- What does the score measure?
- What does the score predict outside the evaluation?
- When should a decision rely on it?
Four manuscripts from this work are under review at NeurIPS 2026 workshops, where I also serve as an invited reviewer. This GitHub holds the research code behind them. Each repository states its reproduction tier, which quantities are recomputed from source and which are regenerated from archived outputs.
| Project | Stack |
|---|---|
| One Capability or Many? Do frontier AI benchmarks measure one capability or several? Factor analysis and leave-one-benchmark-out prediction over a hash-pinned snapshot of 421 model configurations (103 in the economic-dense predictive subset). No distinct economic factor, yet modest but reliable incremental prediction, pooled ΔMSE 0.037 with a 95% interval of 0.019 to 0.055. Hypotheses pre-specified, analysis plan deposited retrospectively. |
Python EFA nested CVpre-specified |
| Three Ways CTT Misleads for LLM Judges Do the standard reliability statistics still answer the right question when the rater is a language model? A 210-item judge bank, reliability estimators, simulation studies and figure scripts. Quantities measured on the bank recompute exactly. Simulations are statistically reproduced from disclosed seeds. Figures regenerate from the archived grid. |
Python psychometricsMIT + CC BY 4.0 |
| Frontier Safety Framework Corpus When a frontier AI developer revises its safety framework, how much of the change can a reader identify from the developer's own account of it? A versioned, hash-pinned corpus of 52 documents and 19 labelled version pairs, each with the provider's revision account, plus the coding behind the silent revision rate. The corpus verifies against its manifest by SHA-256. Statistics recompute from the released coding sheet, which is itself an archived output. |
Python content analysishash-pinned MIT + CC BY 4.0 |
| A Parliament Language Model A 273k-parameter GRU implemented from scratch in PyTorch on 2.1M characters of Hansard, under a hard parameter budget and a no-attention rule, with controlled experiments on context length and capacity. A token-matched arm and a position-wise evaluation cut the headline context effect from 0.398 to 0.041. |
PyTorch deep learningfrom scratch |
| AI Risk and UK Wages How real wages evolved in occupations scored as high automation risk after 2016. A descriptive occupation-year panel of ONS automation-risk scores and occupation-level ASHE wage data, 367 occupations, 2014 to 2023, with two-way fixed effects. |
Stata panel datatwo-way FE descriptive |
Also worth a look. Agent-based labour market · double ML · ABM-ACE · insurance demand · Marshall Investment Fund site
benchprobe, the analysis toolkit behind these studies (factor structure, reliability, grader agreement), is in development.
- One Capability or Many? Preprint, arXiv 2608.29420. Under review, NeurIPS 2026 Trust-AI-Eval Workshop. analysis plan, deposited retrospectively · code
- Three Ways Classical Test Theory Misleads for LLM Judges. Under review, NeurIPS 2026 JUDGe Workshop. code and data
- The Price of Intelligence. Preprint, arXiv 2608.29843. Under review, NeurIPS 2026 EconML Workshop. pre-registered validity audit · dataset
- A Score Should Travel With Its Repair History. Position paper, SocArXiv preprint. Under review at two NeurIPS 2026 workshops, AI for Meta-Science and AI & Science.
- Silent Revision. Measuring undisclosed change in the safety frameworks of frontier AI developers. Preprint, arXiv 2609.08789. corpus · code and coding
- From Advisor to Voting Teammate. Workshop paper, Workshop on Human-Agent Collaboration at CHI 2026. An agent-based simulation, co-authored.
- When Should Neural Data Inform Welfare? Minor revision invited, UCL Journal of Economics. arXiv
- The Unassembled Validity Argument. How harness-dependent instability in MMLU propagates into the leaderboards and capability claims built on it. BSc dissertation, STS Best Dissertation Prize. Manuscript in preparation.
Invited reviewer, NeurIPS 2026 Trust-AI-Eval and EconML workshops. Core contributor, EvalEval Coalition Science of Evaluations, on validity and evidentiary standards.