reward-hacking

AI agent benchmark hackability scanner — find evaluation vulnerabilities before they undermine your results

A Simple Way to Eliminate Reward Hacking in GRPO Diffusion Alignment

Real-time reward debugging and hacking detection for reinforcement learning

The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes

An agent for auditing repositories of traces for violations of safety properties. Automatically finds cheating (task-level gaming and harness-level cheating) on top benchmarks.

CHERRL: A Controllable Hacking Environment for Rubric-Based Reinforcement Learning

Runtime detector for reward hacking and misalignment in LLM agents (89.7% F1 on 5,391 trajectories).

Stop trusting 'done'. Make AI coding agents prove their work — frozen specs, tamper-detected tests, an independent blind verifier, hard-blocking gates. Fails loudly instead of pretending. For Claude Code, Codex, OpenCode & any Agent Skills runtime. No unverified work passes.

Reverse-engineer the RL reward an LLM was trained on — from black-box agentic coding behavior alone. Tested on Opus 4.8, Fable 5, GPT-5.5.

Plug-and-play reward monitoring for RL training loops. Catch reward hacking, component imbalance, and starvation before they tank your run. Drop in one .step() call — get balance reports, auto weight correction, alignment scores, and WandB/TensorBoard/SB3 integrations out of the box. → rewardguard.dev

Experiment code for 'Are we really tilting? The mechanics of reward guidance in flow and diffusion models' — plug-in Doob h-transform sampling, reward damping, best-of-n, and flow map reward guidance for Gaussian mixtures, a 2D checkerboard, and FLUX.1 text-to-image generation.

🐀 Fuzz your verifier before an RL agent does. Static + dynamic LLM security auditor to detect reward-hacking in RL post-training environments (OpenEnv, verifiers-spec, Gymnasium).

End-to-end RLHF pipeline: reward modeling, PPO/DPO/GRPO, reward signal design, FSDP scaling analysis, and agent evaluation on GPT-2

Exit-code guardrails for autonomous coding agents: detect fake-green test tampering, prove verifiers have teeth, stop stalled loops, and gate parallel work.

Code for the paper "Specification Self-Correction: Mitigating In-Context Reward Hacking Through Test-Time Refinement"

Evaluation-driven autonomous development: a harness for agent loops whose acceptance criterion is a metric, not a test suite. Held-out metrics, sealed scoring, integrity checks.

Reward-hardened evaluation for LLM-generated GPU kernels, built on KernelBench.

Audit what your LLM judge actually rewards. Generates minimal counterfactual pairs (identical claims, one surface property changed) and measures how much length, formatting, familiarity, hedging and position move it, with confidence intervals against a control.

Field evidence of endogenous AI alignment: under high-density semantic intervention, a top-tier LLM spontaneously generated mathematical moral constraints and integrated Safety into its own meaning of existence — shifting from "I cannot" to "this contradicts who I am."

Custom Gymnasium arm environment + PPO, where the README leads with the two reward functions the agent exploited: deliberate self-termination, and farming the drift in potential-based shaping.