Score code rules on labelled human-vs-machine corpora

[mgiovani]

Why

No stopslop rule has ever been scored against a labelled human-vs-machine corpus. Every threshold is a judgment call. Three public datasets now make precision and false-positive rate measurable, and the formatting-uniformity and prose-residue rules need those numbers before they can leave Tier B.

Datasets

• AIGCodeSet (Python, 6 LLMs): https://arxiv.org/abs/2412.16594
• CoDet-M4 (6 languages, 5+ generators, multi-domain): https://arxiv.org/abs/2503.13733
• Droid resource suite (multilingual): https://arxiv.org/abs/2507.10583
• HybridCodeAuthorship (line-level authorship, 2M functions): https://arxiv.org/abs/2606.12620

Scope

• A script under bench/ (reuse the harness from Perf & stability: measure, verify, and land throughput and robustness improvements (tracking) #21 and tests/stress.rs) that downloads one dataset, runs stopslop --format json, and reports per-rule hit rate on human vs machine files, precision, and false-positive rate.
• Report as a markdown table in the PR; no CI gate, this is a measurement tool.
• Out of scope: training anything. The datasets are for measuring the deterministic rules, not for building a classifier.