CollapseGuard

A drift/collapse early-warning and minimum-grounding controller for self-improving model loops.

When a model is trained on its own output — synthetic-data flywheels, self-training (ReST/STaR), agentic self-improvement — it drifts, and the rare capabilities die first, silently. The aggregate benchmark you watch is a lagging indicator: it looks healthy for several generations and then falls off a cliff, after the rare skills are already gone. CollapseGuard is the gauge that warns before the cliff, and the dial that tells you the minimum real-data "grounding" needed to keep the capabilities you refuse to lose.

It is model-agnostic: it never touches your model. It operates on a length-K distribution over modes (capabilities, classes, task types) measured each generation, plus a fixed healthy reference p*. Turning your model's output into that vector is the only code you write (see collapseguard.oracle).

Status: early scaffold (v0.0.1). The engine is ported clean-room from a validated Wright–Fisher model of model collapse (closed-form checks); see Provenance below.

Install

pip install collapseguard          # numpy-only core
pip install collapseguard[viz]     # + matplotlib for the demo figure
pip install collapseguard[io]      # + pandas/pyarrow for parquet + to_frame()

Quickstart

One object in your loop — Guard.observe returns diagnostics, a warning, and a ready-to-apply action:

from collapseguard import Guard

guard = Guard(reference=p_star, batch_size=n)   # p_star: healthy distribution over K capabilities
for samples in my_self_training_loop:           # one generation at a time
    advice = guard.observe(p_hat=measure(samples))        # or samples=..., classify=...
    if advice.warning:                          # fires while the benchmark still looks fine
        print("early warning:", advice.reason)
        g = advice.recommended_grounding        # inject this fraction of real data next generation
        counts = advice.allocation              # per-mode real-sample plan (tail-targeted)

The two halves are also usable directly (CollapseMonitor, GroundingController), and collapseguard.oracle turns raw outputs / correctness records into the p_hat vector.

Command line

Already logged your per-generation mode distributions? Replay them without writing code:

collapseguard watch --distributions run.npy --reference p_star.npy --batch-size 300 --figure out.png

(run.npy is a (generations × modes) matrix; .csv and .parquet also accepted.)

What it gives you

• CollapseMonitor — per-generation diagnostics (forward-KL to the reference, rare-tail mass, effective diversity, per-rarity-band survival) and a warning that fires off the tail signal, with a measured lead time over the aggregate benchmark.
• GroundingController — the actionable recipe from the drift threshold m·p*ᵢ ≳ 1:
  • min_grounding(must_keep, n) — the minimum uniform grounding fraction, set by the rarest capability you refuse to lose (not by average loss);
  • tail_targeted(must_keep, n) — the (much smaller) fraction if you can oversample real data from the rare modes;
  • allocate(g, n, policy) — per-mode real-sample counts (uniform vs tail-targeted).

Demo

python examples/synthetic_demo.py

Runs an exactly-known collapse process and shows (1) the dry lineage's rare tail collapsing while the benchmark still reads healthy, (2) the monitor warning several generations early, and (3) the grounding recipe rescuing the tail — with tail-targeting achieving the same protection for less. Writes examples/collapseguard_demo.png.

Scope (honest)

CollapseGuard is a gauge and a grounding dial, not a model-merging tool or an optimizer. It needs a trusted reference p* and a way to label outputs by capability/mode — that labeling is the real adoption cost, and it is stated up front. The current release validates on the exact synthetic process; image and language-model demos are face-validity work in progress.

Provenance

The diagnostic engine (heterozygosity/effective-diversity, forward-KL, tail-band survival) and the grounding threshold come from a research project modelling model collapse literally as the Wright–Fisher genetic-drift process, with three closed-form predictions validated to <0.1%. The functions are copied here clean-room so CollapseGuard is fully self-contained. Collapse-as-drift is established in the literature (e.g. Shumailov et al., Nature 2024; Riis 2026); CollapseGuard's contribution is the instrument: leading indicators plus the rarest-capability grounding recipe.

License

MIT © 2026 Giorgio Gilestro / Gilestro Lab.