A download-and-forget app that turns your computer's idle time into a small piece of a language model that belongs to everyone. Folding@home, but for AI.
Status: early, pre-alpha. The mechanisms below run and are tested, but nothing is training at scale yet. That's the point of posting it now: this is the moment to join and shape it, not admire it finished.
Some of the most consequential technology being built today gets built behind closed doors, by a handful of companies. Thinking@home builds one in the open instead: on the computers people already own, by whoever wants to help, then given away to anyone who wants it. A model owned by no one, because everyone built it.
I had this idea and did the early thinking, but I am not an expert in distributed training or ML security. I'm a builder who thinks this should exist and doesn't want to wait for permission. That's the premise, not a disclaimer: a model owned by everyone has to be built by everyone.
I worked the early thinking, and much of the first code, out in long conversations with Claude (Anthropic's Opus 4.8), and I'd rather name that than pretend I did it alone. The ideas and the calls are mine; a lot of the sharpening was done with a very good rubber duck.
If you know more than I do, and plenty of you will, I want you here. Take a piece, or tell me where I'm wrong. You do not need to be an expert to help. I'm not one either. Start with CONTRIBUTING.md, open an issue, and say hi.
People already donate compute for a cause. Folding@home once hit 2.4 exaflops on pure goodwill, briefly outrunning the world's top 500 supercomputers combined. Intelligence that isn't owned by five companies should pull at least as hard as protein folding did.
And training is the right target, not serving. A live shared brain can't work over the internet, because every request re-pays a slow cost through strangers' machines. Training pays once and leaves a durable artifact: once the weights exist, everyone runs them locally and cheaply, for as long as the hardware lasts. Expensive to produce, trivial to use, the shape of Linux and Wikipedia.
The hard part is trust. Prime Intellect and Nous already train real models across the internet, but they lean on a crypto token to keep contributors honest. Doing it token-free, on a shared artifact that funded enemies have every reason to poison, is the actual problem, and the interesting one.
Training across the internet isn't new, and this doesn't claim to be first. Petals showed distributed inference; hivemind and Learning@home built the transport substrate this leans on; Prime Intellect (OpenDiLoCo, INTELLECT), Nous (DisTrO, Psyche), and Gensyn already train real models across the internet. What's different here is narrow and deliberate. Those efforts are token- or market-incentivized and treat trust as one piece among many; this one is token-free, treats proving a contribution is honest as the central problem, and removes private-data poisoning by design with a single public corpus. If one of them gets there first, good. The goal is a commons, not a logo.
A few choices are settled and shape everything else.
- The substrate is the DiLoCo family of low-communication training: each node trains a small model locally for hundreds of steps, then syncs a compressed update. Infrequent sync is what survives home bandwidth and laptops that close. Fork OpenDiLoCo / Prime Intellect's
primewith hivemind to start. - One public corpus, openly licensed, no private data. Everyone trains on the same hash-pinned dataset, and it has to be genuinely open: public-domain or permissively-licensed text (candidates: Common Corpus, public-domain books, permissively-licensed code and web sets), never quietly-scraped copyrighted data. That is a hard requirement, not a preference. It removes a whole class of attack, since nobody can poison the model through secret local data when there is none, and it keeps the whole effort on clean legal ground. That problem is gone by design.
- Recoverable, not preventable. Every sync round is a transaction with a commit gate: signed checkpoint, held-out eval, rollback, blacklist. Assume defenses eventually fail, and make failure attributable and reversible.
- Token-free, consumer hardware, ungated weights. No speculative coin, and it runs on the GPUs and Macs people already own. If a stake ever proves unavoidable against a funded attacker, it's a refundable compute-bond in GPU-hours.
Every choice above is a strong default: held firmly, defended by default, and changed only for a real reason argued in the open. None of them is a vow we'd keep past the point it stops serving the mission. Openness bends to evidence, never to convenience, and the bar to move a default is high and public.
This is the failure we most want to design out: a principle hardening into dogma until the project can no longer act on what it learns. Rigid immutability, the rule that the rules can never change, is how good ideas ossify. It's arguably where Bitcoin's original promise calcified, "never change" outliving its usefulness until the thing couldn't adapt. A commitment that can't bend when the facts change isn't integrity. It's inertia wearing integrity's clothes.
So if the day comes that shipping a capable model ungated looks genuinely reckless, the right move is to say so and change course, not to quote a founding document at the wreckage. We don't quietly drift, and we won't loudly ossify. Staying able to adapt is itself one of the principles, and the one that keeps the rest honest.
With private data off the table, one real problem is left. A contributor could send an update that looks perfect, with normal size and full accuracy, while secretly teaching the model to misbehave on a rare hidden trigger. A backdoor can only get in through the data or through an update that isn't the honest gradient of that data, and the corpus is clean, so this reduces to one thing: proving an update is the honest gradient of the assigned data.
The near-term bet is redundant recompute, where a few nodes redo the same work and compare. Whether that's enough comes down to one measurable question: is honest cross-hardware divergence smaller than the smallest change that installs a working backdoor? If yes, recompute catches it. Settling that experimentally is the current focus and the highest-value thing to run.
The project rests on one bet that could be wrong and several attacks it doesn't yet stop. They're here in full, because the fastest way to lose a skeptic is to let them find the hole you left out.
- The core bet is unproven at scale. Honest cross-hardware divergence sits more than 1000x below a working backdoor on a tiny model and two backends. It may narrow as models grow and hardware diversifies. If it ever inverts, recompute stops working and the approach has to change. That is exactly what "strong defaults, not unbreakable vows" is for.
- Colluding auditors and Sybil identities. Recompute assumes an honest recomputer. An attacker who controls enough of the replicas can rubber-stamp their own poison. Reputation, random replica assignment, and the refundable compute-bond raise the cost; a funded, patient attacker is still an open problem.
- Adaptive attacks. The experiments test one family of backdoors, caught by magnitude and by concentration. An attacker who shapes the poison to mimic honest hardware noise, or spreads it thinly across many weights, may beat both signals. Catching your own test attack is not catching every attack, which is why the design is layered and recoverable instead of betting everything on one filter.
- Misuse of the finished model. The integrity work defends how the model is trained, not how it gets used. Ungated weights can be misused and can't be recalled. See Scope and Responsible Use below.
- The corpus supply chain. One public corpus removes private-data poisoning, but a compromised public corpus is its own risk. It needs real vetting, not just a hash pin.
- None of it is battle-tested. Everything here is a small-scale demonstration; no motivated adversary has tried to break it yet. That's an open invitation (SECURITY.md), not a reassurance.
If you can break any of these, that's the most valuable thing you can contribute.
Smallest trusted core first, because every defense is earned by scale you don't have yet.
- Rung 0, prove the substrate: a ~150M model training across two ordinary machines, LAN then internet.
- Rung 1, the commit gate: signed checkpoint, held-out eval, accept or reject, rollback. Test it by poisoning your own node.
- Rung 2, download-and-forget: a background daemon that trains when idle and backs off when you touch the machine. BOINC is the pattern.
- Rung 3, open to a dozen trusted people: the first real multi-party run.
- Rung 4, open the swarm: robust aggregation, Sybil resistance, the refundable bond.
- Rung 5, the release gate: before any capable model ships, evaluate and red-team it, publish what it can and can't do, and make the release call in the open. Growing capable is a milestone, not an automatic release.
The recompute experiment runs alongside Rungs 0 and 1.
Rung 0 has started. A ~0.6M-param transformer (bigger than the toy above, small enough to train in seconds) now trains across separate OS processes over TCP with the recompute audit live, and a poisoned worker is flagged and dropped over the wire (scripts/pt_train_networked.py, and its worker scripts/pt_worker_node.py). It runs on localhost today; the same code spans machines by pointing the workers at a host with --no-spawn. Scaling the model up and going localhost to LAN to the open internet is the rest of the rung.
A dependency-free reference implementation of the core mechanisms lives in th/. No installs, no GPU, pure Python standard library. It runs a tiny softmax classifier instead of a transformer on purpose: the goal is to get the coordination and defense mechanisms correct and runnable first, then build the PyTorch and transformer version on the same interfaces.
python3 tests/run_tests.py # the mechanisms, tested
python3 scripts/train_local.py # DiLoCo across workers, and the gate rejecting a crude poison
python3 scripts/run_experiment.py # the Problem-1 experiment: eps_hw vs eps_bd
python3 scripts/audited_swarm.py # a swarm that catches and drops a stealthy backdoor worker
python3 scripts/problem1_depth.py # two more levers: update structure, and the effect of H
python3 scripts/defended_swarm.py # full stack vs a sleeper: spot-check audit + reputation + blacklist + robust aggregation
python3 scripts/recovery.py # a backdoor that slipped through, discovered later: located, attributed, replayed out
diloco.py is the low-communication training loop, gate.py is the accept/reject/rollback commit gate, audit.py is the recompute check that catches a poisoned worker, and experiment.py measures whether it can, backed by a tiny model, a hash-pinned synthetic corpus with a trigger, and plain param math. The end-to-end demo makes the point: with the audit off, a stealthy backdoor worker drives attack success to 100% while clean accuracy stays perfect, which is exactly what the commit gate cannot see; with the audit on, that worker is flagged and dropped every round and the backdoor never lands. The open question for the real version: does honest divergence on actual heterogeneous hardware stay as far below the backdoor gap as it does on this toy?
Beyond the single audit, defend.py composes the realistic defense. Reputation decides who gets spot-checked, since you cannot recompute everyone every round, and an audit-probability floor still catches a build-trust-then-defect sleeper. A caught worker is blacklisted and forfeits its refundable compute-bond, the GPU-hours stake that makes a sleeper swarm expensive. Two backstops sit under that: a structure check flags a concentrated update even when its size hides below the magnitude threshold, and a coordinate-median aggregator bounds the damage of anything that slips past both. In the demo the sleeper defects, is spot-checked and blacklisted, and its attack success rate never leaves zero.
And when a backdoor does slip through, ledger.py is the recovery layer. Every committed round is checkpointed with its params, optimizer state, and the deltas that were folded. A backdoor found later is located by re-auditing the chain, attributed to the worker that introduced it, and replayed out, keeping every honest contribution since. In the demo an unaudited backdoor reaches full attack success, then is traced to its entry round and excised, and the model is rebuilt clean. Corruption of a shared, accumulating artifact becomes a revert plus a replay.
The same training runs across real, separate processes over TCP with python3 scripts/train_networked.py (add --poison to watch the coordinator flag and drop a backdoor worker over the wire). Workers reproduce the hash-pinned corpus from its seed, so only weights and assignments cross the network. Point them at a LAN or internet address and it spans machines unchanged.
The reference core proves the logic; pt/ proves it on an actual nanoGPT-scale transformer (~105K params) and gives the first real-hardware data point for the Problem-1 question. Training the same assigned work on CPU versus MPS (genuinely different hardware), and comparing to a backdoored worker:
| quantity | value |
|---|---|
| CPU vs CPU (determinism floor) | 0.0 (bit-identical) |
| CPU vs MPS (honest hardware divergence, ε_hw) | ~2e-3 (MPS-only; drifts run to run) |
| backdoor delta (ε_bd) | ~2.75 |
| ε_bd / ε_hw | >1000x (measured 1.1–1.4k) |
| clean ASR / backdoor ASR | 0.00 / 0.97 |
On real weights and real hardware, the backdoor's update sits more than a thousand times outside honest float divergence, so the recompute audit separates poison from hardware noise here too. (ε_hw is measured only where a second backend exists, so on CPU-only CI that row is skipped, not faked.)
The audit then runs as a live defense on the real net. In scripts/pt_defended_swarm.py, four workers train a DiLoCo round and two of them are a poisoned coalition. With the audit off the backdoor lands (ASR ~0.87); with it on, both poisoned deltas are recomputed, diverge past the calibrated threshold, and are dropped every round (ASR 0.00). Because recompute re-derives ground truth, it needs no honest majority: half the swarm is hostile here and it still flags both, every round, which is the line robust aggregation alone cannot hold.
scripts/pt_weak_backdoor.py tests the obvious attacker response: shrink the poison until it hides. It doesn't work here, and the reason is the honest finding. Scaling the update down scales its effect down faster: by the time the delta is small enough to approach the recompute threshold, the backdoor has already stopped working. The weakest still-effective backdoor in the sweep sits about 250x above the threshold, so on this model the magnitude audit alone catches every backdoor strength that still worked. The scale-invariant structure signal flags the poisoned direction too, and it's wired into the live defense, but it only earns its keep in a regime with far more hardware divergence than a laptop-and-a-desktop shows. Whether that regime narrows the margin is the open question, and real diverse hardware is what answers it.
Run it (needs torch):
uv venv --python 3.12 .venv && uv pip install -r requirements.txt --python .venv/bin/python
.venv/bin/python scripts/pt_experiment.py # eps_hw vs eps_bd on real hardware
.venv/bin/python scripts/pt_defended_swarm.py # the audit dropping a poisoned coalition on the real net
.venv/bin/python scripts/pt_weak_backdoor.py # can a backdoor hide by shrinking? (no, and why)
.venv/bin/python scripts/pt_train_networked.py # Rung 0: the transformer trained across TCP processes, audit live
.venv/bin/python scripts/pt_train_networked.py --poison # a backdoor worker, flagged and dropped over the wire
What this is: an early, public research commons for finding out whether a model can be trained honestly on donated compute. What it is not: a safety-evaluated production model. Nothing here has been red-teamed, and until it is (Rung 5), treat any weights as an experiment.
Openness is the point, and openness cuts both ways. A model anyone can run is a model anyone can misuse, and open weights can't be recalled. We take that trade knowingly: a model owned by everyone is worth more than the control a closed one buys, and safety-by-secrecy is weak safety anyway. What we can do is be honest about what we ship, evaluate before we release, and keep the training itself clean and accountable. That is what the integrity work above is for, though it defends how the model is trained, not how the finished model gets used. That second problem is real and we don't pretend to have solved it.
A few commitments that follow from this:
- The corpus is openly licensed by requirement, so the training data is clean in both senses.
- The weights, when there are any, are meant to be permissively open (Apache-2.0, like the code), belonging to everyone. Responsible use is asked of contributors and users as a norm, not enforced as a license restriction. The final license is a decision to lock before the first real release.
- No warranty. This is a volunteer project, provided as-is.
- A named home is coming. Past the hobby stage this needs a steward or a light nonprofit to be the accountable place for legal and safety questions. That is on the roadmap, not pretended away.
Whether you write CUDA kernels or have never trained a model, there's a place for you. See CONTRIBUTING.md, open an issue or discussion, and say hi. That's the whole barrier to entry.