An open market for work, where the grade is settled by validator consensus — and no reading a validator would accept can change who gets paid.
-1f6feb){kind=icon}
A sponsor posts a task, a prize and a rubric written in plain language. Anyone enters by staking a bond and submitting work, or a URL the contract fetches itself. Grading is permissionless: every entry is read under validator consensus, and the prize is paid to the highest eligible score.
This repository is the product around that contract, not only the contract. The arena is deployed on studionet and carries real activity; a public board renders every scorecard straight out of contract storage; and the scripts that opened the arena, entered work, appealed a grade and settled the prize are the same ones anyone can point at their own.
| the contract | contracts/proving_ground.py — 15 methods, lint-clean, deployed and verified |
| the product | a self-contained board built from contract views alone, published on GitHub Pages |
| the proof | 118 direct tests and 11 on-chain tests; six direct and one on-chain exist only to pin §2 |
| the activity | real tasks, real entries, a real appeal, a real payout — nothing mocked |
Bounty platforms already do the first part. The reason this one is a contract is the second: the grade has to be a number a stranger can defend, and it has to be the same number whichever honest validator happened to read the work. Four things make it that.
The reader never sees a total to anchor on and is never asked for a score. It is asked which rung of a published ladder the work sits on, and for nothing else:
| rung | label | what it means | worth |
|---|---|---|---|
| 0 | fails the brief | does not deliver what was asked, however well written | 0 |
| 1 | poor | attempts the brief and misses most of it | 20 |
| 2 | weak | delivers part of the brief, with gaps a reader would trip on | 40 |
| 3 | adequate | delivers what the brief asked for, and nothing beyond it | 60 |
| 4 | strong | delivers it well and is usable as it stands | 80 |
| 5 | exemplary | delivers it completely and could hardly be done better | 100 |
get_arena() publishes that table, so a solver reads the same six sentences the grader was handed,
and the board renders it from chain rather than restating it. Everything else is arithmetic the contract performs after the
answer comes back:
| what happened | who decided it | effect on the score |
|---|---|---|
| the reader placed the work on rung n | the model | n × 20 points |
| a preferred criterion is unsatisfied | the sandbox | -15, once per criterion |
| a mandatory criterion is unsatisfied | the sandbox | rung 0, score 0, entry disqualified |
| the work overlaps an earlier entry by 80% or more | the sandbox | rung 0, score 0, entry disqualified |
Both columns are recorded. judged_score keeps what the rung was worth before deductions, score
is what the contract settled on, and the difference is on chain for the solver to read. An entry
read as strong that lands at 65 names the preferred criterion that cost the fifteen points. One
that lands at 0 names the mandatory criterion it broke, and loses its rung with it: an entry code
threw out carries rung 0 whatever the reader thought of the writing, because there is then nothing
left for a reader to have been right or wrong about.
The criteria themselves come from the rubric: a model compiles each sentence of it into a Python
expression over a closed set of facts about the submission (work_len, urls, code_blocks,
words, lines, sentences, digits, has_url, plus the task's own fields), and those
expressions are evaluated inside gl.vm.spawn_sandbox with a restricted builtins map. Anything
containing __ or import is dropped before it gets there, along with anything over 400
characters, and no more than 24 checks survive. A criterion the model could not compile is not
silently satisfied: it simply goes back to the reader as prose.
Models cannot count characters and cannot be trusted to notice a missing link. They are good at writing the code that checks both. So that is what they are asked for.
When no sandbox is available the contract does not fall back to unsandboxed eval. It marks
the round [no deterministic pass] so the degradation is visible on chain, and the whole rubric
falls to the reader.
This is the property the whole design turns on, and an earlier version of this contract did not have it. That version compared the reader's score with a tolerance of ten points, which is a sensible thing to do to a noisy measurement and a fatal thing to do to this number: two validators could each accept the other's 82 and 74, and the same field of entries ranked under those two readings can hand the prize to two different solvers. A tolerance on a value that decides an outcome is not tolerance for noise. It is an outcome left undecided.
So there is no tolerance anywhere in the contract. Exactly one judgment crosses from a model into storage — the rung — and it is compared exactly. Everything that decides an outcome is a function of that rung and of facts code established:
score = max(0, tier * 20 - 15 * preferred_criteria_missed)
and 0 outright if code disqualified the entry
eligible = passed and not disqualified
winner = highest score, then earliest filing, then lowest entry number
appeal won = the rung moved, or the gate flipped, or the disqualification changed
The ranking is a total order, so nothing is left to luck either: two entries on the same score are separated by when they were filed, and two filed in the same second by their entry number.
The consequence is testable, so it is tested. Thirteen readings of one strong entry, spread over the
seven field names a model reaches for (merit, tier, rung, grade, score, rating, value)
— the rung as 4, "4", "4.0", "4/5", "strong", "Strong.", " 4 " and 4 again under
a second name, and the hundred-point scale the contract never asked about as 80, 99, "88/100",
"85%" and 84 — are every one of them accepted by a validator that read the work as
strong, and every one of them:
- stores the same grade — same rung, same label, same score, same gate, same violation count, same disqualification flag;
- leaves the same contest — same winner, same winning score, both claimable balances to the atto, both outcomes; in either filing order, and with a tie broken the same way whichever order the entries were graded in;
- cannot win an appeal by rewording — thirteen appeals against one grade, thirteen bonds into the slash pool, the score unmoved at 80.
A reading on a different rung is not accepted at all: 3 and 5 both disagree, and the round is
thrown away and re-run against a fresh set rather than settled by whichever validator happened to be
leading. A leader reporting a score that does not follow from the rung it claims — 81 where
the arithmetic says 80 — is refused by every honest validator, and so is one that reports the
right score under the wrong rung.
The reader is also allowed to be imprecise without being wrong. A rung named rather than numbered,
written as a fraction, or answered on the hundred-point scale is quantised onto the ladder before it
is compared, which is why all thirteen wordings above land on the same rung instead of failing the
round. judged_score still records what the reader reached for; it has no effect on any outcome.
Every entry is compared against every earlier entry on the same task, inside the same sandbox pass, and the comparison is lexical token overlap after stopword removal — a percentage, computed with integer arithmetic.
That is a deliberate choice over embeddings. An embedding model is floating point and
hardware-sensitive, so two honest validators could disagree about whether an entry is a copy, and
consensus would fail on the one question that decides who gets paid. A token overlap of 89% is
89% on every machine, which makes a duplicate a shared fact rather than an opinion, and lets
duplicate_of be a field validators must agree on exactly.
Order of filing decides, not order of grading. The original is never the copy, even when the copy is graded first.
award() calls no model at all. It ranks the eligible entries by the score consensus already
settled, breaks ties by who filed first, moves the prize, returns the bonds, slashes the bonds of
disqualified entries, and writes the win into a reputation record. It is permissionless: a sponsor
who dislikes the outcome cannot refuse to settle, and after the deadline anyone can call it.
A task nobody entered, or one where every entry was disqualified or failed the gate, is voided and the prize returns to the sponsor untouched.
Grading is permissionless for the same reason: a sponsor cannot bury an entry by declining to look at it, and a solver can always have their own work read.
The arena below is on studionet and everything in it was produced by the live validator set. No answer in this section was mocked, and the only value ever pinned was the clock, once, so a contest could reach its own deadline in a single call.
contract 0x8f404B5BD6B940109473D32c93910835eD9C2086
explorer https://explorer-studio.genlayer.com/address/0x8f404B5BD6B940109473D32c93910835eD9C2086
terms entry bond 5 GEN | appeal bond 20 GEN | appeal window 6 h
merit ladder 6 rungs x 20 points | soft deduction 15 | duplicate threshold 80 %
state 3 tasks | 4 entries | 2 solvers | 1 awarded, 2 open
250 GEN in prize escrow | 10 GEN in bonds | 25 GEN in the slash pool | 0 unclaimed
board https://abstrusimad.github.io/proving-ground/dashboard/
The arithmetic overruled a reader that had already been kind to an entry. A launch note 846
characters long went in against a rubric whose first line was a 600-character limit. The reader
placed it on rung 3, adequate, worth 60 points, and wrote why: "a lengthy preamble that spends
too much time on backstory rather than quickly telling a developer what the API does". The sandbox
had already measured work_len <= 600 as VIOLATED, so the entry lost the rung with the points:
read as 3/5 'adequate' (60 points), thrown out by code: rung 0/5 'fails the brief', scored 0
reason: required criteria not met, established deterministically: note must be at most 600 characters
Then the appeal proved the property this contract exists to have. The solver staked 20 GEN and
argued the limit should not outweigh an otherwise complete note. Round 2 went out to a rotated
validator set, which compiled the rubric on its own and phrased it differently — urls >= 1
and code_blocks >= 2 where round 1 had written has_url and 'https' in work and
code_blocks >= 1 — read the work more warmly in prose ("clearly explains what the
Settlement API does [...] understandable to a developer seeing the product for the first time"),
and landed on the same rung: 3, adequate, again zeroed by the same measured violation. A
different set, different compiled expressions, a different rationale, and an outcome identical to
the atto. The appeal bond was not returned: it went to the slash pool, where it funds the appeals
that do move a rung.
Then the contest settled itself. award ran with no model in it: the entry read as strong
took the 200 GEN prize at a score of 80, its 5 GEN bond came back, the disqualified entry's bond was
slashed, and the winner withdrew 205 GEN for real. The losing solver's record now carries a
disqualification, which is a public number and nothing more: no reputation ever reaches a grader, so
a well regarded solver and a newcomer are read exactly the same way.
A second contest caught a copy the reader was happy with. Task 2 asked for a migration guide
off the v0 payouts endpoint. The first entry went in at rung 4, strong, worth 80. The second solver
submitted the same guide with every sentence reworded — took a batch for accepted a batch,
polling loop for poll loop — and the reader liked it exactly as much, placing it on rung 4
too and writing that it "delivers a clear migration guide [...] usable as-is for someone performing
the migration". Every compiled check passed. What stopped it was measurement: the sandbox compared
it against the entries already stored on this task and returned 86 % overlap, over the 80 %
threshold, so the entry was disqualified at zero with duplicate of submission #2 (86% overlap). The
corpus a copy is measured against is contract storage, so it grows with every entry and no model is
asked whether two texts are the same.
The scorecard board is rendered from
exactly this state, one card per entry, showing the rung the validators agreed on, the points it was
worth, what the contract settled, and every compiled check behind it. It is built from
activity.json, which is nothing but the contract's own views read back, and the page it produces
makes no network calls at all.
grade(submission_id)
|
|-- materialise from storage: the task, the entry, every earlier entry on the task
|
+-- gl.vm.run_nondet_unsafe(leader_fn, validator_fn)
|
| leader_fn, and independently every validator's own run:
|
|-- 1. fetch the work URL, if the entry gave one [web]
|-- 2. compile the rubric into Python expressions [llm]
|-- 3. one sandbox pass: [deterministic]
| evaluate every expression against the work
| measure overlap against every earlier entry
|-- 4. read the work and place it on a rung, with the [llm]
| sandbox result handed over as ground truth
| it is told it may not contradict
+-- 5. arithmetic: points, deductions, gate, duplicate [deterministic]
|
+-- validator_fn compares its own run against the leader's,
field by field, with no tolerance on any of them
Nothing in step 3 or step 5 depends on a model agreeing with itself, and step 4 is the only place a judgement of quality enters — where it produces one small integer. A validator does not check that the leader produced well-formed JSON; it performs the whole pipeline again, on its own fetch of the work and its own sandbox pass, and then compares.
What has to match, and what does not:
| field | agreement required | why |
|---|---|---|
tier |
exact | the one judgment the model makes, and the thing every outcome follows from |
passed |
exact | it is the eligibility gate: binary, and it decides who can win |
disqualified |
exact | code decided it; two honest runs cannot differ |
duplicate_of |
exact | a reproducible integer, measured not guessed |
violations |
exact | a count code produced, deducted at a fixed rate |
score |
exact | a function of the fields above, checked anyway: it is the number that pays |
judged_score |
none | derived from the rung, recorded, and used by nothing |
rationale |
none | wording is never compared, at all |
There is no field in that table a validator is asked to be approximately right about. A validator
whose own run placed the work one rung away from the leader's does not agree a little: it disagrees,
the round is discarded and the entry is graded again by a fresh set. A leader that hides a
disqualification, invents one, invents a duplicate_of, flips passed, or reports a score its own
claimed rung does not produce is rejected by every honest validator, because each of them recomputed
all of it from code and from its own reading rather than reading it off the leader's answer.
The rule in one line: the arena tolerates any wording of a reading and no disagreement about the reading itself. Nothing a validator will sign changes who wins or what they are paid.
Failure paths need consensus too, or a leader could escape a grade it did not like by failing. Errors are prefixed, and the validator compares them by class:
| prefix | example | how validators compare |
|---|---|---|
[EXPECTED] |
This entry was already graded |
deterministic: messages must be identical |
[EXTERNAL] |
Work URL returned 404 |
deterministic: identical, so 404 and 403 disagree |
[TRANSIENT] |
Work host unavailable (503) |
agree if both sides failed transiently |
[LLM_ERROR] |
No merit rung in the response |
never consensus: always disagree, rotate the set |
A leader claiming the work host was down when the validator can reach it disagrees, and the entry is graded by a set that can read the work.
post_task -----> submit -----> grade -----> [ appeal ] -----> award -----> withdraw
prize into bond into consensus one regrade no model pull, never
escrow escrow agrees on a under a bond runs here push
rung, exactly
| method | who may call it | value | what it does |
|---|---|---|---|
post_task(title, spec, rubric, closes_in_s) |
anyone | the prize | opens a task; the prize is escrowed until it settles |
submit(task_id, work, work_url) |
anyone but the sponsor, once per task | the bond | enters work, or an https URL the graders fetch themselves |
grade(submission_id) |
anyone | — | settles a score under consensus |
appeal(submission_id, argument) |
anyone, once per grade, inside the window | the appeal bond | forces exactly one regrade |
award(task_id) |
anyone, after close and after every appeal window | — | pays the winner, or voids the task |
withdraw() |
anyone owed | — | collects prizes, refunded bonds and bounties |
| method | returns |
|---|---|
get_arena() |
the founding terms, the four balances, and the counts |
get_task(id) / get_tasks() |
one task in full, or the whole board without the prose |
get_submission(id) / get_submissions(task_id) |
an entry, with its outcome and its appeal state |
get_grades(id) |
every round on an entry: judged score, settled score, compiled checks, violations, similarity, work digest |
get_reputation(address) |
entries, grades, wins, disqualifications, best and average score, GEN earned |
get_leaderboard() |
the field, ranked deterministically so two readers agree |
get_claimable(address) |
what the arena owes one address |
get_grades is the part a solver argues against: the compiled expression that failed, the reason
sentence code wrote, and a digest of the exact bytes that were read.
A grade can be appealed once, by anyone, not only the solver. Spotting an overrated entry is worth as much to the field as defending an underrated one, and the sponsor is not the only party with an interest in a score being right.
The regrade runs the whole pipeline again under consensus. Then:
- the reading moved — a different rung, or the gate flipped, or the disqualification changed — the appellant gets the bond back plus a bounty out of the pool;
- the reading held — the bond goes into that pool.
There is no threshold in that test, which is what makes the appeal market safe to leave open to anyone. A second set that reads the work the same way has not moved anything, however differently it words its rationale, so an appeal cannot be farmed by paying for rewordings of a reading that already stands: thirteen of them are filed in one test, and all thirteen bonds are slashed.
Successful oversight is funded by unsuccessful oversight, never out of the prize. The prize a sponsor escrowed is the prize the winner receives, whatever happens in the appeal market around it. An appeal filed after the window, or a second appeal on the same grade, is refused on chain.
An arena may be opened with the address of an EVM contract on the Elastic Chain side. When a task
is awarded, the contract calls onAward(uint256 taskId, address winner, uint256 score) on it, fire
and forget: the award is already final on the GenLayer side, so a failure over there must not be
able to unwind it, and the registry can always re-read get_task().
That is what makes the reputation record more than a leaderboard. It is written by consensus, readable synchronously by any other GenLayer contract, and it cannot be written by hand — so a hiring contract, a delegation vault or a higher-stake arena can gate on it.
Four balances, and one invariant the test suite asserts after every operation:
prize_escrow + bond_escrow + slash_pool + claimable_total == everything ever paid in
- prize escrow — prizes of open tasks. Leaves only at
award, to the winner or back to the sponsor. - bond escrow — entry bonds. Refunded at
award, or slashed into the pool if the entry was disqualified. - slash pool — slashed entry bonds and lost appeal bonds. The only source of appeal bounties.
- claimable — what the contract owes named addresses.
Withdrawals are pull, never push: the balance is zeroed before the transfer is emitted, and one
claimant cannot reach another's. All value is atto-scale u256, and every balance crosses the ABI
as a decimal string because atto values overflow a JSON number.
tests/direct/ 118 tests, in-memory GenVM, ~10s
tests/integration/ 11 tests against studionet, real validators, real consensus
Direct mode runs the contract inside a local GenVM with the LLM and the web pinned, which is what
makes 118 tests take ten seconds. Two things are unstubbed on purpose: gl.vm.spawn_sandbox is
executed for real, because stubbing it would hide both the compiled checks and the duplicate
detection — the two things the design rests on — and the EVM bridge is recorded, so a
test can assert that an award really crossed the boundary, and only when a task was actually won.
The suite is organised by claim, not by method:
| section | what it pins down |
|---|---|
| founding, posting, entering | every argument that must be refused, by its exact revert message |
| grading | the pipeline, the digest, the fetched URL, the degraded path |
| the arithmetic the model does not get to argue with | strong and "passed" on an over-length entry still settles as rung 0, score 0, disqualified |
| copying, caught by code | similarity measured in the sandbox; the original is never the copy; a rule break is not filed as a copy |
| what validators have to agree on | every field that must match exactly, and a rung away rejected rather than averaged |
| no reading a validator accepts can change who is paid | thirteen wordings of one reading, one grade, one winner, one payout — in both filing orders, and unappealable by rewording |
| failures need consensus too | the four error classes, including a leader that claims a host is down |
| the appeal market | the bounty, the slash, the window, the one-appeal rule |
| awarding | ranking, ties by filing time, voiding, and every premature call |
| getting paid | pull payments, zeroed balances, and what a stranger is owed |
| the money adds up | the invariant above, asserted after every step of a full contest |
The consensus tests reach into the captured validator function and hand it a doctored leader result, which is how a single-process test can prove what a dishonest leader would fail on:
leader_result, _leader_fn, validator_fn = direct_vm._captured_validators[-1]
doctored = dict(leader_result)
doctored.update({"passed": False}) # the leader hides the pass
assert validator_fn(gl.vm.Return(doctored)) is FalseThe on-chain suite is the same claims where they cost real time: real validators, mocked only in what they are told the model and the web said, so the sandbox, the arithmetic, the fetch, the escrow and the revert paths all execute inside GenVM under consensus.
pip install -r requirements.txt
genvm-lint check contracts/proving_ground.py # pinned runner, storage, decorators
gltest tests/direct/ -q # everything, in memory
gltest tests/integration/ -v -s # the quick on-chain checks
gltest tests/integration/ -m slow -v -s # the long consensus scenarioscp .env.example .env # put a funded studionet key in private_key_wallet
python scripts/deploy_studionet.py # deploys, verifies, writes deployment.json
python scripts/seed_activity.py # posts real tasks and settles one for real
python scripts/snapshot.py # reads the arena back into activity.json
python scripts/build_dashboard.py # renders dashboard/index.html from that snapshotThe founding terms live at the top of scripts/deploy_studionet.py: the arena name, the entry
bond, the appeal bond, the appeal window, and the EVM registry address (empty for none). They are
constructor arguments, fixed at deployment: an arena cannot quietly raise the cost of appealing it
after the fact.
scripts/seed_activity.py is not a demo harness. It posts real tasks with real prizes, enters real
work, and grades it against the live validator set with no mocked answers anywhere. The one thing
it pins is the clock, and only for award, because a contest cannot settle until submissions have
closed and every grade has outlived its appeal window; Studio accepts genvm_datetime per
transaction, so the run reaches the end of a contest in one call instead of in a day.
The page under dashboard/ is built from activity.json and nothing else: every scorecard on it is
a value read back out of the contract, and the file it produces has no network calls in it at all.
Rebuilding it after new activity is python scripts/snapshot.py && python scripts/build_dashboard.py.
Things that cost time, written down so they do not cost it again.
- The runner is pinned.
py-genlayer:testandlatestare local development aliases and are rejected by every network. The first line of the contract is a version hash. - Storage is declared, not assigned. Class-level annotations create the slots;
__init__only fills them.self.items = []inside__init__creates a Python list that never reaches storage. - Enums are stored as
str, money as atto-scaleu256, and every balance leaves through the ABI as a decimal string. - Nothing that reaches storage is touched inside a nondeterministic block. The task, the entry
and the whole corpus of earlier entries are materialised into plain dicts before
run_nondet_unsafe, so the sandbox never reaches into storage. - A revert reason is not in
stderr. It is inreceipt["consensus_data"]["leader_receipt"][0]["result"], as{"status": "rollback", "payload": "..."}. The integration suite asserts against that payload, which is why a test that expects a refusal cannot pass on an unrelated failure. - A leader that says SUCCESS has not changed anything yet. If the validators refuse to agree
with the round, the transaction applies nothing: the entry is still pending and carries no new
grade, while the leader receipt reads
SUCCESSthroughout. The seeding script therefore checks the state rather than the receipt, and asks again if no round landed — which is not a double spend, because the refused transaction wrote nothing. - Outgoing value executes at FINALIZED, not ACCEPTED. A test that checks a balance after
withdraw()has to wait for the later status or it reads the balance from before the transfer. - Floats are avoided in consensus paths. Similarity is integer arithmetic and deductions are integer subtraction, because two validators on different hardware must reach the same number.
- Studio rate limits at 60 requests a minute and rejects a sender with more than 32 in-flight transactions. Both scripts and the on-chain suite retry those, and only those: an ambiguous gateway failure on a transaction submission is raised rather than retried, so nothing here can sign the same transaction twice.
contracts/proving_ground.py the contract, 1457 lines, pure ASCII
scripts/deploy_studionet.py deploy, verify, record the address
scripts/seed_activity.py post, enter, grade, appeal, award, withdraw for real
scripts/_rpc.py retry the gateway, never the transaction
scripts/snapshot.py read the arena back into activity.json
scripts/build_dashboard.py render that snapshot into a self-contained page
dashboard/ the scorecard page: template, and the built copy
tests/direct/ 118 tests, in-memory GenVM
tests/integration/ 11 tests against studionet
deployment.json the deployed address and its founding terms
activity.json a snapshot of what is on chain, as read back
MIT.