Skip to content

feat(m1): verify MILESTONE1.md matches the trinity.m1 code + offline workflow guard - #513

Open
minion1227 wants to merge 1 commit into
James-CUDA:mainfrom
minion1227:minion_m1_conformance
Open

feat(m1): verify MILESTONE1.md matches the trinity.m1 code + offline workflow guard#513
minion1227 wants to merge 1 commit into
James-CUDA:mainfrom
minion1227:minion_m1_conformance

Conversation

@minion1227

Copy link
Copy Markdown
Contributor

The gap

The Milestone-1 drop (feat(m1): …) added the miner-facing contract in docs/MILESTONE1.md — the < 1,000,000 param budget, 1/day rate, ≥ 0.02 win margin, the pack layout, and the 0.7·domain + 0.3·difficulty composite. Those same numbers live independently in trinity/m1/constants.py and the pack/metrics code, and nothing checks the two agree. A doc edit or a constant change could split them silently — and a miner who follows the document would then build a pack the validator rejects (or trains against the wrong composite).

What's here

src/trinity/m1/conformance.py parses MILESTONE1.md and asserts it against the code:

Check Doc Code
param_budget < 1,000,000 constants.MAX_HEAD_PARAMS
rate_limit 1 … per day RATE_LIMIT_MAX_SUBMISSIONS / RATE_LIMIT_WINDOW_DAYS
win_margin ≥ 0.02 constants.WIN_MARGIN
composite_weights 0.7·domain + 0.3·difficulty constants.DOMAIN_WEIGHT
pack_layout_* files under submissions/<miner>/m1/ what save_milestone1_pack writes

All pass on the shipped doc today — this is a guard, not a bug fix, and I'd rather say so than dress it up. Its value is prospective and it's cheap to keep: test_a_changed_param_budget_in_the_doc_is_caught, …_win_margin_…, …_composite_weight_… and …_phantom_pack_file_… show a real drift is caught rather than the guard being vacuous.

One documentation gap it surfaces

save_milestone1_pack can write W_k.npy (attentive packs with a key projection), but the doc's Pack-layout block lists only config.json / W_domain.npy / W_diff.npy / attention_query.npy. That's a file the code emits that the document omits. It's reported as a note, not a failure — an undocumented-but-real optional file is a doc gap, not a code defect. If you'd like the doc to list it, that's a one-line follow-up; this PR just makes the omission visible instead of letting it hide.

Also guards the offline workflow

Beyond the doc-parity checks, the tests exercise the pack → preflight path MILESTONE1.md prescribes (steps 2–3), which had no end-to-end coverage:

  • a pack round-trips through disk, and the default 5-domain penultimate pack is the documented ≈ 10,240 params
  • it clears every offline gate (run_m1_gates)
  • the < 1,000,000 boundary is exact — 999,990 accepted, 1,000,000 rejected
  • the ≥ 0.02 king/challenger decision matches decide_m1_winner (no-king, exactly-at-margin, a hair under)
  • CODE_PACK_FILES cannot drift from the packer: a test saves a required-only pack and an attentive+key pack and asserts the file set on disk equals the declared list

Scope

Detection only. No M1 behaviour, constant, or doc is changed. m1/ and the milestone scripts are untouched by every other open PR.

Verification

  • pytest tests/test_m1_conformance.py -q25 passed
  • existing test_m1_gates / test_m1_leaderboard / test_m1_metrics13 passed, unchanged
  • mypy on the new module → clean; the new files are ruff-clean
  • import trinity.m1.conformance leaves torch out of sys.modules

Note on CI: main is currently ruff-red independent of this PRsrc/trinity/m1/metrics.py:6 has an unused from typing import Iterable that shipped with the M1 commit, so ruff check src/ exits 1 before pytest runs. That is a one-line fix in a file this PR does not touch; every open PR is red on it until it lands. (Flagged separately.)

🤖 Generated with Claude Code

…workflow guard

MILESTONE1.md is the miner-facing contract: the < 1,000,000 param budget,
1/day rate, >= 0.02 win margin, pack layout, and 0.7*domain + 0.3*difficulty
composite. Those numbers also live independently in trinity.m1.constants and
the pack/metrics code, and nothing checked the two agree. A doc edit or a
constant change could split them, and a miner following the document would
then build a pack the validator rejects.

trinity/m1/conformance.py parses MILESTONE1.md and asserts:
- rules: documented limits == constants.MAX_HEAD_PARAMS / RATE_LIMIT_* / WIN_MARGIN
- scoring: documented composite weights == constants.DOMAIN_WEIGHT
- pack layout: every documented file is one the packer writes; every required
  file is documented. A file the packer CAN write but the doc omits is reported
  as a note, not a failure -- W_k.npy (attentive key projection) is exactly that
  today, a documentation gap surfaced rather than silently accepted.

tests also cover the offline pack -> preflight workflow the doc prescribes
(steps 2-3): pack round-trips through disk, the default 5-domain pack is the
documented ~10,240 params, it clears every offline gate, the < 1,000,000
boundary is exact (999,990 ok / 1,000,000 rejected), and the >= 0.02 king
margin decision matches decide_m1_winner. CODE_PACK_FILES cannot drift from the
packer -- a test saves required + attentive packs and asserts the file set.

Pure stdlib, no torch/network; import leaves torch out of sys.modules.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant