Skip to content

Submit taillred final model - #258

Merged
tmimmanuel merged 4 commits into
mini-router:mainfrom
taillred:sn74-luckydicer
Jul 30, 2026
Merged

Submit taillred final model#258
tmimmanuel merged 4 commits into
mini-router:mainfrom
taillred:sn74-luckydicer

Conversation

@taillred

@taillred taillred commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Miner

  • Branch: sn74-taillred
  • Miner name: taillred

What changed

Submitted the trained coordinator head (best_theta.npy) and training summary (summary.json) from a 24-generation sep-CMA-ES evolution on mmlu and math500 benchmarks using the openrouter model pool.

Benchmark & setup

  • Benchmark: mmlu, math500
  • Provider: openrouter
  • Models config: configs/models.openrouter.yaml

MiniBridge identity used for evaluation

  • owner_id: taillred
  • key_id: taillred

Final model bundle

  • submissions/final_model/best_theta.npy included
  • submissions/final_model/summary.json included
  • python utility/validate_submission.py --dir submissions/final_model passes locally

Evaluation score

  • Local eval score: 75.25
  • Benchmark settings used to obtain it: mmlu, math500, openrouter provider, 32 max items

Caveats or known limits

Theta length (14336) is larger than the current default spec (13312); the validator emits a warning but this is expected for the benchmark configuration used.

@taillred taillred changed the title Submit luckydicer mmlu final model Submit taillred final model Jul 30, 2026
@tmimmanuel tmimmanuel added the submission Final submission bundle label Jul 30, 2026
@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown

Submission PR body check passed

All required submission fields are present.

@taillred
taillred marked this pull request as ready for review July 30, 2026 15:23
@github-actions github-actions Bot added the miner Miner contribution PR label Jul 30, 2026
@tmimmanuel

Copy link
Copy Markdown
Collaborator

MiniRouter evaluation result

Submission for taillred on mmlu, math500 completed with status completed.

field value
submission b50662c0-e259-4639-a51d-d9348f7614f0
PR #258
repo mini-router/minirouter
benchmark mmlu, math500
status completed
phase completed
progress 1/1
score 0.7375 (73.75%)
started 2026-07-30T15:29:14.753423+00:00
finished 2026-07-13T15:54:01.309752+00:00

Metrics

metric value
duration_seconds 1,486.55s
cost_usd $0.0795
cost_calls 80
cost_prompt_tokens 92032
cost_completion_tokens 57292
cost_missing False
TRINITY 1.0
execution_mode local_cpu
local_fallback False

@tmimmanuel
tmimmanuel merged commit 739b3f2 into mini-router:main Jul 30, 2026
10 of 15 checks passed
@tmimmanuel tmimmanuel added the king This pr updated the benchmark score. label Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

king This pr updated the benchmark score. miner Miner contribution PR submission Final submission bundle

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants