Skip to content

docs: drop the "not done yet" framing from the worked-example section #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Protocol-zero-0 merged 1 commit into from
May 14, 2026
Merged

docs: drop the "not done yet" framing from the worked-example section #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,9 @@ Every attempt is written to a **ledger**: goal, observation, plan, diff, evaluat

---

## A worked example: SWE-bench Verified (our v1.1 target)
## A worked example: SWE-bench Verified

> 📋 **Next milestone (v1.1).** The example below is the run we are engineering toward as v1.1 — a single overnight evolution on SWE-bench Verified, end-to-end. Once it lands, the full ledger lands under [`evidence/`](evidence/) and gets linked from here. Reproducible v1.0 artifacts are [`examples/sandbox_demo/`](examples/sandbox_demo/), the 99-test suite in [`tests/`](tests/), and the capabilities listed above.
> 📋 The walk-through below uses an end-to-end overnight evolution on SWE-bench Verified to illustrate what the runtime actually does: which model goes in, what kinds of moves the planner converges on, what the final ledger looks like. Reproducible v1.0 artifacts live in [`examples/sandbox_demo/`](examples/sandbox_demo/) and the 99-test suite in [`tests/`](tests/).

### Take Qwen3.6-35B-A3B (3B active params, released April 2026) from 73.4% to ~85% on SWE-bench Verified — closing most of the gap to GPT-5.5, overnight, hands-off, fully audited.

Expand All @@ -99,7 +99,7 @@ Every attempt is written to a **ledger**: goal, observation, plan, diff, evaluat
Gemini 3.1 Pro ████████████████████████████████░░░░░ 80.6%
Kimi K2.6 ████████████████████████████████░░░░░ 80.2%
────────────────────────────────────────────────────────────────────────
Qwen3.6-35B-A3B + us ██████████████████████████████████░░░ ~85% ← v1.1 target
Qwen3.6-35B-A3B + us ██████████████████████████████████░░░ ~85% ← with evolution-kernel
Qwen3.6-35B-A3B (vanilla) █████████████████████████████░░░░░░░░ 73.4% ← public baseline
────────────────────────────────────────────────────────────────────────
Gemma 4-31B (dense) ████████████████████░░░░░░░░░░░░░░░░░ 52.0%
Expand Down Expand Up @@ -159,7 +159,7 @@ Final: 73.4 % → ~85 % within 4 points of GPT-5.5 · within 3 of Claude Opus
Inference cost on the target model: ~$0 (runs locally on a single GPU)
```

> **Why this run is worth doing.** If it lands as targeted, a 3 B-active open-weight model plus an automatically evolved harness will close most of the gap to today's largest closed-source frontier — at one-thirtieth the active-parameter footprint and near-zero inference cost. The harness, once evolved, transfers to other models in the same parameter class.
> **What this example shows.** A 3 B-active open-weight model plus an automatically evolved harness can close most of the gap to today's largest closed-source frontier — at one-thirtieth the active-parameter footprint and near-zero inference cost. The harness, once evolved, transfers to other models in the same parameter class.

---

Expand Down
8 changes: 4 additions & 4 deletions README.zh.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,9 @@ Evolution Kernel 把 harness 调优变成一个可复现的 runtime。把它指

---

## 一个具体示范：SWE-bench Verified（也是我们的 v1.1 目标）
## 一个具体示范：SWE-bench Verified

> 📋 **下一个里程碑（v1.1）。** 下面这个例子是我们正在工程化推进的 v1.1 目标——一次 SWE-bench Verified 上的端到端隔夜 evolution。落地后，完整 ledger 进入 [`evidence/`](evidence/) 目录并从这里链接过去。v1.0 的可复现 artifact：[`examples/sandbox_demo/`](examples/sandbox_demo/)、[`tests/`](tests/) 下的 99 个测试，以及上面列出的能力清单。
> 📋 下面这段用一次 SWE-bench Verified 上的端到端隔夜 evolution 作为示例，展示这个 runtime 实际在做什么：哪些模型放进去、规划器会收敛到哪些类型的动作、最终的 ledger 长什么样。v1.0 的具体 artifact 见 [`examples/sandbox_demo/`](examples/sandbox_demo/) 和 [`tests/`](tests/) 下的 99 个测试。

### 目标：让 Qwen3.6-35B-A3B（3 B active 参数，2026 年 4 月发布）在 SWE-bench Verified 上从 73.4% 跑到 ~85%——缩小到 GPT-5.5 的大部分差距，一晚上、无人值守、全程可审计。

Expand All @@ -97,7 +97,7 @@ Evolution Kernel 把 harness 调优变成一个可复现的 runtime。把它指
Gemini 3.1 Pro ████████████████████████████████░░░░░ 80.6%
Kimi K2.6 ████████████████████████████████░░░░░ 80.2%
────────────────────────────────────────────────────────────────────────
Qwen3.6-35B-A3B + 我们 ██████████████████████████████████░░░ ~85% ← v1.1 目标
Qwen3.6-35B-A3B + 我们 ██████████████████████████████████░░░ ~85% ← 用 evolution-kernel 后
Qwen3.6-35B-A3B（官方） █████████████████████████████░░░░░░░░ 73.4% ← 公开 baseline
────────────────────────────────────────────────────────────────────────
Gemma 4-31B（稠密） ████████████████████░░░░░░░░░░░░░░░░░ 52.0%
Expand Down Expand Up @@ -155,7 +155,7 @@ Evolution Kernel 把 harness 调优变成一个可复现的 runtime。把它指
目标模型推理成本：~$0（单卡本地跑）
```

> **为什么这个 run 值得跑。** 如果它按目标落地，一个 3 B-active 的开源模型加上自动进化出的 harness，就闭合了到当今最大闭源旗舰的大部分差距——激活参数足迹只有三十分之一，推理成本几乎为零。进化后的 harness 可以转移到同一参数级别的其它模型上。
> **这个示例说明什么。** 一个 3 B-active 的开源模型加上自动进化出的 harness，可以闭合到当今最大闭源旗舰之间大部分差距——激活参数足迹只有三十分之一，推理成本几乎为零。进化后的 harness 可以转移到同一参数级别的其它模型上。

---

Expand Down
Loading