feat(#431): 新增 reward 评分端点 /admin/api/fine-tune/reward/score - #434
Merged
Conversation
…vernance docs Addresses the #1 risk flagged by all three competitor audits (fusion-mlx_vs_omlx / rapid-mlx / llama-cpp): maturity-signal inversion. The code is capable but the outward signals undersold it. - pyproject: classifier 3-Alpha -> 4-Beta (matches v0.8.12 + full-modality) - README: fix stale v0.5.11 badge -> live PyPI badge; fix false 'Tests-1200+' -> 'Tests-377 active' (real: 678 files, 301 quarantined in debt_modules.txt); add CI + stars badges; add scope/maturity statement (macOS-only, single-maintainer, seeking contributors) - RELEASE.md: fix the release flow (bump -> CHANGELOG -> PR -> tag -> release -> publish.yml -> PyPI OIDC + homebrew tap auto-bump; remotes table; CI stall squash --admin guidance) - CONTRIBUTING.md: setup/test/lint/PR flow; 'what we need most' lists test-debt cleanup + tool_calling parsers + benchmark data to directly counter bus-factor=1; fail-visible rule restated - ROADMAP.md: strategic moats + phased plan (Phase 0-3 from fusion-mlx-enhance.md) + explicit out-of-scope (no cross-platform, no self-built GGUF quant, no MXFP8 train) + live model matrix stub Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…5 items) Phase-1 test-debt paydown (P1-3). A scan of the 262 quarantined entries in tests/unit/debt_modules.txt found 15 files that already pass 100% green on a full run — their prod code was fixed since the Rapid-MLX migration but their debt lines were never removed (stale debt). Reactivated (360 passed, 3 skipped, 0 failed on full run): test_embedding (89), test_hot_cache (41), test_mcp_config (39), test_mcp_manager (31), test_smart_router (25), test_model_aliases (24), test_grammar (24), test_reranker_causal_lm (23), test_capabilities_field (7), test_install_detection (14), test_server_auth_ordering (12), test_responses_chat_template_kwargs (15), test_alias_recommended_sampling (9), test_cli_info (4), test_models_command_layout (3) - debt_modules.txt: 262 -> 247 excluded; header count comment corrected to reality (was stale '269/326'); Phase-1 rescue note added - README: Tests badge 377 active -> 431 files | 8742 items (truthful per the fail-visible rule; old '1200+' claim was already removed in Phase 0) Active test items collected: 8742 -> 9105 (+363, zero new code). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Audit R5 "tool_parser 矩阵落后 4×" premise was outdated (v0.8.10): fusion already ships fusion_mlx/tool_parsers/ with 21 parsers + delegates to mlx-lm 0.31.3 native 10 families. Real defects were boundary-translation bugs, fixed here: - anthropic_adapter: translate UI-TARS `point`→`coordinate` (R6-M2) on /v1/messages when tool name == "computer"; was passing raw `point` through. - responses_adapter: _is_computer_use_tool only accepted dict, not pydantic ResponsesTool — request_uses_computer_use returned False post-validation, so /v1/responses never emitted `computer_call` items. - aliases.json: add 9 ui-tars-* aliases (1.5-7b 4/6/8bit, 7b dpo/sft, 72b-dpo-4bit) the lane-parity tests expected. - test_ui_tars_lane_parity: re-point R10-C2 source inspection from the routes_internal/chat shim to api.openai_routes where the fast-path SSE helper migrated in the r10-B refactor. ui_tars suite: 24 fail → 6 fail (+18). Remaining 6 are pre-existing TestLaneInjectionParity AST checks for un-wired maybe_inject_ui_tars_system_prompt (dead code, never called) — separate lane-completeness task. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the "near-term goal" stub with a real matrix derived from aliases.json (81 aliases, ~24 families) + vendored patches/ inventory + mlx-lm 0.31.3 (119 arches) / mlx-vlm 0.5.0 (~60 arches) upstream sets. Status legend (deterministic, no model loading): - ✅ Tested: has a fusion alias, covered by CI - 🟡 Custom patch: runs via fusion_mlx/patches/ (cutting-edge arch) - 🟦 Upstream: mlx-lm/mlx-vlm native, runs but no alias (unverified) - ❌ No: GGUF (rejected) or unsupported arch Tables: text LLMs (parser + spec-decode flags), vision LLMs, specialized modalities (embedding/reranker/diffusion/UI-TARS). Mark Phase 0 + P1-3 + P1-4 + P1-5 rows done in the near-term table. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
run_bench.py drives the live server's /v1/chat/completions with a fixed prompt, temperature=0, top_p=1, measuring tok/s + TTFT + wall (incl cold start). README documents method, Ollama/mlx-lm comparison, honest results table. First run: 7 models 0.6B-27B, 1.5-168 tok/s. 3 models skipped (weights not downloaded) reported not hidden. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Import drifted during the rapid-mlx migration: the test imported audio_to_wav_bytes from fusion_mlx.engine.audio_utils (singular) but the canonical module is fusion_mlx.engines.audio_utils (plural), which all production callers (tts.py, sts.py) use. Singular module never exported it -> EOFError on the test's wav roundtrip. Re-pointed the import; 5/5 green. Removed from debt_modules.txt (294->293). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dahai80
force-pushed
the
fix/reward-score-431
branch
from
August 9, 2026 04:17
a820394 to
da3e93c
Compare
闭合 RLSL Phase1->Phase2 循环:
- POST /admin/api/fine-tune/reward/score
请求 {model_id, adapter_name, prompt, completions:[str]}
响应 {rewards:[float], model_id, adapter_name}
- 新增 fusion_mlx/training/reward_score.py: standalone load-and-evict,
加载 reward adapter (LoRA + value head), 对每个 (prompt, completion)
取 last-token hidden -> value_head 投影返回标量 reward。复用
reward.py 的 _ValueHead 与 _score 前向逻辑(非可微推理版)。
- 校验 adapter_config.json reward_model=true, 否则 400。
- fine_tune_route.py 新增路由, 同 logprob 端点的 load-and-evict 模式,
不进推理池。
附带修复 (CI 阻塞): tests/unit/test_ui_tars_lane_parity.py I001
import 排序 (ruff --fix), 否则 ruff check fusion_mlx/ tests/ 在
所有 PR 上 fail。
测试: tests/test_fine_tune_route.py 新增 TestRewardScoreEndpoint
(happy path/缺 model_id/缺 completions/adapter 不存在), 26 passed。
Closes #431
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dahai80
force-pushed
the
fix/reward-score-431
branch
from
August 9, 2026 04:20
da3e93c to
52cb3dc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
闭合 RLSL Phase1 RM (#424) → Phase2 GRPO (#363) 循环的最后一块拼图: reward 评分端点。
端点
加载 Phase 1 训练的 reward adapter (LoRA + value head), 对每个
(prompt, completion)取 last-token hidden → value_head 投影返回标量 reward。len(rewards) == len(completions)。实现
fusion_mlx/training/reward_score.py— standalone load-and-evict 模块, 复用reward.py的_ValueHead与_score前向逻辑 (非可微推理版)。校验adapter_config.json的reward_model=true, 否则 400。fine_tune_route.py新增路由, 同logprob端点的 load-and-evict 模式, 不进推理池, 避免流冲突 (bug(training): GRPO 训练作业失败 There is no Stream(gpu, 6) in current thread #430)。验证
tests/test_fine_tune_route.py新增TestRewardScoreEndpoint(happy path / 缺 model_id / 缺 completions / adapter 不存在)pytest tests/test_fine_tune_route.py— 26 passedruff check新文件 — clean验收对照 (#431)
reward_endpoint传给 GRPO, GRPO 能跑通真实 reward-guided 更新 (待 fusion-trainer 闭环集成测试验证)关联
Closes #431
🤖 Generated with Claude Code