feat(#424): 接入 reward 训练 HTTP 路由与 server 启动装配

[dahai80]

背景

#424 reward-model 训练。PR #427 已落地 reward_service.py + 14 项 route/service 单测,但单测 import 的路由处理函数此前未提交进 fine_tune_route.py,server 启动也未装配 RewardService,导致对外 HTTP 端点缺失。本 PR 补齐这层。

改动

• fusion_mlx/admin/fine_tune_route.py
  • set_reward_context(pool, service=None) / _get_reward_service() / _create_reward_job(request_body) 共享辅助
  • 5 个路由: POST/GET/GET{id}/POST cancel/DELETE on /admin/api/fine-tune/reward/jobs
  • 校验 model_id 与 preference_pairs[{prompt,chosen,rejected}] schema
• fusion_mlx/server.py
  • 启动期实例化 RewardService 单例,经 set_reward_context 注入 engine pool + event loop,镜像 DPO 装配块

端到端验证(真实模型)

Qwen3-0.6B-4bit, 3 个 preference pairs, 4 iters:

status=completed
progress: {iter:4, loss:0.0003, reward_margin:7.98, acc_chosen:1.0}
adapter: adapters.safetensors(5.8MB) + adapter_config.json{reward_model:true}

测试

• pytest tests/ -k reward → 17 passed, 7 skipped
• pytest tests/ -k "reward and route" → 14 passed (test_reward_route.py)
• ruff check / black --check → clean

Closes #424

🤖 Generated with Claude Code