Skip to content

[P2] Reward function + rollout MDP #14

Description

@duckyquang

Phase: P2
Owner(s): Davin


Goal: Define the RL reward and rollout interface Quang's GRPO imports.

Where: marc/train/reward.py, marc/train/rollout.py, tests/test_reward.py.

Parallel track: Day 1 of P2. Build + test with fake trajectories before GRPO trainer exists.

  1. Implement compute_reward(trajectory, G, checker, cas) → float.

  2. Implement run_rollout(policy, G, K) → trajectory wrapping solve steps.

  3. Unit test: fake trajectory with energy decreasing → positive shaping reward.

  4. Unit test: checker accept → terminal reward fires.

  5. Document API in module docstring for Quang.

Done when: pytest tests/test_reward.py passes; Quang imports without changes.

Push to: PR davin/p2-rewardmain (merge before or same day as GRPO).


Auto-created from the MARC task board (index.html). Update the board if scope changes.

Metadata

Metadata

Assignees

Labels

P2 · checker RLPhase 2 — checker fine-tuning / GRPOmath-heavyMath-heavy modeling taskparallel-trackCan start in parallel — no blocking wait

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions