Skip to content

[RL][Phase 4] Verify C++ and Python training parity #1076

Description

@utilForever

Parent: #1056
Epic: #1052

Goal

Detect drift between the native and Python training frontends while keeping one model implementation.

Scope

  • Use one fixed CPU model seed and mini-batch
  • Compare forward outputs, component losses, gradients, and one simple optimizer step
  • Define explicit numerical tolerances
  • Test tiny-batch overfitting in both frontends

Non-goals

  • Bitwise equality across GPUs or platforms
  • Optimizer-state interchange
  • Benchmarking frontend speed

Acceptance criteria

  • Forward outputs, losses, and gradients match within documented CPU tolerances.
  • One equivalent optimizer step produces matching weights within tolerance.
  • Both frontends overfit the fixed batch to the agreed threshold.
  • The parity test reports the first mismatched tensor clearly.

Verification

Run the cross-runtime parity test from a clean RL-enabled build.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-rlCategory: Search, self-play, reinforcement learning, and training.P-importantPriority: Other work depends on this, or it is low-level and critical.T-featureType: New capability or supported behavior.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions