Skip to content

[RL][Phase 5] Define the self-play sample format #1077

Description

@utilForever

Parent: #1057
Epic: #1052

Goal

Define the minimum versioned training sample produced by self-play and consumed by both trainers.

Scope

  • Store observation, legal mask, normalized MCTS visit-policy target, outcome, and schema metadata
  • Define dtype, shape, ordering, and player perspective
  • Provide deterministic serialization and validation
  • Keep the format independent of trainer language

Non-goals

  • A general-purpose dataset framework
  • Long-term migration across arbitrary schemas
  • Storing complete search trees

Acceptance criteria

  • C++ and Python read the same sample values.
  • Policy targets are normalized and assign no mass to illegal actions.
  • Outcomes use the observation player's perspective.
  • Invalid versions, shapes, masks, or non-finite values are rejected.

Verification

Add C++ and Python round-trip tests using one small golden sample.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-rlCategory: Search, self-play, reinforcement learning, and training.P-importantPriority: Other work depends on this, or it is low-level and critical.T-featureType: New capability or supported behavior.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions