Recipe 59: Observation and action selection #59

Recipe 59 of 60 · Level 5 (Expert) · Games & simulation · Choice

Use case

Choose the next permitted observation and subsequent legal action in a partially observed game, then compare complete trajectories with a rule-based baseline.

What to build

The folder recipes/59-observation-and-action-selection/ with notebook.ipynb, README.md, and fixtures/, meeting the recipe contract. Start from python tools/new_recipe.py 59 and the template in recipes/_template/.

Build notes

• In a partially observed game, choose the next permitted observation, then, in a later request, a legal action.
• Python enumerates permitted observations and legal actions.
• Compare complete trajectories with a rule-based baseline over seeded games.

Decision design

• Choice. Options are a fixed set supplied by Python, including the fallback outcome this use case names. Show the choice, the per-option probabilities, and the confidence for individual examples before aggregating.

Level 5 scope

Controlled optimization studies or closed-loop simulations requiring budgets, baselines, and held-out or trajectory evaluation.

Acceptance criteria

• recipes/59-observation-and-action-selection/notebook.ipynb runs top to bottom offline, with no key and no network, and its outputs are committed
• Live mode is opt-in (JEV_COOKBOOK_LIVE=1 with TYPESAFE_API_KEY) and uses the same question definitions as the offline run
• Fixtures are synthetic, include the hard cases named in the build notes, carry gold labels, and pass the fixture validator
• Every response fixture records its provenance, and the notebook states which mode it ran in
• Evaluation: Trajectory outcomes against the baseline over fixed seeds, with zero illegal moves.
• Budgets, seeds, and frozen configurations are recorded in the notebook output
• A baseline is run under the same conditions, and any held-out split is used exactly once after choices are frozen
• Results are reproducible from the committed seeds and fixtures
• No statement about Jev's quality, latency, or cost appears without recorded live inference; in a synthetic run, metrics are labelled as pipeline checks
• recipes/59-observation-and-action-selection/README.md says what the recipe teaches, how to run it, and what was and was not measured live
• The pull request changes only recipes/59-observation-and-action-selection/ and CI is green

Dependencies

• Blocked by Foundation: CI executes every notebook offline and validates fixtures #69 (CI executes every notebook offline and validates fixtures)
• Blocked by Foundation: simulation and state primitives #70 (simulation and state primitives)
• Builds on Recipe 36: Card-game action selection #36 (Card-game action selection). Merge those first and reuse their patterns; do not copy their fixtures.
• Tracked in Epic: Level 5 · Expert recipes #79

Sources

Read these before writing the questions.

• S02 · TypeSafe AI: Primitives (Questions): Choice, Noul, Score, independent questions, and dependent follow-up calls.
• S03 · TypeSafe AI: Confidence: Distribution-derived confidence and task-specific thresholds informed by observed outcomes.
• S04 · TypeSafe AI: How to build with TypeSafe: Narrow semantic judgments integrated with workflows, deterministic logic, and side effects controlled by code.
• S07 · TypeSafe AI: Jev 1.13 jaggedness: Version-specific limitations affecting arithmetic, context selection, adversarial content, and generation.

Out of scope

• Changes to shared code, the catalog, or the root README. If the shared package is missing something, open a foundation issue and link it here.
• Real side effects of any kind. Actions are simulated.