Skip to content

[baseball] make the AI's off-plate pitches actually miss the zone - #397

Draft
gazure wants to merge 1 commit into
mainfrom
worktree-fix-flaky-pitcher-test
Draft

[baseball] make the AI's off-plate pitches actually miss the zone#397
gazure wants to merge 1 commit into
mainfrom
worktree-fix-flaky-pitcher-test

Conversation

@gazure

@gazure gazure commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Problem

the_ai_pitcher_throws_strikes_and_balls_in_a_sane_mix is flaky on main — it failed in roughly 1 run in 4 (measured 3/12, then 2/12 across 24 runs), always by overshooting the upper bound (76–79% vs. the asserted 75% ceiling).

It is not just unlucky sampling. ai_pitch_plan aims in the zone 60% of the time and "just off the plate" the rest, but the off-plate branch sampled a rectangle spanning x ±1.55 by y 1.05..4.05 — which fully contains the strike zone (x ±0.83, y 1.6..3.4). The zone is ~32% of that rectangle's area, so:

in-zone rate = 0.60 + 0.40 × 0.32 ≈ 0.73

The AI was throwing ~73% strikes when the mix intends 60%, landing right on the test's ceiling — hence a coin-flip failure rather than a clean pass or a clean fail.

Fix

Aim past one chosen edge (inside/outside/high/low) instead of sampling a rectangle around the plate, so a pitch meant to miss always misses. All four cases stay within AIM_LIMIT_*, so the_ai_pitcher_never_aims_somewhere_unreachable still holds. HBP_X is -1.75, beyond AIM_LIMIT_X of 1.7, so hit-by-pitch was never reachable by aim and is unaffected.

Also adds a_pitch_meant_to_miss_actually_misses, which pins the invariant directly rather than leaving a statistical band as the only thing standing between this bug and CI.

Verification

  • Strike rate back to ~60%, comfortably inside the asserted 0.45..=0.75
  • Baseball suite 30/30 runs green (was ~25% failure)
  • cargo +nightly fmt --all -- --check clean; CI clippy reports no errors
  • 99 baseball + 26 twotris tests passing

🤖 Generated with Claude Code

`ai_pitch_plan` aims in the zone 60% of the time and "just off the plate"
the rest. The off-plate branch sampled a rectangle spanning x ±1.55 by
y 1.05..4.05, which fully contains the strike zone — so about a third of
those intended misses finished over the plate for a called strike, putting
the real in-zone rate near 73% rather than the intended 60%.

That sat right on the 75% ceiling asserted by
`the_ai_pitcher_throws_strikes_and_balls_in_a_sane_mix`, which failed in
roughly 1 run in 4 (measured: 3/12, then 2/12).

Aim past one chosen edge instead, so a pitch meant to miss always misses.
Strike rate returns to ~60% and the test passed 30/30 runs. Adds
`a_pitch_meant_to_miss_actually_misses` to pin the invariant directly
rather than relying on the statistical band to catch it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant