M51: Crazy Fruits — dense all-action regression fixes missed 4-in-a-row opportunities (+117% over random, all gates green) - #39
Merged
Conversation
…the 3-vs-4 ranking Root cause (4-agent investigation): the DQN loss regressed only the chosen action's Q, so ranking a 3-match above an available 4-match cost zero loss; the +40/+60/+100 creation bonus was in the training target but in no observation feature (the per-action planes were fire-only). - deterministicValueShaped in the .pg (refill-free cascade + creation weights) + third per-action obs plane: 928 -> 1040 (parity pin unchanged 481681208 - no rule change) - DqnOptions.DenseTargets/DenseTargetWeight: gamma=0-only dense regression; every legal action regressed toward its shaped deterministic value, so a wrong ORDERING finally costs loss; dense mass normalized per supervised entry - web stale-ckpt guard: input width != observation width -> expectimax fallback (replaces the dropped lambda re-rank) - Lab: --dense / --dense-weight / --probe N (opportunity diagnostics) - cf9train (400k, gamma=0 + shaping + dense): FIRST net to pass ALL gates - 5666.4 +/- 155.2 = +117.2% over random (cf5train +54.9%), gap-share 91% (bar 64%), created 9.57 / fired 10.39; probe: random-class behavior (17.6% raw take) -> search-class (35.5% raw / 54.9% opportune ~ e1's 55.7%); combo take 48% -> 66%; shipped to wwwroot, round-over bar ~5 650 - probe lesson recorded in the PRD: creation-chasing is not optimal play (specials-greedy takes 91% and scores 2000 pts worse); gate is relative to expectimax-1 - tests: 3 dense-trainer tests (three-arm never-sampled ranking, NaN unsupervised, gamma guard) + 3 engine tests + obs plane assertions Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Owner report on the shipped M50 net: on boards where the same fruit offers both a 3-in-a-row and a 4-in-a-row, the AI takes the 3-match — forfeiting the striped special. A 4-agent investigation found the root cause is the loss, not the reward: DQN regressed only the chosen action's Q, so ranking a 3-match above an available 4-match cost exactly zero loss. The creation bonus lived in the training target but in no observation feature, and the per-action input planes were fire-only.
Full analysis, probe data, and the gate post-mortem:
docs/prd/CRAZY_FRUITS_RANKING_PRD.md(PLAN M51).What
DqnOptions.DenseTargets, γ=0-only): every legal action is regressed toward its creation-shaped deterministic value each update — the loss that makes a wrong ranking costly. Targets are read off the observation planes the engine already computes, so training cost is unchanged.deterministicValueShapedin the single-source.pg+ a third per-action observation plane (obs 928 → 1040). Parity pin unchanged (481681208) — no game-rule change.--dense,--dense-weight, and--probe N(opportunity-behavior diagnostics).Results (cf9train, 400k steps, 500-episode gate protocol)
First Crazy Fruits net to pass ALL gates:
Behavior probe: raw creating-take-rate 17.6% → 35.5% (expectimax-class), combo take-rate 48% → 66%. The pre-registered escalation (DAgger from expectimax-2) was not needed.
Shipped: new 1040-input checkpoint (
wwwroot/models/crazyfruits.dqn.ckpt), round-over bar → ~5 650. Live-verified on the running host: net tier active, zero console errors.🤖 Generated with Claude Code