fix(mccfr): normalize sampling distribution used for importance reach#57
Merged
Conversation
sampling_distribution/sampling() produced unnormalized weights: smoothing is added per-edge in the numerator but once in the denominator, temperature scales only the numerator, and the curiosity floor adds mass — so the per-infoset sum Z is > 1. The external sampler draws via WeightedIndex, which normalizes internally, so the true draw probability is weight(a)/Z, but sampling_reach treated the raw weight(a) as the sampling probability. That mis-scales counterfactual values by a path-dependent product of Z, compounding worst in deep (postflop) trees. Normalize by Z at the single source (sampling_distribution) and route the per-edge sampling() through the same, so sampling_reach / ancestor_reach / recursed_value all use the true draw probability. The draw itself is unchanged (WeightedIndex renormalizes regardless), so this corrects only the importance term; Kuhn still converges to the analytical Nash equilibrium. Fixes #53.
krukah
added a commit
that referenced
this pull request
Jul 13, 2026
Asserts sampling_distribution sums to ~1 across Kuhn infosets. Fails on the pre-#57 code (Z ≈ 1.04 for J|Bet), where the per-edge smoothing pseudocount and curiosity floor left the distribution unnormalized; passes with the normalized sampling. Locks in the #53 fix. Co-authored-by: andyafter <3354145+andyafter@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #53 — thanks @andyafter for the sharp catch and the function-level pointers.
bug
sampling_distribution()/sampling()weren't normalized. Intended form:The external sampler draws opponent actions via
WeightedIndex, which normalizes internally — so the true probability it drewaisw(a)/Z. Butsampling_reach()used the raww(a)as the importance term inpayoff · relative_reach / sampling_reach. Result: counterfactual values mis-scaled by a path-dependentΠ Z, compounding asZ^depth.sizing (live β=2, ε=.05)
≈ KK=5→ ~2800×)≈ 1.3/node≈ 1Worst early (β dominates) and persistent postflop (ε-floor on rarely-played sizes); compounds with depth — which is why postflop was the wobble.
fix
Normalize by
Zat the single source (sampling_distribution) and route the per-edgesampling()through the same weight, sosampling_reach/ancestor_reach/recursed_valueall use the true draw probability.The draw is unchanged —
WeightedIndexrenormalizes regardless — so this touches only the importance correction, not sampling behavior. Kuhn still converges to the analytical Nash equilibrium.note
This is a code fix; existing blueprints were trained under the biased correction, so regenerate (
--mode reset) to get the corrected strategy — the fingerprint is unchanged, so old tables still load but encode the old fixpoint.