fix(grpo): per-env starvation guard in the difficulty-filter buffer - #73
Open
flaviusburca wants to merge 1 commit into
Open
fix(grpo): per-env starvation guard in the difficulty-filter buffer#73flaviusburca wants to merge 1 commit into
flaviusburca wants to merge 1 commit into
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
flaviusburca
pushed a commit
that referenced
this pull request
Aug 4, 2026
…gating - harbor.py: per-step compose teardown (_teardown_harbor_projects) after every Harbor job incl. timeout path — a 64-rollout repo group otherwise exhausts the docker network pool mid-group; close() handles harbor>=0.20 '<trial>__env' project naming (was silently never matching) - scripts/harbor_sweeper.sh: standing GC for leaked compose containers/networks, per-trial build images, and LRU eviction of unused base images below 80GB free - heldout_probe.py: --lanes subset runs; _wait_repo_quiet() gates the repo lane on a quiet Harbor window (concurrent campaign repo groups starve probe agent loops -> harbor phase timeouts -> every episode quarantines to 0.0) - trainer.py: reorder_micros_for_vtc_guard (upstreamed as PR #75, issue #74) - buffer.py: per-env starvation guard (upstreamed as PR #73) - MISSION.md: launch steps 4/6/7 (harbor env, lane-liveness gate, GC), corrected probe baselines, vtc root-cause facts
An env whose every example gets classified easy/hard becomes silently unschedulable: sample_examples() only deals envs with a non-empty NORMAL pool, and the global normal_pool_min_examples floor cannot trip while other envs stay populated. An outage that scores one env's whole registry under hard_threshold therefore removes that env from training with no log line and no error (observed on a live campaign: a 45-step harness outage zeroed one lane's rewards and the lane stopped being dealt). _recycle_examples_if_needed now returns ALL of a weighted env's easy/hard examples to the normal pool whenever that env's normal pool is empty; difficulty re-classifies as fresh results arrive. The global floor path is unchanged. Contract change: sampling no longer raises when a weighted env still has easy/hard examples — it recycles them instead (test updated); the ValueError remains for the truly-empty case.
flaviusburca
force-pushed
the
fix/buffer-per-env-starvation
branch
from
August 4, 2026 18:10
42420a9 to
894288c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
With
online_difficulty_filtering, an env whose every example gets classified easy/hard becomes silently unschedulable:sample_examples()deals only envs with a non-empty normal poolnormal_pool_min_examples) is global — it cannot trip while other envs stay populatedSo an outage that scores one env's whole registry under
hard_thresholdremoves that env from training with no log line and no error. Observed on a live campaign: a 45-step harness outage zeroed one lane's rewards; after resume, the lane's registry sat in the hard pool and every other lane kept the global floor satisfied — the lane was never dealt again.Fix
_recycle_examples_if_needednow returns all of a weighted env's easy/hard examples to the normal pool whenever that env's normal pool is empty (logged). Difficulty re-classifies as fresh results arrive. The global-floor path is unchanged.Contract change: sampling no longer raises when a weighted env still holds easy/hard examples — it recycles them instead. The
ValueErrorremains for the truly-empty case. One existing test encoded the old behavior and was updated; a new test constructs the starvation state directly (full env keeps the global floor satisfied, starved env must recycle).Tests
tests/grpo/test_buffer.py: 6 passed (1 new, 1 rewritten to the new contract).🤖 Generated with Claude Code