feat: GavelFGD+ — Placement-Opportunity-Aware LP + FGD scoring improvements - #12
Open
Turquoise-T wants to merge 78 commits into
Open
feat: GavelFGD+ — Placement-Opportunity-Aware LP + FGD scoring improvements#12Turquoise-T wants to merge 78 commits into
Turquoise-T wants to merge 78 commits into
Conversation
- Add task arrival order option (trace, ascending, descending) - Implement distribution shift awareness - Fix bugs in evaluating fgd rate
- remove explicit tiebreaking logic - apply integer quantization and more precise formula
Replace developer setup guide with project overview, embedded result figures (Gavel replication, FGD standalone, Alibaba split, Cluster H), key findings, reproduction instructions, and updated project structure. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Generated with PaperBanana (Gemini native image generation). Architecture diagram shows Gavel LP + FGD placement two-phase loop. Node topology diagram explains uniform vs mixed node fragmentation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Approved - Rewrite README with results figures, GIF, and submission-ready structure
Add standalone FGD replication code/results and update README comparison section
Reorganize results into three sequential parts: 1. FGD algorithm diagram + standalone replication 2. Gavel policy diagram + Philly replication (Figs 9/10/11) 3. Integration architecture + topology comparison + Cluster H results Each part follows: explanatory diagram -> experiment setup -> results graph. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Approved - Rebase gavelfgdrepl: add architecture diagrams to README
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR introduces GavelFGD+, which improves upon base
GavelFGD through
coordinated changes at both the placement and allocation
layers.
Key Insight
Fragmentation is caused by placement decisions, not
allocation decisions.
The previous LP fragmentation penalty was architecturally
unsound: penalizing
allocation to fragmented GPU types cannot reorganize jobs
already placed on
those nodes. GavelFGD+ fixes this by replacing the
backward-looking penalty
with a forward-looking Placement-Opportunity-Aware (POA)
bonus.
Changes
Placement layer:
packing)
Allocation layer (POA LP bonus):
After each round's lease-extension phase, compute per-type
fit fractions:
poa[type][demand] = fraction of nodes with ≥ demand free
GPU slots
LP objective becomes:
max min_j(throughput_j) + μ · Σ_{j,t} poa[t][demand_j] ·
x[j,t]
This steers allocation toward GPU types where jobs can
currently be placed,
creating a correct feedback loop between the allocation and
placement layers.
The bonus naturally vanishes at high utilization when fit
fractions → 0
everywhere, reverting to pure Gavel fairness under
saturation.
Results (Alibaba cluster, 7 load levels, 2 seeds)
At 85.4% GPU utilization:
reduction
Effect is load-dependent by design: dormant at low
utilization (POA scores
uniform everywhere), strongest at high utilization (POA
scores differentiated).