Skip to content

feat: GavelFGD+ — Placement-Opportunity-Aware LP + FGD scoring improvements - #12

Open
Turquoise-T wants to merge 78 commits into
jiayu/gavelfgdexperimentfrom
main
Open

feat: GavelFGD+ — Placement-Opportunity-Aware LP + FGD scoring improvements#12
Turquoise-T wants to merge 78 commits into
jiayu/gavelfgdexperimentfrom
main

Conversation

@Turquoise-T

Copy link
Copy Markdown
Owner

Overview

This PR introduces GavelFGD+, which improves upon base
GavelFGD through
coordinated changes at both the placement and allocation
layers.

Key Insight

Fragmentation is caused by placement decisions, not
allocation decisions.
The previous LP fragmentation penalty was architecturally
unsound: penalizing
allocation to fragmented GPU types cannot reorganize jobs
already placed on
those nodes. GavelFGD+ fixes this by replacing the
backward-looking penalty
with a forward-looking Placement-Opportunity-Aware (POA)
bonus
.

Changes

Placement layer:

  • Paper-accurate FGD sigmoid scoring
  • Popularity threshold raised to 95% (more aggressive
    packing)
  • Buddy tiebreak for equal-score placement options

Allocation layer (POA LP bonus):
After each round's lease-extension phase, compute per-type
fit fractions:
poa[type][demand] = fraction of nodes with ≥ demand free
GPU slots

LP objective becomes:
max min_j(throughput_j) + μ · Σ_{j,t} poa[t][demand_j] ·
x[j,t]

This steers allocation toward GPU types where jobs can
currently be placed,
creating a correct feedback loop between the allocation and
placement layers.
The bonus naturally vanishes at high utilization when fit
fractions → 0
everywhere, reverting to pure Gavel fairness under
saturation.

Results (Alibaba cluster, 7 load levels, 2 seeds)

At 85.4% GPU utilization:

  • Frag rate: 2.01% (GavelFGD) → 0.47% (GavelFGD+), 76.7%
    reduction
  • JCT: 22,275s → 21,810s, 2.1% improvement

Effect is load-dependent by design: dormant at low
utilization (POA scores
uniform everywhere), strongest at high utilization (POA
scores differentiated).

clubzip and others added 30 commits January 31, 2026 02:03
- Add task arrival order option (trace, ascending, descending)
- Implement distribution shift awareness
- Fix bugs in evaluating fgd rate
clubzip and others added 30 commits March 1, 2026 00:43
- remove explicit tiebreaking logic
- apply integer quantization and more precise formula
Replace developer setup guide with project overview, embedded result
figures (Gavel replication, FGD standalone, Alibaba split, Cluster H),
key findings, reproduction instructions, and updated project structure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Generated with PaperBanana (Gemini native image generation).
Architecture diagram shows Gavel LP + FGD placement two-phase loop.
Node topology diagram explains uniform vs mixed node fragmentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Approved - Rewrite README with results figures, GIF, and submission-ready structure
Add standalone FGD replication code/results and update README comparison section
Reorganize results into three sequential parts:
1. FGD algorithm diagram + standalone replication
2. Gavel policy diagram + Philly replication (Figs 9/10/11)
3. Integration architecture + topology comparison + Cluster H results

Each part follows: explanatory diagram -> experiment setup -> results graph.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Approved - Rebase gavelfgdrepl: add architecture diagrams to README
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants