From NVIDIA Megatron-LM for visibility - #18

Open

RaymondLi0 wants to merge 7649 commits into

bigcode-project:multi-query-attentionfrom

This pull request is big! We're only showing the most recent 250 commits

Commits on Jul 23, 2026

Batch-invariant train/inference logprob parity (#5897)
wdykas
authored
51e915a
Move FSDP model weight sync to optimizer post-step (#5949)
wujingyue
authored
15d4b34
fix: allow mtp_num_layers=0 with overlap_moe_expert_parallel_comm (#5912)
cuichenx
authored
acd0e35
Port Multi-Latent Attention to `HybridModel` (#4452)
janEbert
authored
4464d1c
docs(skills): clarify container::lts is the older LTS PyTorch base (#6008)

ko3n1g
and
claude
authored
4f72cc0
[main] add thd sequence packing dispatcher support for main (#5008)
HaochenYuan
authored
81770cb

Commits on Jul 24, 2026

Commits on Jul 25, 2026

Commits on Jul 26, 2026

fix(inference): MCORE-536 report dropped prompt token lengths (#6051)
svcnemo-autobot
authored
7ec1bce

Commits on Jul 27, 2026

Commits on Jul 28, 2026

Commits on Jul 29, 2026

Commits on Jul 30, 2026

Update copy-pr-bot.yaml [skip ci]
github-actions[bot]
committed
542ff53
Reorganize a few docs (#6109)
Phlip79
authored
9436656
Perf: skip per-param copy_ dispatch in the MXFP8 param copy-back (#6094)
fanshiqing
authored
8e57bb6
test: add async scheduling inference coverage (#6104)

authored
354b5d2
test(ci): AUT-1173 set functional tests to ft_launcher (#6128)
svcnemo-autobot
authored
ab16414
Fix --freeze-all-layers for new model builder path + unit tests (#5926)
AAnoosheh
authored
ff6b92a
Fix inference overflow into dummy blocks (#5950)

tdene
and
jalbericiola
authored
3fd8f33
fix(tests): prefer staged release assets (#6149)
ko3n1g
authored
175b4cc
Fix Megatron FSDP optimizer under-update (#5976)

wujingyue
and
svcnvidia-nemo-ci
authored
ce8a8a7
Rebind MFSDP v2 sharded grads once per gradient reduction (#6041)

wujingyue
and
claude
authored
9d416ee
Fix MLA dynamic inference decode flag (#4902)
cuichenx
authored
0081c84
Normalize MFSDP data-parallel axes at fully_shard (#6067)
wujingyue
authored
dc3c9ab
Add Agent Compose package and runtime interface skeleton (#5937)
ISEEKYAN
authored
ab20a60
Fix GTP full-iteration CUDA-graph capture regression (#6077)

prajwal1210
and
claude
authored
ec53920
Harden dynamic prefix cache allocation - mamba mostly (#6091)
wdykas
authored
d04e7a0
ci(cache): AUT-1188 harden container cache selection (#6160)
svcnemo-autobot
authored
8543ae4

Commits on Jul 31, 2026

Update copy-pr-bot.yaml [skip ci]
github-actions[bot]
committed
6513e3e
ci(nvskills): AUT-1188 fix reusable workflow startup failure (#6162)
svcnemo-autobot
authored
b19b1f4
Add MFSDP runtime schedule design (#6031)
wujingyue
authored
d3519d5
Feature/enable inference optimized qwen moe (#5700)

authored
8339200
Bs invar moe (#4871)

authored
57deb43
chore: update pytest version to 9.1.1 in pyproject.toml and uv.lock (#6170)
balasaajay
authored
4ac5ad0
Fix LRU block accounting for resumed requests (#5995)

lmcafee-nvidia
and
santhnm2
authored
ee38649
[Main] update moe single weight unit test (#6035)
zhongbozhu
authored
d378bfb
fix ckpt conversion issue (#6151)
dimapihtar
authored
801d7e6
Add determinism developer documentation (#5718)
Connor-XY
authored
07fdc08
Add MFSDP v2 MCore integration with FullyShardedOptimizer (#5865)

wujingyue
and
svcnvidia-nemo-ci
authored
313f6e9

Commits on Aug 1, 2026

Update copy-pr-bot.yaml [skip ci]
github-actions[bot]
committed
4d129d8
Add streaming replies for dynamic engine (#5727)
nvcsathe
authored
1addfbd
deps: Update urllib3 to version 2.7.0 (#6182)
balasaajay
authored
2cbe71d
deps: Update black dependency to version 26.3.0 (#6180)
balasaajay
authored
f2d4dfa
Add meta-parameter support (#5369)
wujingyue
authored
1678db5
Fix tied parameters (#6164)
wujingyue
authored
714c24a

Commits on Aug 2, 2026

chore(ci): AUT-1231 bump preflight workflow template (#6198)
svcnemo-autobot
authored
9829b3f

Commits on Aug 3, 2026

Commits on Aug 4, 2026

[feat] Add reduce-scatter-with-fp32-accumulation support for GTP (#6200)
fanshiqing
authored
23b2ff2
chore(ci): AUT-1298 bump community workflow to v1.8.7 (#6219)
svcnemo-autobot
authored
5adbee2
fix: pretrain_hybrid.py --help (#6211)

paulwambergue
and
ilml
authored
0896be0
refactor(ci): migrate triage consumer to Cerno (#6250)

thomasdhc
and
codex
authored
22dfc28
Validate NIXL CUDA transports and share transfer agents (#6221)
wdykas
authored
46fe6d2
Add prefix-cache lifecycle hooks (#5860)
nvcsathe
authored
3fe1519

Commits on Aug 5, 2026

Commits on Aug 6, 2026

Commits on Aug 7, 2026

chore(ci): bump community workflow to v1.8.8 (#6322)
ko3n1g
authored
d62ff04
fix(moe): free activations immediately in full recompute mode (#5197)

authored
07a0de6
Allocate main_grad on the reduce-scatter stream (#6187)
wujingyue
authored
fa0758c
Encapsulate symmetric-memory AVG in DBuffer (#6169)
wujingyue
authored
cde3b5c
fix(ci): remove community workflow PAT (#6350)
ko3n1g
authored
c4968aa
Add multimodal argument and config plumbing (#6332)

matthieule
and
trintamaki
authored
628c2a5
Fix FSDP activation recompute prefetch (#6153)

shjwudp
and
wujingyue
authored
07a2319
Add Nemotron 3.5 Lightning GB200 nightly test (#6312)
Phlip79
authored
d2715d7

Commits on Aug 8, 2026

Update copy-pr-bot.yaml [skip ci]
github-actions[bot]
committed
a6fb162
Exit non-zero when --exit-on-missing-checkpoint triggers (#6321)

shanmugamr1992
and
claude
authored
e04ed19
Divide expert gradients by the expert-parallel size (#5956)

wujingyue
and
claude
authored
f8ec15e
[feat] GTP+MTP (#6242)
fanshiqing
authored
79e6b6a
fix(mtp): scatter the MTP embedding when the model's embedding does not (#6256)

DanialTaheri
and
claude
authored
67700f7
Apply the layerwise param layout on the ModelBuilder DDP path (#6286)

deepakn94
and
claude
authored
0ced206
test: AUT-1363 prevent recipe fetch-token exposure (#6364)
svcnemo-autobot
authored
14499ef
fix(ci): AUT-1396 use one global external queue worker (#6366)
svcnemo-autobot
authored
ccf686e
Add generic interface for SSM inference (#5382)

santhnm2
and
shanmugamr1992
authored
d12f6c8

Commits on Aug 9, 2026

Add PyTorch DCP checkpoint save/load for MFSDP v2 (#6024)

ahmadki
and
wujingyue
authored
6518b75

Commits on Aug 10, 2026

Commits on Aug 11, 2026

Bump FLA to 0.5.1 and integrate GDN2 kernels (#5765)
xuantengh
authored
0c1fc4c
Update docstring for moe_router_topk_scaling_factor (#2178)

JavaZeroo
and
guihong-nv
authored
67a355e
Add construction-scoped MFSDP v2 fully shard options (#6127)
wujingyue
authored
78901d8
Support explicit Hugging Face finetune data files in finetune.sh (#6413)
jenchen13
authored
5e86dfa
Add megatron.core.telemetry base layer and observability guide (#6415)
deepakn94
authored
d7d7a6a
[NCCL EP] Support eager mode and overflow handling (#6229)
YangFei1990
authored
8971d19
Gate expensive work behind log levels (#5805)

shurkat-nvidia
and
claude
authored
db57c15
Fixes for SP + MTP all-gathers (#5748)

authored
8018459
TE Layernorm dtype guard with fp32 residuals (#6272)
mkhona-nvidia
authored
7a76dff
Treat LayerWise bucket_size as a soft minimum and fill would-be padding with real parameters (#5415)

deepakn94
and
claude
authored
e0ccd26
Gated Delta Product (GDP) implementation (#6074)

authored
8e70f69
fix(mfsdp): refresh V2 compute weights after optimizer step (#6336)

wujingyue
and
claude
authored
1d402ad
[core] Clarify GPT output_processor callback contract (#5739)

authored
c3d44dc
Validate MFSDP module phase transitions (#6359)
wujingyue
authored
03318a8
Refine MFSDP v2 configuration validation (#6333)
wujingyue
authored
80cda5d
Add hybrid layer config definitions (#6410)
Phlip79
authored
f3eb0df
Propagate RL runtime config updates to module configs (#6423)
Phlip79
authored
da18ed6
[fix] GTP+recompute: keep adjacent GTP recompute weights off the same gather buffer (#6407)
fanshiqing
authored
e22bcb0
Add MFSDP v2 design document (#6431)
wujingyue
authored
49a49a9

Commits on Aug 12, 2026

Commits on Aug 13, 2026

Skip attention mask materialization in RL training (#5885)
tdene
authored
5436289
Avoid unnecessary MoE router host synchronization (#6432)
JF-D
authored
a34880f
Refactor GDP dynamic inference onto the shared SSM interface (#6443)

authored
3cd8cf3
test(fsdp): mark default overlap check flaky (#6493)
wujingyue
authored
3ce4b4a
Paged stashing no longer assumes single layer config (#6419)
Phlip79
authored
301e0d6
Respect `profile_ranks` on log interval memory snapshots (#6354)
maanug-nv
authored
5714a20
Add unit tests for GDP inference (#6505)

authored
fecb29f
[fix] fix GTP+DCP ckpt saving/loading for GDP module (#6503)
fanshiqing
authored
8ba5c60
feat(fsdp): support expert parallelism in MFSDP v2 adapter (#6450)

wujingyue
and
svcnvidia-nemo-ci
authored
281075d
Complete clamped SwiGLU across expert paths (#5940)
FDecaYed
authored
2a75ac1
RL: unify rollout generation into one pipeline (#6495)
tdene
authored
87bf520
Avoid passing -1 sentinels into inference kernels (#6481)

tdene
and
jalbericiola
authored
487bb39
RL: bound per-env wandb keys to the char limit (#6477)

tdene
and
jalbericiola
authored
09921c0
Remove torch cuda empty cache (#5678)

shurkat-nvidia
and
claude
authored
2119719
[fix] GTP+DDP potential data-race: publish DDP params before GTP prefetch reads them (#6388)

fanshiqing
and
JF-D
authored
87cc590
Cover symmetric memory in CUDA graph test (#5987)

wujingyue
and
claude
authored
d7430d5
Fix tool call reasoning boundary (#6411)
nvcsathe
authored
f7b2209
Update Nemotron 3.5 Lightning golden values (#6445)
Phlip79
authored
0ee415b
Handle hidden chunked prefill during async overlap (#6442)
lmcafee-nvidia
authored
94939b8
Thread socket argument through MInf API (#6513)
tdene
authored
f9cc544
Update release notes page (#6520)
Phlip79
authored
69e282e
test(fsdp): verify optimizer CUDA graphs in v2 (#6498)
wujingyue
authored
3d0956f
Clean up SSM ops directory structure (#6444)

santhnm2
and
shanmugamr1992
authored
de4a82e
Add read/write ledger for bulky inference metadata (#6125)
tdene
authored
d918967

Commits on Aug 14, 2026

Commits on Aug 15, 2026

ci: AUT-1408 replace cicd PAT with GitHub App tokens (#6360)

svcnemo-autobot
and
ko3n1g
authored
669d6ed
fix: AUT-1501 abort when SSO list download fails (#6516)
svcnemo-autobot
authored
51f6a39
Fix missing golden values updates (#6528)
janEbert
authored
e79110c
Support HFSDP deferred DP-outer gradient reduction (#5972)

authored
e60e42f

Commits on Aug 17, 2026

Commits on Aug 18, 2026

ci: Update workflow to include NeMo RL testing (#6548)
balasaajay
authored
d7e2c13
Add data-schedule utilities and GPTDatasetConfig fields for sequence packing (#5903)

ilml
and
claude
authored
574e04e
Enhance optimizer state loading with runtime overrides (#3720)

authored
59ff171
test(mfsdp): cover v2 training memory (#6567)
wujingyue
authored
35ae46c
fix(ci): avoid JET expansion of HOME in setup scripts (#6599)
balasaajay
authored
5d50b16
RL: isolate placeholder rollouts from GRPO rewards (#6475)

tdene
and
jalbericiola
authored
32756e3
tests: Update cerno.yml with linear issue status (#6304)
balasaajay
authored
ae5ec14
Enable GTP for heterogeneous MIMO training (#6570)
yashaswikarnati
authored
cfb8e46
test(mfsdp): separate overlap test (#6591)
wujingyue
authored
dcb2bfd