Skip to content

(DeepSeek-R1) MLA for decode#5

Draft
almaslof wants to merge 62 commits into
mainfrom
feat/merge_mla_for_decode
Draft

(DeepSeek-R1) MLA for decode#5
almaslof wants to merge 62 commits into
mainfrom
feat/merge_mla_for_decode

Conversation

@almaslof

@almaslof almaslof commented Jun 9, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

@almaslof almaslof changed the title Merge mla for decode from BruceXcluding:aiter_customer (DeepSeek-R1) MLA for decode Jun 10, 2025
almaslof and others added 25 commits June 11, 2025 06:54
@almaslof
...
2560af2
@PanJason @almaslof
Fix torch profiler bugs for bench_offline_throughput.py (sgl-project#…
dfa836d 
…6557)
@zhyncs @Alcanderian
@Qiaolin-Yu @Fridge003 @mickqian @ispobock @almaslof
chore: upgrade flashinfer v0.2.6.post1 jit (sgl-project#6958)
e6e36a0 
Co-authored-by: alcanderian <alcanderian@gmail.com>
Co-authored-by: Qiaolin Yu <qy254@cornell.edu>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
@zhyncs @almaslof
cleanup tmp dir (sgl-project#7007)
fbc8cfe
@zhyncs @almaslof
chore: update pr test xeon (sgl-project#7008)
99f45ce
@fzyzcjy @almaslof
Fix cutlass MLA gets almost zero accuracy (sgl-project#6998)
38c03b9
@saienduri @almaslof
Update amd nightly models CI. (sgl-project#6992)
b095bd5
@ishandhanani @almaslof
feat: add direct routing strategy to DP worker (sgl-project#6884)
3cd75fd
@Fridge003 @almaslof
Fallback to lower triton version for unfound fused moe configs (sgl-p…
e749da7 
…roject#7013)
@Edenzzzz @almaslof
Fix torchvision version for Blackwell (sgl-project#7015)
3fe88c5
@merrymercy @almaslof
Simplify prepare_extend_after_decode (sgl-project#6987)
8368a97
@emmanuel-ferdman @almaslof
Migrate to assertEqual (sgl-project#6741)
8a91eff 
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
@Qiaolin-Yu @almaslof
Fix torch version in blackwell dockerfile (sgl-project#7017)
09a8302
@zhyncs @almaslof
chore: update pr test xeon (sgl-project#7018)
ab8ce6e
@Fridge003 @almaslof
Update default settings for blackwell (sgl-project#7023)
b7629a7
@fzyzcjy @almaslof
Support both approximate and exact expert distribution collection (sg…
4537d8a 
…l-project#6964)
@ByronHsu @almaslof
Add decode req pool (sgl-project#6980)
f10c4eb
@YouNeedCryDear @almaslof
[CI] Add CI workflow for sgl-router docker build (sgl-project#7027)
1b58b85
@yudian0504 @almaslof
Fix fused_moe triton configs (sgl-project#7029)
596b0ad
@yanbing-j @mingfeima @almaslof
CPU: map changes from developing branch in sgl-kernel (sgl-project#6833)
3d7f5b8 
Co-authored-by: mingfeima <mingfei.ma@intel.com>
@zhyncs @almaslof
chore: bump v0.4.7 (sgl-project#7038)
cd0055e
@merrymercy @almaslof
Update README.md (sgl-project#7040)
6512d58
@Swipe4057 @almaslof
[Docker] Upgrading base image from 24.04 to 24.12 (sgl-project#7043)
dea7bd9
@zhyncs @almaslof
fix 24.12 docker (sgl-project#7045)
8b750e7
@merrymercy @almaslof
Minor cleanup of fa3 backend (sgl-project#6999)
67e9184
HanHan009527 and others added 30 commits June 11, 2025 08:01
@HanHan009527 @almaslof
[fix] libmlx5.so already in base image (sgl-project#7060)
0e30b1e
@Fridge003 @almaslof
Fix test_lora.py CI (sgl-project#7061)
eb69c92
@fzyzcjy @almaslof
Tiny fix cutlass_mla_get_workspace_size stub incorrect signature (sgl…
d39e088 
…-project#7057)
@fzyzcjy @almaslof
Add sanity checks when a test file is not added to CI (sgl-project#6947)
5dcc822
@zhyncs @almaslof
Revert "Add sanity checks when a test file is not added to CI (sgl-pr…
3d5fa78 
…oject#6947)" (sgl-project#7063)
@Xu-Wenqing @almaslof
Fix missing tool call id if tool call index >0 in streaming tool call…
297d056 
… output. (sgl-project#7049)

Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
@zhyncs @almaslof
chore: update dev docker (sgl-project#7064)
f0f2e05
@kyle-pena-kuzco @almaslof
Open AI API hidden states (sgl-project#6716)
d472996
@zhyncs @almaslof
fix arm sgl-kernel link issue (sgl-project#7066)
4908433
@b8zhong @cinjon @almaslof
[Feature] Add Logit Bias (sgl-project#6579)
01ca49a 
Co-authored-by: Cinjon Resnick <cinjon.resnick@gmail.com>
@merrymercy @almaslof
Improve perf tuning docs (sgl-project#7071)
e789217
@binarycrayon @almaslof
Frontend language separate reasoning support (sgl-project#6031)
41b0736
@merrymercy @almaslof
Do not run frontend_reasoning.ipynb to reduce the CI load (sgl-projec…
3bc0883 
…t#7073)
@merrymercy @almaslof
Simplify the heuristics for setting --mem-fraction-static (sgl-projec…
5d25efd 
…t#7054)
@Ximingwang-09 ximing.wxm @almaslof
update doc (sgl-project#7046)
6665ccc 
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
@merrymercy @almaslof
Clean up docs for server args and sampling parameters (generated by g…
00fd64e 
…rok) (sgl-project#7076)
@Fridge003 @almaslof
Fix GGuf and add back test_gguf.py (sgl-project#7067)
eba1685
@almaslof
Cherry pick decode_attention.py from aiter_cistomer
ef5870f
@almaslof
ok
e1f8763
@almaslof
...
bbfd3eb
@almaslof
...
ac9c1bd
@almaslof
mla.py qo_indptr shape error
254969f
@almaslof
Error: nhead=32 not supported
9d548df
@almaslof
Merge remote-tracking branch 'origin/main' into feat/merge_mla_for_de…
7b38f37 
…code
@almaslof
Merge remote-tracking branch 'origin/main' into feat/merge_mla_for_de…
f7948c8 
…code
@almaslof
stash before merge
b296bea
@almaslof
Merge remote-tracking branch 'origin/main' into feat/merge_mla_for_de…
88377cb 
…code
@almaslof
OK run
3f30b04
@almaslof
set aiter commit
e0f958d
@almaslof
...
9ba95d7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.