Add context parallelism support - #38

Draft

rlrs wants to merge 7 commits into

mainfrom

31-context-parallelism

Commits on Nov 10, 2025

changes for context parallel gemma, todos for flex attn
rlrs
committed
eaed9ab
changes for supporting cp for gemma w/ flex attn
rlrs
committed
3051ee2
fix non-cp parallelism
rlrs
committed
ba14de0
small changes to debug gemma model
rlrs
committed
d9800e6
llama context parallelism

oliverkinch
authored and
rlrs
committed
c7969e9

Commits on Nov 11, 2025

import devicemesh
oliverkinch
committed
9b7f766

Commits on Nov 12, 2025

Ignore data and models folders
oliverkinch
committed
39b5f2a