Skip to content

Context parallel does not support Qwen3.5 GatedDeltaNet #12051

Description

@MegaGimen

Unsloth does not support context parallel for Qwen3.5 GatedDeltaNet, and it does not shard that hybrid model with FSDP2.

Qwen3.5-35B-A3B has 40 decoder layers, and 30 of them are GatedDeltaNet. The convolution and the gated delta rule are recurrent in time, so a rank that only holds a slice of the sequence cannot run them. Ring-style rotation of K and V cannot run that recurrence either. The other 10 layers are full attention. On one 96GB GPU the bf16 weights already occupy about 65 GiB, and the first LoRA step runs out of memory at sequence length 32768. The same thing happens at 65536.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions