Skip to content

[GLM5.2] Use official IndexShare support - #1917

Draft
yapdianang wants to merge 1 commit into
NovaSky-AI:mainfrom
yapdianang:fix/glm52-indexshare
Draft

[GLM5.2] Use official IndexShare support#1917
yapdianang wants to merge 1 commit into
NovaSky-AI:mainfrom
yapdianang:fix/glm52-indexshare

Conversation

@yapdianang

Copy link
Copy Markdown

Goal

Train GLM-5.2 with its published shared attention layout without runtime patches.

What changed

  • Update Megatron Bridge to its official GLM-5.2 support commit.
  • Update Megatron Core to the exact commit recorded by that Bridge version.
  • Refresh the locked dependencies and the generated parallelism reference.

The previous versions created a separate attention indexer on every layer. GLM-5.2 instead creates full indexers on selected layers and shares them with nearby layers. The new pair reads index_topk_freq and index_skip_topk_offset from the official model configuration and builds that layout directly.

Verification

bash format.sh
uv lock --check

A live zai-org/GLM-5.2-FP8 B300 training test is in progress. This PR stays draft until model load, backward calculation, optimizer update, checkpoint reload, and sampling after weight sync pass.

Signed-off-by: Ubuntu <23232359+yapdianang@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant