Skip to content

feat(llm): support Ling-3.0-tiny with vLLM and SGLang - #5645

Merged
qinxuye merged 1 commit into
xorbitsai:mainfrom
Minamiyama:ENH/models/llm/ling-3.0-tiny-more
Oct 7, 2026
Merged

qinxuye merged 1 commit into
xorbitsai:mainfrom
Minamiyama:ENH/models/llm/ling-3.0-tiny-more

Conversation

@Minamiyama

Copy link
Copy Markdown
Collaborator

Enable vLLM and SGLang for the existing Ling-3.0-tiny BF16, FP8 and INT4 variants from Hugging Face and ModelScope.

  • Add backend-specific virtualenv requirements: vllm==0.29.0 and sglang==0.5.19, matching Ling-3.0-flash.
  • Reuse the existing BailingMoeV3ForCausalLM backend adapters.
  • Add regression tests for engine discovery, backend matching, launch-class selection and dependency resolution.
  • Regenerate the English model documentation.

@XprobeBot XprobeBot added this to the v3.x milestone Oct 7, 2026

@qinxuye qinxuye left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Checked engine discovery, launch-class selection, dependency resolution and reuse of the existing Ling-3.0 backend adapters for BF16/FP8/INT4. All 60 Ling-3.0 tiny/flash regression tests passed locally. GPU model inference was not run locally.

@qinxuye
qinxuye merged commit bf0c701 into xorbitsai:main Oct 7, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants