Skip to content

Commit 44413a9

Browse files
localai-org-maint-botlocalai-org-maint-bot
authored
feat(gallery): add Qwen3.8 27B variants (#11519)
Add the official Q4_K_M and Q8_0 GGUF files with the shared vision projector. Include an MTP variant for speculative decoding. Assisted-by: Codex:gpt-5 Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
1 parent 3a3c311 commit 44413a9

1 file changed

Lines changed: 159 additions & 0 deletions

File tree

‎gallery/index.yaml‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,163 @@
11
---
2+
- &qwen3-8-27b
3+
name: "qwen3.8-27b-q4"
4+
variants:
5+
- model: qwen3.8-27b-q4-mtp
6+
- model: qwen3.8-27b-q8
7+
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
8+
urls:
9+
- https://huggingface.co/Qwen/Qwen3.8-27B
10+
- https://huggingface.co/ggml-org/Qwen3.8-27B-GGUF
11+
description: |
12+
Qwen3.8-27B is Qwen's dense 27B vision-language model for reasoning,
13+
coding, tool use, and long-running agent tasks. It accepts text, images,
14+
and video, and it supports a native context window of 262K tokens.
15+
16+
This default entry uses the official Q4_K_M GGUF and Q8_0 vision
17+
projector. The linked variants add MTP speculative decoding or use the
18+
higher-quality Q8_0 model.
19+
license: "apache-2.0"
20+
tags:
21+
- llm
22+
- gguf
23+
- cpu
24+
- gpu
25+
- qwen
26+
- reasoning
27+
- thinking
28+
- coding
29+
- agent
30+
- tools
31+
- vision
32+
- multimodal
33+
- long-context
34+
icon: https://qianwen-res.oss-cn-beijing.aliyuncs.com/logo_qwen.jpg
35+
last_checked: "2026-08-14"
36+
overrides:
37+
backend: llama-cpp
38+
context_size: 262144
39+
function:
40+
automatic_tool_parsing_fallback: true
41+
grammar:
42+
disable: true
43+
known_usecases:
44+
- chat
45+
- vision
46+
mmproj: llama-cpp/mmproj/qwen3.8-27b/mmproj-Qwen3.8-27B-Q8_0.gguf
47+
options:
48+
- use_jinja:true
49+
parameters:
50+
min_p: 0
51+
model: llama-cpp/models/qwen3.8-27b/Qwen3.8-27B-Q4_K_M.gguf
52+
presence_penalty: 0
53+
repeat_penalty: 1
54+
temperature: 1
55+
top_k: 20
56+
top_p: 0.95
57+
template:
58+
use_tokenizer_template: true
59+
files:
60+
- filename: llama-cpp/models/qwen3.8-27b/Qwen3.8-27B-Q4_K_M.gguf
61+
uri: huggingface://ggml-org/Qwen3.8-27B-GGUF/Qwen3.8-27B-Q4_K_M.gguf
62+
sha256: 31629f53165ab6a7dad8c9847dcfd1fdf55829dac1e6e748f4a68581b0033d34
63+
- filename: llama-cpp/mmproj/qwen3.8-27b/mmproj-Qwen3.8-27B-Q8_0.gguf
64+
uri: huggingface://ggml-org/Qwen3.8-27B-GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf
65+
sha256: 2e968a6af97ce35d8971890b257b9b7edabf20ad91450501fa53162a19ee33eb
66+
- !!merge <<: *qwen3-8-27b
67+
name: "qwen3.8-27b-q4-mtp"
68+
variants: []
69+
description: |
70+
Qwen3.8-27B with the official Q4_K_M model and Q4_0 MTP draft model.
71+
MTP speculative decoding can increase generation speed by proposing
72+
multiple tokens for the target model to verify.
73+
tags:
74+
- llm
75+
- gguf
76+
- gpu
77+
- qwen
78+
- reasoning
79+
- thinking
80+
- coding
81+
- agent
82+
- tools
83+
- vision
84+
- multimodal
85+
- long-context
86+
- mtp
87+
- speculative
88+
overrides:
89+
backend: llama-cpp
90+
context_size: 262144
91+
draft_model: llama-cpp/models/qwen3.8-27b/mtp-Qwen3.8-27B-Q4_0.gguf
92+
function:
93+
automatic_tool_parsing_fallback: true
94+
grammar:
95+
disable: true
96+
known_usecases:
97+
- chat
98+
- vision
99+
mmproj: llama-cpp/mmproj/qwen3.8-27b/mmproj-Qwen3.8-27B-Q8_0.gguf
100+
options:
101+
- use_jinja:true
102+
- spec_type:draft-mtp
103+
- spec_n_max:6
104+
- spec_p_min:0.75
105+
parameters:
106+
min_p: 0
107+
model: llama-cpp/models/qwen3.8-27b/Qwen3.8-27B-Q4_K_M.gguf
108+
presence_penalty: 0
109+
repeat_penalty: 1
110+
temperature: 1
111+
top_k: 20
112+
top_p: 0.95
113+
template:
114+
use_tokenizer_template: true
115+
files:
116+
- filename: llama-cpp/models/qwen3.8-27b/Qwen3.8-27B-Q4_K_M.gguf
117+
uri: huggingface://ggml-org/Qwen3.8-27B-GGUF/Qwen3.8-27B-Q4_K_M.gguf
118+
sha256: 31629f53165ab6a7dad8c9847dcfd1fdf55829dac1e6e748f4a68581b0033d34
119+
- filename: llama-cpp/models/qwen3.8-27b/mtp-Qwen3.8-27B-Q4_0.gguf
120+
uri: huggingface://ggml-org/Qwen3.8-27B-GGUF/mtp-Qwen3.8-27B-Q4_0.gguf
121+
sha256: 051a1764cff8c4f3ee6ae8b00593a0364c7539c67fa50ffc58f3f96509fca38e
122+
- filename: llama-cpp/mmproj/qwen3.8-27b/mmproj-Qwen3.8-27B-Q8_0.gguf
123+
uri: huggingface://ggml-org/Qwen3.8-27B-GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf
124+
sha256: 2e968a6af97ce35d8971890b257b9b7edabf20ad91450501fa53162a19ee33eb
125+
- !!merge <<: *qwen3-8-27b
126+
name: "qwen3.8-27b-q8"
127+
variants: []
128+
description: |
129+
Qwen3.8-27B in the official Q8_0 GGUF format. This variant provides higher
130+
model fidelity for hosts with enough memory.
131+
overrides:
132+
backend: llama-cpp
133+
context_size: 262144
134+
function:
135+
automatic_tool_parsing_fallback: true
136+
grammar:
137+
disable: true
138+
known_usecases:
139+
- chat
140+
- vision
141+
mmproj: llama-cpp/mmproj/qwen3.8-27b/mmproj-Qwen3.8-27B-Q8_0.gguf
142+
options:
143+
- use_jinja:true
144+
parameters:
145+
min_p: 0
146+
model: llama-cpp/models/qwen3.8-27b/Qwen3.8-27B-Q8_0.gguf
147+
presence_penalty: 0
148+
repeat_penalty: 1
149+
temperature: 1
150+
top_k: 20
151+
top_p: 0.95
152+
template:
153+
use_tokenizer_template: true
154+
files:
155+
- filename: llama-cpp/models/qwen3.8-27b/Qwen3.8-27B-Q8_0.gguf
156+
uri: huggingface://ggml-org/Qwen3.8-27B-GGUF/Qwen3.8-27B-Q8_0.gguf
157+
sha256: f5c702d8820d36fb55985bb238fc83ee3a313e920f4b752a437c3a6a9e14e4c8
158+
- filename: llama-cpp/mmproj/qwen3.8-27b/mmproj-Qwen3.8-27B-Q8_0.gguf
159+
uri: huggingface://ggml-org/Qwen3.8-27B-GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf
160+
sha256: 2e968a6af97ce35d8971890b257b9b7edabf20ad91450501fa53162a19ee33eb
2161
- &twil-lm3
3162
name: "twil-lm3-q4"
4163
variants:

0 commit comments

Comments
 (0)