Skip to content

Commit 7621939

Browse files
gallery: add Qwythos 27B variants (#11292)
Add the recommended Q4_K_M build and an MTP-enabled variant with the shared vision projector. Tag the existing Qwythos 9B MTP entry so serving-feature ranking recognizes it. Assisted-by: Codex:gpt-5 Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
1 parent cff69a0 commit 7621939

1 file changed

Lines changed: 94 additions & 0 deletions

File tree

gallery/index.yaml

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1007,6 +1007,99 @@
10071007
sha256: 05d4475a956030be87b099865d6552a541a476db8cc3e266fcfa7c5a24846248
10081008
uri: https://huggingface.co/unsloth/Inkling-Small-GGUF/resolve/main/mmproj-BF16.gguf
10091009
variants: []
1010+
- &qwythos-27b-v1
1011+
name: "qwythos-27b-v1"
1012+
variants:
1013+
- model: qwythos-27b-v1-mtp
1014+
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
1015+
urls:
1016+
- https://huggingface.co/empero-ai/Qwythos-27B-v1
1017+
- https://huggingface.co/empero-ai/Qwythos-27B-v1-GGUF
1018+
description: |
1019+
Qwythos-27B-v1 is an Apache-2.0 dense 27B reasoning and agentic model
1020+
derived from Qwen3.5-27B. It supports tool use, vision through the included
1021+
projector, and a one-million-token context window. This entry uses the
1022+
recommended Q4_K_M GGUF quantization; an MTP-enabled build is available as
1023+
a variant for hosts with recent llama.cpp support.
1024+
license: "apache-2.0"
1025+
tags:
1026+
- llm
1027+
- gguf
1028+
- vision
1029+
- multimodal
1030+
- reasoning
1031+
- thinking
1032+
overrides:
1033+
backend: llama-cpp
1034+
function:
1035+
automatic_tool_parsing_fallback: true
1036+
grammar:
1037+
disable: true
1038+
known_usecases:
1039+
- chat
1040+
mmproj: llama-cpp/mmproj/Qwythos-27B-Q4_K_M/mmproj-Qwythos-27B-F16.gguf
1041+
options:
1042+
- use_jinja:true
1043+
parameters:
1044+
model: llama-cpp/models/Qwythos-27B-Q4_K_M/Qwythos-27B-Q4_K_M.gguf
1045+
repeat_penalty: 1.05
1046+
temperature: 0.6
1047+
top_k: 20
1048+
top_p: 0.95
1049+
template:
1050+
use_tokenizer_template: true
1051+
files:
1052+
- filename: llama-cpp/models/Qwythos-27B-Q4_K_M/Qwythos-27B-Q4_K_M.gguf
1053+
sha256: 817aa895cd3b6e0680c7aa8da62ec393f3125ff0ab676f8f06a8fc759d20e3d5
1054+
uri: huggingface://empero-ai/Qwythos-27B-v1-GGUF/Qwythos-27B-Q4_K_M.gguf
1055+
- filename: llama-cpp/mmproj/Qwythos-27B-Q4_K_M/mmproj-Qwythos-27B-F16.gguf
1056+
sha256: 82af886db3806d5bf634145fe431dca9577235fab39720ac144b16c1b4fcb0a9
1057+
uri: huggingface://empero-ai/Qwythos-27B-v1-GGUF/mmproj-Qwythos-27B-F16.gguf
1058+
- !!merge <<: *qwythos-27b-v1
1059+
name: "qwythos-27b-v1-mtp"
1060+
variants: []
1061+
description: |
1062+
Qwythos-27B-v1 MTP is the Q4_K_M build with its native multi-token
1063+
prediction head enabled for faster speculative decoding. It also includes
1064+
the shared vision projector and supports tool use and long-context
1065+
reasoning.
1066+
tags:
1067+
- llm
1068+
- gguf
1069+
- vision
1070+
- multimodal
1071+
- reasoning
1072+
- thinking
1073+
- mtp
1074+
overrides:
1075+
backend: llama-cpp
1076+
function:
1077+
automatic_tool_parsing_fallback: true
1078+
grammar:
1079+
disable: true
1080+
known_usecases:
1081+
- chat
1082+
mmproj: llama-cpp/mmproj/Qwythos-27B-MTP-Q4_K_M/mmproj-Qwythos-27B-F16.gguf
1083+
options:
1084+
- use_jinja:true
1085+
- spec_type:draft-mtp
1086+
- spec_n_max:6
1087+
- spec_p_min:0.75
1088+
parameters:
1089+
model: llama-cpp/models/Qwythos-27B-MTP-Q4_K_M/Qwythos-27B-MTP-Q4_K_M.gguf
1090+
repeat_penalty: 1.05
1091+
temperature: 0.6
1092+
top_k: 20
1093+
top_p: 0.95
1094+
template:
1095+
use_tokenizer_template: true
1096+
files:
1097+
- filename: llama-cpp/models/Qwythos-27B-MTP-Q4_K_M/Qwythos-27B-MTP-Q4_K_M.gguf
1098+
sha256: 51cce0271e7ccf25d5892ec8d731bd21e7e80c9a78e291d2f473c7337656ecaa
1099+
uri: huggingface://empero-ai/Qwythos-27B-v1-GGUF/Qwythos-27B-MTP-Q4_K_M.gguf
1100+
- filename: llama-cpp/mmproj/Qwythos-27B-MTP-Q4_K_M/mmproj-Qwythos-27B-F16.gguf
1101+
sha256: 82af886db3806d5bf634145fe431dca9577235fab39720ac144b16c1b4fcb0a9
1102+
uri: huggingface://empero-ai/Qwythos-27B-v1-GGUF/mmproj-Qwythos-27B-F16.gguf
10101103
- name: "qwythos-9b-v2"
10111104
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
10121105
urls:
@@ -1019,6 +1112,7 @@
10191112
- vision
10201113
- multimodal
10211114
- reasoning
1115+
- mtp
10221116
overrides:
10231117
backend: llama-cpp
10241118
function:

0 commit comments

Comments
 (0)