Skip to content

Commit b92da46

Browse files
feat(gallery): add Ling 3.0 Tiny variants
Add Q4_K_M and Q8_0 GGUF builds for the 7.9B-parameter hybrid reasoning MoE model. Assisted-by: Codex:gpt-5
1 parent 4058194 commit b92da46

1 file changed

Lines changed: 80 additions & 0 deletions

File tree

gallery/index.yaml

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,84 @@
11
---
2+
- &ling-3-0-tiny
3+
name: "ling-3.0-tiny-q4"
4+
variants:
5+
- model: ling-3.0-tiny-q8
6+
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
7+
urls:
8+
- https://huggingface.co/inclusionAI/Ling-3.0-tiny
9+
- https://huggingface.co/bloomer010/Ling-3.0-tiny-GGUF
10+
description: |
11+
Ling-3.0-tiny is InclusionAI's MIT-licensed hybrid reasoning MoE model
12+
with 7.9B total parameters and 1.3B active parameters per token. It
13+
targets reasoning, coding, instruction following, and agentic tasks with
14+
a native 131K-token context window.
15+
16+
This default entry uses the Q4_K_M GGUF. A higher-quality Q8_0 model is
17+
available as a variant.
18+
license: "mit"
19+
tags:
20+
- llm
21+
- gguf
22+
- cpu
23+
- gpu
24+
- moe
25+
- reasoning
26+
- thinking
27+
- coding
28+
- agent
29+
- tools
30+
- long-context
31+
last_checked: "2026-08-18"
32+
overrides:
33+
backend: llama-cpp
34+
context_size: 131072
35+
function:
36+
automatic_tool_parsing_fallback: true
37+
grammar:
38+
disable: true
39+
known_usecases:
40+
- chat
41+
options:
42+
- use_jinja:true
43+
parameters:
44+
model: llama-cpp/models/ling-3.0-tiny/Ling-3.0-tiny-Q4_K_M.gguf
45+
temperature: 1
46+
top_k: 20
47+
top_p: 0.95
48+
template:
49+
use_tokenizer_template: true
50+
files:
51+
- filename: llama-cpp/models/ling-3.0-tiny/Ling-3.0-tiny-Q4_K_M.gguf
52+
uri: huggingface://bloomer010/Ling-3.0-tiny-GGUF/Ling-3.0-tiny-Q4_K_M.gguf
53+
sha256: 23e0f9e8f5ad8c44079b80a2a6c98fc507d2471fd906079467cca73d9fd81e10
54+
- !!merge <<: *ling-3-0-tiny
55+
name: "ling-3.0-tiny-q8"
56+
variants: []
57+
description: |
58+
Ling-3.0-tiny in the higher-quality Q8_0 GGUF format. This variant
59+
preserves more model fidelity for hosts with enough memory.
60+
overrides:
61+
backend: llama-cpp
62+
context_size: 131072
63+
function:
64+
automatic_tool_parsing_fallback: true
65+
grammar:
66+
disable: true
67+
known_usecases:
68+
- chat
69+
options:
70+
- use_jinja:true
71+
parameters:
72+
model: llama-cpp/models/ling-3.0-tiny/Ling-3.0-tiny-Q8_0.gguf
73+
temperature: 1
74+
top_k: 20
75+
top_p: 0.95
76+
template:
77+
use_tokenizer_template: true
78+
files:
79+
- filename: llama-cpp/models/ling-3.0-tiny/Ling-3.0-tiny-Q8_0.gguf
80+
uri: huggingface://bloomer010/Ling-3.0-tiny-GGUF/Ling-3.0-tiny-Q8_0.gguf
81+
sha256: 326221bdef72525bd6178b072d3827cd5fb5248a55e8c44eb400417f401733bc
282
- &ornith-1-0-9b
383
name: "ornith-1.0-9b-q4"
484
variants:

0 commit comments

Comments
 (0)