Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ Keep changes focused and preserve Colibri's dependency-free default CPU path.
it into `main`. This keeps `main` clean instead of taking every PR one at a time.

Every PR — on either branch — is reviewed for a clean build (0 warnings), the oracle
(32/32 TF + 20/20 greedy), and its own targeted validation before merge.
(~30-32/32 TF depending on floating-point near-ties + 20/20 greedy), and its own
targeted validation before merge.

## Local checks

Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,8 +157,9 @@ math on the unified-memory GPU.

### Faithful model, compressed state

The forward pass is validated **token-exact against a `transformers` oracle**
(teacher-forcing 32/32). MLA attention stores a compressed KV state — 576
The forward pass is validated against a `transformers` oracle (teacher-forcing
typically 30-32/32; two tiny-oracle positions are floating-point near-ties and
toolchain-dependent). MLA attention stores a compressed KV state — 576
floats/token instead of 32,768 (**57× smaller**) — and persists it across
restarts (`.coli_kv`): conversations reopen warm with zero re-prefill,
byte-identical to an uninterrupted session. DSA sparse attention (GLM-5.2's
Expand Down
7 changes: 4 additions & 3 deletions README.zh-TW.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,9 +101,10 @@ VRAM/RAM/硬碟層級長條,以及角落的即時迷你大腦。</em></p>

### 忠實模型,壓縮狀態

前向傳遞已透過 `transformers` oracle 驗證為**逐 token 完全一致**
(teacher-forcing 32/32)。MLA 注意力儲存壓縮後的 KV 狀態——每個 token 為 576 個
浮點數,而非 32,768 個(**縮小 57×**)——並跨重新啟動持久保存
前向傳遞已透過 `transformers` oracle 驗證(teacher-forcing 通常為
30-32/32;tiny oracle 中有兩個位置是浮點數近似平手,結果會受工具鏈影響)。
MLA 注意力儲存壓縮後的 KV 狀態——每個 token 為 576 個浮點數,而非 32,768 個
(**縮小 57×**)——並跨重新啟動持久保存
(`.coli_kv`):對話可暖啟恢復,不需重新 prefill,結果與不中斷的工作階段
逐位元組相同。DSA 稀疏注意力(GLM-5.2 的 lightning indexer)已忠實實作,
並透過強制選取所有 key,驗證可精確重現稠密注意力。
Expand Down
2 changes: 1 addition & 1 deletion c/colibri.c
Original file line number Diff line number Diff line change
Expand Up @@ -6666,7 +6666,7 @@ int main(int argc, char **argv){
fprintf(stderr,
"ERROR: no PROMPT given, so this is oracle self-test mode — but ref_glm.json is the TINY\n"
" model's oracle (max token %d) and your model's vocab is %d. Nothing to validate here.\n"
" Engine self-test: SNAP=./glm_tiny TF=1 ./glm 64 16 16 (expect 32/32)\n"
" Engine self-test: SNAP=./glm_tiny TF=1 ./glm 64 16 16 (expect ~30-32/32; FP near-ties are toolchain-dependent)\n"
" Real generation: PROMPT=\"Hello\" NGEN=32 SNAP=<model> ./glm 64\n"
" or: python coli chat --model <model>\n"
" REF_FORCE=1 to run the comparison anyway (meaningless).\n"
Expand Down
2 changes: 1 addition & 1 deletion c/setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ make -s colibri ARCH="${ARCH:-native}"
# 3) self-test sull'oracolo tiny, se presente
if [ -d glm_tiny ] && [ -f ref_glm.json ]; then
r=$(SNAP=./glm_tiny TF=1 ./colibri 64 16 16 2>/dev/null | grep -oE "[0-9]+/[0-9]+ positions" || true)
echo " engine self-test: ${r:-?} (expected 32/32)"
echo " engine self-test: ${r:-?} (expected ~30-32/32; FP near-ties are toolchain-dependent)"
fi

# 4) info macchina (la velocità dipende da QUESTI due numeri, non dalla GPU)
Expand Down
2 changes: 1 addition & 1 deletion docs/benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ cycle will heat cheaper drives. Monitor drive temperature and health.
## Test your machine, in order

```bash
cd c && ./setup.sh # build + architecture self-test (expects 32/32)
cd c && ./setup.sh # build + architecture self-test (expects ~30-32/32)

# 1) measure YOUR disk the way the engine uses it (parallel 19 MB random reads):
gcc -O2 -fopenmp iobench.c -o iobench
Expand Down
8 changes: 5 additions & 3 deletions docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,11 +96,13 @@ cd colibri/c
self-test. When it prints:

```
engine self-test: 32/32 (expected 32/32)
engine self-test: 32/32 (expected ~30-32/32; FP near-ties are toolchain-dependent)
```

the engine is working correctly. (On Windows Option A you already have the
binary — you can skip this step.)
the engine is working correctly. Some toolchains report 30/32 or 31/32 because
two tiny-oracle positions are floating-point near-ties; this is still a valid
self-test result. (On Windows Option A you already have the binary — you can
skip this step.)

---

Expand Down
2 changes: 1 addition & 1 deletion docs/windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ make glm.exe ARCH=native # banner prints "idot: avx-vnni"
# Verify (tiny model, 2.4 MB):
pip install torch transformers safetensors huggingface_hub
python tools/make_glm_oracle.py # generate tiny oracle
SNAP=./glm_tiny TF=1 ./glm.exe 64 16 16 # expect "32/32 positions"
SNAP=./glm_tiny TF=1 ./glm.exe 64 16 16 # expect "~30-32/32 positions"

# Run with real model:
SNAP=D:\glm52_i4 ./glm.exe 64 4 16 # batch inference
Expand Down
Loading