[Bug]: The Engine Exited While Loading : `missing model.norm.weight`

[xbais]

Commit or version

153e6f4

Environment

My system :

No CUDA / Discrete GPU available.

Reproduction steps

1. Followed the instructions on Github to setup and download the model
2. Tried to start a chat, specifying the model location using --model flag : ./coli chat --model ./colibri_data/glm52_i4

Expected behavior

The chat should load in an interactive terminal session.

Actual behavior and logs

Logs : I am not sure how to get the logs.

Actual behavior :The program tries to "wake the giant", but fails with error : missing model.norm.weight

[ZacharyZcR]

model.norm.weight is the model's final layer-norm — the engine loads it right at the start (colibri.c: m->final_norm = ld(m,"model.norm.weight")), before any layer. "missing" means the tensor isn't present in the safetensors files in your model directory. That's almost always an incomplete download, not an engine bug or a wrong --model path (a wrong path gives a different error).

Why it happens: the GLM-5.2 int4 container is ~372 GB split across 100+ .safetensors shards. If the download stalled partway — the Hugging Face xet backend is known to silently hang or drop connections (see #452) — some shards never arrive, and model.norm.weight happens to live in one of the missing ones. A single missing shard produces exactly this: the engine finds most tensors, then dies on the first one from the gap.

Check which shard it needs and whether it's there. Your model directory should have model.safetensors.index.json (the map of tensor → shard):

cd ./colibri_data/glm52_i4
python3 -c "import json; m=json.load(open('model.safetensors.index.json'))['weight_map']; print('model.norm.weight is in:', m['model.norm.weight'])"

That prints the shard filename (e.g. model-00123-of-00144.safetensors). Confirm that file exists and isn't truncated (compare its size to the others — a stalled download often leaves a 0-byte or half-size file). More broadly, compare your shard count to the total the index expects:

ls model-*.safetensors | wc -l          # how many you have
python3 -c "import json; print(len(set(json.load(open('model.safetensors.index.json'))['weight_map'].values())))"   # how many there should be

If the first number is smaller, the download is incomplete.

Fix: re-run the download with the flaky backend disabled, which also resumes/repairs missing files:

HF_HUB_DISABLE_XET=1 hf download <the-repo-you-used> --local-dir ./colibri_data/glm52_i4

Two side notes: your build is 153e6f4, which is a few releases behind — worth updating once you're running, though it's not the cause here (a missing tensor is a container problem, not an engine one). And if you're picking a container fresh, the group-scaled mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp is the recommended one (better quality than the per-row containers — #455). If your shard count checks out complete and it still says missing, paste the output of the two commands above and it becomes a real bug worth digging into.

[JustVugg]

@ZacharyZcR nailed it — this isn't an engine bug, it's an incomplete download. model.norm.weight is the final layer-norm the engine loads first; "missing" means the shard it lives in never fully arrived. The GLM-5.2 int4 container is ~372 GB across 100+ .safetensors shards, and the Hugging Face xet backend is known to silently hang or drop mid-transfer (#452) — leaving a gap.

Please run the two checks from Zach's comment (they take a second):

cd ./colibri_data/glm52_i4
ls model-*.safetensors | wc -l
python3 -c "import json; print(len(set(json.load(open('model.safetensors.index.json'))['weight_map'].values())))"

If the first number is smaller than the second, the download is incomplete — re-run it with the flaky backend off (it resumes and repairs the missing shards):

HF_HUB_DISABLE_XET=1 hf download <the-repo-you-used> --local-dir ./colibri_data/glm52_i4

If the shard count matches and it still says missing, that would be a real bug — paste the two numbers and we'll dig in. (Side note: for a fresh download the group-scaled mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp is the recommended container — better quality than per-row, #455.)

[xbais]

Hello @JustVugg and @ZacharyZcR ! Thanks a lot for the help. For my installation, the ./colibri_data/glm52_i4 directory doesnt have any files that match the expression model-*.safetensors. Instead, here are the contents:

_meta                   out-00041.safetensors  out-00085.safetensors
config.json             out-00042.safetensors  out-00086.safetensors
generation_config.json  out-00043.safetensors  out-00087.safetensors
out-00000.safetensors   out-00044.safetensors  out-00088.safetensors
out-00001.safetensors   out-00045.safetensors  out-00089.safetensors
out-00002.safetensors   out-00046.safetensors  out-00090.safetensors
out-00003.safetensors   out-00047.safetensors  out-00091.safetensors
out-00004.safetensors   out-00048.safetensors  out-00092.safetensors
out-00005.safetensors   out-00049.safetensors  out-00093.safetensors
out-00006.safetensors   out-00050.safetensors  out-00094.safetensors
out-00007.safetensors   out-00051.safetensors  out-00095.safetensors
out-00008.safetensors   out-00052.safetensors  out-00096.safetensors
out-00009.safetensors   out-00053.safetensors  out-00097.safetensors
out-00010.safetensors   out-00054.safetensors  out-00098.safetensors
out-00011.safetensors   out-00055.safetensors  out-00099.safetensors
out-00012.safetensors   out-00056.safetensors  out-00100.safetensors
out-00013.safetensors   out-00057.safetensors  out-00101.safetensors
out-00014.safetensors   out-00058.safetensors  out-00102.safetensors
out-00015.safetensors   out-00059.safetensors  out-00103.safetensors
out-00016.safetensors   out-00060.safetensors  out-00104.safetensors
out-00017.safetensors   out-00061.safetensors  out-00105.safetensors
out-00018.safetensors   out-00062.safetensors  out-00106.safetensors
out-00019.safetensors   out-00063.safetensors  out-00107.safetensors
out-00020.safetensors   out-00064.safetensors  out-00108.safetensors
out-00021.safetensors   out-00065.safetensors  out-00109.safetensors
out-00022.safetensors   out-00066.safetensors  out-00110.safetensors
out-00023.safetensors   out-00067.safetensors  out-00111.safetensors
out-00024.safetensors   out-00068.safetensors  out-00112.safetensors
out-00025.safetensors   out-00069.safetensors  out-00113.safetensors
out-00026.safetensors   out-00070.safetensors  out-00114.safetensors
out-00027.safetensors   out-00071.safetensors  out-00115.safetensors
out-00028.safetensors   out-00072.safetensors  out-00116.safetensors
out-00029.safetensors   out-00073.safetensors  out-00117.safetensors
out-00030.safetensors   out-00074.safetensors  out-00118.safetensors
out-00031.safetensors   out-00075.safetensors  out-00119.safetensors
out-00032.safetensors   out-00076.safetensors  out-00120.safetensors
out-00033.safetensors   out-00077.safetensors  out-00121.safetensors
out-00034.safetensors   out-00078.safetensors  out-mtp-00000.safetensors
out-00035.safetensors   out-00079.safetensors  out-mtp-00001.safetensors
out-00036.safetensors   out-00080.safetensors  out-mtp-00002.safetensors
out-00037.safetensors   out-00081.safetensors  tokenizer.json
out-00038.safetensors   out-00082.safetensors  tokenizer_config.json
out-00039.safetensors   out-00083.safetensors
out-00040.safetensors   out-00084.safetensors

Then, counting the safetensors ...

ls out-*.safetensors | wc -l

.... gives a count of 125