ValueError: invalid literal for int() with base 10: 'layers'

[Tradunsky]

Bug: Number of layers parsed incorrectly when layer names not defined.

Root cause: the model layers key from the index file after split by "." will have the following components: "model" [0], "layers" [1], "" [2].
In some cases, like Qwen/Qwen3.6-27, the elements after split: "ge_model" [0], "layers" [1], "" [2], since the code does an interesting heuristic to cut a prefix.

When layer_names parameter is missing, it uses the correct index after split by dot - 2, but when this parameter is defined, it uses 1.

I could not locate the index file spec in safetensors docs: https://huggingface.co/docs/safetensors/en/index

I checked the following models:
https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/model.safetensors.index.json
https://huggingface.co/timdettmers/guanaco-33b-merged/blob/main/pytorch_model.bin.index.json
https://huggingface.co/Qwen/Qwen3-235B-A22B/blob/main/model.safetensors.index.json
https://huggingface.co/moonshotai/Kimi-K3/blob/main/model.safetensors.index.json (different key arrangement)
https://huggingface.co/zai-org/GLM-5.2/blob/main/model.safetensors.index.json

My understanding that there must be a model index file that actually will result in the index 1 to correctly point to a layer number, but I could not find such.

Question: Overall, the parsing logic and the format are loosely defined, which leads to brittle parsing logic. Do you know if there is a case where it would require to take the element at 1 position after the key split by dot or we can simply fix it by changing the split index to 2?

Reproduction code

from airllm import AutoModel

MAX_LENGTH = 128
# just pass a hugging face repo id — works with almost any popular model:
model = AutoModel.from_pretrained("Qwen/Qwen3.6-27B")

# go bigger with the exact same one line:
#model = AutoModel.from_pretrained("Qwen/Qwen3-235B-A22B")     # 235B, runs in ~3GB
#model = AutoModel.from_pretrained("deepseek-ai/DeepSeek-V3")  # 671B, runs in ~12GB

# or use a model's local path...
# model = AutoModel.from_pretrained("/home/ubuntu/.cache/huggingface/hub/models--Qwen--Qwen3-32B/snapshots/...")

input_text = [
    'What is the capital of United States?',
    #'I like',
]

input_tokens = model.tokenizer(input_text,
                               return_tensors="pt",
                               return_attention_mask=False,
                               truncation=True,
                               max_length=MAX_LENGTH,
                               padding=False)

generation_output = model.generate(
    input_tokens['input_ids'].cuda(),
    max_new_tokens=20,
    use_cache=True,
    return_dict_in_generate=True)

output = model.tokenizer.decode(generation_output.sequences[0])

print(output)

Traceback

using generic AirLLM streaming model for architecture: Qwen3_5ForConditionalGeneration
Fetching 14 files: 100%|██████████| 14/14 [00:00<00:00, 319.98it/s]
Traceback (most recent call last):
File "/home/pc/test/model.py", line 5, in <module>
model = AutoModel.from_pretrained("Qwen/Qwen3.6-27B")
File "/home/pc/test/.venv/lib/python3.14/site-packages/airllm/auto_model.py", line 61, in from_pretrained
return class_(pretrained_model_name_or_path, *inputs, **kwargs)
File "/home/pc/test/.venv/lib/python3.14/site-packages/airllm/airllm_base.py", line 131, in __init__
self.model_local_path, self.checkpoint_path = find_or_create_local_splitted_path(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
    model_local_path_or_repo_id,
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                              ...<3 lines>...
hf_token=hf_token,
^^^^^^^^^^^^^^^^^^
delete_original=delete_original)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pc/test/.venv/lib/python3.14/site-packages/airllm/utils.py", line 554, in find_or_create_local_splitted_path
return Path(hf_cache_path), split_and_save_layers(hf_cache_path, layer_shards_saving_path,
                                                  ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                                  compression=compression, layer_names=layer_names,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
delete_original=delete_original, repo_id=model_local_path_or_repo_id, hf_token=hf_token)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pc/test/.venv/lib/python3.14/site-packages/airllm/utils.py", line 284, in split_and_save_layers
n_layers = len(set([int(k[len(layer_names['layer_prefix']):].split('.')[1]) for k in index.keys() if layer_names['layer_prefix'] in k]))
~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: 'layers'

Debug screenshots

[HEETMEHTA18]

I dug into this and can point to the exact root cause + a fix.

The crash comes from air_llm/airllm/utils.py, lines 282–284:

​python if layer_names is None: n_layers = len(set([int(k.split('.')[2]) for k in index.keys() if 'model.layers' in k])) else: n_layers = len(set([int(k[len(layer_names['layer_prefix']):].split('.')[1]) for k in index.keys() if layer_names['layer_prefix'] in k])) ​

This assumes every checkpoint key containing "model.layers" starts with model.layers.N..., so the layer index always sits at split('.')[2]. That breaks for checkpoints where the causal LM is nested under a wrapper module — e.g. a key like language_model.model.layers.0.self_attn.q_proj.weight still matches the 'model.layers' in k filter, but k.split('.')[2] is 'layers', not '0', giving exactly:

​ ValueError: invalid literal for int() with base 10: 'layers' ​

Repro:
​python k = 'language_model.model.layers.0.self_attn.q_proj.weight' int(k.split('.')[2]) # -> ValueError: invalid literal for int() with base 10: 'layers' ​

Worth noting the codebase already handles this same class of problem correctly elsewhere — the per-expert MoE key parser in airllm_base.py (~line 660) uses .find() + an .isdigit() guard instead of a fixed split index. This bug is just that same fix not having been applied to the layer-counting code in utils.py.

I've got a fix that applies the same defensive pattern (tested against both a normal flat checkpoint and a nested language_model.model.layers... checkpoint) — opening a PR for it now.