adding-models.md

Adding a New Model

Models are registered in ModelRegistry. Adding a new model requires only a configuration entry — no code changes.

Step 1: Download the Model

Place MLX-format model files in your models directory (default: ~/models/):

# Example: download a model
bash scripts/download-smollm3.sh ~/models

Step 2: Register in ModelRegistry

Add an entry to the built-in catalog or register at runtime:

runtime.registry.register({
  name: 'MyModel',
  path: 'MyModel-7B-mlx',        // relative to modelsDirectory
  architecture: 'Llama',
  contextSize: 8192,
  parameters: '7B',
  quantization: '4bit',
  tokenizer: 'Llama',
  description: 'My custom model',
});

Or edit src/models/ModelRegistry.ts for permanent registration.

Step 3: Set as Default (Optional)

Update runtime.config.json:

{
  "defaultModel": "MyModel"
}

Step 4: Load and Use

const runtime = await Runtime.create();
await runtime.loadModel('MyModel');

const result = await runtime.generate({
  messages: [{ role: 'user', content: 'Hello!' }],
});

Model Metadata Fields

Field	Required	Description
name	Yes	Unique identifier used in API calls
path	Yes	Directory name or absolute path
architecture	Yes	Model architecture (SmolLM3, Qwen3, etc.)
contextSize	Yes	Maximum context window
parameters	Yes	Parameter count (e.g. "3B", "7B")
quantization	Yes	Quantization level (4bit, 8bit, fp16)
tokenizer	Yes	Tokenizer identifier
description	No	Human-readable description

Pre-registered Models

Name	Architecture	Parameters	Context
SmolLM3	SmolLM3	3B	8192
Qwen3	Qwen3	4B	32768
Gemma	Gemma	2B	8192
Phi	Phi	3.8B	4096
DeepSeek	DeepSeek	7B	8192

Download the corresponding MLX-format weights before use.