📊 Hardware Benchmarks — run edge0 on your device & share results

[shuk777]

Help us map how edge0 runs across real hardware! Run the benchmark on your machine and post your numbers as a reply. We'll curate submissions into a results table in the repo — with your handle credited.

Supported platform (today): macOS — Apple Silicon, Python / MLX

The one-command benchmark for this campaign currently supports macOS only (Apple Silicon, via the Python/MLX runtime).

cd python
python3.12 -m venv .venv && .venv/bin/pip install -e '.[dev,fetch]'
export EDGE0_8B_MODEL=$PWD/models/edge0-8b     # or EDGE0_35B_MODEL
BENCH_LONG=1 .venv/bin/python examples/bench.py edge0-8b

Paste this submission block as your reply

tier:            edge0-8b | edge0-35b
runtime:         python-mlx
device:          (e.g. MacBook Air M2)
chip:            (e.g. Apple M2)
ram_gb:
os:              (e.g. macOS 15.1)
storage:         internal SSD | external USB-SSD
mlx_version:     (pip show mlx | grep Version)
decode_tok_s:
prefill_tok_s:   (BENCH_LONG, long-context)
peak_active_gib:
on_battery:      yes | no
reporter:        @your-handle
notes:           anything unusual (throttling, background load, short vs long)

Example (a real submission):

tier: edge0-8b | runtime: python-mlx | chip: Apple M5 | ram_gb: 16
os: macOS 26.3.2 | decode_tok_s: 11.8 (long) / 13.5 (short)
prefill_tok_s: 231 cold / 314 warm | peak_active_gib: 3.19 | reporter: @shuk777

Windows / iOS / Android — PRs welcome!

These platforms don't have a community-ready benchmark yet (no one-command script that prints the submission block above). We'd love help building them — each is scoped as a good-first-issue:

• Windows: Benchmark: Windows — one-command script that prints the community submission block #139
• iOS: Benchmark: iOS — on-device benchmark via Edge0PhoneProbe that prints the submission block #140
• Android: Benchmark: Android — on-device benchmark producing the community submission block #141

Pick one, follow its checklist, and your script becomes the official way people benchmark edge0 on that platform.

Credit

Curated results will land in docs/benchmarks/ via PR, and every reporter is credited by handle. Want your name in the repo? Add your own row — we'll help.

Notes

• Numbers vary a lot with RAM and storage — low-memory machines are expected to be slower (expert pages get evicted and re-read from SSD). That's useful data too!
• Short vs long prompt differ a lot; please run BENCH_LONG=1 for a meaningful prefill number.

[Raphaeal19]

tier: edge0-8b
runtime: python-mlx
device: Macbook Pro M4 Pro
chip: Apple M4 Pro
ram_gb: 24gb
os: macOS Tahoe 26.6
storage: internal SSD
mlx_version: 0.32.1
decode_tok_s: 14.55 / 27.17
prefill_tok_s: 370 / 1086
peak_active_gib: 3.44GiB
on_battery: no
reporter: @Raphaeal19
notes:

This bench was ran with BENCH_LONG=1
[lora] applied=153 not_found=0 skipped=0 scale=2.0
[edge0-8b] prerouter installed: 16 heads, start=7, K=8
[edge0-8b] built in 0.4s
[bench] tier=edge0-8b ntok=200 warmup=10
prompt=3292 tok prefill=8.90s (370 tok/s) decode=200/13.74s tok/s=14.6 mlx_allocator_peak=2.68 GiB
prompt=3292 tok prefill=3.03s (1086 tok/s) decode=200/7.36s tok/s=27.2 mlx_allocator_peak=3.44 GiB
[bench] mean tok/s=20.9 mlx_allocator_peak<=3.44 GiB