freetoken

I Ran Qwen 3.8 27B on Just 4GB VRAM! (FreeToken Tutorial) - Edge-native MoE serving engine for local LLMs on low VRAM GPUs

Reproducible Apple-Silicon (Apple MLX) deployment and one-token verification kit for the FreeToken edge-native MoE serving API — pinned, loopback-only, schema-validated evidence.

Forked from FlashML-org/FreeToken, with AMD Strix Halo (gfx1151) ROCm support and kernel tuning

Reproducible FreeToken MoE expert-offload compatibility lab and benchmarks for NVIDIA Turing sm_75.

Unofficial FreeToken fork: Turing (RTX 20 series) support, image input over the OpenAI API, MTP speculative decoding, 64k context on an RTX 2060 6 GB, and layer-split serving over two consumer GPUs (Qwen3.8-Flash-Next with 128k context on two RTX 3060 12 GB, no NCCL). Written by Claude Fable 5.1.