KernelGPT is a from-scratch implementation of a GPT (Generative Pre-trained Transformer) written in pure C, running directly on bare metal x86 hardware.
It boots a heavily modified vesion of MooseOS solely to train a neural network. No Linux, no Python, no PyTorch, no Standard Library (libc) — just raw pointers, math, and attention mechanisms.
It is heavily inspired off Andrej Karpathy's MicroGPT.
KernelGPT takes MooseOS's source code, and removes unneeded functions like filesystems, ATA support, GUI, etc, leaving only a mostly bare-bones kernel. KernelGPT sits directly inside the kernel.
Inspiration for the LLM design choice was taken from Andrej Karpathy's MicroGPT. This meant implementation for a GPT-2 style LLM, while keeping Karpathy's choice of switching Layernorm for RMSNorm and GELU for ReLU. To merge the AI into MooseOS, KernelGPT uses an arena allocator for easy memory allocation, and initialises the FPU for floating-point arithmetic.
MicroGPT’s names.txt was converted into a C header before runtime so KernelGPT can access the vocabulary directly in memory, avoiding any filesystem dependencies.
To run the project, simply do
cd kernelgpt/
make runYou would need QEMU to run the project. Since this is a modified version of MooseOS, please check its README for the comprehensive usage guide.
For Mac users, I have provided a brew.txt with the packages you need to install.
MIT License. Based on Andrej Karpathy's MicroGPT.