A 64-bit x86 operating system engineered from the ground up.
Vailism OS is an experimental operating system written in C and x86_64 Assembly, developed with the goal of understanding — and controlling — the complete software stack between the processor and the user.
This is not a Linux distribution, desktop skin, or collection of shell scripts. The objective is to build the system itself from bare metal:
operating-system osdev x86_64 kernel bare-metal limine-bootloader assembly c qemu paging interrupts low-level
Vailism is designed as a layered system. Hardware-dependent mechanisms remain below the kernel’s portable core, while userspace interacts with the machine through explicit kernel interfaces.
VAILISM OS
x86_64 Operating System
│
▼
┌───────────────────┐
│ Physical Machine │
│ x86_64 CPU │
└─────────┬─────────┘
│
BIOS / UEFI
│
▼
┌───────────────────┐
│ Limine │
│ Boot Protocol │
└─────────┬─────────┘
│
CPU state / memory map / framebuffer
│
▼
┌──────────────────────────┐
│ Kernel Entry │
│ kmain │
└────────────┬─────────────┘
│
┌──────────────────────┼──────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Architecture │ │ Drivers │ │ Memory │
│ Layer │ │ │ │ Subsystem │
├─────────────────┤ ├─────────────────┤ ├─────────────────┤
│ GDT │ │ Serial (COM1) │ │ Physical Memory │
│ IDT (256 gates) │ │ Framebuffer │ │ Paging (4-Level)│
│ Exceptions 0-31 │ │ Keyboard (PS/2) │ │ Virtual Memory │
│ IRQs (8259 PIC) │ │ Timer (8254 PIT)│ │ Kernel Heap │
│ CPU Control │ │ Storage (ATA) │ │ Allocators │
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘
│ │ │
└──────────────────────┼──────────────────────┘
│
▼
┌──────────────────────────┐
│ Kernel Core │
├──────────────────────────┤
│ Scheduler │
│ Processes / Threads │
│ IPC │
│ VFS / Filesystems │
│ System Call Interface │
└────────────┬─────────────┘
│
Ring 0 → Ring 3
│
▼
┌──────────────────────────┐
│ Userland │
├──────────────────────────┤
│ libc │
│ Shell │
│ Core Utilities │
│ Applications │
└────────────┬─────────────┘
│
▼
┌──────────────────────────┐
│ Vailism UI │
├──────────────────────────┤
│ Framebuffer │
│ Compositor │
│ Window Manager │
│ Mouse / Input │
│ Applications │
└──────────────────────────┘
┌─────────────────────────────────────────────────┐
│ DEVELOPMENT INFRASTRUCTURE │
├─────────────────────────────────────────────────┤
│ Cross Compiler │ Linker │ Make │ QEMU │ GDB │
│ Serial Debug │ ISO │ CI │ Tests │
└─────────────────────────────────────────────────┘
Vailism is developed around clear engineering principles:
A subsystem should first exist in a form that makes its interaction with the hardware understandable. Abstractions are introduced when they solve a real architectural problem — not simply because a framework makes the code shorter.
The kernel owns privileged operations. Userspace interacts with the kernel through defined interfaces rather than directly manipulating hardware.
USERSPACE
│
│ system call
▼
┌───────────────┐
│ KERNEL │
├───────────────┤
│ Scheduler │
│ Memory │
│ Filesystem │
│ Drivers │
└───────┬───────┘
│
▼
HARDWARE
The project deliberately exposes the mechanisms normally hidden by modern operating systems:
- CPU privilege levels (Ring 0 vs Ring 3)
- 4-level page tables & translation lookaside buffers (TLB)
- Interrupt descriptors and hardware IRQs
- Physical memory frames and virtual address spaces
- Device registers and port I/O
- Context switching & register frames
- System calls & executable loading
The initial boot path is intentionally simple and deterministic:
Power On
│
▼
Firmware (BIOS / UEFI)
│
▼
Limine Bootloader
├── Load 64-bit ELF Kernel
├── Discover Physical Memory Map
├── Initialize Linear Graphical Framebuffer
└── Enter 64-bit Long Mode
│
▼
Vailism Kernel Entry (kmain)
│
├── CPU & Architecture Initialization (GDT, TSS, IST)
├── Interrupt Subsystem (IDT, Exceptions 0-31, 8259 PIC)
├── Device Drivers (Serial COM1, Framebuffer, PIT Timer, PS/2 Keyboard)
├── Memory Management (PMM Bitmap, VMM 4-Level Paging, Kernel Heap)
├── Kernel Core Services (Scheduler, Multitasking)
└── Userspace Transition (Ring 3)
The bootloader is kept outside the kernel’s responsibilities. Vailism consumes the standard Limine protocol and focuses on what happens after control reaches the kernel.
- Architecture: x86_64 Long Mode (64-bit).
- Privilege Level: Ring 0 (Supervisor Execution).
- Virtual Memory Base: Higher-half kernel mapped at
0xffffffff80000000.
- Protocol: Limine Boot Protocol (v8 specification).
- Firmware: Dual boot support for both UEFI and legacy BIOS.
- GDT (Global Descriptor Table): 64-bit Kernel Code (
0x08), Kernel Data (0x10), User Data (0x18), User Code (0x20), and 64-bit TSS (0x28). - TSS (Task State Segment): Configured with
RSP0kernel stack and dedicatedIST1stack for safe Double Fault exception recovery. - IDT (Interrupt Descriptor Table): 256 interrupt gates handling all 32 CPU exceptions (
#DE,#DF,#GP,#PF) with full CPU register state preservation (struct registers). - Interrupt Controller: 8259 PIC remapped to vectors
0x20..0x2F(IRQs 0..15). - Timer Subsystem: 8254 PIT configured to 100 Hz (10 ms ticks) driving uptime tracking and
timer_sleep(). - Keyboard Driver: PS/2 controller driver on IRQ1 (Vector 33) decoding Scancode Set 1 with Shift/Caps Lock/Enter/Backspace support.
- Physical Memory Manager (PMM): Bitmap page frame allocator tracking 4 KiB frames across all physical RAM.
-
Virtual Memory Manager (VMM): 4-level x86_64 page tables (
PML4$\rightarrow$ PDPT$\rightarrow$ PD$\rightarrow$ PT) with page mapping, unmapping, and TLB invalidation (invlpg). -
Kernel Heap Allocator: Coalescing free-list allocator providing
kmalloc(),kfree(),kcalloc(), andkrealloc()with 16-byte alignment and block splitting starting at0xffffffffc0000000.
- Serial Port (COM1
0x3F8): 16550 UART driver operating at 38400 baud. Serial output provides rock-solid diagnostics to the host terminal even during early kernel faults. - Graphical Framebuffer Console: Direct VRAM linear framebuffer rendering 32-bit ARGB color with an embedded 8x16 VGA bitmap font engine, cursor tracking, and terminal scrolling.
Vailism/
├── Makefile # Cross-compilation & ISO packaging
├── linker.ld # Memory layout & higher-half section mapping
├── limine.conf # Bootloader menu configuration
├── limine-bin/ # Vendored Limine binary boot assets
│
├── src/
│ ├── arch/
│ │ └── x86_64/
│ │ ├── gdt_flush.asm # GDT reload & 64-bit far return
│ │ └── interrupts.asm # ISR & IRQ assembly stubs, context save/restore
│ │
│ ├── include/
│ │ ├── types.h # Standard integer types (uint64_t, size_t, bool)
│ │ ├── io.h # Port I/O inline assembly (inb, outb, io_wait)
│ │ ├── limine.h # Official Limine boot protocol specification
│ │ ├── serial.h # 16550 UART driver header
│ │ ├── framebuffer.h # Linear RGB graphics & terminal header
│ │ ├── font8x16.h # Embedded 8x16 VGA bitmap font table
│ │ ├── gdt.h # GDT & TSS structures
│ │ ├── idt.h # IDT descriptors & register state structs
│ │ ├── pic.h # 8259 PIC controller definitions
│ │ ├── timer.h # 8254 PIT timer driver header
│ │ ├── keyboard.h # PS/2 keyboard driver header
│ │ ├── string.h # Freestanding string & memory operations
│ │ ├── memory.h # Page macros & address translation helpers
│ │ ├── pmm.h # Physical Memory Manager header
│ │ ├── vmm.h # Virtual Memory Manager & paging header
│ │ └── heap.h # Kernel Heap allocator header
│ │
│ └── kernel/
│ ├── main.c # Kernel entry point (kmain)
│ ├── serial.c # Serial port driver implementation
│ ├── framebuffer.c # Screen clearing, pixel plotting, font rendering
│ ├── gdt.c # GDT and TSS initialization
│ ├── idt.c # IDT gate setup and exception dispatcher
│ ├── pic.c # 8259 PIC remapping and EOI signalling
│ ├── timer.c # PIT timer tick handler and sleep functions
│ ├── keyboard.c # PS/2 scancode decoder and interactive echo
│ ├── string.c # Freestanding memset, memcpy, strlen, strcmp
│ ├── pmm.c # Physical memory bitmap frame allocator
│ ├── vmm.c # 4-level paging table walker & mapper
│ └── heap.c # Dynamic kernel heap allocator implementation
│
├── build/ # Generated build artifacts & bootable ISO
└── README.md
Vailism is developed and tested inside virtual machines. The host operating system is never modified during development.
┌──────────────────────────────────────┐
│ Host: macOS / Linux │
│ │
│ Compiler → Linker → ISO Generation │
│ │ │
│ ▼ │
│ QEMU │
│ │ │
│ ▼ │
│ Vailism OS │
└──────────────────────────────────────┘
One-command automated setup via Homebrew:
make setup-mac
# Or directly: ./scripts/setup_macos.shManual prerequisite installation:
brew install llvm nasm qemu xorriso# Build bootable ISO and launch in QEMU with scaled Cocoa window & serial stdio
make run
# Launch directly in macOS Fullscreen mode (16:10 scaled)
make run-fullscreen
# Build and run in headless mode (serial output only in terminal)
make run-nographic
# Clean build artifacts
make cleanVailism utilizes a canonical higher-half kernel memory layout:
Virtual Address Space
──────────────────────────────────────────────
0xFFFFFFFFFFFFFFFF
│
│ Kernel Space (Code, Data, Heap)
│
0xFFFFFFFFC0000000 ← Kernel Heap Base (kmalloc pool)
│
0xFFFFFFFF80000000 ← Kernel Virtual Base (ELF Image)
│
│ Higher-Half Direct Map (HHDM)
│
0xFFFF800000000000 ← All Physical RAM mapped 1:1
│
│ Unmapped / Guard Pages
│
0x00007FFFFFFFFFFF ← User Space Ceiling (Ring 3)
│
│ User Applications & Stack
│
0x0000000000000000
The memory subsystem is divided into three distinct layers:
Physical Memory
│
▼
Physical Memory Manager (PMM Bitmap)
│
▼
Page Tables / Virtual Memory Manager (4-Level Paging)
│
▼
Kernel Heap / Allocators (kmalloc / kfree)
Applications
│
▼
System Call API
│
▼
VFS
┌──────┴──────┐
▼ ▼
FAT Ext2
│ │
└──────┬──────┘
▼
Block Device
│
▼
Hardware
- Architecture Layer: Handles x86_64 GDT, TSS, IDT, CPU exceptions, IRQs, APIC, context switching, and assembly stubs.
- Memory Manager: Controls physical page allocation, 4-level page tables, virtual address spaces, and kernel heap.
- Scheduler: Manages threads, processes, context switches, preemption, and scheduling policies.
- Drivers: Provides abstract interfaces to hardware (UART, Framebuffer, Keyboard, PIT Timer, Storage, Mouse).
- Filesystem (VFS): Separates generic POSIX-like file operations (
open,read,write,close) from concrete disk filesystem implementations (FAT32, Ext2).
RING 3
┌────────────────────┐
│ Shell │
│ Applications │
│ libc │
└─────────┬──────────┘
│
SYSTEM CALL
│
▼
RING 0
┌────────────────────┐
│ Kernel │
│ │
│ Scheduler │
│ Memory Manager │
│ Filesystem │
│ Drivers │
└────────────────────┘
The privilege boundary is fundamental. Applications in Ring 3 cannot:
- Modify kernel memory or page tables
- Directly execute privileged CPU instructions (
cli,sti,invlpg,mov cr3, ...) - Directly manipulate device I/O ports
- Interfere with another process's memory space
All services are mediated through the System Call Interface.
- Phase 1 — Kernel Bring-Up: Limine boot, x86_64 entry, higher-half mapping, 16550 UART serial driver, linear framebuffer console, 8x16 font renderer, QEMU runner.
- Phase 2 — CPU & Interrupts: GDT, 64-bit TSS, IDT (256 gates), 32 CPU exception handlers, 8259 PIC remapping, 8254 PIT timer (100 Hz), PS/2 keyboard driver with interactive echo.
- Phase 3 — Memory Management: Physical Memory Manager (PMM bitmap), 4-level paging Virtual Memory Manager (VMM), Kernel Heap allocator (
kmalloc,kfree,kcalloc,krealloc). - Phase 4 — Execution & Multitasking: Process & Thread Control Blocks (PCB/TCB), CPU context switching (
context_switch), preemptive timer-driven round-robin scheduler, non-blockingthread_sleep(), cooperativeyield(), thread lifecycle management. - Phase 5 — Storage & Filesystem: ATA/IDE hard disk driver (LBA28 PIO), Virtual Filesystem (VFS) abstraction, In-Memory Root Filesystem (RamFS), file descriptors, POSIX file API (
open,read,write,close,mkdir). - Phase 6 — Userspace & Interactive Shell: Fast x86_64
syscall/sysretMSR interface (SYS_read,SYS_write,SYS_open,SYS_close,SYS_yield,SYS_sleep,SYS_reboot), full interactive kernel shell with built-in commands (help,clear,echo,ls,cat,mkdir,touch,write,mem,uname,reboot). - Phase 7 — Graphics, Mouse & Desktop GUI: PS/2 mouse driver on IRQ12 (Vector 44), double-buffering compositor, 2D graphics primitives (gradients, shadows, outlines, font), floating window manager with draggable windows ("System Monitor", "Kernel Terminal", "File Explorer"), and live taskbar telemetry.
Every major subsystem follows the same progression:
┌──────────────┐
│ Understand │
│ the hardware │
└──────┬───────┘
│
▼
┌──────────────┐
│ Minimal │
│implementation│
└──────┬───────┘
│
▼
┌──────────────┐
│ Test in QEMU │
└──────┬───────┘
│
▼
┌──────────────┐
│ Debug through│
│ serial / GDB │
└──────┬───────┘
│
▼
┌──────────────┐
│ Formalize API│
│& abstraction │
└──────┬───────┘
│
▼
┌──────────────┐
│ Integrate │
│ with kernel │
└──────────────┘
The project favors working vertical slices over disconnected components. For example, the scheduler is proven by: $$\text{Timer Interrupt} \longrightarrow \text{Scheduler Invoked} \longrightarrow \text{Thread Selected} \longrightarrow \text{Context Switched} \longrightarrow \text{Thread Executes}$$
Kernel development fails differently from application development: invalid operations can cause triple faults, silent CPU resets, or corrupted memory tables.
Vailism maintains dual independent debugging paths:
Kernel
/ \
/ \
▼ ▼
Framebuffer Serial (COM1)
│ │
Visual Display Machine Logs
│
▼
GDB
- Framebuffer: For visual presentation, interactive shell, desktop compositor, and terminal output.
- Serial (COM1): Rock-solid stream of telemetry and kernel crash traces.
- QEMU + GDB: For instruction-level single-stepping and memory inspection.
Vailism is deliberately not:
- A Linux distribution or modified Linux kernel
- A collection of userland scripts
- A desktop theme or terminal wrapper
- A toy demonstration that stops at "Hello World"
The project is dedicated to engineering a complete, self-contained operating system from bare silicon.
Current Milestone: All 7 Core Phases Complete & Operational!
The system currently runs:
The ultimate goal is to understand and control every stage of execution:
POWER
│
▼
FIRMWARE
│
▼
BOOTLOADER
│
▼
KERNEL ENTRY
├── CPU (GDT / IDT)
├── MEMORY (PMM / VMM / Heap)
├── INTERRUPTS (PIC / APIC / Exceptions)
├── DEVICES (Serial / Framebuffer / Keyboard / Timer)
├── STORAGE (ATA / VFS)
└── SCHEDULER (Processes / Threads)
│
▼
SYSCALL (Ring 3 Transition)
│
▼
USERLAND (libc / Shell / Utilities)
│
▼
APPLICATION (Window Manager / GUI)
Thanks to the contributors who have helped build and improve Vailism OS:
ashbornn29
Vailism OS is licensed under the MIT License.