Skip to content
bishalramdamPublic

About

A minimal Linux system built from scratch: mainline kernel + my own init and shell, booting on aarch64 QEMU

Resources

Stars

2 stars

Watchers

0 watching

Forks

Latest commit

Β 

History

92 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

BishOS 🐧

Build ISOs

A minimal Linux operating system built modularly like Lego blocks: combining the mainline Linux kernel (compiled from source) with custom and open-source userspace components (/init, shell, standard utilities) to learn operating system fundamentals from the ground up.

Builds for two architectures from the same source:

x86_64 (default) arm64
Kernel image bzImage Image
ISO firmware UEFI and legacy BIOS UEFI
QEMU machine PC (qemu-system-x86_64) virt (qemu-system-aarch64)
Acceleration on Apple Silicon none (TCG software emulation) HVF hardware virtualization
Serial console ttyS0 (16550 UART) ttyAMA0 (PL011 UART)
Network card Intel e1000 virtio-net

🧱 The Lego Block Architecture

+-------------------------------------------------------------+
| 1. The Linux Kernel (x86_64 bzImage / arm64 Image)          |
|    - Manages hardware, CPU scheduling, memory, and devices  |
+-------------------------------------------------------------+
                              β”‚
                     boots and executes
                              β–Ό
+-------------------------------------------------------------+
| 2. The Initramfs (Initial RAM Filesystem / rootfs)          |
|    - SVR4 newc cpio.gz archive unpacked into memory         |
|    - Contains root directory structure (/bin, /proc, /sys)  |
+-------------------------------------------------------------+
                              β”‚
                  launches PID 1 (first process)
                              β–Ό
+-------------------------------------------------------------+
| 3. Early Init (initramfs stage, PID 1)                      |
|    - Mounts /proc, /sys, /dev                               |
|    - Finds a persistent ext4 root on vda/sda/nvme0n1        |
|    - switch_root into it and hands over to /sbin/init       |
|    - No disk? Keeps running from RAM instead                |
+-------------------------------------------------------------+
                              β”‚
                    switch_root (MS_MOVE)
                              β–Ό
+-------------------------------------------------------------+
| 4. Real Root Filesystem (persistent ext4 disk)              |
|    - Minimal static C init (PID 1) + BusyBox userland       |
|    - DHCP networking, login shell, graceful shutdown        |
|    - Files written here survive reboots                     |
+-------------------------------------------------------------+

Two boot paths

The diagram above is how the ISO boots, and it has to: a live image has no root partition, so its initramfs is the root. That is the one situation an initramfs genuinely solves, and it is the only place bishos-install can run.

An installed system no longer uses any of it. Every driver needed to reach the root -- ahci, sd, usb-storage, ext4 -- is built into the kernel, so the kernel mounts the root itself and there is nothing left for an initramfs to do:

kernel  ->  mounts root by PARTUUID, waits for it with rootwait
        ->  /bin/init (BusyBox), which reads /etc/inittab
        ->  OpenRC brings up the services
        ->  getty on tty1  ->  login  ->  .bash_profile  ->  sway

root= names a GPT PARTUUID, not the filesystem label the initramfs used. The kernel reads the partition table before any filesystem driver exists, so a label stored inside a filesystem cannot be read at that point; the kernel accepts only PARTUUID= and PARTLABEL=. PARTUUID also stays unambiguous once a second partition labelled BISHOS exists on another disk.

rootwait replaces the 45-second wait init.c did by hand, and waits as long as it takes rather than giving up.

Both paths are in the boot menu, so the old one is always one keypress away. /sbin/init is still the init.c binary and the default entry names init=/bin/init explicitly, which is what lets the two coexist with nothing on disk renamed.

πŸš€ Quickstart

Prerequisites

Tool Purpose Install (macOS)
Docker Desktop Linux build environment (ELF static binaries) Docker Desktop
QEMU Hardware emulator to boot the OS brew install qemu
Make Build automation Built-in

Build & Run

  1. Build everything β€” compiles Linux 6.18.46 from source (checksum-pinned), compiles init, and packs the initramfs:

    make all              # x86_64
    make ARCH=arm64 all   # arm64 (native + HVF-accelerated on Apple Silicon)

    The kernel source lives in a Docker volume; the first build compiles from scratch (~10-25 min), later builds are incremental.

  2. Boot in QEMU (creates a 1 GB persistent ext4 disk on first run):

    make run              # x86_64 (software-emulated on Apple Silicon)
    make ARCH=arm64 run   # arm64 (near-native speed on Apple Silicon)
  3. First boot sets the machine up. It asks for a username (default bishal), a hostname (default BishOS), and then a password for root and for your new account. Nothing is baked into the image -- an ISO carrying its author's username and machine name would put a stranger's details on your computer. Then log in, and use sudo when you need root:

    sudo apk add python3     # install something
    sudo su                  # become root

    Only a persistent root has accounts. A RAM-only boot (the ISO with no BishOS disk attached) drops straight into a root shell instead, because nothing written to /etc there would survive to be asked for again.

  4. SSH in, once you have asked for it. BishOS does not listen on port 22 until you say so. In the guest:

    sudo touch /etc/bishos/ssh.enabled

    Then from another terminal on the host:

    ssh -p 2222 you@localhost

    make run forwards that port into the guest. The answer is remembered across reboots, and sudo rm /etc/bishos/ssh.enabled && sudo pkill sshd takes it back. Root is refused over SSH by design -- log in as your own account and use sudo.

  5. Exit QEMU: Press Ctrl + A, release, then press X.

    The disk at build/$ARCH/bishos-disk.img persists across boots and is never overwritten by a rebuild. It defaults to 5 GB and is sparse, so it costs host disk only as it fills.

    make ARCH=arm64 disk-update               # refresh init and /etc, keep your data
    make ARCH=arm64 disk-grow                 # bigger, keeping your data
    make ARCH=arm64 DISK_SIZE=20G disk-grow   # any size you like
    make ARCH=arm64 disk-reset                # wipe and start clean

    Growing works in place because the image is a whole-disk ext4 filesystem with no partition table -- there is no partition to move before resizing.

    disk-update is the one to remember. Because a rebuild never touches an existing image, a disk in daily use goes on booting whatever init it was created with, and new features simply do not appear -- which looks exactly like new features being broken. It replaces /sbin/init and the BishOS configuration under /etc, and leaves accounts, home directories and installed packages alone.

Persistent live USB

init looks for its root on any block device -- whole disks and partitions alike -- and adopts the first ext4 filesystem labelled BISHOS. That is the whole mechanism, so a USB stick gains persistence by giving it such a partition.

Do not dd the ISO and try to add a partition afterwards. That is documented widely and does not work here: grub-mkrescue leaves a GPT whose secondary table overlaps the last partition, so sgdisk -e aborts with Aborting write of new partition table and the disk keeps reporting itself as 27 MB. sgdisk -n 0:0:0 then makes a 1.5 KiB partition in a gap rather than a 28 GB one at the end.

Install to it properly instead. On Linux, with /dev/sdX the stick -- check with lsblk -o NAME,SIZE,MODEL,TRAN first, because getting this wrong destroys the machine you are typing on:

sudo wipefs -a /dev/sdX
sudo sgdisk -Z /dev/sdX
sudo sgdisk -n 1:0:+512M -t 1:ef00 -c 1:BISHOSEFI \
            -n 2:0:0     -t 2:8300 -c 2:BISHOS /dev/sdX
sudo partprobe /dev/sdX
sudo mkfs.vfat -F32 -n BISHOSEFI /dev/sdX1
sudo mkfs.ext4 -L BISHOS /dev/sdX2

No bootloader needs installing: the ISO already contains one, and it is the same binary that draws the menu when the ISO is booted directly. Copy the whole thing onto the EFI partition -- 27 MB into 512 MB, and FAT is case-insensitive, so the ISO's /efi/boot/bootx64.efi lands exactly where firmware looks for /EFI/BOOT/BOOTX64.EFI on a removable device:

sudo mkdir -p /mnt/iso /mnt/esp
sudo mount -o loop,ro bishos-x86_64_v1.0.0.iso /mnt/iso
sudo mount /dev/sdX1 /mnt/esp
sudo cp -r /mnt/iso/. /mnt/esp/
sudo umount /mnt/iso /mnt/esp && sync

The label must be exactly BISHOS -- it is compared with strcmp. Anything else is ignored, which is what stops BishOS from adopting the internal drive of whatever machine it is booted on. Check it with sudo blkid -s LABEL -o value /dev/sdX2 | cat -A; a trailing space will show up as BISHOS $ and will not match.

The partition is not enough on its own. An empty filesystem with the right label gets as far as /dev/sdX2 has no /sbin/init, staying on initramfs: init mounts it, finds nothing to hand over to, and falls back to RAM. Boot the stick and put a system on it from BishOS itself:

bishos-install

It finds the BISHOS partition, asks before overwriting it, copies the running system across, and -- using the apk bundle carried on the boot media -- adds sudo, openssh and gcompat, which needs the network. Reboot and the next boot switches into the stick and asks for a username, a hostname and passwords, because there is now somewhere to keep them.

To install again over an existing one, boot the "BishOS (live -- ignore any installed system)" menu entry first. Once a stick has BishOS on it, init finds that root and switches into it, so the live system -- the only place the installer can run from -- is otherwise unreachable. That entry passes bishos.rootwait=0, which stops init looking. Then:

bishos-install --force

Secure Boot must be off, since this GRUB is unsigned.

USB storage can take a surprisingly long time to appear. One stick failed its first descriptor read at SuperSpeed, retried, and did not present a block device until 23 seconds into the boot -- so init waits 45 seconds for a root before falling back to RAM, printing its progress as it goes. bishos.rootwait=N on the kernel command line changes that.

The VM gets 2 GB of RAM by default (make MEMORY=4G run to change it). That figure also sizes /tmp: it is a tmpfs, so it defaults to half of RAM and lives in RAM. If an installer dies with No space left on device while df -h / shows plenty free, /tmp is what filled up -- either raise MEMORY or point the installer at disk with TMPDIR=/var/tmp.

  1. Build a bootable ISO -- GRUB on both architectures, same layout and the same grub.cfg; only the kernel binary differs:

    make ARCH=arm64 iso   # -> output/arm64/bishos-arm64_v1.0.0.iso   (UEFI)
    make iso              # -> output/x86_64/bishos-x86_64_v1.0.0.iso (UEFI + BIOS)

    VERSION in the Makefile is the single source of truth: it names the ISO and is compiled into init's banner, so a booted system always reports the version of the image it came from.

  2. Run things at boot -- init reads /etc/bishos/services:

    <name>  <action>  <command...>
    
    respawn   start at boot, restart whenever it exits
    once      run at boot, do not restart
    console   respawn, and give it the controlling terminal
    

    A service that dies immediately is retried with a growing delay rather than in a tight loop. With no table present, init falls back to a login shell on the console, which is what it always did.

    Anything a service prints is collected by init and written to /var/log/messages under that service's own name, alongside the kernel's own log, and it survives reboots:

    sudo tail -20 /var/log/messages
  3. Install software -- the persistent root ships with apk:

    sudo apk add python3     # or gcc, git, vim, curl, tmux, ...
    python3 --version

    sudo, because you are logged in as an ordinary user and installing software is root's business. A RAM-only boot has no accounts and no sudo, and the console there is root already. Installed packages survive reboots, because the root is a real disk. Prebuilt ISOs for both architectures are attached to every CI run and to tagged releases. In VMware Fusion: New VM -> drag the ISO in -> "Other Linux 6.x 64-bit Arm". The same shell lands on the serial port in QEMU and on the screen in VMware -- the kernel gives /dev/console to the last console= that exists.


πŸ—ΊοΈ Roadmap & Progress

  • Phase 1: Proof of Concept (The 1-Program OS)

    • Custom static C /init running as PID 1
    • newc cpio.gz initramfs generation
    • Direct kernel boot in QEMU with serial console
  • Phase 2: Interactive Shell & Core Utilities

    • Integrate static BusyBox
    • Mount pseudofilesystems (/proc, /sys, /dev)
    • Interactive sh prompt with controlling TTY & job control
    • User accounts & shell profile (/etc/passwd, /etc/group, /etc/profile, /etc/hostname)
  • Phase 3: Kernel Compilation from Source

    • Linux 6.18.46 (LTS) from kernel.org, pinned by SHA-256
    • Cross-compiled for x86_64 / built natively for arm64 in Docker
    • Custom minimal defconfig (currently stock defconfig per arch)
  • Phase 4: Networking & Internet Connectivity

    • NIC driver built into the kernel (e1000 on x86_64, virtio-net on arm64)
    • Auto-configuration of eth0 and default gateway
    • Google DNS resolution (8.8.8.8, 8.8.4.4)
    • Live Internet connectivity, ICMP ping, and HTTP web fetching
  • Phase 5: Multi-Architecture Support

    • Single Makefile drives both arches (make ARCH=arm64 ...)
    • Shared kernel source tree, per-arch out-of-tree (O=) build dirs
    • Hardware-accelerated arm64 boot on Apple Silicon via HVF
  • Phase 6: Real Hypervisors & Init Lifecycle

    • UEFI-bootable ISO via GRUB (make ARCH=arm64 iso), verified on QEMU + EDK2 firmware and VMware Fusion
    • DHCP auto-configuration (udhcpc) with static QEMU fallback
    • Graceful shutdown: poweroff/reboot/halt signal PID 1 (SIGUSR2/SIGTERM/SIGUSR1), which terminates and reaps all processes, syncs, and calls reboot(2)
    • One GRUB config for every architecture; x86_64 ISOs carry both a UEFI and a legacy-BIOS boot path
    • CI builds and boot-tests both ISOs on native runners and attaches them to tagged releases
    • Semantic versioning: one VERSION names the ISO and the banner

  • Phase 7: Persistent Root Filesystem

    • Raw ext4 disk image built with mke2fs -d (no loop device, no privileged container)
    • switch_root out of the initramfs into the real root, carrying /proc, /sys, /dev across with MS_MOVE
    • Graceful fallback to running from RAM when no disk is attached (so the ISO and direct-kernel boots still work)
    • The root is found by scanning every block device for the BISHOS label, so it can live on a disk, a USB partition or an SD card -- and a foreign disk is never adopted
    • Read-only remount on shutdown so ext4 stays clean
    • make disk-update refreshes init and the BishOS configuration on an existing disk without touching accounts, home directories or installed packages. A rebuild never overwrites a disk that already exists -- which protects your data, and quietly leaves a machine in daily use booting whatever init it was created with
  • Phase 8: Package Manager

    • Alpine's apk (static binary) on the persistent root, with Alpine's signing keys so downloads are verified, not blindly trusted
    • Repositories pinned to a specific Alpine release
    • CA certificate bundle shipped, so HTTPS can actually be verified
    • Installs dynamically-linked software: the first package pulls in musl, which provides the loader the whole repository needs
    • Clock kept correct by ntpd, which matters because TLS validates certificate dates -- a wrong clock fails looking like a CA problem
  • Phase 9: Accounts

    • /etc/shadow ships locked -- !, not a hash -- so no default password is ever committed, and nothing can log in until one is set
    • First boot on a persistent root asks for a username and a hostname, creates the account with adduser, puts it in wheel, and asks for both passwords once, writing SHA-512 crypt hashes to /etc/shadow. No account and no machine name ship in the image: an OS that arrives already called BishOS, with its author's username on it, is somebody else's machine
    • The console runs login, not a root shell, so a session starts by saying who you are
    • sudo on the disk, with wheel allowed in a /etc/sudoers.d drop-in. It lives on the root and not in the initramfs because busybox has no sudo applet and the real one is dynamically linked
    • A RAM-only boot deliberately keeps the old root shell: /etc there is rebuilt from the initramfs every boot, so a password set on the ISO could never be asked for again
  • Phase 10: Remote Access

    • sshd supervised from the service table, on a machine that already had the two things it needs: ptys, and passwords to check
    • Off until asked for. A service that listens because it happens to be installed is a service nobody decided to run. Port 22 opens only while /etc/bishos/ssh.enabled exists, the answer survives reboots, and it takes effect in seconds without one
    • Host keys generated on the machine the first time ssh is actually turned on -- never shipped in the image, and never generated at all for a machine that never wants them. A host key in an ISO is the same private key on every install, which is what host keys exist to prevent
    • PermitRootLogin no, via a /etc/ssh/sshd_config.d drop-in that survives reinstalling the package. root is the account name every scanner tries first, and your own account plus sudo is a better road in
    • make run forwards host port 2222 to the guest, because QEMU's user-mode networking otherwise gives the guest no inbound route: ssh -p 2222 you@localhost
  • Phase 11: Logging & the Console

    • Kernel log persisted to /var/log/messages by syslogd and klogd, capped by rotation so a log cannot fill the root filesystem
    • Every non-console service gets its stdout and stderr on a pipe that init forwards to syslogd under that service's own name. Most daemons never call syslog(3) -- they just print -- so without this their output scrolled off the console and was gone
    • The console session runs on the real tty the kernel names in /sys/class/tty/console/active, not on /dev/console. The latter is a redirector, not the session's controlling terminal, so tcsetpgrp against it fails -- which a shell only discovers on the way out, and every shutdown ended with can't set tty process group
    • The console session is sent SIGHUP before the general SIGTERM at shutdown: an interactive shell ignores SIGTERM by design, and SIGHUP is what actually means "your terminal is going away"
  • Phase 12: A Screen That Stays On

    • Network settings read from /etc/bishos/network and applied by /etc/bishos/net-up, rather than being decided in init.c where changing an address meant a recompile, a rebuilt initramfs and a reboot. sudo /etc/bishos/net-up re-applies them on the spot
    • The old compiled-in path is still in init, used when the script is missing, unrunnable or fails -- a typo in a config file should not cost you the network you would need to fetch the fix
    • DNS comes from the DHCP server. The lease used to be discarded and Google's resolvers written on every renewal, so a router's own DNS, and every local name only it could resolve, simply did not exist
    • DRM_FBDEV_EMULATION, DRM_SIMPLEDRM and SYSFB_SIMPLEFB. x86_64's defconfig builds DRM_I915 in, and a DRM driver evicts the firmware framebuffer when it probes -- without these it then provides no console, so on any machine with Intel graphics the screen lights up, the driver loads, and it goes black
    • Invisible to every test: QEMU has no Intel GPU, so i915 never probes, efifb is never evicted, and CI passed for four releases while the ISO was unusable on the hardware it was built for
    • Boot menu entries for diagnosing it on a machine you cannot iterate on: nomodeset to rule graphics in or out, and quiet dropped so the boot narrates itself
  • Phase 13: A Desktop, and a Machine That Builds Itself

    • Sway on Wayland with a browser, sound and fonts -- which needed four things a distribution normally arranges before you ever log in: device permissions for /dev/dri and /dev/input, a udev daemon, a seat manager, and XDG_RUNTIME_DIR
    • First boot puts the new account in seat, video, render, input and audio. The username is chosen at first boot, so it cannot be written into /etc/group in advance
    • ~/.cache on tmpfs, so a browser stops writing constantly to flash
    • Self-hosting. BishOS compiles its own kernel. A full build is about two hours on a 2014 i5 with the source on a USB stick; changing one option afterwards is under four minutes, which is the difference between kernel work being practical and being avoided

πŸ“‚ Project Layout

β”œβ”€β”€ .github/workflows/
β”‚   └── build-iso.yml   # CI: builds + boot-tests both ISOs, publishes releases
β”œβ”€β”€ Makefile            # Build and run automation (ARCH=x86_64 | arm64)
β”œβ”€β”€ Dockerfile.kernel   # Kernel build container (native gcc + x86_64 cross-toolchain)
β”œβ”€β”€ Dockerfile.iso      # ISO packaging container (GRUB + xorriso, per-arch)
β”œβ”€β”€ grub/
β”‚   └── grub.cfg        # Boot menu -- one config, every architecture
β”œβ”€β”€ etc/                # System configuration overlay, copied into the rootfs
β”‚   β”œβ”€β”€ bishos/         # Everything init reads or runs
β”‚   β”‚   β”œβ”€β”€ services    # The service table: what runs at boot, and how
β”‚   β”‚   β”œβ”€β”€ console     # Console session: first-boot passwords, then login
β”‚   β”‚   β”œβ”€β”€ net-up      # Applies the network settings below
β”‚   β”‚   β”œβ”€β”€ network     # Interface, mode, addresses, DNS policy
β”‚   β”‚   β”œβ”€β”€ ntp-step    # Writes the corrected clock back to the RTC
β”‚   β”‚   β”œβ”€β”€ sshd        # Waits for ssh.enabled, makes host keys, runs sshd
β”‚   β”‚   β”œβ”€β”€ devperms    # Device groups: there is no udev to set them
β”‚   β”‚   β”œβ”€β”€ udev        # Starts eudev, then tells it to look at the hardware
β”‚   β”‚   └── tmpfs-cache # Puts ~/.cache in RAM instead of on flash
β”‚   β”œβ”€β”€ passwd          # root and sshd only -- your account is made on first boot
β”‚   β”œβ”€β”€ shadow          # Ships LOCKED -- "!", never a hash. Set on first boot
β”‚   β”œβ”€β”€ group           # Group definitions, including wheel
β”‚   β”œβ”€β”€ sudoers.d/
β”‚   β”‚   └── wheel       # Members of wheel may run anything, with a password
β”‚   β”œβ”€β”€ ssh/sshd_config.d/
β”‚   β”‚   └── bishos.conf # PermitRootLogin no, as a drop-in the package cannot undo
β”‚   β”œβ”€β”€ hostname        # Machine name; chosen on first boot, BishOS by default
β”‚   β”œβ”€β”€ hosts           # Local loopback resolution
β”‚   β”œβ”€β”€ profile         # Login shell environment, aliases, colors
β”‚   β”œβ”€β”€ resolv.conf     # Fallback nameservers; DHCP overwrites this per lease
β”‚   └── udhcpc/
β”‚       └── default.script # DHCP event handler: address, route, nameservers
β”œβ”€β”€ dotfiles/           # Desktop config: sway, launcher, switcher, shell.
β”‚   └── install.sh      # Symlinks them into place; the files stay here
β”œβ”€β”€ tools/
β”‚   └── centre-tilde.py # Moves a console font's ~ down into the letter body
β”œβ”€β”€ src/
β”‚   └── init.c          # PID 1: pseudo-filesystems, switch_root, service
β”‚                       # supervision, log collection, shutdown
β”œβ”€β”€ build/              # Scratch, gitignored -- make clean deletes all of it
β”‚   β”œβ”€β”€ x86_64/
β”‚   β”‚   β”œβ”€β”€ bzImage     # Compiled Linux kernel (x86_64)
β”‚   β”‚   β”œβ”€β”€ rootfs/     # Root filesystem staging area
β”‚   β”‚   └── initramfs.cpio.gz
β”‚   └── arm64/
β”‚       β”œβ”€β”€ Image       # Compiled Linux kernel (arm64)
β”‚       β”œβ”€β”€ rootfs/
β”‚       β”œβ”€β”€ initramfs.cpio.gz
β”‚       └── bishos-disk.img  # Persistent ext4 root (survives rebuilds)
β”œβ”€β”€ output/             # Finished ISOs, gitignored. make clean leaves these
β”‚   β”œβ”€β”€ x86_64/
β”‚   β”‚   └── bishos-x86_64_v1.0.0.iso
β”‚   └── arm64/
β”‚       └── bishos-arm64_v1.0.0.iso
β”œβ”€β”€ TODO.md             # What is done, what is declined and why, what is left
β”œβ”€β”€ CLAUDE.md           # Instructions for Claude Code, and the traps this
β”‚                       # project has already fallen into
β”œβ”€β”€ LICENSE             # MIT
└── README.md

πŸ“ Notes

Why apk and not apt

BishOS builds its userland against musl (Alpine's toolchain), while every Debian .deb -- including apt and dpkg themselves -- is compiled against glibc and expects perl, a /var/lib/dpkg database, and Debian's filesystem layout. Running real apt therefore means replacing the whole userland with Debian's, which would make BishOS "Debian with a custom kernel and init" rather than a system built from parts.

The natural fit is Alpine's apk, which is what BishOS ships: a self-contained static binary that speaks to the same Alpine mirrors the build already uses for BusyBox. Its packages are built for exactly this userland.

How installing dynamic software works

Every binary BishOS builds itself is static -- it carries its own copy of the library code, which is why it runs on a system with no libraries at all. Alpine's packages are dynamic: they expect to find a loader at /lib/ld-musl-<arch>.so.1 when they start, and on a freshly built BishOS that file does not exist.

The way out is that musl is itself a package. The first apk add installs it, the loader appears, and from then on the entire Alpine repository is installable. So apk add python3 is not a special case -- it is apk pulling musl, then Python's dependencies, then Python.

One thing a from-scratch system does not get for free: a CA certificate bundle. Without /etc/ssl/certs/ca-certificates.crt there is nothing on disk to check a server's certificate against, and apk refuses every download with TLS: server certificate not trusted. BishOS ships the bundle (175 KB) in the rootfs.

Note that BusyBox's own wget never uses it. BusyBox delegates TLS to its ssl_client applet, whose minimal TLS implementation encrypts but has no certificate verification compiled in -- hence TLS certificate validation not implemented. It is a note, not an error: the transfer works, it is just unauthenticated. For HTTPS that actually verifies the peer, install a real client once apk is up (apk add curl); it, and anything else from the repository such as Python's ssl module, will use the shipped bundle.

apk lives on the persistent root and deliberately not in the initramfs: the initramfs is loaded into RAM in full on every boot, so every megabyte is a permanent cost, and a package manager whose installs vanish at the next reboot would be pointless.

Changing the network settings

They live in /etc/bishos/network, which is shell syntax because /etc/bishos/net-up sources it:

IFACE=eth0
MODE=dhcp                    # or: static
FALLBACK_ADDRESS=10.0.2.15   # used by static, and when DHCP finds nobody
FALLBACK_NETMASK=255.255.255.0
FALLBACK_GATEWAY=10.0.2.2
FALLBACK_DNS="8.8.8.8 8.8.4.4"
USE_DHCP_DNS=yes             # no = ignore the DHCP server's nameservers

Re-apply without rebooting:

sudo /etc/bishos/net-up

That is the reason this is a script and not C. Networking is the part of a machine most likely to need changing on the machine itself, and the old arrangement -- addresses compiled into init -- meant every experiment cost a recompile, a rebuilt initramfs and a reboot.

Every value here also exists as a default compiled into init. If this file is deleted or broken, the machine still comes up on the network exactly as it did before the file existed. That duplication is deliberate: a typo should not cost you the network you would need to fetch the fix.

init prints what actually happened, including the nameserver it ended up with rather than the one it hoped for:

Networking: DHCP (DNS: 10.0.2.3)

The console font, and where the tilde sits

Console fonts draw ~ near the top of the cell. That is typographically right -- the tilde began as a spacing diacritic, the mark over the n in "senor" -- but in a shell prompt, where ~ means your home directory and sits beside ordinary letters, it reads as though it has floated off.

No stock font avoids it at a readable size. Measuring the tilde's vertical centre across all 374 fonts in kbd-misc and font-terminus: the best-placed (0.44, near centred) are all 8x16, far too small on a real monitor, and every font 20px or taller sits between 0.22 and 0.27. So pick whichever you prefer:

sudo apk add kbd kbd-misc font-terminus
sudo setfont /usr/share/consolefonts/solar24x32.psfu.gz   # 24x32, best stock at 0.27

or move the glyph, which is the only way to get both size and placement:

python3 tools/centre-tilde.py \
    /usr/share/consolefonts/ter-132b.psf.gz ter-132b-centred.psf.gz 9
sudo cp ter-132b-centred.psf.gz /usr/local/share/consolefonts/

/usr/local/share rather than /usr/share/consolefonts, which belongs to the package and would lose the file on upgrade.

Either way, load it from the service table so it applies before the login prompt rather than after it:

consolefont  once  /usr/sbin/setfont /usr/local/share/consolefonts/ter-132b-centred.psf.gz

once runs it at boot and does not restart it. The kernel's own fbcon=font:TER16x32 still draws everything before userspace exists, so early boot messages keep the built-in font whatever this says.

Running a desktop

Sway works, with a browser, sound and fonts. What it needs is not obvious, because a distribution normally arranges all of it before you ever log in.

sudo apk add sway seatd foot mesa mesa-dri-gallium eudev \
             wmenu swaybg i3status wl-clipboard grim slurp \
             font-dejavu font-noto chromium \
             pipewire pipewire-pulse wireplumber alsa-utils

The installed system runs OpenRC now, so seatd is a service rather than a line in /etc/bishos/services:

sudo apk add openrc busybox-openrc seatd-openrc eudev-openrc
sudo rc-update add seatd default

Four more packages, none of which a distribution would make you think about:

sudo apk add dbus coreutils font-noto-all polkit udisks2
sudo rc-update add dbus default
sudo rc-update add polkit default

dbus because a session bus is what notifications, the system tray and browser keyrings all need -- and sway is started as dbus-run-session sway in .bash_profile so everything it launches shares one. Without it, mako sat in the config for weeks and never once ran.

coreutils because a surprising amount of software calls GNU-only options. env -S, which nwg-drawer uses to launch everything, is not in busybox, so the app grid opened perfectly and started nothing.

font-noto-all because fourteen scripts had no font at all and any window title in them rendered as boxes.

polkit and udisks2 to mount disks from the file manager -- with a rule in etc/polkit-1/ authorising the wheel group, because polkit normally decides by asking logind and there is no logind here.

Software that needs glibc

BishOS is musl, and some software will not run on it at any price. Widevine -- the DRM every streaming service requires -- is a proprietary glibc binary, so Netflix in Chromium reports that "secure streaming" is unavailable and no setting fixes it.

Rebuilding the system around glibc would mean replacing 865 packages and 4 GB of /usr, which is really "install Debian and copy the dotfiles across". Flatpak avoids all of it: a Flatpak application runs against its own bundled glibc runtime and never touches the host's libraries.

sudo apk add flatpak xdg-desktop-portal xdg-desktop-portal-gtk xdg-desktop-portal-wlr
flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak install flathub com.brave.Browser
flatpak override --user --env=LIBVA_DRIVER_NAME=i965 com.brave.Browser

Netflix plays. Three things were needed that are not obvious:

CONFIG_FUSE_FS in the kernel. flatpak downloads through revokefs-fuse so an unprivileged user can write to the system repository. Without /dev/fuse it fails with Can't create temporary directory.

xdg-desktop-portal-gtk. The wlr portal provides only Screenshot, ScreenCast and Secret; Settings and FileChooser come from the GTK backend, and without it Brave cannot open a file dialog.

LIBVA_DRIVER_NAME=i965. The runtime picks iHD, which is for newer Intel GPUs; Haswell needs i965 or video decodes on the CPU.

One wrinkle worth knowing: a Flatpak app's window app_id often differs from its application ID, so a dock that looks icons up by app_id finds nothing and draws a broken-image X. Brave reports brave-browser while its icon is com.brave.Browser.svg. The .desktop file bridges the two -- which is why launchers get it right and docks do not:

ln -sf /var/lib/flatpak/exports/share/icons/hicolor/scalable/apps/com.brave.Browser.svg \
       ~/.local/share/icons/hicolor/scalable/apps/brave-browser.svg

Four things had to be solved that no package installs for you:

Device permissions. devtmpfs creates device nodes as root:root 600, so /dev/dri and /dev/input belong to root alone and a compositor cannot open the GPU. devperms sets the groups.

udev. libinput does not read /dev/input -- it asks udev which devices are keyboards and mice. With no udev database it finds nothing whatever the permissions say, and reports "no input devices" as though the hardware were absent. eudev is started and then told to look at hardware that appeared before it did, which is all of it.

A seat manager. seatd lets a compositor use the screen and input devices without running as root. Without -g seat its socket is owned by root:root and members of the group still cannot open it.

XDG_RUNTIME_DIR. Wayland keeps its sockets in a private per-user directory and refuses to start without one. /etc/profile creates it.

Chromium needs --ozone-platform=wayland, so it talks to the compositor rather than looking for X11, and CONFIG_USER_NS=y in the kernel or its sandbox cannot start. Brave is not an option: it is not packaged for Alpine, the prebuilt binaries are glibc-linked, and building it needs around 100 GB.

Building BishOS on BishOS

The machine compiles its own kernel, which turns a change from a cross-machine chore into a four-minute loop.

sudo apk add gcc make flex bison perl bc \
             openssl-dev elfutils-dev ncurses-dev linux-headers musl-dev

linux-headers is the one that surprises people. musl does not ship /usr/include/asm -- those headers belong to the kernel rather than the C library -- so building a kernel needs headers from a kernel.

make -j4 bzImage      # the image
make -j4 modules      # anything set to =m
sudo make modules_install

make bzImage alone does not build modules, and modules_install then stops with No rule to make target 'modules.order'.

When a program that is definitely there reports "not found"

Sooner or later something gets installed by curl ... | sh, or downloaded as a release tarball, and BishOS says this:

$ ./hello
sh: ./hello: not found
$ ls -la hello
-rwxr-xr-x    1 root     root         70432 hello

The file is right there, executable, owned correctly. The message is still not found, and it is not lying -- it is just not talking about the file.

Almost everything prebuilt on the internet is linked against glibc, and a dynamic binary names the loader it needs inside itself:

$ readelf -l hello | grep interpreter
[Requesting program interpreter: /lib/ld-linux-aarch64.so.1]

BishOS is a musl system. Its loader is /lib/ld-musl-aarch64.so.1, so /lib/ld-linux-aarch64.so.1 genuinely does not exist -- and when the kernel cannot find a binary's interpreter it returns ENOENT, which the shell reports about the thing you named rather than the loader you never mentioned. That is the whole trick: not found means the loader, not the file.

readelf -l is how to confirm it, and the fix is one package:

sudo apk add gcompat        # 131 KB

gcompat provides /lib/ld-linux-*.so and maps the glibc symbols onto musl. The same binary then runs:

$ ./hello
hello from a glibc binary

gcompat ships on every persistent root, so in practice the failure above is one you will only meet on a disk built before this or on a RAM-only boot. It costs 131 KB against a 5 GB filesystem, and the failure it prevents is the single most misleading message in Unix -- a bad trade to make anyone debug twice.

Worth knowing what it is, though: a translation layer that maps glibc symbols onto musl, not glibc itself. Most things work; something unusual can start and then misbehave in a way that failing outright would not have hidden. If a prebuilt binary behaves strangely rather than refusing to run, gcompat sitting underneath it is worth suspecting early. Anything from the Alpine repository is unaffected -- those are musl binaries and never touch it.

About

A minimal Linux system built from scratch: mainline kernel + my own init and shell, booting on aarch64 QEMU

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Used by

Contributors

Languages