@@ -9,8 +9,13 @@ host-managed SCSI ZBC / SMR disks and NVMe Zoned Namespaces, and building up to
99ZLFS -- a small log-structured filesystem that runs directly on those devices.
1010
1111This is research and bring-up work, not production-ready storage code. ZLFS is
12- a mountable read-write filesystem today, with the limitations listed under
13- "ZLFS Status And Direction" below.
12+ a mountable read-write filesystem today -- files, directories, ` rename ` ,
13+ garbage collection with a circular log, and crash-safe commits, all validated
14+ on a QEMU NVMe ZNS VM -- with the limitations listed under "ZLFS Status And
15+ Direction" below. ` functional_testing.md ` tracks, per feature, what is
16+ VM-validated, what is pushed and awaiting evidence, and every defect each
17+ verification round caught. The fork is periodically rebased onto the OpenBSD
18+ mainline (last: July 2026).
1419
1520## Goals
1621
@@ -39,25 +44,34 @@ Implemented prototype pieces:
3944- initial NVMe ZNS reporting and zone management path
4045- QEMU/OpenBSD VM validation workflow
4146- experimental raw sequential write gate for one cached zone descriptor
42- - ZLFS on-disk format v1 (` sys/sys/zlfs.h ` ): little-endian, CRC32C
47+ - ZLFS on-disk format v2 (` sys/sys/zlfs.h ` ): little-endian, CRC32C
4348 checksums, a ZNS-compatible superblock generation log ping-ponged
44- across zones 0-1 (no conventional zone required), plus checkpoint,
45- inode-map, inode, and fixed-size directory-entry structures
49+ across zones 0-1 (no conventional zone required), a checkpoint that
50+ carries a multi-block inode map (~ 256000 inodes at a 4 KB block),
51+ inodes with direct plus single-indirect block pointers, and fixed-size
52+ directory entries
4653- ZLFS registered with the VFS (` option ZLFS ` , ` vfs_init.c ` typenum 20,
4754 ` MOUNT_ZLFS ` ), mountable read-write
4855- ` newfs_zlfs(8) ` creates a real filesystem (superblock log, root
4956 directory with a sample file, inodes, inode map, checkpoint) using the
5057 dkzone ioctls and the validated raw sequential write path
5158- ` mount_zlfs(8) ` to mount the filesystem
5259- ZLFS read path (` sys/zlfs/ ` ): superblock-log discovery, checkpoint and
53- inode-map load, a per-mount vnode cache, real ` lookup ` /` readdir ` /` read `
54- over on-disk inodes, directories, and direct data blocks
60+ inode-map load, a per-mount vnode cache, buffer-cache-backed
61+ ` lookup ` /` readdir ` /` read ` over on-disk inodes, directories, and both
62+ direct and indirect data blocks
5563- ZLFS log-structured write path: an in-kernel raw zoned-write primitive
5664 (` dk_zone_write_kern ` in ` sd(4) ` , a direct ` WRITE(16) ` that bypasses
57- the buffer cache and the host-managed write gate), an append-only log
58- allocator, and ` create ` /` write ` /` fsync ` /truncate with a commit that
59- flushes a fresh segment (data, inodes, inode map, checkpoint) and then
60- a generation N+1 superblock as the atomic commit point
65+ the buffer cache and the host-managed write gate), a circular log
66+ allocator, and the full namespace -- ` create ` /` write ` /` fsync ` /truncate,
67+ ` mkdir ` /` rmdir ` , ` unlink ` , and ` rename ` (including directories across
68+ parents) -- with a commit that flushes a fresh segment (data, inodes,
69+ inode map, checkpoint), issues a cache flush, and then appends a
70+ generation N+1 superblock as the atomic, durable commit point
71+ - ZLFS garbage collection: a three-pass liveness scan (durable
72+ checkpoint re-read from disk, in-core map, open vnodes) reclaims fully
73+ dead zones with a zone reset, the superblock zones recycle by
74+ ping-pong reset, and inode numbers of removed files are reused
6175- minimal ` ZBD ` kernel config (` sys/arch/arm64/conf/ZBD ` ) covering only
6276 the QEMU virt machine for fast development rebuilds
6377
@@ -113,6 +127,19 @@ Tested so far:
113127 exact contents. This exercises the whole cycle: write, commit as a new
114128 segment, generation N+1 superblock append, remount, superblock-log
115129 discovery, checkpoint and inode-map load, and read-back.
130+ - The full namespace passed a six-part VM suite: files and subdirectories,
131+ ` mv ` of files into subdirectories, directories across parents,
132+ same-directory renames, rejection of a directory moved into its own
133+ subtree, ` rm ` /` rmdir ` , an indirect-block (>48 KB) file surviving
134+ remount, and hierarchy persistence across remounts.
135+ - Garbage collection was validated by ` zlfs-gc-churn.sh ` : 150 zone-fills
136+ on a 126-zone device completed with no ` ENOSPC ` -- impossible without
137+ reclaim -- with ` df ` showing the cleaner reclaim ~ 6.6 GB in one pass
138+ (98% -> 18%) and a keeper file bit-identical after the churn and after
139+ a remount that finds the circular log head mid-device.
140+ - The v2 format was validated by ` zlfs-manyfiles.sh ` : 700 files (past
141+ the old single-block 512-inode ceiling, spanning two inode-map
142+ blocks) intact across a remount, and their removal persisted.
116143- The next cross-transport milestone is to run the same ` dkzone-vm-smoke.sh `
117144 flow against a SCSI ZBC or host-managed SMR target. The
118145 ` dkzone-scsi-zbc-smoke.sh ` wrapper prints target evidence, refuses
0 commit comments