Skip to content

Commit c87f1db

Browse files
committed
Update README for the ZLFS bring-up milestone
Document the on-disk format v1, VFS registration, the validated newfs_zlfs end-to-end run on the QEMU ZNS VM, the minimal ZBD development kernel, and the remaining sequence to a read-only mount.
1 parent af371e8 commit c87f1db

1 file changed

Lines changed: 70 additions & 33 deletions

File tree

README.md

Lines changed: 70 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,15 @@ Implemented prototype pieces:
3434
- initial NVMe ZNS reporting and zone management path
3535
- QEMU/OpenBSD VM validation workflow
3636
- experimental raw sequential write gate for one cached zone descriptor
37-
- ZLFS on-disk format draft (`sys/sys/zlfs.h`) with a ZNS-compatible
38-
superblock generation log ping-ponged across zones 0-1
37+
- ZLFS on-disk format v1 (`sys/sys/zlfs.h`): little-endian, CRC32C
38+
checksums, and a ZNS-compatible superblock generation log ping-ponged
39+
across zones 0-1 (no conventional zone required)
40+
- ZLFS registered with the VFS (`option ZLFS`, `vfs_init.c` typenum 20,
41+
`MOUNT_ZLFS`); all operations still return `EOPNOTSUPP`
3942
- `newfs_zlfs(8)` prototype that lays down the ZLFS superblock log using
4043
the dkzone ioctls and the validated raw sequential write path
44+
- minimal `ZBD` kernel config (`sys/arch/arm64/conf/ZBD`) covering only
45+
the QEMU virt machine for fast development rebuilds
4146

4247
Tested so far:
4348

@@ -71,6 +76,14 @@ Tested so far:
7176
disk is refused by `dkzone-scsi-zbc-smoke.sh` as NVMe-backed `sd(4)`, and
7277
the normal VirtIO boot disk `/dev/rsd0c` is refused before build/mutation
7378
because it is not marked `zoned` in `dmesg`.
79+
- `newfs_zlfs sd1c` ran end to end on the QEMU ZNS VM: zone enumeration
80+
and geometry validation (128 zones of 131072 LBAs), superblock zones
81+
reset, the write gate armed by a fresh zone report, the generation-0
82+
superblock written at LBA 0 through the raw sequential write path and
83+
read back with a passing checksum. A `hexdump` of the device confirmed
84+
the on-disk bytes field by field against `sys/sys/zlfs.h` (magic
85+
`0x54BDCC01`, version, block size, UUID, geometry, root inode,
86+
little-endian CRC32C in the final struct slot).
7487
- The next cross-transport milestone is to run the same `dkzone-vm-smoke.sh`
7588
flow against a SCSI ZBC or host-managed SMR target. The
7689
`dkzone-scsi-zbc-smoke.sh` wrapper prints target evidence, refuses
@@ -200,6 +213,25 @@ target LBA matches the cached write pointer for the last reported sequential
200213
zone and the transfer fits within that zone. The lowercase `-w` option probes
201214
the rejection path and expects the write to fail with `EROFS` or `EINVAL`.
202215

216+
## Minimal Development Kernel
217+
218+
`sys/arch/arm64/conf/ZBD` is a minimal kernel config for the QEMU virt VM:
219+
only the devices this VM uses (PL011 uart, ECAM PCIe via ACPI, virtio,
220+
NVMe/ZNS) plus FFS, ZLFS and the debug options, which makes cold builds and
221+
relinks several times faster than GENERIC.MP. It also carries the arm64
222+
link floor -- SoC uarts, acpiec, com and i2c glue that `machdep.c`,
223+
`acpi.c` and `dsdt.c` reference unconditionally -- documented in the config.
224+
225+
```sh
226+
cd /usr/src/sys/arch/arm64/compile/ZBD
227+
make obj && make config && make clean && make -j4
228+
doas make install && doas reboot
229+
```
230+
231+
Plain `make` suffices after editing `.c` files; re-run
232+
`make config && make clean` only after changing the config file or
233+
`sys/conf/files`. This kernel boots on the QEMU virt machine only.
234+
203235
## QEMU NVMe ZNS Target
204236

205237
QEMU 11 can expose an emulated NVMe Zoned Namespace. The current development VM
@@ -265,37 +297,42 @@ that can present SCSI ZBC semantics.
265297
8. Validate zone reset/open/close/finish operations on SCSI ZBC and NVMe ZNS.
266298
9. Evaluate filesystem and buffer-cache implications before enabling general
267299
writable host-managed devices.
268-
10. Start a ZLFS prototype only after the raw write-pointer contract is stable.
269-
270-
## ZLFS Direction
271-
272-
ZLFS is intentionally behind the raw write-pointer milestone. The filesystem
273-
work should start only after the kernel and `dkzone-write-seq.sh` demonstrate:
274-
275-
```text
276-
reset zone -> write exactly at WP -> continue from cached WP
277-
report WP advanced -> reject stale write
278-
```
279-
280-
The QEMU ZNS VM now demonstrates that sequence with two consecutive 8-sector
281-
writes and a final reported write pointer of LBA 16. The next confidence step
282-
is to repeat the same contract on a SCSI ZBC or host-managed SMR target.
283-
284-
The first ZLFS prototype should be userland-first:
285-
286-
1. Define an append-only record format with magic, type, length, generation,
287-
and checksum.
288-
2. Build `zbfs_mkfs`, `zbfs_dump`, and `zbfs_check` against a scratch zoned
289-
raw device.
290-
3. Write checkpoint records sequentially instead of assuming that zone 0 is
291-
conventional.
292-
4. Add tiny userland file write/read tools over data, inode, directory, and
293-
checkpoint records.
294-
5. Move to a read-only kernel mount only after crash recovery and checkpoint
295-
scanning are understandable in userland.
296-
6. Add single-writer append support, then fsync/checkpoint, then cleaner
297-
policy.
298-
7. Integrate with VFS and buffer cache only after write ordering is explicit
300+
10. Grow ZLFS from the validated raw write-pointer contract: on-disk format
301+
and `newfs_zlfs` are done; next are the in-kernel zone report API and a
302+
read-only mount.
303+
304+
## ZLFS Status And Direction
305+
306+
The raw write-pointer gate that ZLFS was waiting for is proven on the QEMU
307+
ZNS VM (reset -> write exactly at WP -> continue from cached WP -> report WP
308+
advanced -> reject stale write), so filesystem work has started:
309+
310+
- The on-disk format lives in `sys/sys/zlfs.h`. All multi-byte fields are
311+
little-endian and every on-disk structure carries a CRC32C checksum. The
312+
superblock is a generation-numbered log ping-ponged across zones 0-1
313+
rather than a rewrite-in-place block, because NVMe ZNS namespaces have no
314+
conventional zones; the header documents append rules, mount discovery,
315+
the block-size bootstrap, and the both-zones-full crash recovery case.
316+
- `sbin/newfs_zlfs` creates the filesystem today: it enumerates zones,
317+
validates geometry, resets the superblock zones, and writes the
318+
generation-0 superblock through the same gated raw write path the smoke
319+
tests validate. Run `newfs_zlfs -N sd1c` for a read-only dry run.
320+
- The kernel skeleton (`sys/zlfs/`) registers with the VFS but rejects all
321+
operations, so an `option ZLFS` kernel is inert until mount lands.
322+
323+
Remaining sequence to a usable filesystem:
324+
325+
1. In-kernel zone report API: the dkzone ioctls copy zone descriptors to
326+
userland pointers, so `zlfs_mount` cannot reuse them; the report path
327+
needs a kernel-buffer variant with the ioctls as thin wrappers.
328+
2. Read-only `zlfs_mount`: superblock-log discovery per `sys/sys/zlfs.h`,
329+
mount argument plumbing (`struct zlfs_args`, `vfc_datasize`), and zone
330+
allocator state initialized from a full zone report.
331+
3. Root vnode, `zlfs_vget`, and directory reads over the checkpoint and
332+
inode map (their on-disk formats are the next format additions).
333+
4. Single-writer append support, then fsync/checkpoint, then cleaner
334+
policy, with crash recovery exercised in the VM at each step.
335+
5. Integrate with the buffer cache only after write ordering is explicit
299336
and tested.
300337

301338
## Non-Goals For The Current Prototype

0 commit comments

Comments
 (0)