Skip to content

Commit f8664fe

Browse files
committed
ZLFS: de-drift the docs; link the repo from the slides
README still described the pre-roadmap state in places: a single- indirect ~2 MB file cap, whole-file buffering, the copying cleaner and the concurrency-safe commit as future work, and 'no garbage collection' under non-goals -- all long since implemented and VM-validated. Bring the read/write/GC bullets, the limitations list and the non-goals in line with the code, and replace the completed 'remaining sequence' with a pointer to the stacked f1/f2/f3 branches awaiting incremental VM validation. docs/zlfs-design.md section 5 similarly still called the copying cleaner future work; describe the implemented dirty-overlay compaction instead. The slides now link the repository (slide 1 eyebrow and a footer line on the roadmap slide, with the evidence log alongside).
1 parent 1fd758d commit f8664fe

3 files changed

Lines changed: 38 additions & 28 deletions

File tree

README.md

Lines changed: 27 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -407,13 +407,14 @@ Kernel (`sys/zlfs/`):
407407
per-zone allocator state, all validated against device geometry with
408408
every on-disk field bounded before use.
409409
- Read: a per-mount vnode cache, `lookup`/`readdir`/`read` over on-disk
410-
inodes, directories, and both direct and single-indirect data blocks.
410+
inodes, directories, and data blocks through the direct, single- and
411+
double-indirect pointers.
411412
- Write: a raw zoned-write primitive (`dk_zone_write_kern`, a direct
412413
`WRITE(16)` bypassing the buffer cache and the host-managed write gate),
413414
an append-only log allocator, and `create`/`write`/`fsync`/truncate,
414-
`mkdir`/`rmdir`, `unlink`, and `rename`. Regular files grow past the
415-
twelve direct blocks through a single indirect block, and the in-core
416-
file buffer grows on demand rather than reserving the maximum up front.
415+
`mkdir`/`rmdir`, `unlink`, and `rename`. Regular files reach ~1 GB
416+
through single- and double-indirect trees, and writes land in a sparse
417+
per-block dirty overlay rather than a whole-file buffer.
417418
Directories nest: `..` resolves to the real parent, directory link
418419
counts track subdirectories, and `rename` reparents `..` (rejecting a
419420
move of a directory into its own subtree). A commit flushes all dirty inodes as a
@@ -432,32 +433,32 @@ Kernel (`sys/zlfs/`):
432433
the durable checkpoint (re-read from disk, not trusted from memory),
433434
from the in-core inode map, and from in-core vnodes (covering
434435
unlinked-but-open files), then resets any written zone reachable from
435-
none of them. Only fully dead zones are reclaimed, and any read
436-
failure aborts the scan without reclaiming, so a crash at any point
437-
still recovers everything the durable checkpoint references.
436+
none of them. Fully dead zones are reset; mixed zones are compacted —
437+
the sync path relocates their live blocks through the dirty overlay so
438+
the zone goes dead and a later pass resets it. Any read failure
439+
aborts the scan without reclaiming, so a crash at any point still
440+
recovers everything the durable checkpoint references.
438441

439442
Current limitations (documented in the code; each is a natural next step):
440443

441-
- Single indirect block only: files are capped at
442-
`ZLFS_NDADDR + block_size/8` blocks (about 2 MB at a 4 KB block size);
443-
no double or triple indirect blocks yet.
444-
- Each open file or directory is buffered whole in core, so the maximum
445-
file size is also bounded by available memory; there is no block-level
446-
buffer-cache integration.
447-
- The cleaner reclaims only fully dead zones; zones holding a mix of
448-
live and superseded blocks are not compacted (a copying cleaner is
449-
future work), so space in mixed zones is reclaimed only once
450-
everything in them is superseded.
444+
- Files are capped at ~1 GB (direct + single- + double-indirect);
445+
triple-indirect support is implemented and awaiting VM validation on
446+
the `zlfs-f1-tripleind` branch.
447+
- Directories are rewritten whole on commit and capped at
448+
`(12 + block_size/8) * block_size` (about 2 MB at a 4 KB block size).
451449
- Reads go through the buffer cache. This is coherent with the raw
452450
zoned writes because the log never overwrites a live LBA; the cache
453451
is purged whenever a zone reset makes cached blocks stale.
454-
- The commit path is not yet safe against concurrent vnode operations.
452+
- Special files, symlinks, hard links, and advisory locks are not
453+
implemented (`EOPNOTSUPP`).
455454

456-
Remaining sequence toward a general-purpose filesystem:
457-
458-
1. Double/triple indirect blocks (files past ~2 MB).
459-
2. A copying cleaner (compact mixed live/dead zones).
460-
3. Concurrency-safe commit.
455+
The original bring-up roadmap — indirect blocks, per-block commit, the
456+
copying cleaner, and the concurrency-safe commit — is complete and
457+
VM-validated (see `functional_testing.md`). Work beyond it lives on
458+
stacked branches awaiting incremental VM validation: `zlfs-f1-tripleind`
459+
(triple-indirect blocks), `zlfs-f2-powerfail` (power-cut simulation and
460+
torn-write recovery), and `zlfs-f3-fsck` (`fsck_zlfs(8)`, an offline
461+
consistency checker).
461462

462463
Regular files no longer buffer their whole contents in memory: writes
463464
land in a sparse per-block dirty overlay, partially covered blocks are
@@ -467,8 +468,9 @@ file no longer rewrites the file.
467468

468469
## Non-Goals For The Current Prototype
469470

470-
- Not production-ready: no garbage collection, limited file and directory
471-
sizes, and durability caveats above.
471+
- Not production-ready: experimental on-disk format, limited directory
472+
sizes, and validation so far only on QEMU NVMe ZNS (no physical SMR
473+
or cache-backed hardware yet).
472474
- No filesystem-level zoned allocation policy beyond append-at-write-pointer.
473475
- No promise of on-disk format or ABI stability before review.
474476
- No attempt to support drive-managed SMR specially; those already appear as normal disks.

docs/zlfs-design.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -116,8 +116,14 @@ resets any written data zone reached by none:
116116
3. **Open vnodes**: unlinked-but-open files live in neither map.
117117

118118
Any read failure aborts the scan (reclaim nothing). The log head is
119-
never reset while it has room. Only fully dead zones are reclaimed —
120-
compacting mixed zones (a copying cleaner) is future work. After any
119+
never reset while it has room. Fully dead zones are reset directly;
120+
mixed zones go through the **copying cleaner**: when free zones run
121+
low, the sync path (which holds no vnode locks) picks the least-live
122+
written zone and pulls every live data block owned by any inode into
123+
that inode's dirty overlay, marking inodes whose *metadata* sits in
124+
the victim for a full indirect-tree rewrite (`zn_relocate`) — so the
125+
next commit relocates everything, the zone goes dead, and a later
126+
pass resets it. After any
121127
reset the device buffer cache is purged (`vinvalbuf`): ZLFS writes
122128
bypass the cache, and reset+reuse is the only way an LBA's contents
123129
change, so this single invalidation point keeps cached reads coherent.

docs/zlfs-slides.html

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,11 +43,12 @@
4343
.t-next{border:1px solid var(--mut);color:var(--mut)}
4444
.counter{position:fixed;right:16px;bottom:14px;font:11px ui-monospace,Menlo,monospace;color:var(--mut)}
4545
code{font:500 .92em ui-monospace,Menlo,monospace;color:var(--acc)}
46+
a{color:var(--acc);text-underline-offset:3px}
4647
@media (prefers-reduced-motion:no-preference){html{scroll-behavior:smooth}}
4748
</style>
4849

4950
<section class="slide" id="s1">
50-
<p class="eyebrow">Slide 1/5 · OpenBSD-src fork · xsub/OpenBSD-src</p>
51+
<p class="eyebrow">Slide 1/5 · OpenBSD-src fork · <a href="https://github.com/xsub/OpenBSD-src">github.com/xsub/OpenBSD-src</a></p>
5152
<h1>What is this project for? <em>Native zoned storage for OpenBSD.</em></h1>
5253
<p class="lead">Modern high-capacity drives — host-managed SMR and NVMe <b>ZNS</b> — refuse random writes: data may only land on each zone's <em>write pointer</em>. OpenBSD had no native support for them at all.</p>
5354
<div class="cols">
@@ -116,6 +117,7 @@ <h1>Where we are, <em>what remains</em></h1>
116117
<li><span class="tag t-next">BEYOND</span><span>POSIX gaps — symlinks, hard links, advisory locking (flock/lockf)</span></li>
117118
<li><span class="tag t-next">BEYOND</span><span>Validation on a physical SMR drive (so far: QEMU NVMe ZNS)</span></li>
118119
</ul>
120+
<p class="legend" style="margin-top:20px">Code, docs &amp; tests: <a href="https://github.com/xsub/OpenBSD-src">github.com/xsub/OpenBSD-src</a> · evidence log: <a href="https://github.com/xsub/OpenBSD-src/blob/main/functional_testing.md">functional_testing.md</a></p>
119121
</section>
120122
<div class="counter" id="ctr">1 / 5</div>
121123
<script>

0 commit comments

Comments
 (0)