Skip to content

docs(block): name the split packages after what the tree already calls them - #2417

Merged
marmos91 merged 1 commit into
developfrom
docs/block-dataflow-naming
Sep 8, 2026
Merged

docs(block): name the split packages after what the tree already calls them#2417
marmos91 merged 1 commit into
developfrom
docs/block-dataflow-naming

Conversation

@marmos91

@marmos91 marmos91 commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Renames the planned block data-flow split from pier/crane/ferry/harbour to journal / carver / syncer / engine, across the RFC and both planning docs.

Why

Three of the four packages already exist under good names, and the fourth is the verb used at every call site. The split now introduces no new vocabulary at all:

was now already in the tree as
pier journal pkg/block/journal, and the "DFSJRN1\0" format stamp (segment.go:18)
crane carver carve.go, CarveChunk, carveMu, two carve_dispatch.go
ferry syncer engine/syncer.go already declares type Syncer
harbour engine pkg/block/engine, already the composition layer

syncer is the interesting one: today Syncer owns both transport and carve dispatch, so the split narrows it and the name becomes accurate rather than merely inherited.

What needed rewriting rather than substituting

The docs argued for the nautical names, so a find-and-replace would have left nonsense behind:

  • §11 decision 1 rebuilt. It previously decided pier and explicitly rejected journal — "in filesystems that means a metadata write-ahead log; this is the data store itself, and the misnomer is part of why GC was read as touching remote refcounts." That objection is kept and recorded, not quietly dropped: it is accepted because the package already carries the name, so the misnomer exists regardless, and renaming would desynchronise the package from its own format magic.
  • The doc.go metaphor ("bytes park in a container until a boat carries them out") rewritten to state the Sync/Flush distinction directly.
  • The harbour rationale in the master plan rewritten — it argued against harbourmaster on authority grounds, which no longer parses once the package is engine.

Structural changes beyond the rename

  • pkg/block/chunker folds into carver. It is already FastCDC + gear table + Params — precisely the three files the layout lists under carver/ — so keeping both would be two packages for one job, and Params is a format contract that belongs with the code that carves by it. Ten files import it today; noted as mechanical but not free, to be done with the extraction.
  • A measured sizing note. Carve is CPU-bound, but not where the plan implies: on an EPYC 7543, 53% of carve CPU is large-buffer allocation and zeroing under packRuns, against 23% BLAKE3 and 13% FastCDC. Pooling those buffers needs no format change and no dedup trade, and is worth ~4× what removing content-defined chunking would buy. This corrects the impression left by "FastCDC plus BLAKE3 plus dedup dominate" — that phrasing omits the leading term, and BenchmarkCarve stubs the oracle with missDeduper, so it never measured dedup at all.

Verification

rg for pier|crane|ferry|harbour|piertest|cranetest|ferrytest across the repo returns nothing. go build ./... and go vet ./pkg/block/journal/ clean. The design-plan file is renamed to match the reference the RFC already carries.

Docs and one test comment only — no production code.

…s them

The block data-flow split was planned as pier/crane/ferry/harbour. Three of
those packages already exist under good names and the fourth is the verb used
at every call site, so the split now introduces no new vocabulary: journal,
carver, syncer, engine.

journal keeps its name and its DFSJRN1 format stamp; the design plan's own
objection to it is recorded rather than dropped. carver absorbs pkg/block/
chunker, which is already the same three files. syncer is the type engine
already declares, narrowed to transport once carve dispatch leaves it.

Also records the measured carve breakdown: buffer zeroing dominates, not
chunking or hashing.
Copilot AI lite review requested due to automatic review settings September 7, 2026 20:55
@marmos91 marmos91 self-assigned this Sep 7, 2026
@github-project-automation github-project-automation Bot moved this to Todo in DittoFS Sep 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

There are a couple of concrete doc inconsistencies/typos introduced in the updated planning docs that should be corrected to avoid confusing readers.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates DittoFS’s block data-flow documentation to use existing in-tree package names (journal / carver / syncer / engine) instead of the previously planned nautical naming, and adjusts related plan text accordingly.

Changes:

  • Renames the planned split terminology across the RFC and planning docs to match existing package/type names.
  • Rewrites affected rationale sections where a simple find/replace would be misleading.
  • Updates one benchmark comment to reference the “design plan” rather than the prior “pier design” wording.
File summaries
File Description
pkg/block/journal/transition_bench_test.go Updates a benchmark comment to match the renamed design-plan terminology.
docs/internals/rfc-block-dataflow.md Renames the RFC’s library split vocabulary to journal/carver/syncer.
.planning/2026-09-01-journal-library-design-PLAN.md Renames the design plan vocabulary and rewrites supporting narrative for the new naming.
.planning/2026-09-01-block-dataflow-MASTER-PLAN.md Renames master-plan references and updates sequencing/terminology for the new package names.
Review details

Suppressed comments (1)

.planning/2026-09-01-journal-library-design-PLAN.md:485

  • The rationale sentence is self-contradictory: it says the package is named syncer “rather than syncer”, which looks like a leftover from the earlier ferry vs syncer discussion and makes the naming justification read incorrectly.
  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .planning/2026-09-01-block-dataflow-MASTER-PLAN.md
@marmos91
marmos91 merged commit 5ca935f into develop Sep 8, 2026
44 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in DittoFS Sep 8, 2026
@marmos91
marmos91 deleted the docs/block-dataflow-naming branch September 8, 2026 07:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants