Skip to content

feat(storage): make per-piece writeback configurable - #2006

Merged
gaius-qi merged 5 commits into
dragonflyoss:mainfrom
cparadis-nvidia:feat/configurable-storage-writeback
Aug 25, 2026
Merged

feat(storage): make per-piece writeback configurable#2006
gaius-qi merged 5 commits into
dragonflyoss:mainfrom
cparadis-nvidia:feat/configurable-storage-writeback

Conversation

@cparadis-nvidia

Copy link
Copy Markdown
Contributor

Description

Adds a storage.writeback config option that controls how the storage triggers writeback of written piece ranges:

storage:
  writeback: inline   # default — current behavior, await sync_file_range per piece write
  #           background — enqueue ranges to a dedicated flusher task (fire-and-forget)
  #           off        — rely on the kernel vm.dirty_* thresholds only

The default is inline, so existing deployments see zero behavior change. Hosts whose download bandwidth exceeds the disk's sustained write bandwidth can opt into background and recover the full download rate (measurements in the linked issue: 0.25 → 1.07 GiB/s on a host with ~1.19 GiB/s network ingress and ~266 MiB/s NVMe).

Implementation notes:

  • The per-piece writeback moves from io.rs (which returns to pure I/O, no signature changes, no test churn) to the content layer, which owns both the configuration and the flusher lifecycle.
  • In background mode, Content::new spawns a single flusher task fed by a small bounded channel; the write path does a try_send and moves on. A full channel drops the hint — safe, because sync_file_range is purely advisory and kernel writeback covers dropped ranges. The single consumer also means at most one blocked syscall at a time, instead of piling blocked threads onto the blocking pool at high piece rates.
  • Lifecycle is owned, not global: when Content drops, the last Sender drops, the flusher's recv() returns None, and the task exits.
  • The GC-time and copy-time writeback from Enhance storage management with page cache optimizations and readahead #1981 is unchanged — only the two per-piece write-path calls are affected.
  • On non-Linux targets all modes are no-ops (the existing sync_file_range wrapper is already cfg-gated to Linux).

Related Issue

Fixes #2005

Motivation and Context

SYNC_FILE_RANGE_WRITE only initiates writeback, but the syscall blocks while the device writeback queue is congested. Awaiting it inline after every piece write therefore paces the whole cold download at disk write speed on hosts where the network outruns the disk, and it flushes each range immediately, so dirty pages never accumulate and the kernel vm.dirty_* thresholds never get a chance to absorb download bursts in RAM. Full analysis, measurements, and reproduction steps are in #2005.

The background mode preserves what the inline call was added for — a steady drain and bounded dirty pages while the disk keeps up — and degrades gracefully to kernel-threshold writeback when it can't, instead of pacing the download.

Screenshots (if appropriate)

N/A

@codecov

codecov Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.40506% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.68%. Comparing base (ff17c0b) to head (05f0cbf).
⚠️ Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
dragonfly-client-storage/src/content_linux.rs 87.50% 5 Missing ⚠️
dragonfly-client-storage/src/content.rs 95.83% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2006      +/-   ##
==========================================
+ Coverage   53.59%   53.68%   +0.09%     
==========================================
  Files         100      100              
  Lines       26908    26976      +68     
==========================================
+ Hits        14420    14482      +62     
- Misses      12488    12494       +6     
Files with missing lines Coverage Δ
dragonfly-client-config/src/dfdaemon.rs 91.94% <100.00%> (+0.12%) ⬆️
dragonfly-client-storage/src/io.rs 99.47% <ø> (-0.01%) ⬇️
dragonfly-client-storage/src/content.rs 99.04% <95.83%> (-0.96%) ⬇️
dragonfly-client-storage/src/content_linux.rs 81.24% <87.50%> (+0.87%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Replace the hardcoded sync_file_range call after every piece write with a
configurable WritebackMode (sync, async, off). The default mode is async,
which enqueues written ranges to a background task so the write path is
never stalled by disk speed. Sync preserves the old behaviour of awaiting
sync_file_range inline. Off leaves dirty-page writeback entirely to the
kernel thresholds.

Signed-off-by: Gaius <gaius.qi@gmail.com>
Reword doc comments for `WritebackMode`, `Writeback`, and the queue
capacity constant to be more accurate and concise.

Signed-off-by: Gaius <gaius.qi@gmail.com>
@gaius-qi gaius-qi added the enhancement New feature or request label Aug 25, 2026
@gaius-qi gaius-qi added this to the v2.6.0 milestone Aug 25, 2026
Update all dragonfly-client crate versions from 1.5.1 to 1.5.2.

Signed-off-by: Gaius <gaius.qi@gmail.com>
@gaius-qi
gaius-qi force-pushed the feat/configurable-storage-writeback branch from 904ec94 to ddc2320 Compare August 25, 2026 08:06
The inline comments restated what the assertions already express clearly.

Signed-off-by: Gaius <gaius.qi@gmail.com>
…ests

The comment described the sleep's purpose redundantly; the code is self-explanatory without it.

Signed-off-by: Gaius <gaius.qi@gmail.com>
@gaius-qi
gaius-qi enabled auto-merge (squash) August 25, 2026 08:10
@gaius-qi
gaius-qi merged commit ea54cfc into dragonflyoss:main Aug 25, 2026
7 checks passed

@gaius-qi gaius-qi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inline sync_file_range after every piece write caps cold download throughput at disk write speed

4 participants