Skip to content

fix(transfer): interleave itemize in flist order (sync incremental)#6568

Merged
oferchen merged 1 commit into
masterfrom
fix/incremental-itemize-flist-order
Jul 14, 2026
Merged

fix(transfer): interleave itemize in flist order (sync incremental)#6568
oferchen merged 1 commit into
masterfrom
fix/incremental-itemize-flist-order

Conversation

@oferchen

Copy link
Copy Markdown
Owner

Problem

The receiver emits itemize (-i) rows in the wrong order versus upstream rsync 3.4.4. Upstream itemizes in a single flist-index-order walk (generate_files, generator.c:2329-2344 -> recv_generator -> itemize, generator.c:1480-1483), so a directory row lands immediately before its children's rows. For the flist a/ a/f1 b/ b/f2 upstream prints:

.d a/
>f a/f1
.d b/
>f b/f2

oc's two-phase receiver instead emitted every directory (.d) row first, then every file (>f) row:

.d a/
.d b/
>f a/f1
>f b/f2

Prior fix

PR #6560 fixed this for the run_pipelined receiver driver by adding a deferral mechanism: defer_itemize (a context flag), emit_or_record_itemize (buffers a row keyed by its flist index instead of writing it immediately), and flush_itemize_rows (drains the BTreeMap in ascending index order just before finalization). The candidate-selection and pipeline record sites already route through emit_or_record_itemize.

However, the sync incremental-flist driver run_pipelined_incremental was not wired to it: its inline per-directory creation loop still called emit_itemize directly and it never flushed, so it kept emitting dir-block-then-file-block.

This change

Extends the exact same #6560 mechanism to the sync incremental driver:

  • Sets self.defer_itemize = true at driver start (mirrors run_pipelined).
  • Routes the inline directory-creation loop's itemize through emit_or_record_itemize, keyed by the entry's flist index (obtained via .enumerate()), instead of emit_itemize.
  • Calls flush_itemize_rows before finalize_transfer to drain in flist-index order.

No new interleave logic is introduced - it reuses the shared helper (DRY). The driver's responsibility is unchanged; only itemize emission is routed through the shared deferral path.

Scope

  • Only the sync incremental path (run_pipelined_incremental) is touched.
  • The async variants (run_pipelined_incremental_async, run_sync_async, any *_async) are intentionally out of scope: those receiver paths are default-off and gated.

Test

Adds incremental_deferred_itemize_rows_interleave_in_flist_index_order, mirroring #6560's run_pipelined test but driving the incremental driver's own record sites (the create_directory_incremental loop plus build_files_to_transfer). It asserts the buffered rows are keyed by flist index and drain interleaved (.d a/, >f a/f1, .d b/, >f b/f2), not batched. It fails if the batch emission returns.

Validation

  • cargo fmt -p transfer -- --check - clean
  • cargo check -p transfer --all-features --tests - clean
  • cargo clippy -p transfer --all-features --all-targets --no-deps -- -D warnings - clean
  • new test passes

The sync incremental-flist receiver driver run_pipelined_incremental
emitted itemize (-i) rows in two batches - every directory (.d) row from
its inline directory-creation loop, then every file (>f) row from the
transfer pass - instead of upstream's single flist-index-order walk where
a directory row immediately precedes its children.

#6560 fixed this for run_pipelined by adding defer_itemize /
emit_or_record_itemize / flush_itemize_rows: itemize rows are buffered
keyed by flist index and drained in index order just before finalization.
The candidate and pipeline record sites already route through that helper,
but the incremental driver's own per-directory loop still called
emit_itemize directly and never flushed, so it stayed in batch order.

Route the incremental driver through the same mechanism: enable
defer_itemize, record each directory row under its flist index via
emit_or_record_itemize, and flush in index order before finalize. The
async incremental path is intentionally out of scope (default-off).
@github-actions github-actions Bot added the bug Something isn't working label Jul 13, 2026
@oferchen oferchen merged commit 3b2f193 into master Jul 14, 2026
65 checks passed
@oferchen oferchen deleted the fix/incremental-itemize-flist-order branch July 14, 2026 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant