fix: take the tree checkout window from the two versions

.changeset/tree-checkout-window.md

@@ -0,0 +1,5 @@
+---
+"loro-crdt": patch
+---
+
+Fix tree checkouts that depended on the path taken. Checking out from a version to a concurrent one could keep a tree node the target version does not have, or, with a subscriber, panic with `get_index_by_tree_id(..).unwrap()` in `tree_state.rs` when the diff created a node's child without the node. The tree diff now retreats and replays from the smallest lamport that differs between the two versions instead of from a replay base, which could be a non-critical meet that skipped earlier concurrent ops. Tree checkouts also no longer search the DAG for a replay base.

AGENTS.md

@@ -29,6 +29,8 @@ Loro is a Rust CRDT workspace with JS/WASM packaging and a MoonBit codec.
   [context/shallow-snapshot-style-redaction.md](context/shallow-snapshot-style-redaction.md).
 - Rich-text insert positions next to style anchors, the insert cursor cache,
   and delete `start_id`s: [context/richtext-insert-positions.md](context/richtext-insert-positions.md).
+- Tree diff cache transitions and why their lamport window comes from the two
+  versions, not the replay base: [context/tree-checkout-window.md](context/tree-checkout-window.md).
 - User-facing Loro usage, sync, editor integration, and performance guidance:
   [skills/loro/SKILL.md](skills/loro/SKILL.md).
 - Pure TypeScript runtime indexes, complexity contracts, benchmarks, and remaining gaps:

context/tree-checkout-window.md

@@ -0,0 +1,79 @@
+# Tree Checkout Window
+
+Verified against code 2026-09-28.
+
+`TreeDiffCalculator` (`crates/loro-internal/src/diff_calc/tree.rs`) computes
+tree diffs for checkouts and for imports in `DiffMode::Import`. It keeps one
+`TreeCacheForDiff` per tree in the history cache (`TreeOpGroup` in
+`history_cache.rs`): the tree's move ops of `current_vv`, grouped by target.
+
+## Invariant
+
+Each cached op carries `effected`: whether the cycle check let it apply
+(`TreeCacheForDiff::apply`). That flag depends on every op ordered before it
+by (lamport, peer), so the cache must always hold ops applied in that order. A
+node's parent is its last effected op (`get_parent_with_id`).
+
+## Transitions
+
+`checkout` (moves the cache to `from` without diffs) and `checkout_diff`
+(records `TreeDeltaItem`s from `from` to `to`) use the same window:
+
+1. `min_lamport_of_version_diff(a, b)`: the smallest lamport among ops that are
+   in exactly one of the two versions. Lamports grow along a peer's ops, so it
+   reads only the first op of each per-peer span of the version-vector
+   difference, O(peers).
+2. Retreat every cached op at or above it, newest first. `checkout_diff` emits
+   the inverse of each effected op.
+3. Apply every op of the target version at or above it, oldest first.
+
+Ops below the window are in both versions and stay as they are. Both passes
+walk `TreeOpGroup::ops` (ordered by `IdLp`), which also holds the seeded
+shallow-root entries, so they never scan nodes that did not change.
+
+The retreat pass ends at `TreeCacheForDiff::max_lamport`: no cached op has a
+greater lamport (`apply` and the shallow-root seeding raise it,
+`retreated_from` lowers it to just below the window). The group holds the
+whole history, so without that bound a checkout between two old versions
+walked every later tree op and grew linearly with the history (about 3.8 ms
+per checkout at 300k later moves). The forward pass ends at the target's max
+lamport.
+
+## Why not the replay base
+
+Until 2026-09-28 the window started at the change-start lamport of
+`find_replay_base(from, to)`. In `Checkout` mode that base is the meet even
+when the meet is not a critical version (`MeetAsBase::Valid`), and an op
+concurrent with the meet can have a lower lamport than the meet. Example
+(`crates/loro/tests/tree_checkout_path.rs`): `0@3` creates a node at lamport 1
+and is concurrent with `2@2` (lamport 2); checking out from `[3@2]` to
+`[1@1]` used the meet `[2@2]`, the window started at 2, and the creation was
+never retreated. In the other direction the forward pass skipped a node's
+creation but applied its child's, and `TreeState::apply_diff_and_convert`
+panicked on the child's `Create` (with a subscriber; without one the state was
+wrong). `docs/critical-version-spec.md` Q7 had argued the window was safe.
+
+The cache's own `checkout` had the same problem with a window taken from the
+target frontiers.
+
+## Shallow documents
+
+The cache of a shallow document starts at the shallow root, seeded with the
+root's nodes (`record_shallow_root_state`). `TreeCacheForDiff::init_current_vv`
+sets `current_vv` to the root version before the first transition, so the
+window never reaches trimmed ops. The seeding itself only covers the seeded op
+IDs, which is why `current_vv` has to be replaced.
+
+## Tests
+
+- `crates/loro/tests/tree_checkout_path.rs`: the two repros.
+- `retreat_bound_follows_the_cached_version` (unit test in `tree.rs`) and the
+  ignored `crates/loro/tests/perf_tree_checkout_old_versions.rs` (run with
+  `--release -- --ignored`): the retreat bound.
+- `crates/fuzz/tests/checkout_path.rs`: random concurrent edits on all
+  container kinds, every ordered pair of versions (including heads no peer
+  had), compared with a document that imported only that version's changes,
+  with and without a subscriber (events are mirrored). Wider sweeps:
+  `CHECKOUT_PATH_SEEDS=0..300 cargo test --release -p fuzz --test checkout_path`.
+- `crates/loro/tests/issue.rs`: `checkout_across_non_critical_meet_stays_canonical`
+  and `checkout_with_low_lamport_concurrent_branch_stays_canonical`.

crates/fuzz/tests/checkout_path.rs

@@ -0,0 +1,242 @@
+//! Checkouts must not depend on the path taken.
+//!
+//! Several peers make concurrent edits (tree create/move/delete, moves into
+//! and out of deleted subtrees, metadata edits, plus map/list/movable
+//! list/text/counter edits) and sync at random. Then one document walks
+//! through every ordered pair `(a, b)` of the collected versions:
+//! `checkout(a)`, then `checkout(b)`. After each checkout its value must equal
+//! a fresh document that imported only the changes of that version, and the
+//! value rebuilt from its events must equal its state. The versions include
+//! heads that no peer ever had, so the transitions cross concurrent branches.
+//!
+//! The seeds are fixed; `CHECKOUT_PATH_SEEDS=start..end` runs others. The
+//! `loro.js` differential fuzz found the tree cases (loro-dev/loro#1141): the
+//! other fuzzers only check out versions from one peer's own history, which
+//! are ordered by inclusion. See `context/tree-checkout-window.md`.
+
+use fuzz::actor::{assert_value_eq, Actor};
+use loro::{
+    ExportMode, Frontiers, LoroDoc, LoroMap, LoroText, LoroTree, LoroValue, TreeID, TreeParentId,
+    ID,
+};
+use rand::{rngs::StdRng, seq::SliceRandom, Rng, SeedableRng};
+
+const PEERS: u64 = 3;
+const ACTIONS: usize = 40;
+const MAX_VERSIONS: usize = 24;
+
+fn random_parent(tree: &LoroTree, rng: &mut StdRng) -> TreeParentId {
+    let nodes = tree.nodes();
+    if nodes.is_empty() || rng.gen_bool(0.3) {
+        TreeParentId::Root
+    } else {
+        TreeParentId::Node(*nodes.choose(rng).unwrap())
+    }
+}
+
+fn random_node(tree: &LoroTree, rng: &mut StdRng) -> Option<TreeID> {
+    tree.nodes().choose(rng).copied()
+}
+
+fn random_edit(doc: &LoroDoc, rng: &mut StdRng) {
+    let tree = doc.get_tree("tree");
+    let value = rng.gen_range(0..100);
+    // Errors are expected: moving a node below itself, editing a deleted
+    // node's metadata, out-of-range indexes after a sync, and so on.
+    let _ = match rng.gen_range(0..16) {
+        0..=2 => {
+            let parent = random_parent(&tree, rng);
+            tree.create(parent).map(|_| ())
+        }
+        3..=5 => match random_node(&tree, rng) {
+            // Includes moves into deleted subtrees and moves that revive
+            // nodes under deleted ancestors.
+            Some(target) => {
+                let parent = random_parent(&tree, rng);
+                tree.mov(target, parent)
+            }
+            None => Ok(()),
+        },
+        6 => match random_node(&tree, rng) {
+            Some(target) => tree.delete(target),
+            None => Ok(()),
+        },
+        7..=8 => match random_node(&tree, rng) {
+            Some(target) => tree
+                .get_meta(target)
+                .and_then(|meta| meta.insert(&format!("k{}", value % 3), value)),
+            None => Ok(()),
+        },
+        9 => match random_node(&tree, rng) {
+            Some(target) => tree.get_meta(target).and_then(|meta| {
+                meta.insert_container("text", LoroText::new())?
+                    .insert(0, "t")
+            }),
+            None => Ok(()),
+        },
+        10 => {
+            let map = doc.get_map("map");
+            if rng.gen_bool(0.3) {
+                map.delete(&format!("k{}", value % 3))
+            } else if rng.gen_bool(0.3) {
+                map.insert_container(&format!("k{}", value % 3), LoroMap::new())
+                    .and_then(|m| m.insert("v", value))
+            } else {
+                map.insert(&format!("k{}", value % 3), value)
+            }
+        }
+        11 => {
+            let list = doc.get_list("list");
+            if !list.is_empty() && rng.gen_bool(0.4) {
+                list.delete(rng.gen_range(0..list.len()), 1)
+            } else {
+                list.insert(rng.gen_range(0..=list.len()), value)
+            }
+        }
+        12 => {
+            let list = doc.get_movable_list("movable");
+            let len = list.len();
+            match rng.gen_range(0..4) {
+                0 if len > 1 => list.mov(rng.gen_range(0..len), rng.gen_range(0..len)),
+                1 if len > 0 => list.delete(rng.gen_range(0..len), 1),
+                2 if len > 0 => list.set(rng.gen_range(0..len), value),
+                3 => list
+                    .insert_container(rng.gen_range(0..=len), LoroMap::new())
+                    .and_then(|m| m.insert("v", value)),
+                _ => list.insert(rng.gen_range(0..=len), value),
+            }
+        }
+        13 => {
+            let text = doc.get_text("text");
+            let len = text.len_unicode();
+            if len > 0 && rng.gen_bool(0.3) {
+                let pos = rng.gen_range(0..len);
+                text.delete(pos, 1)
+            } else {
+                text.insert(rng.gen_range(0..=len), "ab")
+            }
+        }
+        14 => doc.get_counter("counter").increment(value as f64),
+        _ => {
+            // A burst of local edits gives this peer higher lamports than the
+            // others, so concurrent branches end up with very different
+            // lamport ranges.
+            (0..3).try_for_each(|_| {
+                doc.get_map("burst").insert("k", value)?;
+                doc.commit();
+                Ok(())
+            })
+        }
+    };
+    doc.commit();
+}
+
+fn random_version(doc: &LoroDoc, rng: &mut StdRng) -> Frontiers {
+    let vv = doc.oplog_vv();
+    let mut ids = Vec::new();
+    for _ in 0..rng.gen_range(1..=2) {
+        let peers: Vec<_> = vv.iter().filter(|(_, end)| **end > 0).collect();
+        let Some((peer, end)) = peers.choose(rng) else {
+            break;
+        };
+        ids.push(ID::new(**peer, rng.gen_range(0..**end)));
+    }
+    let vv = doc.frontiers_to_vv(&Frontiers::from(ids)).unwrap();
+    doc.vv_to_frontiers(&vv)
+}
+
+fn value_at(all: &LoroDoc, version: &Frontiers) -> LoroValue {
+    let vv = all.frontiers_to_vv(version).unwrap();
+    let doc = LoroDoc::new();
+    doc.set_hide_empty_root_containers(true);
+    doc.import(&all.export(ExportMode::updates_till(&vv)).unwrap())
+        .unwrap();
+    doc.get_deep_value()
+}
+
+fn check_checkout(doc: &LoroDoc, version: &Frontiers, expected: &LoroValue, context: &str) {
+    doc.checkout(version).unwrap();
+    assert_value_eq(
+        expected,
+        &doc.get_deep_value(),
+        Some(&mut || format!("{context}, checkout to {version:?}")),
+    );
+}
+
+fn run(seed: u64) {
+    let mut rng = StdRng::seed_from_u64(seed);
+    let peers: Vec<LoroDoc> = (1..=PEERS)
+        .map(|peer| {
+            let doc = LoroDoc::new();
+            doc.set_peer_id(peer).unwrap();
+            doc
+        })
+        .collect();
+    for _ in 0..ACTIONS {
+        let i = rng.gen_range(0..peers.len());
+        if rng.gen_bool(0.25) {
+            let j = rng.gen_range(0..peers.len());
+            if i != j {
+                let update = peers[j]
+                    .export(ExportMode::updates(&peers[i].oplog_vv()))
+                    .unwrap();
+                peers[i].import(&update).unwrap();
+            }
+        } else {
+            random_edit(&peers[i], &mut rng);
+        }
+    }
+
+    let all = LoroDoc::new();
+    for peer in peers.iter() {
+        all.import(&peer.export(ExportMode::all_updates()).unwrap())
+            .unwrap();
+    }
+    let mut versions = vec![Frontiers::default(), all.oplog_frontiers()];
+    while versions.len() < MAX_VERSIONS {
+        let version = random_version(&all, &mut rng);
+        if !versions.contains(&version) {
+            versions.push(version);
+        }
+    }
+    let expected: Vec<LoroValue> = versions.iter().map(|v| value_at(&all, v)).collect();
+
+    // `Actor` mirrors the document from its events.
+    let with_events = Actor::new(100);
+    let plain = LoroDoc::new();
+    let updates = all.export(ExportMode::all_updates()).unwrap();
+    for doc in [with_events.loro.as_ref(), &plain] {
+        doc.set_hide_empty_root_containers(true);
+        doc.import(&updates).unwrap();
+    }
+    with_events.check_tracker();
+    for (a, b) in (0..versions.len()).flat_map(|a| (0..versions.len()).map(move |b| (a, b))) {
+        for (i, context) in [(a, "from"), (b, "to")] {
+            let context = format!(
+                "seed {seed}, {context} of {:?} -> {:?}",
+                versions[a], versions[b]
+            );
+            check_checkout(&with_events.loro, &versions[i], &expected[i], &context);
+            with_events.check_tracker();
+            check_checkout(&plain, &versions[i], &expected[i], &context);
+        }
+    }
+}
+
+#[test]
+fn checkout_between_any_two_versions_is_path_independent() {
+    let seeds: Vec<u64> = match std::env::var("CHECKOUT_PATH_SEEDS") {
+        Ok(range) => {
+            let (start, end) = range
+                .split_once("..")
+                .expect("CHECKOUT_PATH_SEEDS=start..end");
+            (start.parse().unwrap()..end.parse().unwrap()).collect()
+        }
+        Err(_) => (0..24).collect(),
+    };
+    let failed: Vec<u64> = seeds
+        .into_iter()
+        .filter(|&seed| std::panic::catch_unwind(|| run(seed)).is_err())
+        .collect();
+    assert!(failed.is_empty(), "failed seeds: {failed:?}");
+}

crates/loro-internal/AGENTS.md

@@ -29,7 +29,9 @@ over graceful degradation.
   `src/handler/movable_list_apply_delta.rs` applies movable-list diffs for
   `apply_diff` and undo:
   [../../context/movable-list-apply-diff.md](../../context/movable-list-apply-diff.md).
-- `src/diff_calc/`: diff calculation when moving between versions.
+- `src/diff_calc/`: diff calculation when moving between versions. The tree
+  calculator's cache transitions:
+  [../../context/tree-checkout-window.md](../../context/tree-checkout-window.md).
 - `src/container/richtext/`: text state with style anchors. Where local inserts
   go next to anchors, the insert cursor cache, and delete `start_id`s:
   [../../context/richtext-insert-positions.md](../../context/richtext-insert-positions.md).
@@ -65,16 +67,17 @@ coverage under `crates/fuzz` and ask before running long fuzz targets.
 - Replay-base selection uses Eg-walker terminology (arXiv:2409.14252 §3.5):
   a version V is **critical** when every event outside `Events(V)` happened
   after all of `Events(V)` — no concurrency crosses the cut. Non-`Checkout`
-  diff modes and the tree calculator's lamport windows are only sound when
-  the base satisfies this; `dag.rs`/`oplog.rs` enforce it via the
+  diff modes are only sound when the base satisfies this (the tree
+  calculator's `Checkout` path takes its window from the two versions
+  instead); `dag.rs`/`oplog.rs` enforce it via the
   `ImportGreaterUpdates` entry check and, on conservative retreat, the
   multi-head fixpoint `OpLog::latest_critical_version_below_meet`, with
   the `latest_single_head_critical_version` descent as fallback. Do not
   use "LCA" or "common ancestor" in new code or docs: the meet of two
   versions is generally NOT a safe replay base, and a caller that needs a
   critical version must ask for one (loro-dev/loro#1095). Read
-  `docs/critical-version-spec.md` before touching `find_replay_base`, diff
-  modes, or `diff_calc/tree.rs` windows.
+  `docs/critical-version-spec.md` before touching `find_replay_base` or diff
+  modes.
   `OpLog::iter_from_replay_base_causally` may bypass the DAG when the
   concurrency is register-only (`docs/diff_calc.md`, "Register-only
   concurrency"); keep that check container-granular and history-based.