fix(loro.js): include reattached child containers' state in diff() and revertTo()

.changeset/loro-js-diff-reattached-child.md

@@ -0,0 +1,11 @@
+---
+"loro.js": patch
+---
+
+`diff()` and `revertTo()` in `loro.js` now restore containers that the range makes reachable again: a map key set back to an older child, a revived list element, or a revived tree node with its whole subtree and metadata, including a node moved out of a deleted ancestor. Their whole state is included, together with nested children, in parent-before-child order, as Rust does. Containers that stay reachable, such as a moved movable-list child, keep only their own changes. `diff()` leaves out containers that are unreachable at the target version.
+
+Mergeable children follow Rust main (loro-dev/loro#1134). `diff()` reports a re-activated mergeable child with its full state. `applyDiff` applies it incrementally by default, and `applyDiff(diff, { fullState: true })` aligns the receiver's existing child with it: maps per key, counters by difference, text and lists by common prefix and suffix, and trees by node ID. `revertTo` and undo apply only the actual change, so the child keeps its identity. A local re-ensure emits only the parent marker. Import and checkout events now carry the full state of a re-activated child, as in loro-crdt. **An event mirror (subscribe → `applyDiff`) should therefore pass `{ fullState: true }` for `import` and `checkout` batches.** With the default, the full state is applied on top of the mirror's hidden copy.
+
+Before writing anything, `applyDiff` checks every list delta against the list lengths the batch produces, including a hidden mergeable child's own length. It also dry-runs every tree item, so an out-of-range list delta or a tree item that cannot apply rejects the whole batch. Other malformed entries (text ranges, counter values, a diff whose type does not match its container) can still leave earlier entries applied, as before and as in Rust. Tree items apply in order, as in Rust: a create of a node that is alive moves it, a move of a missing or deleted node creates a new one, and a delete of a deleted node does nothing. Map children are always set as new containers. Tree diffs list moves first (parent first), then the topmost deletes, then revived subtrees, with each index counted in the tree the earlier items leave. A node that is hidden at the target version is reported as deleted. On a movable list, `applyDiff` moves an existing child container instead of creating an empty copy. Diffs, imports and checkouts of deep trees take linear time.
+
+Undo can revert movable-list moves, including several moves of the same element. The element returns next to its old physical neighbour when that neighbour has not moved since. The undo manager keeps only the tracked ranges its stacks can still use, at a commit cost independent of the stack length. Undoing a move writes a move op, and loro.js still places a remote move by its index rather than its causal position, so a move concurrent with an insert can diverge from Rust (the MovableList model rework, loro-dev/loro#1132).

AGENTS.md

@@ -34,6 +34,8 @@ Loro is a Rust CRDT workspace with JS/WASM packaging and a MoonBit codec.
   and delete `start_id`s: [context/richtext-insert-positions.md](context/richtext-insert-positions.md).
 - Tree diff cache transitions and why their lamport window comes from the two
   versions, not the replay base: [context/tree-checkout-window.md](context/tree-checkout-window.md).
+- loro.js tree diff item order, index counting, apply resolution, and batch dry-run:
+  [context/loro-js-tree-diff.md](context/loro-js-tree-diff.md).
 - User-facing Loro usage, sync, editor integration, and performance guidance:
   [skills/loro/SKILL.md](skills/loro/SKILL.md).
 - Pure TypeScript runtime indexes, complexity contracts, benchmarks, and remaining gaps:

context/loro-js-tree-diff.md

@@ -0,0 +1,75 @@
+# loro.js Tree Diffs: Item Order, Indexes, and Apply
+
+Verified against code 2026-09-29.
+
+How `loro.js` builds and applies tree diff items, and why. The Rust side is in
+[tree-checkout-window.md](tree-checkout-window.md) and `crates/loro-internal/src/handler.rs`
+(`Handler::apply_diff`, `Self::Tree`).
+
+## Items apply in order
+
+A tree diff is a sequence. Each item's `index`/`oldIndex` refers to the tree that the
+earlier items leave, as in Rust. `LoroDoc.#applyTreeDiff` (`loro-js/src/runtime/document.ts`)
+applies items in order. It defers an item only while its parent does not exist yet.
+
+Resolution follows Rust's `apply_diff`:
+
+- A `create` of a node that is alive here (the node and all its ancestors are not
+  deleted) moves it.
+- A `move` of a node that is missing or deleted here creates a new node and remaps its
+  meta map.
+- A `delete` of a node that is already deleted does nothing.
+
+Placement differs from Rust on purpose. Rust places a node at its `fractionalIndex` and
+ignores `index`; loro.js uses `index`. Equal positions (concurrent creates) tie on the new
+op's writer, so position placement reorders them. In a random oracle (300 concurrent
+histories), position placement lost 30 of 1,200 applies and 47 of 1,200 reverts that
+index placement gets right. Consequence: a receiver whose *other* siblings differ from the
+source (for example a doc with a different hidden mergeable tree, applied without
+`fullState`) orders new nodes differently from Rust.
+
+## How `diff()` orders items
+
+`#completeTreeRevivals` rewrites every tree diff that `diff()` and import/checkout events
+produce (via `#completeDiffEntries`):
+
+1. **Moves of live nodes, parent first by depth at `to`.** A node leaves a subtree before
+   that subtree is deleted, and no move makes a cycle.
+2. **Deletes, topmost only.** These are the diff's own deletes, plus the old copies of
+   moved-in nodes that a revived subtree recreates. A node that is hidden at `to` but was
+   alive at `from` is emitted as a delete, whatever its recorded item was. A node alive at
+   neither version is dropped.
+3. **Revived subtrees as creates, parent first.** A revived node's whole subtree is
+   recreated, because `diff()` has full-state semantics.
+
+`#sequenceTreeItems` counts each index against the tree the earlier items leave. Only
+parents that exist at `from` need counting; a recreated parent gets its children in
+order. Under such a parent, the children at any moment are:
+
+- the unchanged ones, counted with the tree's rank query (`LoroTree._childRank`, O(log n))
+  minus the changed nodes currently there;
+- the present *slots* of changed nodes: a slot at `from` for each changed node that was
+  there, and a slot at `to` for each moved or revived node placed there. These are kept in
+  a Fenwick tree.
+
+The cost is O(changed · log n), independent of the sibling count. A single move among
+100k siblings must stay about 1 ms. Enumerating siblings instead cost 50–75 ms per diff or
+event.
+
+## Validation before writing
+
+`applyDiff` dry-runs every tree item with `validateTreeItems`, using the same order,
+deferral and resolution as `#applyTreeDiff`. It throws what applying would throw: a
+missing parent, a move below itself, or an index outside the parent's children. A
+failing batch therefore writes nothing (`loro-js/tests/tree-diff-order.test.ts`). Keep
+the two functions in step when changing either one.
+
+## Pitfalls
+
+- Hidden checks walk to the root. Pass a memo (`#isTreeRecordHidden(tree, record,
+  cache)`, `#treeNodeAliveAt(..., cache)`), or deep chains go quadratic in `diff()` and in
+  import/checkout events with subscribers.
+- Build fractional-index hex strings flat (`fractionalIndexHex`). Appending per byte
+  leaves rope strings that a 100k-node diff keeps alive, and GC dominates.
+- Rust applies a TS diff by fractional index (see above). Diffs with equal positions can
+  therefore come out ordered differently in Rust than in loro.js.

context/mergeable-containers.md

@@ -1,6 +1,6 @@
 # Mergeable Container Context
 
-Verified against code 2026-09-28.
+Verified against code 2026-09-29.
 
 Mergeable containers let two peers independently create the same child container
 under a map key and converge to one deterministic container id. The source of
@@ -123,6 +123,14 @@ full-state diff. So the mode is explicit:
   difference is written, keeping char/element/`TreeID` identity.
 - **UndoManager** composes remote events with `ContainerDiff::change()`, so a peer can still
   undo its edit after a remote undo re-activates the child.
+- **loro.js** mirrors this in `loro-js/src/runtime/document.ts`. `#completeDiffEntries`
+  adds full state for a re-activated child, both in `diff()` and in import/checkout events
+  (`#emit`). `revertTo` sets `#changesOnlyDiff`, which skips full state for mergeable
+  children. `applyDiff(diff, { fullState: true })` aligns through `#fullStateTargets` and
+  `#alignFullState`. `#validateDiffBatch` checks a hidden mergeable list's deltas against
+  its hidden length (re-ensuring resurfaces it). An event mirror passes `{ fullState: true }`
+  for `import`/`checkout` batches, as with loro-crdt (`ApplyDiffOptions.fullState`). Tests:
+  `loro-js/tests/mergeable-diff-alignment.test.ts`, with values taken from the Rust WASM build.
 
 Which batches may carry the flag:
 

loro-js/src/codec/bytes.ts

@@ -252,11 +252,14 @@ export function compareBytes(a: Uint8Array, b: Uint8Array): number {
   return a.length - b.length;
 }
 
+const HEX_BYTES = Array.from({ length: 256 }, (_, byte) =>
+  byte.toString(16).padStart(2, "0"),
+);
+
 export function bytesToHex(bytes: Uint8Array): string {
   let output = "";
-  for (const byte of bytes) {
-    output += byte.toString(16).padStart(2, "0");
-  }
+  for (let index = 0; index < bytes.length; index += 1)
+    output += HEX_BYTES[bytes[index]!];
   return output;
 }
 

loro-js/src/runtime/containers.ts

@@ -70,6 +70,15 @@ export interface SequenceMoveMeta {
   readonly beforeNext: CodecId | undefined;
   readonly afterPrevious: CodecId | undefined;
   readonly afterNext: CodecId | undefined;
+  /**
+   * The element physically after the moved one before the move, deleted or
+   * not; `null` when it was last. Undo puts the element back there, so undone
+   * moves and restored deletions keep their relative order. Undefined when
+   * unknown (e.g. metadata rebuilt from a snapshot).
+   */
+  readonly beforePhysicalNext?: CodecId | null | undefined;
+  /** The element physically before it, deleted or not; `null` when it was first. */
+  readonly beforePhysicalPrevious?: CodecId | null | undefined;
 }
 
 export type CausalVersion = ReadonlyMap<bigint, number>;
@@ -2660,6 +2669,11 @@ export class LoroMovableList<T = unknown> extends LoroList<T> {
     return this._sequence.atVisible(pos)?.id.peer.toString();
   }
 
+  /** Physical placement for the next `_applyMove` of `element` (used by undo). */
+  _physicalMoveHint:
+    | { readonly element: SequenceElement; readonly before: SequenceElement | undefined }
+    | undefined;
+
   _applyMove(
     from: number,
     to: number,
@@ -2670,12 +2684,31 @@ export class LoroMovableList<T = unknown> extends LoroList<T> {
     if (element === undefined) return;
     const beforePrevious = this._sequence.previousVisible(element)?.id;
     const beforeNext = this._sequence.nextVisible(element)?.id;
-    this._sequence.moveVisible(from, to);
+    const physical = this._sequence.physicalIndexOf(element);
+    const beforePhysicalNext =
+      physical === undefined
+        ? undefined
+        : (this._sequence.atPhysical(physical + 1)?.id ?? null);
+    const beforePhysicalPrevious =
+      physical === undefined
+        ? undefined
+        : physical === 0
+          ? null
+          : (this._sequence.atPhysical(physical - 1)?.id ?? null);
+    const hint = this._physicalMoveHint;
+    this._physicalMoveHint = undefined;
+    if (hint !== undefined && hint.element === element) {
+      this._sequence.moveBefore(element, hint.before);
+    } else {
+      this._sequence.moveVisible(from, to);
+    }
     if (operation === undefined) return;
     const meta: SequenceMoveMeta = {
       ...operation,
       beforePrevious,
       beforeNext,
+      beforePhysicalNext,
+      beforePhysicalPrevious,
       afterPrevious: this._sequence.previousVisible(element)?.id,
       afterNext: this._sequence.nextVisible(element)?.id,
     };
@@ -2945,6 +2978,25 @@ export class LoroTree<
     );
   }
 
+  /** How many current children of `parent` sort before `key`, in O(log n). */
+  _childRank(
+    parent: CodecId | undefined,
+    key: Pick<TreeNodeRecord, "position" | "writer" | "id">,
+  ): number {
+    this._ensureHydrated();
+    return (
+      this._children
+        .get(treeParentKey(parent))
+        ?._lowerBoundBy((record) => compareTreeRecords(record, key as TreeNodeRecord)) ??
+      0
+    );
+  }
+
+  _childCount(parent: CodecId | undefined): number {
+    this._ensureHydrated();
+    return this._children.get(treeParentKey(parent))?.size ?? 0;
+  }
+
   _childrenOf(parent: CodecId | undefined): TreeNodeRecord[] {
     this._ensureHydrated();
     return this._children.get(treeParentKey(parent))?.values() ?? [];