fix(loro.js): keep root-time map values and tree nodes when a shallow doc retreats

[zxch3n]

Stacked on #1126; #1128 builds on this. Merge #1126 → #1127 → #1128 back to back. Without this PR, #1126 guards root-time Map/Tree winners with a full rebuild, which costs 83–88 ms for every return to the root of a 32k-Map shallow document; this PR makes it 0.014 ms.

Summary

This is the loro.js counterpart of Rust #1124. Same symptom: when the root commit holds more than one op, checkout(root) on a shallow replica drops a map value that was rewritten after the root. It also affects tree nodes, and getLastEditor returns undefined.

Minimal repro on main: map.set("a",1); map.set("z",0); commit (root), then map.set("a",2); commit. After a shallow import, checkout(root) returns {"z":0} instead of {"a":1,"z":0}.

Root cause

The cause is different from Rust. loro.js rebuilds the shallow root state by replaying, so its exported winner metadata is already correct. The broken part is retreat: #applyVersionTransition looks up the winner of a map key or tree node in the retained op history (latestIncludedOperation). The op that wrote the root-time value was trimmed, so nothing was found, and the key or node was removed.

Fix

When no retained op wins at the target, Map and Tree retreats fall back to the shallow root state (#shallowRootMapRecord and #shallowRootTreeNode). The root store container index, and each map's keys or tree's nodes, are indexed on first lookup and cached per root store. A retreat only touches the containers it changes, which matches the per-map lazy seeding of Rust #1120.

Two more retreat cases are fixed here:

• Retained delete with a trimmed placement: when a retained delete wins but the node's placement op was trimmed, the node falls back to its root placement and stays deleted. Before, it stayed alive.
• Rust shallow snapshots without a latest state: a Rust shallow export with a short retained tail encodes state = E. The root-store index is now built from the root hydration, so the first fallback does not format every root container id: 0.6 ms at 32k Maps, down from 10.1 ms.

context/loro-js-performance.md documents the design.

Rust parity table (#1124 scenarios)

Scenario	loro.js before	after
checkout(root) after an equal-value rewrite / revertTo (Map, child Map, mergeable Map)	key dropped	fixed
same for Tree node placement	node dropped	fixed
re-export at the same root (2 hops) and at a later root	lost after retreat	fixed
getLastEditor at the root	undefined	root-time writer
exported shallow root metadata	already correct (replayed)	unchanged
MovableList value_id / getLastEditorAt	not fixed here: getLastEditorAt/getLastMoverAt are stubs that return the creator, and MovableList snapshots drop position/last-set ids. This needs a separate MovableList position-model change (see the audit report)

Validation

• New loro-js/tests/shallow-root-map-winner.test.ts ports shallow_checkout_equal_value.rs without MovableList, with 0 and 300 padding ops and with/without revertTo. It compares a shallow replica with an imported full-history replica for checkout at every version, retreat in reverse order, diff+applyDiff for every version pair, re-export, undo/redo, and last editor. On the base branch 9 of 11 tests fail; with this PR all pass.
• pnpm -C loro-js check (format, lint, typecheck, tests, build) passes with 290 tests.
• shallow-root-map-winner.test.ts also covers the retained-delete case above and a Rust-written state = E fixture (shallow-empty-state.blob): root-time values and getLastEditor across checkouts.
• bench:complexity 1000,2000,4000,8000 against origin/main: no systematic regression. The only entry flagged in one run, text-tail-checkout-subscribed at 8000, measured 0.15–0.35 ms in the other 5 runs, so it was a GC outlier.
• First fallback retreat, 32k child Maps (median of 5 alternating runs, load1 about 5), round 3 79170146 vs now efb4e90e:
  • Rust writer (state = E): 0.603 vs 0.604 ms.
  • loro.js writer: 0.596 vs 0.585 ms.
  • Later retreats: 0.019 ms.
• CI (build and loro-js jobs) runs on this stacked PR.

Inherited from #1126

This PR includes #1126's snapshot-checkout changes, including the known limitations and the #1136/#1137 integration notes in #1126's description.

Rust: #1124 (equal-value winner metadata), #1120 (per-map lazy root seeding).

🤖 Generated with Claude Code

[zxch3n]

Follow-up on the review of 8efeeb15 → now 54125375. The updated #1126 and main are merged in.

• F4: the retained-delete fix moved here from fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata #1128 (commit 1bb76a83), so this PR is correct on its own. After a shallow import, checkout(root) gives ["0@1","1@1"] and checkout(deleted) gives ["1@1"], matching Rust for Rust→loro.js, loro.js→loro.js, and loro.js→Rust. Before this change the result was ["0@1","1@1"] both times. A regression test is in shallow-root-map-winner.test.ts.
• First fallback: it no longer formats and indexes every root container. It reuses the root entry index that the import already builds to merge the root and latest states (#shallowRootEntryIndex, from fix(loro.js): correct checkout after a snapshot import #1126). These direct fallbacks also replace fix(loro.js): correct checkout after a snapshot import #1126's temporary rebuild guard for trimmed winners.
• Fallback cost. Setup: a shallow doc with N child Maps, all written in the root commit. After the one-time replay, one child is edited, and then the doc retreats to the root. "Cold" is the first such retreat; "warm" is the median of 100.

N	8efeeb1 cold	now cold	warm (both)
1,024	0.40–0.46 ms	0.33–0.41 ms	~0.017 ms
8,192	1.8–2.1 ms	0.34–0.37 ms	~0.017 ms
32,768	8.1–11.1 ms	0.41–0.48 ms	~0.017 ms

• Inherited from fix(loro.js): correct checkout after a snapshot import #1126: the F1 and F3 fixes and the perf changes. Median first checkout of a 262k-op Text: 147.62 → 56.75 ms. 32k child Maps: 156.91 → 57.58 ms, with peak RSS 285.7 → 234.4 MiB (main: 235.6 MiB). Methodology and details are in the fix(loro.js): correct checkout after a snapshot import #1126 comment.
• Checks: pnpm -C loro-js check passes (268 tests). bench:complexity against main flags 0 of 284 cases.
• Unchanged: MovableList value, last-editor, and last-mover metadata are still not covered, as the description says.

[github-actions]

WASM Size Report

• Original size: 3272.76 KB
• Gzipped size: 1087.19 KB
• Brotli size: 760.88 KB

[zxch3n]

Round-2 follow-up: 54125375 → 880bd5f6. The updated #1126 (B1/B2 fixes, CI) is merged in.

• N1: first fallback for Rust shallow snapshots without a latest state. A Rust shallow export with a short retained tail encodes state = E, so the import replays from the root store and never built the root-entry index. The first fallback then formatted every root container id. The index is now filled from the keys that the root-store hydration already computes (aead1634). Adds a Rust-written E-state fixture test (shallow-empty-state.blob): root-time values and getLastEditor across checkouts.

first fallback retreat, 32k child Maps (5 alternating runs)	5412537	now
Rust-written (state = E)	10.1 ms	0.81 ms
loro.js-written (latest state present)	0.67 ms	0.69 ms
later retreats, both writers	0.019 ms	0.019 ms

• CI: fix(loro.js): keep root-time map values and tree nodes when a shallow doc retreats #1127 now gets build and loro-js checks. On 880bd5f6, build and loro-js pass.
• Checks: pnpm -C loro-js check passes (282 tests). Inherits fix(loro.js): correct checkout after a snapshot import #1126's B1/B2 fixes and numbers.

[zxch3n]

Round-3 update: 880bd5f6 → 79170146. This only merges #1126 1c5ff489, which brings in main 27982a55. No loro.js code changed.

• Rust recheck:
  • The Rust-written E-state fixture (shallow-empty-state.blob) regenerates byte-identical with Rust at 27982a55.
  • The F4 repro (Rust and loro.js, as writer and reader in all four combinations) is unchanged.
• First fallback retreat (ms, median of 5 alternating runs, load1 median 10.0):

	round 2 880bd5f6	now 79170146
Rust-written (state = E), 32k child Maps	0.587	0.589
loro.js-written, 32k child Maps	0.584	0.602
later retreats, both writers	0.019–0.021	0.019–0.021

• Inherited from fix(loro.js): correct checkout after a snapshot import #1126: the first-checkout, bench:complexity, and B4 numbers are in fix(loro.js): correct checkout after a snapshot import #1126's round-3 comment. Against round-2 heads, 0 of 852 complexity entries are flagged. Against main, 1 is: a first-call constant of about 0.15 ms, and the fix(loro.js): correct checkout after a snapshot import #1126 comment has the targeted recheck.
• Checks: pnpm -C loro-js check passes with 282 tests.
• CI on 79170146: build and loro-js pass.

[zxch3n]

Round-4 follow-up: 79170146 → efb4e90e. This merges main 29553d5b through #1126, plus #1126's NEW-B1/NEW-B2/P2 fixes (see #1126's comment). This PR's own code is unchanged.

• Description: updated with the merge order (fix(loro.js): correct checkout after a snapshot import #1126 must not ship without this PR), the retained-delete and state = E fallbacks, and current numbers.
• Checks: pnpm -C loro-js check passes with 289 tests.
• First fallback retreat, 32k child Maps (median of 5 alternating runs, load1 about 5):

	round 3 79170146	now
Rust writer (state = E)	0.603 ms	0.604 ms
loro.js writer	0.596 ms	0.585 ms
later retreats	0.019 ms	0.019 ms

• bench:complexity: 0 of 852 entries flagged against main and against the round-3 heads.
• CI on efb4e90e: build and loro-js pass.

[zxch3n]

Round-5 follow-up: efb4e90e → ea561f8f. This merges #1126's R4-B1 fix (see #1126's comment); this PR's own code is unchanged.

• Checks: pnpm -C loro-js check passes with 290 tests.
• Rust check: the deleted-mark paths give hlXo, as in Rust.

CI on ea561f8f: build and loro-js pass.