fix(loro.js): correct checkout after a snapshot import

[zxch3n]

Base of a stack: #1126 → #1127 → #1128. Merge the three back to back (see "Merge order").

Summary

On main, checkout/diff on a loro.js document imported from a full or shallow snapshot can return wrong state, throw, or persist a wrong latest state:

• A non-root container that has not been read yet is hydrated from the encoded latest state after the doc moved to an older version ({"a":2,"b":3} instead of {"a":1}).
• Retreating over a delete restores nothing, because snapshot state has no tombstones (root list: [] instead of [1]).
• A full replay touches lazily encoded containers, which then hydrate on top of the replayed ops and throw duplicate sequence id N@P.
• After a replay to an earlier version, moving forward over a delete skips it.
• Replaying Rust-written rich text replaces its snapshot state. loro.js does not count style anchors in positions, so the replay can lose styles or move text, and the next export persists that.
• Two concurrent deletes of one character throw event diff position N is out of range during checkout, leaving a half-updated state.

Design

Details and rationale: context/loro-js-performance.md.

• Per-container completion. A Text or List hydrated from a snapshot is recorded in #snapshotSequences. The first transition that crosses one of its snapshot operations replays only that container from its own history, once (#prepareSnapshotTransition, #completeSnapshotSequence). Unrelated containers stay as they are, and the lazy SSTable is kept.

• Unreplayable Text. When the replay does not reproduce the snapshot state (visible ids, plus the delta when styled), the replay is discarded and the container keeps its snapshot state. Transitions move it separately: by id when the installed state already has the forward operations, otherwise by rebuilding from the snapshot state plus later operations.

• Full rebuilds check styled history. Before a full replay (the non-incremental fallback, shallow export, forkAt), every hydrated Text that has or had styles is compared first:

  • styles in its snapshot state or its shallow root state;
  • or a mark operation in its history (#markedTexts).

  A mark whose characters were all deleted leaves no style, but its anchors still shift later Rust positions. Plain text is not checked, so it pays nothing extra.

• MovableList stays on main's path. Its Rust snapshot names moved elements by position id, and its hydrated state has no move or value history. A transition that crosses its snapshot operations therefore replays to the target, as on main. The context doc explains why the other options were worse.

• forkAt/fork keep a snapshot state only when the fork's version includes it. An older fork replays its own operations, as on main.

• Transitions deduplicate sequence elements by id (SequenceElementSet).

• Atomicity. Nothing is left half-updated when something throws:

  • checkout/attach restores its version and state (#transitionTo, including its preparation);
  • a failed completion reinstalls the snapshot state;
  • diff restores the current state when it, or its move back, throws.

• CI. .github/workflows/rust.yml runs for any PR base and adds a loro-js job (pnpm -C loro-js check).

Merge order

Do not release this PR without #1127. On its own, this PR guards root-time Map/Tree winners of a shallow document with a full rebuild. The round-3 review measured 83–88 ms for every return to the root of a 32k-Map shallow document; #1127 makes that 0.014 ms. Merge #1126, #1127, and #1128 back to back, or merge #1128's branch.

Known limitations

• Older versions of Rust rich text are approximate when loro.js cannot replay it: text deleted before the snapshot cannot be restored.
  • Random Rust histories (table below) show more mismatching older versions and revertTo results than main, while main corrupts the latest state instead.
  • The anchor model (fix(loro.js): keep rich-text style anchors in the text sequence #1137) makes such text replayable; the round-3 review found 0 mismatches with it.
• "Latest is exact" is qualified: it means the snapshot state plus later operations, as loro.js applies them.
  • An operation concurrent with a delete that the snapshot already applied can land at another position than in Rust: loro.js: an insert concurrent with a delete baked into an imported snapshot lands at the wrong position #1163, also on main.
• MovableList matches main:
  • A move or set imported right after a Rust snapshot import, before any checkout, can resolve the wrong element.
  • loro.js's MovableList replay can differ from Rust under concurrent moves, so the latest state can change at the first checkout that crosses such operations.
  • Once the list is replayed, later transitions are incremental and can show a wrong older version where main is right. Round-4 seed 22: after checking out v11, v12, then v13, the list shows [11], where Rust, main, and a full-history document show [9]. The latest state and revertTo are right.
  • All three are left to the MovableList model work (test(loro.js): add a Rust differential suite for MovableList #1132 and follow-ups).
• Shallow export gaps, also on main:
  • After an update is imported while detached, a shallow export on the live document can change its latest state (round-4 s=1299: qqhhbbbcccbb, Rust qqhhbbbbbccc). This happens for plain text too, and fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136 fixes the plain-text case.
  • A shallow export that throws midway leaves the live document at the root or in between: #encodeShallowSnapshot rebuilds without a restore.

Integration with #1136 / #1137 (whichever lands second resolves)

• fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136 adds its own SequenceElementSet with the same semantics; keep one. There is also one comment conflict in document.ts.
• fix(loro.js): keep rich-text style anchors in the text sequence #1137, trial-merged onto fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata #1128: 2 conflict hunks in containers.ts and 3 in document.ts.
  • LoroText._styleIndex: it is mutable TextStyleIndex<TextStyleMeta> here and readonly TextStyleIndex<TextStyle> there. It must stay swappable.
  • LoroText._swapState / TextState must also swap fix(loro.js): keep rich-text style anchors in the text sequence #1137's anchor state (#anchorStarts, and whatever replaces _detachedStyleCounter). Otherwise a completion leaves stale anchors behind.
  • Shallow export: fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata #1128's retention set (#buildStateStore(frontiers, retained)) and fix(loro.js): keep rich-text style anchors in the text sequence #1137's style redaction (#buildStateStore(frontiers, { redacted, only })) need one combined signature, and the export needs both.
  • With fix(loro.js): keep rich-text style anchors in the text sequence #1137, Rust rich text replays, so the unreplayable path becomes rare.

Validation

• Tests:
  • loro-js/tests/snapshot-checkout.test.ts, with Rust fixtures styled-text, rich-text-history, and movable-moves;
  • loro-js/tests/version-transition.test.ts, which covers the concurrent-delete dedupe and fault injection for checkout, completion, and diff.
  • deleted-mark.json covers a Rust text whose marked characters were deleted. The test runs checkout → attach, a live shallow export and its blob, and forkAt(latest); it fails on round-4 8ef6818d (hloX instead of hlXo).
  • pnpm -C loro-js check passes with 277 tests. The fork/MovableList/fault tests fail on round-3 1c5ff489.
• checkout(f) → shallow export at g → attach over every (f, g) of 600 random 2–3-peer Rust Text histories, plus a variant that deletes marked text:

cases where the live latest state changed	main	round 4 8ef6818d	now
plain text (7,586)	194	0	0
marked text (21,104)	7,847	0	0
marked text, variant that deletes marked text (32,588)	15,677	34	0

• The exported blob as Rust reads it is always right now. The only exceptions are 5 histories where loro.js and Rust already read the imported snapshot differently (the anchor gap).
• 150 random 2-peer Text + MovableList seeds (walk + attach, export to Rust, live shallow export, forkAt(latest)): main 110/110/93/93; round 4 0/0/0/0, and 0/0/3/3 with the delete variant; now 0 in both.
• Rust cross-checks (WASM built from main 29553d5b): exports of detached, forked, and shallow documents re-import in Rust as the loro.js value, and the rich-text invariance fuzz passes 240/240.
• Random Rust histories (90 seeds × 3 peers × 24 steps of Text inserts/deletes/marks and MovableList insert/move/set/delete; loro.js imports the snapshot, then checks out every version, forks and reverts at every 6th version, and compares with Rust):

mismatches with Rust	main	round 3 1c5ff489	now
MovableList at checked-out versions (2,160)	91	1,352	56
Text at checked-out versions (2,160)	517	787	787
forkAt Text / MovableList (360)	64 / 7	275 / 183	64 / 7
revertTo Text / MovableList (360)	62 / 45	127 / 212	124 / 41
latest after the checkouts (seeds)	80 (70 Text)	0	37 (all MovableList, loro.js non-convergence)
export re-imported by Rust ≠ loro.js value	0	0	0

Performance

Setup: Apple M5 Pro (18 cores, shared with other sessions), Node 22.23.1.

• Every run is a separate process, and the order rotates between main (29553d5b), the round-3 heads, and the current heads.
• The drivers waited up to 10 minutes in total for load1 ≤ 6 (7 minutes used), then ran anyway; load1 was recorded before every run.

First checkout after a snapshot import (ms, median of 5; load1 4.1–6.0, median 4.9, per-variant medians within 0.2):

case	main*	round 3 1c5ff489	now 8ef6818d
Text 65,536 ops	4.4	24.1	24.2
Text 262,144 ops	9.3	55.2	53.9
Text 262,144 ops, subscribed	9.2	53.7	53.4
32k child Maps, retreat one	47.1	48.0	48.6
32k child Maps, peak RSS (MiB)	234.4	234.0	231.3
unreplayable Rust rich text 65,536	2.5	32.2	31.1
unreplayable Rust rich text 262,144	2.7	69.9	70.1
rich text round trip / subscribed	0.05 / 0.10	0.06 / 0.09	0.06 / 0.09
Rust MovableList 8,192 with one move: first / each later checkout	9.0 / 0.15	17.0 / 2.16 (wrong value)	9.2 / 0.18
Rust MovableList 65,536 with one move: first / each later checkout	35.6 / 0.16	74.5 / 17.1 (wrong value)	38.0 / 0.19

* On main, the Text rows return N−1 characters and the Map row returns the latest value.

bench:complexity 1000,2000,4000,8000 (7 rounds × 7 variants, load1 5.2–6.0): 0 of 852 entries flagged against main, and 0 against the round-3 heads. A flag means over 1.5× the baseline and more than 0.3 ms slower. The largest ratios are entries under 0.2 ms, up to 1.97×.

Round 5: full-rebuild paths, 262,144 characters (5 alternating rounds with main and the round-4 heads, load1 5.0–6.0), in ms:

case	main	round 4	now
loro.js plain text: shallow export / forkAt(latest) / first checkout	387.5 / 138.7 / 7.2	261.6 / 77.6 / 46.2	273.9 / 78.0 / 47.1
Rust rich text (unreplayable): same three	224.4 / 30.0 / 1.5 (wrong)	375.0 / 184.9 / 68.2	377.1 / 190.7 / 67.7
Rust text whose marked characters were deleted: same three	214.7 / 30.7 / 1.4 (wrong)	219.4 / 35.3 / 61.6 (wrong)	372.0 / 160.3 / 66.4

The last row is the price of R4-B1: one replay of that text on a full-rebuild path. Plain text is unchanged.

B4, 259,778 actions (3 rounds, load1 5.6–6.0), in ms. The results are byte-identical.

phase	main	round-3 #1128	now #1128
apply	236.2	233.6	234.2
snapshot export	111.0	113.2	112.3
update export	71.7	72.1	73.4
snapshot import	113.1	114.0	115.9
update import (coalesced inserts)	239.5	169.7	174.0

🤖 Generated with Claude Code

[github-actions]

WASM Size Report

• Original size: 3272.76 KB
• Gzipped size: 1087.19 KB
• Brotli size: 760.88 KB

[zxch3n]

Follow-up on the adversarial review of 0b638203 → now e56d37af. main (#1129, #1130, #1123) is merged in. This supersedes the "replays history once" part of the description.

What changed

• Per-container completion instead of a global replay. Before a checkout, checkoutToLatest, diff, or detached snapshot export transition, #prepareSnapshotTransition prepares only the containers that the transition touches:
  • Lazily encoded containers are hydrated first. Their encoded latest state is their state at the current version.
  • A snapshot-hydrated Text/List/MovableList is rebuilt from its own operations, once, and only when the transition crosses one of the snapshot's operations.
  • Map, Tree, and Counter state needs no rebuild.
  • The lazy SSTable is kept, so untouched containers stay encoded.
  • Consecutive text inserts that continue each other are replayed as one span (coalescedTextInsert).
• The root cause of F1 already exists on main. loro.js reads Text op positions without Rust style anchors. Rust uses entity indexes that count the anchors.
  • On main, a Rust update stream with a mark followed by an insert after it already imports as helloX instead of hellXo.
  • Deletes that loro.js creates after marks also import into Rust incorrectly.
  • The forced replay turned this into data loss for snapshot-imported docs.
  • Fixing the anchor model is out of scope for this PR. Instead, the completion compares the replay with the snapshot state (visible ids, plus the delta when the text is styled). When they differ, the replay serves the transitions, and the snapshot state is put back at every version with the same ops on that container (#settleSnapshotSequences). So the snapshot version, exports, and re-imports into Rust stay exact.
  • Full rebuilds (the fallback path, and shallow export) stash snapshot states the same way.
• F3: touched lazy containers are hydrated before event values are captured.
• A lazy-export bug on main found along the way: once history was loaded (getAllChanges, checkout), a full snapshot export wrote unread nested Maps as empty ({"c":{}} instead of {"c":{"k":"v"}}). Export now always uses the lazy-aware encoder.
• fix(loro.js): export the latest state in a snapshot of a detached document #1130's detached export now uses the same prepared transition. There is no staging replay for lazily imported docs, and snapshot states stay exact.
• F4 on this PR alone: when a shallow doc's Map/Tree winner at the target is a trimmed root-time write, the transition falls back to a rebuild from the shallow root state. fix(loro.js): keep root-time map values and tree nodes when a shallow doc retreats #1127 replaces this with its direct fallback.

Blocking repros

Cross-checked with a local pnpm -C crates/loro-wasm build-dev build.

	main	0b63820	now	Rust
F1 checkout(empty) → latest	styled b	unstyled b; Rust re-import unstyled	styled; Rust re-import styled	styled
Rust mark + insert after it, checkout v1/v2/v3	ello/ellXo/ellXo	hello/helloX/elloX (latest corrupted)	hello/helloX/ellXo	hello/hellXo/ellXo
F3 unread grandchild event	throws duplicate sequence id	[{insert:"abc"}]	[{delete:1}]	[{delete:1}]
F4 checkout(root), then checkout(deleted)	["1@1"], ["1@1"]	["0@1","1@1"], ["0@1","1@1"]	["0@1","1@1"], ["1@1"]	same as now

The middle helloX row is the anchor gap described above. It affects only versions whose ops on that container differ from the snapshot's.

Rebuild count. Measured on an instrumented build with abc → abcX → bcX and a snapshot import:

• Checkouts to V1, V2, V1, V3: cumulative full rebuilds went from 1,1,1,2 to 0,0,0,0, with one completion of the touched container.
• First diff(V2,V1): 3 full rebuilds → 0 (one completion).

Performance

Node 22.23.1, Apple M5 Pro. The machine was under load (load average about 20). Each number is the median of 5 separate processes, run in alternating order. The time is the first checkout after the snapshot import; the import itself is excluded.

first checkout	main*	0b63820	now
Text 1,024 ops	1.54 ms	4.11 ms	4.41 ms
Text 8,192	2.50 ms	13.26 ms	9.43 ms
Text 65,536	4.86 ms	44.83 ms	25.16 ms
Text 262,144	10.55 ms	147.95 ms	57.72 ms
Text 65,536, subscribed	5.06 ms	58.07 ms	25.67 ms
Text 262,144, subscribed	9.93 ms	191.98 ms	58.64 ms
32,768 child Maps, retreat one	54.49 ms	151.96 ms	61.16 ms
same, peak RSS	235.6 MiB	283.6 MiB	234.2 MiB

* main returns wrong results here: N−1 characters, and "tail" for the Map.

• Remaining Text cost: most of it is replaying the one touched container (about 40 of the 57 ms at 262k ops). The rest is history materialization, which main pays as well.
• Steady-state checkouts: 0.10 ms at 64k and 0.40 ms at 256k (0b63820: 0.09/0.36; main: 0.07/0.32).
• B4 trace: apply, export, and snapshot import are unchanged. Importing the trace as one update is about 30% faster (about 195 ms vs 275 ms) thanks to the coalescing. The resulting text and snapshot bytes are identical to main.
• bench:complexity 1000,2000,4000,8000: 7 alternating rounds against main. 0 of 284 cases are both slower than 1.5× main and more than 0.3 ms slower. The largest ratios are sub-0.4 ms noise that does not grow with size.

Validation

• pnpm -C loro-js check passes: 256 tests, plus format, lint, typecheck, and build.
• New tests:
  • A Rust styled-text fixture checked across 3 checkout orders, with the events mirrored into another doc.
  • Detached exports, shallow export, and edits or imports made elsewhere.
  • The unread nested-container event (F3).
  • A spy on _reset asserting that an unread nested sequence is rebuilt without a full rebuild.
  • Lazy export after history is loaded.
• Differential fuzz against a full-history doc: 60 seeds × 50 ops, 3 peers, covering Text with marks, List, MovableList, Map, Tree, and Counter. It runs lazy, eager, subscribed, and local-edit variants (with imports while detached) and compares values, events, diff, and re-exports. Every remaining failure is one of the pre-existing ones listed below.

Not fixed here (all present on main)

• The Rust style-anchor position gap described above. It affects update import and replay of Rust rich text, and needs its own PR.
• Convergence: in the fuzz, loro.js peers do not converge with a replay of the same updates in 13 of 60 seeds.
• Event diffs: some checkouts throw event diff position N is out of range, on full-history docs as well.
• Tree events: the order of tree event items can differ between hydrated and replayed trees.
• Shallow snapshots: shallow snapshots of concurrent histories still differ from full-history docs in some fuzz seeds. There are fewer failing seeds than on main.

[zxch3n]

Round-2 follow-up: e56d37af → 134c7c7b. main (#1124, #1133) is merged in.

B1: the pinned-snapshot mitigation is removed and redone

The stash/pin mechanism is gone. When #completeSnapshotSequence finds that a Text/List replay does not reproduce the snapshot state (Rust rich text, since loro.js does not count style anchors in positions), the container becomes unreplayable:

• The replay is discarded. The snapshot state stays installed and is encoded once. No replay is ever installed for that container, in any path (checkout, diff, detached export, full rebuild fallback, shallow export's latest state, forkAt).
• Transitions that touch it run without its ops and then move it separately:
  • If the installed state already has every forward op (tracked as applied), the Text is toggled by id and style version. This is O(delta), like the snapshot-state transitions on main.
  • Otherwise, for example after an update imported while detached, the container is rebuilt from its snapshot state, and the later ops that the target includes are applied (O(container)).
• Invariant: the latest state, imports, and exports always equal the snapshot state plus later ops, as on main. Older versions of such a text are approximate: text deleted before the snapshot cannot be restored, also as on main. The anchor model itself is left to 60be1650.

B1 repro (Rust WASM oracle)	main	e56d37a	now
pinned → detach → import → attach; Rust re-import	AellXo	AelloX	AellXo / AellXo
same with importBatch	AellXo	AelloX	AellXo
local A edit → checkout earlier → snapshot version	ellXo	elloX	ellXo

New regression tests:

• The B1 paths (import/importBatch/attached, with an event mirror and an export round trip), plus fork/forkAt.
• rich-text-history.json: 6 Rust-generated histories with random marks (all four expand types), inserts, and deletes, followed by remote updates. Random checkout/detach/import/importBatch sequences, with and without a subscriber mirror, must end at the latest state of a doc that only imports.
• Out-of-suite, the same property with the Rust oracle over 60 seeds × 2 modes: this PR passes 120/120. main fails 78/80 and e56d37af 72/80 (both corrupt the latest state via replays).

B2: coalesced inserts and overlapping deletes

• Root cause: a packed Text span returns a new wrapper per findById, and the transition's Set<SequenceElement> deduplicated by object identity. Transitions now deduplicate by id (SequenceElementSet).
• Already on main: this is a pre-existing bug there too. main throws the same RangeError when ab is inserted as one multi-character op. Coalescing just made single-character histories reach it.
• Atomic checkout: checkout/attach now go through #transitionTo. If the transition throws, the checkout version and state are restored before rethrowing.
• Tests: the reporter's repro with 1 and 2 inserts, checking events, value aX, and an export round trip (Rust re-import: aX). A forced failure inside a transition asserts that the version and state are unchanged.
• Side effect: the event diff position N is out of range throws that the fuzz previously attributed to full-history docs are this same bug. They no longer occur.

CI

.github/workflows/rust.yml filtered pull_request to bases main/dev/feat-gc, so the stacked PRs got no checks, and no job ran loro-js check. Now:

• The base filter is dropped.
• A loro-js job runs pnpm install --frozen-lockfile && pnpm -C loro-js check, validated locally with pnpm 10.

The previous push got build + loro-js green on all three PRs.

Validation

• Checks: pnpm -C loro-js check passes with 269 tests.
• Combined with main + fix(loro.js): include reattached child containers' state in diff() and revertTo() #1131 d8cb3917 + fix(loro.js): keep root-time map values and tree nodes when a shallow doc retreats #1127 + fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata #1128: 322 tests pass. All B1/B2/F1–F4 repros, the rich-text fuzz (120/120), and the general fuzz pass.
• General differential fuzz (60 seeds × 50 ops, lazy/eager/subscribed/local-edit): the only remaining differences are 13 seeds where loro.js peers do not converge with a replay of the same updates even without snapshots. They are identical on main.
• Shallow variant: the failing seeds are a subset of main's (39 vs 43).
• B4: importing the trace as one update gives byte-identical text and snapshot to main.

Performance

Apple M5 Pro, Node 22.23.1. Medians of 5 separate processes in alternating order, with load1 8.2–16.4 (median 10.8) during the run.

first checkout after snapshot import	main*	e56d37a	now
Text 65,536 ops	5.3	25.5	28.0
Text 262,144 ops	9.8	57.2	59.3
Text 262,144, subscribed	10.3	58.0	58.7
32k child Maps, retreat one	55.9	56.0	58.1
same, peak RSS (MiB)	234.3	234.9	232.2
unreplayable rich text 65,536	2.6	36.0	33.3
unreplayable rich text 262,144	2.9	87.6	74.2
unreplayable rich text 65,536, round trip	0.05	0.13	0.06
same, subscribed round trip	0.10	15.8	0.09

* On main, the Text rows return N−1 characters and the Map row returns the latest value; both are wrong.

• Unreplayable first checkout: it pays the replay that detects the divergence plus encoding the snapshot state once. Later checkouts match main.
• Rich-text len differs from e56d37af at the older version: 65,536 vs 65,537. The old head restored the deleted character through the replay it then kept. That was the B1 hazard, so this is intended.
• bench:complexity 1000,2000,4000,8000: 7 alternating rounds against main, load1 about 7–10. 0 of 852 size×case entries (3 heads × 284) are both over 1.5× main and more than 0.3 ms slower. The largest ratios are sub-0.2 ms entries at 1–2k.
• B4: 3 alternating rounds, load1 5.4–5.8. apply 227.3 vs 236.7 ms, snapshot export 109.1 vs 105.4, update export 70.0 vs 70.1, snapshot import 113.8 vs 110.3, update import 171.0 vs 234.8 ms (coalesced inserts, byte-identical result).
• CI on 134c7c7b: build and loro-js pass.

[zxch3n]

Round-3 update: 134c7c7b → 1c5ff489. This only merges main at 27982a55 (#1138, #1135).

The merge brings in main's Rust crates, changesets, and context docs only. git diff 134c7c7b 1c5ff489 -- loro-js is empty, so the loro.js code is exactly what round 2 reviewed.

Rechecked against Rust at 27982a55

#1135 changes which side of a style anchor Rust puts new text on, so I rebuilt the WASM oracle from 27982a55 and reran every cross-check.

• Repros: results are identical to round 2.
  • B1 gives AellXo for import, importBatch, attached, and local edit, and the Rust re-import matches.
  • B2 gives aX with one event; the Rust re-import is also aX.
  • F1–F4, detached export, shallow export of styled text, and lazy export are unchanged.
• Fixtures: styled-text.blob, styled-text.expected.json, and shallow-empty-state.blob regenerate byte-identical with the new Rust.
• rich-text-history.json: regenerating it gives different ops, because of the fix: rich-text insert positions near style anchors, delete IDs, and delete events #1135 change above. Both the committed file (written by the previous Rust) and the regenerated one pass snapshot-checkout.test.ts. I kept the committed one, because documents written by older versions are valid inputs too.
• Rust rich-text invariance fuzz with the new oracle: 240/240 pass (120 seeds × 2 modes).
• General differential fuzz (60 seeds) on the fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata #1128 head:
  • Same numbers as the round-2 head: 4,963 checks, 100 differences.
  • Outside the shallow variant, every difference falls in the 12 seeds where loro.js peers don't converge even without snapshots. main has the same 12 seeds, plus 22 other failures of its own, including duplicate sequence id throws.
  • Shallow variant: 47 failing seeds, all of which also fail on main (53 there).

Combined with #1131 ae55ff7a

I merged this stack's top (#1128 b716216c) with #1131 ae55ff7a locally. One conflict came up, in adjacent private-field declarations, and I kept both sides.

• pnpm -C loro-js check passes with 330 tests.
• The repros, the rich-text fuzz (240/240), and the general fuzz give the same results as fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata #1128 alone.
• The reviewer's B3 repro now matches Rust exactly on this combination: 7@1 → 8@1 {k: v}.

Performance

Setup:

• Apple M5 Pro (18 cores, shared with other sessions), Node 22.23.1.
• Every run is a separate process, and the order rotates between main, the round-2 heads, and the current heads.
• The drivers waited up to 10 minutes in total for load1 ≤ 6. It never dropped that low, so every number below was measured under load.
• load1 was recorded before each run. For the checkout matrix it was 7.7–12.8 (median 10.0), and the per-variant medians differ by at most 0.1.

First checkout after snapshot import (ms, median of 5):

case	main*	round 2 134c7c7b	now 1c5ff489
Text 65,536 ops	4.4	23.5	23.8
Text 262,144 ops	9.1	52.0	53.3
Text 262,144 ops, subscribed	9.3	53.0	53.3
32k child Maps, retreat one	53.6	47.7	47.5
32k child Maps, peak RSS (MiB)	234.1	234.2	233.8
unreplayable rich text 65,536	2.3	31.8	32.1
unreplayable rich text 262,144	2.7	70.4	69.5
unreplayable rich text 65,536, round trip	0.05	0.06	0.06
same round trip, subscribed	0.10	0.09	0.09

* On main, the Text rows return N−1 characters and the Map row returns the latest value. Both are wrong, as in round 2.

bench:complexity 1000,2000,4000,8000 (7 alternating rounds × 7 variants, load1 9.6–22.0, median 12.0). A case is flagged when it is over 1.5× the baseline and more than 0.3 ms slower.

• Against the round-2 heads: 0 of 852 entries flagged.

• Against main: 1 of 852, text-repeated-mark-tail-retreat@8000.

  • Times: fix(loro.js): correct checkout after a snapshot import #1126 0.455 → 0.780 ms; fix(loro.js): keep root-time map values and tree nodes when a shallow doc retreats #1127 0.72 ms; fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata #1128 0.70 ms. The round-2 heads measure 0.60–0.64 ms with identical code.
  • An earlier 3-round run flagged a different case, text-full-range-insert-retreat@1000, which the 7-round run does not flag.

• Targeted recheck of the repeated-mark case (7 alternating processes, load1 about 18):

  repeated-mark retreat	main	round 2	now
  8,000 marks, first retreat	1.90	2.04	2.07
  64,000 marks, first retreat	3.81	3.81	3.71

  Warm retreat and restore are equal. The difference is a constant of about 0.15 ms on the first call, not a scaling cost.

B4 (3 alternating rounds, load1 7.8–8.3), main / round-2 #1128 / now #1128, in ms:

phase	main	round-2 #1128	now #1128
apply	249.0	256.4	245.7
snapshot export	115.5	122.7	115.3
update export	74.9	73.4	75.0
snapshot import	125.8	121.5	127.3
update import (coalesced inserts)	253.7	181.9	182.0

Checks

• pnpm -C loro-js check passes with 269 tests.
• CI on 1c5ff489: build and loro-js pass.

[zxch3n]

Round-4 follow-up: 1c5ff489 → 8ef6818d.

Commits:

• ef4c3c30 merges main 29553d5b (no loro.js change).
• e8dca05a is the NEW-B1/NEW-B2/P2 fix.
• 32fa42e5 replaces that fix's MovableList part; see NEW-B2.
• 8ef6818d is docs only.

The PR description is rewritten for the current design. It adds merge order, known limitations, and #1136/#1137 integration notes.

NEW-B1: forkAt/fork before the snapshot version

• A fork now keeps a snapshot state only when its version includes that state's version. An older fork replays its own operations, as on main, so it stays consistent with its oplog.
• With the reviewer's repro and the Rust oracle 29553d5b, every row matches main:

forkAt(v1) / checkout(v1); fork()	main	1c5ff489	now
value	hello	ellXo	hello
after a local insert of Q, then Rust reads the fork's updates	Qhello / Qhello	QellXo / Qhello	Qhello / Qhello
fork.import(snapshot)	elloX	throws duplicate sequence id 7@1	elloX
MovableList forkAt(v0)	[0,1,2]	[9,1,2,0]	[0,1,2]

NEW-B2: Rust MovableList with a move

Decision: a MovableList is not a snapshot sequence at all. It stays on main's path: its hydrated state has no move or value history, so a transition that crosses its snapshot operations replays to the target.

Why: there is no sound way to compare a hydrated MovableList with its replay.

• Its Rust snapshot names a moved element by position id. listItemIds also includes the invisible old positions, and Rust encodes element ids as (peer, lamport), not (peer, counter).

Options tried, on 90 random Rust histories × every version:

option	MovableList versions ≠ Rust
1c5ff489 (move = unreplayable)	1,352
e8dca05a: always accept the replay at the current version, then retreat incrementally	581
complete only lists whose snapshot uses insert ids	167
main	91
now	56

• The incremental options expose loro.js's own MovableList transition bugs with concurrent moves and sets. A full-history loro.js doc on main fails the same versions, for example seed 17 [62,65] vs Rust [62,93]. main's snapshot docs avoid them by replaying to the target.
• The reviewer's repro now matches main and Rust:
  • checkout(v0), revertTo(v0) (with Rust re-import), diff(v1,v0), and forkAt(v0) all give [0,1,2].
  • A checkout round trip followed by the remote update gives [0,7,2].

Cost, as on main:

• A move or set imported right after a Rust snapshot import, before any checkout, still resolves the wrong element ([7,2,0]).
• When loro.js's replay really differs from Rust, the latest MovableList can change at the first crossing checkout: 37 of 90 seeds, vs 38 on main.
• Both are left to the MovableList model work (test(loro.js): add a Rust differential suite for MovableList #1132 and follow-ups).

Performance (8,192 / 65,536 elements, one move; first / each later checkout):

	8,192	65,536
main	9.0 / 0.15 ms	35.6 / 0.16 ms
now	9.2 / 0.18 ms	38.0 / 0.19 ms
1c5ff489 (wrong value)	17.0 / 2.16 ms	74.5 / 17.1 ms

P2 and diff

• A completion that throws at any point reinstalls the snapshot state and leaves the container hydrated.
• #transitionTo prepares inside its try; a failure before the version is set just rethrows, since nothing changed.
• diff restores the current state with a full rebuild when it, or its move back, throws.
• Tests:
  • The reviewer's injection points (the n-th visibleIdRuns/_swapState call during checkout and diff, followed by a retry and a Rust-readable export) reproduce elloX on 1c5ff489 and pass now.
  • A second test injects into setIdRunsVisible/setIdRunsDeleted/setDeleted during diff, covering both the forward move and the move back.

Docs, changeset, issue

• context/loro-js-performance.md is corrected:
  • the latest state is exact only as "the snapshot state plus later ops as loro.js applies them";
  • old versions of unreplayable text are approximate, with the review's numbers;
  • it covers the forkAt rule and the MovableList decision with its measurements.
• The changeset no longer claims that deleted text is restored for unreplayable text.
• The pre-existing concurrent-insert bug (abefX vs Rust abXef, reproduced on main 29553d5b) is now loro.js: an insert concurrent with a delete baked into an imported snapshot lands at the wrong position #1163.

Validation

• Checks: pnpm -C loro-js check passes with 276 tests. The 5 new fork/MovableList tests and the 2 fault tests fail on 1c5ff489.
• Rust oracle 29553d5b: B1, B2, F1–F4, detached/shallow/lazy export, and the rich-text fuzz (240/240) are unchanged. The new movable-moves.json fixture regenerates byte-identical.
• Random Rust histories (90 seeds; the full table is in the description):

mismatches with Rust	main	now
MovableList versions	91	56
forkAt Text / MovableList	64 / 7	64 / 7
revertTo MovableList	45	41
Text versions	517	787
revertTo Text	62	124
export re-imported by Rust	0	0

The Text rows are the documented rich-text trade-off; #1137 removes it.

• General loro.js-written fuzz (60 seeds): every non-shallow failure also fails on main (109 of 109 seed/variant pairs). The shallow failing seeds are a subset of main's (47 vs 53).
• Performance (load1 4.1–6.0; tables in the description):
  • First checkout and rich text are unchanged from round 3: Text 262k 53.9 ms, unreplayable rich text 262k 70.1 ms.
  • bench:complexity: 0 of 852 entries flagged against main and against the round-3 heads.
  • B4 update import: 174.0 vs 239.5 ms on main.
• CI on 8ef6818d: build and loro-js pass.

[zxch3n]

Round-5 follow-up: 8ef6818d → 8bbcd7b4 (R4-B1). main is still 29553d5b.

R4-B1: full rebuilds now check Text that had styles

• #checkSnapshotSequences compares every hydrated Text before a full replay if it is styled in one of three places:
  • its snapshot state;
  • its shallow root state;
  • its history. Mark operations are indexed per container as #markedTexts, filled with the other history indexes and carried over from the staged doc when deferred history loads.
• This is the cheaper variant you suggested: plain text is never checked.
• The lazy branch consults the index before it decodes the state.

Rust-written hlXo fixture (a mark whose characters were all deleted, plus a MovableList with one move):

path	main / round 4	now	Rust re-import
checkout(beforeMove) → attach	hloX	hlXo	hlXo
checkout(textDone) → attach	hloX	hlXo	hlXo
live shallow export at either version (doc and blob)	hloX	hlXo	hlXo
forkAt(latest)	hloX	hlXo	hlXo

The new test (deleted-mark.json) fails on 8ef6818d.

Exhaustive checkout(f) → shallow export at g → attach over every (f, g) of 600 random 2–3-peer Rust histories. I rebuilt this harness because yours was cleaned up.

live latest changed	main	round 4	now
plain text (7,586 cases)	194	0	0
marked text (21,104)	7,847	0	0
marked text, variant that deletes marked text (32,588)	15,677	34	0

• The blob as Rust reads it also changed in 34 cases on round 4, where main is wrong in 15,108; now it is 0. The only remaining blob differences are in 5 histories where loro.js and Rust already disagree when reading the imported snapshot (the anchor gap).
• 150 random 2-peer Text + MovableList seeds, checking walk + attach, export to Rust, live shallow export, and forkAt(latest):

	main	round 4	now
default seeds	110 / 110 / 93 / 93	0 / 0 / 0 / 0	0 / 0 / 0 / 0
delete variant	—	0 / 0 / 3 / 3	0 / 0 / 0 / 0

Docs and description

• The known limitations now also list:
  • the detached-import + live shallow export corruption (s=1299; fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136 fixes the plain-text case);
  • the non-atomic shallow export;
  • MovableList seed 22.
• context/loro-js-performance.md says the same and describes the new check.

Checks

• pnpm -C loro-js check passes in chain order: fix(loro.js): correct checkout after a snapshot import #1126 277, fix(loro.js): keep root-time map values and tree nodes when a shallow doc retreats #1127 290, fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata #1128 311 tests, 0 lint warnings.
• Combined with fix(loro.js): include reattached child containers' state in diff() and revertTo() #1131 ae55ff7a: 338 tests pass. R4-B1's paths give hlXo, and B3 gives Rust's exact tree.
• Unchanged from round 4 (Rust oracle 29553d5b): B1, B2, F1–F4, NEW-B1/NEW-B2, and the rich-text fuzz (240/240).
• The 90-seed random matrix gives the same numbers as round 4 (MovableList versions 56, forks 64/7, exports 0).

Performance

262,144 characters; 5 alternating rounds against main and the round-4 heads; load1 5.0–6.0 (per-variant medians 5.5–5.7). Times in ms as shallow export / forkAt(latest) / first checkout:

case	main	round 4	now
loro.js plain text	387.5 / 138.7 / 7.2	261.6 / 77.6 / 46.2	273.9 / 78.0 / 47.1
unreplayable Rust rich text	224.4 / 30.0 / 1.5 (wrong)	375.0 / 184.9 / 68.2	377.1 / 190.7 / 67.7
Rust text whose marked characters were deleted	214.7 / 30.7 / 1.4 (wrong)	219.4 / 35.3 / 61.6 (wrong)	372.0 / 160.3 / 66.4

• For plain text and the other cases, including 65,536-character text, the new check costs nothing measurable.
• The last row is the one-time replay that makes that text exact.

CI on 8bbcd7b4: build and loro-js pass.