fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata

[zxch3n]

Stacked on #1127, which is stacked on #1126. Merge #1126 → #1127 → #1128 back to back (see #1126 and #1127).

Summary

This ports Rust #1119 and #1123 to loro.js.

On main, LoroDoc.#buildStateStore writes every container ever created into both the shallow root state and the latest state. So a map child deleted before the cut, such as {"secret":"old"}, ships in every shallow snapshot. Re-exporting an older blob at its own root keeps it too.

Fix

• #retainedContainerKeys() is the Rust AliveWalk::Retention rule. It walks from the root containers through visible Map/List children, and follows every tree node's meta, including deleted nodes, because a retained move can revive a node with its old meta.
• Mergeable containers are roots of the walk, as in Rust. Their deterministic id survives a deleted or replaced marker, and re-ensuring the same kind shows their state again. So a hidden mergeable child keeps its data.
• The root state keeps the walk at the root. The latest state keeps the walk at the root and at the latest version, plus every container whose creation id the root version vector does not include. This mirrors retain_created_after_root.
• fix: drop dead containers when re-exporting a shallow doc at its cached root #1123 needs no separate code path: loro.js always rebuilds the root state by replay (it has no cached-root reuse branch), so re-exports of old blobs are pruned too.
• The tree-revival checkout tests found one more retreat gap in the fix(loro.js): keep root-time map values and tree nodes when a shallow doc retreats #1127 fallback: when a retained delete wins but the node's placement op was trimmed, the node stayed alive. It now falls back to the root placement and stays deleted.

context/internal-encoding.md documents this in its retention-set section, with a loro-js paragraph.

Parity with Rust (#1119/#1123 scenarios)

Scenario	loro.js on main	this PR
Map children deleted before the root (0 / 300 retained ops)	kept in root and latest state	dropped
Containers created after the root	kept	kept
Meta of a node revived from a deleted parent, or by a remote Move of a directly deleted node	kept (everything was kept)	kept
Checkout across revivals vs full history, both directions	wrong after the root (retained delete with trimmed placement)	matches
Re-export of old Rust blobs (legacy_* from #1123) and of an old loro.js blob	dead map survives	dropped
Rust pre-#1119 blob without revived meta	imports	imports (structure only, as in Rust)
Tree diff cache seeded with deleted nodes	n/a: loro.js keeps deleted records in its tree state	n/a

Validation

• New loro-js/tests/shallow-snapshot-deleted-containers.test.ts (17 tests) ports every test in shallow_snapshot_deleted_containers.rs except state-only, which loro.js does not implement. It also imports the Rust fixtures from fix: drop containers deleted before the shallow root from shallow and state-only snapshots #1119/fix: drop dead containers when re-exporting a shallow doc at its cached root #1123 and a new blob exported by origin/main loro.js (legacy-shallow-tree-revival-dead-map.ts.blob). On the base branch, 8 tests fail; the revival tests passed before too and now guard against over-pruning.

• Cross-runtime check: Rust main imports shallow blobs exported by this branch (tree revival + dead map, 0 and 300 filler ops). The deep value equals the loro.js value, has_container(dead map) is false, and a Rust re-export at the same root keeps the same value.

• pnpm -C loro-js check (format, lint, typecheck, tests, build) passes with 311 tests. shallow-snapshot-deleted-containers.test.ts also covers hidden mergeable children (switched kind and deleted marker, with 0 and 300 retained ops), and Rust reads them the same.

• Shallow export, 8,192 child Maps with half deleted before the root and 300 retained ops (median of 5 alternating runs, load1 5.8–6.0):

  • main: 138.0 ms, 558,402 B.
  • Round 3 c12ccc99: 110.5 ms, 352,852 B.
  • Now 64d12269: 112.0 ms, 352,852 B.

  The output is 37% smaller. The round-3 review measured a different setup: 387,772 → 267,366 B (−31%).

• Combined with fix(loro.js): include reattached child containers' state in diff() and revertTo() #1131 ae55ff7a: pnpm -C loro-js check passes with 338 tests. On that combination, the round-2 tree revival repro (B3) gives Rust's exact tree.

Integration notes

• fix(loro.js): keep rich-text style anchors in the text sequence #1137: its shallow export redacts dead styles through #buildStateStore(frontiers, { redacted, only }), while this PR passes a retention set (#buildStateStore(frontiers, retained)). The two need one combined signature, and the shallow export needs both. Whichever PR lands second does this.
• Inherited from fix(loro.js): correct checkout after a snapshot import #1126: the snapshot-checkout changes, their known limitations, and the fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136/fix(loro.js): keep rich-text style anchors in the text sequence #1137 notes in fix(loro.js): correct checkout after a snapshot import #1126's description.

Rust: #1119 (merged), #1123 (open). If #1123's final semantics change, this PR should follow.

🤖 Generated with Claude Code

[zxch3n]

Follow-up on the review of 9ef23c75 → now 0d358cb7. The updated #1127, #1126, and main are merged in. #1123's Rust paragraph in context/internal-encoding.md sits before the loro.js one.

• F2: #retainedContainerKeys now treats every mergeable container as a retention root, as Rust's existing_retention_roots does (mergeable cids are root-form). A child hidden by a deleted or different-kind marker kept its deterministic id, but it was exported empty.
  • I tested 8 variants: deleted marker or kind switch, × 0 or 300 retained ops, × loro.js or Rust reader, each with a checkout across the cut.
  • All 8 now restore "hello", the same as main and Rust. Before this change all 8 gave "".
  • Regression tests are in shallow-snapshot-deleted-containers.test.ts, and context/internal-encoding.md and context/mergeable-containers.md are updated.
• Retained-delete tree fix: it now lives in fix(loro.js): keep root-time map values and tree nodes when a shallow doc retreats #1127. Both sides had identical code, so it merged cleanly. The changeset sentence moved there too.
• Shallow export keeps snapshot states (from fix(loro.js): correct checkout after a snapshot import #1126). A shallow export of a Rust rich-text snapshot keeps both the live doc and the blob exact. I verified this with a Rust import.
• Shallow export perf. 8,192 child Maps, half deleted before the root, 300 retained ops, 3 alternating runs:
  • main: 156–167 ms, 558,402 B
  • 9ef23c75: 121–131 ms, 352,852 B
  • now: 122–126 ms, 352,852 B
• First checkout (inherited from fix(loro.js): correct checkout after a snapshot import #1126): a 262k-op Text drops from 149.26 to 56.58 ms. 32k child Maps drop from 155.85 to 56.94 ms, with peak RSS 283.8 → 235.2 MiB.
• Checks: pnpm -C loro-js check passes (289 tests). bench:complexity against main flags 0 of 284 cases. The changesets only bump loro.js (patch).
• Fuzz: in the shallow variant, the failing seeds are a strict subset of main's under the same settings (38 vs 43 of 60 seeds). These are pre-existing loro.js shallow-ordering issues with concurrent histories, and are not addressed here.

[github-actions]

WASM Size Report

• Original size: 3272.76 KB
• Gzipped size: 1087.19 KB
• Brotli size: 760.88 KB

[zxch3n]

Round-2 follow-up: 0d358cb7 → c12ccc99. The updated #1127/#1126 and main are merged in.

• Inherited from fix(loro.js): correct checkout after a snapshot import #1126: the B1/B2 fixes, and the unreplayable-container handling in shallow export. The latest state is rebuilt from the snapshot state. The root state uses the plain replay, as before, because a snapshot state that is later than the root cannot restore text deleted before it; that limitation is unchanged from main.
• Inherited from fix(loro.js): keep root-time map values and tree nodes when a shallow doc retreats #1127: the N1 index.
• CI: fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata #1128 now gets build and loro-js checks. On c12ccc99, build and loro-js pass.
• Checks: pnpm -C loro-js check passes (303 tests).
• Combined with main and fix(loro.js): include reattached child containers' state in diff() and revertTo() #1131 d8cb3917: 322 tests pass. B1/B2/F1–F4, the Rust rich-text invariance fuzz (120/120), and the mergeable/tree retention repros all pass. The reviewer's B3 repro (fix(loro.js): include reattached child containers' state in diff() and revertTo() #1131) also gives the correct tree on that fix(loro.js): include reattached child containers' state in diff() and revertTo() #1131 head.
• Shallow export (8,192 child Maps, half deleted before the root, 300 retained ops; median of 3 alternating runs): main 133.7 ms / 558,402 B, 0d358cb7 103.1 ms / 352,852 B, now 108.6 ms / 352,852 B. The output is unchanged, and the time is within noise.

[zxch3n]

Round-3 update: c12ccc99 → b716216c. This only merges #1127 79170146, which brings in #1126 and main 27982a55. No loro.js code changed.

• Rust recheck with the WASM oracle rebuilt from 27982a55: the F2 repro is unchanged. For both switch-kind and delete-marker, with 0 and 300 retained ops, loro.js and Rust both give {"x":"hello"}.
• Combined with fix(loro.js): include reattached child containers' state in diff() and revertTo() #1131 ae55ff7a:
  • pnpm -C loro-js check passes with 330 tests.
  • The B1/B2/F1–F4 repros pass, and so does the rich-text fuzz (240/240).
  • In the general fuzz, the shallow variant's failing seeds are a subset of main's.
  • The reviewer's B3 repro matches Rust exactly: 7@1 → 8@1 {k: v}.
• Shallow export (8,192 child Maps, half deleted before the root, 300 retained ops; median of 5 alternating runs, load1 11.4–12.5):

	time	size
main	152.1 ms	558,402 B
round 2 c12ccc99	124.5 ms	352,852 B
now b716216c	125.4 ms	352,852 B

• B4 (main / round-2 / now, ms):

phase	main	round 2	now
apply	249.0	256.4	245.7
update import	253.7	181.9	182.0

The other phases are within noise. The full table is in #1126's round-3 comment.

• bench:complexity: against round-2 heads, 0 of 852 entries are flagged. Against main, 1 is: the first-call constant of about 0.15 ms covered in fix(loro.js): correct checkout after a snapshot import #1126's comment.
• Checks: pnpm -C loro-js check passes with 303 tests.
• CI on b716216c: build and loro-js pass.

[zxch3n]

Round-4 follow-up: b716216c → 64d12269. This merges main 29553d5b and #1127, which carries #1126's NEW-B1/NEW-B2/P2 fixes (see #1126's comment). This PR's own code is unchanged.

• Description: updated with the merge order, the mergeable-root retention (F2), current numbers, and the fix(loro.js): keep rich-text style anchors in the text sequence #1137 integration point: #buildStateStore must take both the retention set and the style redaction.
• Checks: pnpm -C loro-js check passes with 310 tests.
• Rust oracle 29553d5b: F2 (hidden mergeable children, switch/delete × 0/300 retained ops, loro.js and Rust readers) and F4 are unchanged.
• Fuzz:
  • Random Rust histories give the same numbers as fix(loro.js): correct checkout after a snapshot import #1126.
  • General fuzz: the shallow variant's failing seeds are a subset of main's (47 vs 53).
• Combined with fix(loro.js): include reattached child containers' state in diff() and revertTo() #1131 ae55ff7a (the same one adjacent-field conflict; I kept both sides):
  • 337 tests pass.
  • B3 gives Rust's exact tree (7@1 → 8@1 {k: v}).
  • NEW-B1/NEW-B2 match main/Rust.
  • The rich-text fuzz passes 240/240.
• Shallow export, 8,192 child Maps with half deleted before the root and 300 retained ops (median of 5 alternating runs, load1 5.8–6.0):

	time	size
main	138.0 ms	558,402 B
round 3	110.5 ms	352,852 B
now	112.0 ms	352,852 B

• B4 update import: main 239.5 ms, round 3 169.7 ms, now 174.0 ms. The other phases are within noise.
• CI on 64d12269: build and loro-js pass.

[zxch3n]

Round-5 follow-up: 64d12269 → e4b01934. This merges #1127, which carries #1126's R4-B1 fix (see #1126's comment); this PR's own code is unchanged.

• Checks: pnpm -C loro-js check passes with 311 tests.
• R4-B1 on this head, which has its own shallow export with retention:
  • The hlXo paths match Rust: checkout → attach, live shallow export and its blob at both versions, and forkAt(latest).
  • The exhaustive checkout → shallow export → attach run with the delete-marked-text variant changes the live state in 0 of 40,174 cases.
  • The 150 random seeds are 0 in both variants.
  • F2 and F4 are unchanged.
• Combined with fix(loro.js): include reattached child containers' state in diff() and revertTo() #1131 ae55ff7a: 338 tests pass. R4-B1 gives hlXo, B3 gives Rust's exact tree, and the rich-text fuzz passes 240/240.
• Performance, 5 alternating rounds, load1 5.0–6.0; shallow export / forkAt(latest), in ms:
  • loro.js plain text, 262,144: 268.1 / 75.6 on round 4, 267.4 / 77.6 now.
  • Rust text whose marked characters were deleted: 218.9 / 34.7 on round 4 (wrong), 368.9 / 162.2 now (right).

CI on e4b01934: build and loro-js pass.