Skip to content

fix(loro.js): keep rich-text style anchors in the text sequence - #1137

Merged
zxch3n merged 41 commits into
mainfrom
fix/loro-js-richtext-anchors
Sep 29, 2026
Merged

zxch3n merged 41 commits into
mainfrom
fix/loro-js-richtext-anchors

Conversation

@zxch3n

@zxch3n zxch3n commented Sep 27, 2026 •

Copy link
Copy Markdown
Member

Summary

loro.js now keeps rich-text style anchors in the text sequence, like Rust, so every Text op position (insert, delete, mark start/end; binary and JSON) means the same thing in both runtimes. This fixes the rich-text divergence analyzed in #1133:

Repro Rust loro.js before loro.js now
"abcd", mark [0,2), insert X at 3, delete 1 — loro.js importing Rust's update acXd acdX acXd
same edits in loro.js, imported by Rust acXd bXcd acXd
"a😀bc", mark [0,3), unmark [0,1), delete(3,1) — Rust importing loro.js JSON a😀c a😀bc a😀c
"abc" (v1), mark all + append d, Rust checks out v1 of loro.js's history abc abd abc

Breaking (major changeset): loro.js reads rich-text history written by 0.2 with Rust's positions and no version marker (decision: option A). In a document edited only with loro.js 0.2, every op after a mark can apply elsewhere (the second row: 0.2 read its own ops as acXd, Rust and now loro.js read bXcd). README "Upgrading from 0.2" gives measured readings (a 0.2 snapshot keeps its state, but its history reads like Rust; updates imported after it and its own shallow-snapshot export switch the document to Rust's reading) and migration paths (reading the final state with 0.2 is the safe one; reading with 1.0 right after importing a 0.2 snapshot is only valid when the snapshot is the final version); the decision is in context/loro-js-richtext-anchors.md.

For each repro, loro.js now writes Rust's JSON ops (tests/richtext-anchors.test.ts, against Rust-written fixtures), and Rust reads loro.js's updates and snapshots, including checkout (crates/loro/tests/loro_js_interop.rs). Ops are not identical to Rust's for every call (known gaps: delete merging, applyDelta unmark order, undo); the state is.

Design

Full write-up: context/loro-js-richtext-anchors.md.

  • Sequence: an anchor is a zero-width TextElement (value: "", anchor: { style, isEnd }); both anchors share a TextStyle. SequenceIndex counts zero-width elements per subtree, so Unicode ↔ entity ↔ UTF-16/UTF-8 conversions stay O(log n). Without anchors every path short-circuits to the old one.
  • Remote ops: Text inserts and anchors use the same Fugue insertion at the entity position of the op's causal view (start anchor at start, end anchor at end + 1), like RichtextDiffCalculator::apply_crdt_op_to_tracker.
  • Local ops follow Rust's TextHandler: find_best_insert_pos for inserts and both mark ends; delete runs split at anchors and written last run first; redundant marks/unmarks skipped; start >= end throws; applyDelta overrides inherited styles like Rust's apply_delta.
  • Styles: still indexed by element ID in TextStyleIndex, but membership now comes from the anchors: a style covers everything physically between its anchors (hidden and later-inserted elements included). New elements take the styles shared by their physical neighbors. A style applies once its end anchor is in the version. Mark events are emitted with the end anchor for the text whose value changed.
  • Checkout: anchors toggle like inserted text; style events from TextStyleIndex.transitions.
  • Snapshots: Rust's text state layout with real anchors (encode_snapshot_fast), and contiguous text merged into one span.
  • Undo: undoing a mark restores the value from just before its op wherever it changed it (one boundary gap, see known gaps).

mark/unmark keep their signatures but now throw when start >= end (as Rust), and the _-prefixed LoroText members that were visible in the type declarations (_insertVisible, _applyMark, _styleRuns, _validateInsertPosition, _unicodePosition, _detachedStyleCounter) are gone; the changeset lists both as breaking. The encoding is unchanged; loro.js now reads and writes it as Rust does. Also covered after review: lazily hydrated nested Text (positions hydrate first), shallow-snapshot redaction of dead style values (Rust's redact_dead_style_values), and checkout events at versions with a mark's start but not its end.

Validation

  • pnpm --dir loro-js check on the merge with main via fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136: 25 files, 414 tests (+1 Rust-only skip) incl. fix(loro.js): correct checkout after a snapshot import #1126's snapshot-checkout suite; format, lint, typecheck, build. Regression tests for the review's B3–B5 run in loro.js and loro-crdt and fail without their fixes; tests/legacy-data.test.ts pins how both read loro.js@0.2.1 fixtures.
  • Differential suite against a WASM build of main ce6498b5, with per-op two-version checkouts: 300 rich-text + 150 plain-text seeds × 120 steps pass (13001+, 14001+) (all expand modes, nested/overlapping styles, emoji/UTF-8, cross-runtime binary and JSON sync, checkout, snapshots, shallow snapshots, cursors, events, undo, revertTo).
  • After the second review (N1, 1856d8c): the suite also writes Rust MovableList inserts and sets (their sets force the replay path), and every imported full snapshot checks out three versions, diffs the last two, and attaches (snapshotCheckout on). Seed 19 of the default 40 fails without the fix (duplicate sequence id); with it, 300 rich + 150 plain seeds × 80 steps and 200 + 100 × 120 steps pass. pnpm --dir loro-js check with the Rust reference required: 25 files, 414 tests + 1 Rust-only skip.
  • cargo test -p loro --test loro_js_interop: 13 passed on main ce6498b5.

Integration with #1126–#1128 (now on main)

  • LoroText._swapState also swaps the anchor registry (#anchorStarts); _styleIndex stays replaceable as a TextStyleIndex<TextStyle>; TextState drops detachedStyleCounter.
  • #buildStateStore(frontiers, retained, redaction) takes fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata #1128's retained-container set and this PR's style redaction.
  • REPLAY_POSITION_GAP is no longer in main's snapshot-checkout.test.ts; nothing to remove.
  • fix(loro.js): correct checkout after a snapshot import #1126's unreplayable-text mechanism is narrowed: the check before every full replay and fork of styled Text (#checkSnapshotSequences, #hasStyleHistory, #markedTexts) existed because loro.js did not count the anchors, so it is removed. The generic fallback stays: a completed replay that differs from the snapshot state still keeps that state. With the anchors, Rust- and loro.js-written histories replay to their snapshot state (0 fallbacks in the whole suite); the fallback now fires only for snapshots whose state disagrees with their history (0.2 snapshots, and loro.js shallow exports with the known retained-range gap), plain and rich alike. Differential seeds with snapshotCheckout on gave the same pass/fail sets before and after the narrowing.

Performance

Node 22, alternating with main and #1136 at 1-minute load 6–9 (September 29):

  • B4: apply 256–278 ms (main 238–240); update export 9 ms, 274,574 B (main 72–73 ms, 1,153,540 B); snapshot 206,553 B (main 309,780 B); heap 91.8 MB (main 107.0 MB).
  • Rust-written styled Text from a snapshot (16k chars, 200 marks, 2k edits): first checkout 24.7–25.0 ms, back 4.1–4.3 ms, fork 11.8–12.2 ms, matching Rust; main 35.9–37.2 / 3.1–3.3 / 29.8–30.5 ms and wrong text. Without the transition-check fix from fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136 these checkouts took 350–410 ms here.
  • bench:complexity stays flat from 1k to 8k except text-repeated-mark-tail-{retreat,restore} (0.46 → 6.06 ms and 0.49 → 8.50 ms; main 0.12–0.50 ms), O(nested anchors) as in Rust and accepted in review.
  • Constant cost of the zero-width counters on plain Text: 8,000 subscribed middle inserts 15.4–15.8 vs 14.3–14.4 ms; text-subscribed-batch, history-commit, history-update-batch-import 12–20% slower at 8k than fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136. A sidecar like the line-break totals would remove it (not done here).
  • Typing 1,000 chars inside a 64k bold range: 4.0–7.4 ms (main 2.8–3.1 ms).

Fixed after the second review

  • N1 (1856d8c): with a snapshot import, a checkout past a mark, then one before it, attach/checkout/diff threw duplicate sequence id when the forward records could not move incrementally (a MovableList set/move/delete). The replay re-inserted anchors the state already held. hasMaterializedSequenceInsertions now counts text-mark/text-mark-end, and diff rebuilds its from state in that case (this also fixes the same throw with plain List inserts, pre-existing on main). Regression: tests/richtext-anchors.test.ts (5→3→attach, 4→3→7, diff, both runtimes) plus the suite's seed 19.

Known gaps

The scope is narrowed to N1 and the docs; the items below are follow-ups.

  • Mark undo boundary: after undoing a before/both mark, text typed at its old start still gets the undone value, because the restoring mark's start anchor goes after the undone one; Rust leaves the text unstyled. A fix (d1663c9, 159/160 cases equal to Rust) was backed out of this PR (b822913) for a follow-up.
  • Checkout into the retained range of a shallow snapshot that loro.js exported diverges (shallowCheckout, off). loro.js uses the requested frontiers as the root, while Rust moves it back to a critical version (#calculateShallowStart). The suite samples a few versions per export (6/40 rich and 5/20 plain seeds at 120 steps). The review's exhaustive check found most multi-peer histories affected (14,276/17,724 non-critical-root versions here, 0 with a critical root). It is pre-existing, and fix(loro.js): root shallow snapshots at the latest critical version #1144 roots shallow snapshots at the latest critical version.
  • Op shape (N4): adjacent deletes in one transaction merge over a narrower range than Rust (#mergeTrailingTextDeletes), and applyDelta writes the unmarks for inherited styles in a different order (#insertOverrides). The state is the same; fix(loro.js): record absent-key map deletes and zero counter increments like loro-crdt #1145 works on local op parity.
  • Rust panics on reading a snapshot of a version that has a mark's start but not its end (unclosed style mark, container_wrapper.rs:198); loro.js writes that state in Rust's layout, so a loro.js fork/forkAt snapshot of such a version hits it too. Rust issue: Snapshot of a version with a mark's start but not its end panics on read ("unclosed style mark") #1165. Only op-level frontiers reach such versions.
  • MovableList history views in documents that also have rich text follow main (test(loro.js): add a Rust differential suite for MovableList #1132); the suite does not compare the list, and leaves out moves and deletes because Rust panics (movable_list_state.rs consistency check) checking out a loro.js snapshot of a list the runtimes already disagree on.
  • Rust's own diff between concurrent versions can disagree with its checkout (a style value on the wrong side; seed 162 at 120 steps). The suite requires only the loro.js diff to reach its target.
  • Undo parity with Rust's UndoManager under remote edits (see fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136).
  • A delete op that covers an anchor (Rust never writes one) hides the anchor, but the style keeps its range; Rust's state would drop the style.

Merge order / rebase plan

  1. fix: rich-text insert positions near style anchors, delete IDs, and delete events #1135 (Rust fixes) — merged.
  2. fix(loro.js): stop a concurrent insert at the end of its origin's subtree #1139 (Fugue subtree fix; fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136 contains it).
  3. fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136 (differential suite + plain-text fixes), base main.
  4. This PR, stacked on fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136; retarget to main after fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136 merges.

#1126–#1128 are on main and merged in (see Integration above). #1131 touches document.ts but not Text positions (resolved in an earlier trial merge). The MovableList rework in progress elsewhere touches sequence-index.ts; the zero-width counters here are additive.

🤖 Generated with Claude Code

zxch3n and others added 2 commits September 28, 2026 04:13
A seeded differential suite now runs the same Text edits, syncs, checkouts,
snapshots, cursors, undo, and revertTo in loro.js and loro-crdt (nodejs WASM)
and compares every step; it skips when no WASM build exists.

It found these divergences, fixed here: Fugue placement after a sibling
subtree, deletes applied by recorded ID instead of position, doubly deleted
text restored twice by checkout events, and cursor semantics. Ops are also
written in Rust's shape (merged inserts, last delete run first).

See context/loro-js-rust-differential.md and context/loro-js-performance.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Rust writes every Text op position as an entity index that counts style
anchors. loro.js kept styles beside the text, so once a document had a mark,
each runtime applied the other's later edits at different places (#1133).

Anchors are now zero-width elements of the Text sequence: remote marks insert
them with Fugue, local edits follow Rust's insert/delete/mark position rules,
styles cover the elements between their anchors, and snapshots, checkout,
events, and undo use them. The differential suite now runs with marks.

See context/loro-js-richtext-anchors.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
zxch3n and others added 14 commits September 28, 2026 04:29
…tree

The indexed Fugue path jumped from the last direct child of `originLeft`
to the origin-right element. Rust's scan stops at the first concurrent
element whose origin is further left, so the insert could land after
that element. Replicas then disagreed depending on import order, and all
of them could disagree with loro-crdt.

Check whether the interval's last element descends from `originLeft` and
otherwise binary-search the boundary. The descent test skips implicit
runs through per-peer sorted explicit counters. Rationale and cost:
context/loro-js-performance.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An element right after its ID predecessor has it as origin-left, so only
run starts can end a sibling's subtree (context/loro-js-performance.md).
Importing a concurrent character after a 512k-character run drops from
~9.5 ms to ~0.2 ms. Adds text-concurrent-insert-after-long-run to
bench:complexity.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Decision and rationale: context/loro-js-rust-differential.md ("Data
written by loro.js 0.2"). The changeset becomes major; README "Upgrading
from 0.2" lists what changes and how to migrate. Fixtures written by
loro.js 0.2.1 pin how loro.js and Rust read its updates, snapshots, and
cursors.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…t events

- Insert positions hydrate a lazily loaded Text first; its anchors decide
  where text goes.
- Shallow snapshots null the values of styles with no text left, like
  Rust's redact_dead_style_values (context/shallow-snapshot-style-redaction.md).
- A transition that only toggles a start anchor starts the text's event
  baseline, so it no longer reports the whole text as inserted.
- The differential suite checks out per-op versions, two in a row.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Visits two versions per checkout, including versions right after a random
op, so transitions between historical versions and mid-change versions
are compared with Rust. README "Upgrading from 0.2" now reflects what a
document loaded from a 0.2 snapshot does after its history is used.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The decision and its reasons are in context/loro-js-richtext-anchors.md
("Data written by loro.js 0.2"). The changeset becomes major and lists
the breaking changes; README "Upgrading from 0.2" covers rich text; a
0.2.1 fixture pins how loro.js and Rust read its rich-text ops.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

zxch3n commented Sep 28, 2026

Copy link
Copy Markdown
Member Author

Review follow-up (head b842514d). Base is now #1136 (b1e4bade), which includes main with #1135; the differential suite runs against a nodejs WASM build of main (27982a5).

Runtime fixes (B3–B5), each with a regression test that runs in loro.js and in loro-crdt (tests/richtext-anchors.test.ts, "rich-text anchors in %s") and fails without its fix:

  • B3 lazy nested Text: #entityFrom hydrates first, so #insertPosition sees the anchors. The review's repro (insert at 0/2/4 into an unread nested Text with full-range bold, expand before) now gives Rust's deltas (Xabcd bold, abXcd bold, abcd bold + X).
  • B4 shallow redaction: redactDeadStyleValues (Rust's redact_dead_style_values) nulls the value of every pair with no text between its anchors at the shallow root, except both-expand pairs; the latest state nulls only the pairs redacted at the root (the whitelist rule). Anchors and IDs stay. Root values now match Rust for none/before/after/both; a style that dies after the root keeps its value in the latest state.
  • B5 half-mark checkout events: anchor ops take the bulk transition path, which records no event, so a transition that only toggled a start anchor had no event state and #emit diffed the text against an empty value (insert:"abc"). The text's event baseline now starts when an anchor op is collected. The differential suite now checks out two versions in a row, each either a commit version or the version right after a random op (mid-change and mid-op versions included); with the fix reverted it fails (seed 9025) within 40 seeds.

Compatibility (B1/B2), decided as option A: loro.js reads all data with Rust's semantics, no version marker.

  • Changeset → major (loro.js 0.2.1 → 1.0.0) and lists the breaking changes, including start >= end now throwing and the removed _-members of LoroText.
  • README "Upgrading from 0.2": what changes, measured readings of 0.2 data, and migration paths. Measured with the published loro.js@0.2.1: updates/JSON read like Rust (bXcd); a snapshot keeps its 0.2 state (acXd) but its history reads like Rust, so a checkout there and back switches to bXcd (Rust itself shows [aX]Xd), and snapshot vs update replicas diverge after new edits. Recommended: read the final state (with 0.2, or right after importing a 0.2 snapshot) and rebuild, or import the old updates everywhere.
  • Decision and reasons: context/loro-js-richtext-anchors.md ("Data written by loro.js 0.2"); fixtures written by 0.2.1 plus tests/legacy-data.test.ts pin the readings in both runtimes.

Performance (Node 22, fresh processes, order reversed in round 2, 1-minute load 5–25; ranges are the two rounds):

main #1137 before (8b67bda) now (b842514)
Concurrent char after a 512k run (review probe) 0.20–0.45 ms 8.9–9.3 ms 0.22–0.28 ms
text-concurrent-insert-after-long-run, 64k → 512k 0.16–0.37 ms 2.43 → 16.50 ms 0.20–0.28 ms
64k: type 1,000 chars inside bold 2.6–4.3 ms 2.2–5.9 ms 2.4–6.6 ms (same-load A/B: 2.4–2.5 vs main 2.9–4.3)
64k: full-range mark / retreat / restore 0.06–0.12 / 0.12–0.14 / 0.02–0.03 ms 0.13–0.14 / 0.23–0.24 / 0.11 ms 0.11–0.15 / 0.22–0.27 / 0.11–0.14 ms
Undo of those 1,000 chars 16.8–21.6 ms 0.44–0.45 ms 0.56 ms
text-repeated-mark-tail-retreat 1k / 8k 0.14 / 0.41 ms 0.51 / 5.96 ms 0.51 / 6.11 ms
B4 apply 239–265 ms 243–259 ms 243–265 ms
B4 update export 71–76 ms, 1,153,540 B 9 ms, 274,574 B 9–11 ms, 274,574 B
B4 snapshot export / import 109–130 / 112–131 ms, 309,780 B 27–30 / 63–64 ms, 206,553 B 26–31 / 63–73 ms, 206,553 B
Heap after B4 apply 107.0 MB 91.7 MB 91.7 MB

bench:complexity at 8k: median now/before 0.96; only the repeated-mark entries grow (O(nested anchors), as in Rust).

The accepted costs (typing inside bold, repeated-mark retreat/restore, O(nested anchors) as in Rust) are recorded in context/loro-js-performance.md with these numbers.

Validation

Not changed here: UndoManager parity with remote edits, the detach/attach import gap, and Rust's own shallow issue (seed 5048) — separate items per the review.

zxch3n and others added 5 commits September 28, 2026 17:27
Both branches fixed the same misordering after the last sibling. Keep
#1139's binary search over origin-left descent and drop this branch's
run-start walk (SequenceIndex.nextPhysicalIdRunStart); the measured
trade-off is in context/loro-js-performance.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

zxch3n commented Sep 28, 2026

Copy link
Copy Markdown
Member Author

Follow-up: merged the updated #1136, which now uses #1139's Fugue fix instead of its own run walk (reasons and measurements in the #1136 comment). Head d0d652d9.

Validation:

  • pnpm check: 22 files, 347 tests.
  • Differential against the main WASM build, with per-op two-version checkouts: 150 rich-text + 75 plain-text seeds × 120 steps pass.

Benchmark rerun (Node 22, fresh processes, order reversed in round 2; waited 120 s for 1-minute load < 8, round 1 ran at 7.4–8.1 and round 2 at 10–17):

main review head 8b67bda3 now d0d652d9
Concurrent char after a 512k run 0.29–0.30 ms 9.7–12.6 ms 0.37–0.39 ms
text-concurrent-insert-after-long-run, 64k → 512k 0.17–0.30 ms 2.81 → 20.74 ms 0.22–0.31 ms
64k: type 1,000 chars inside bold 2.8 ms 4.6–8.5 ms 5.3–5.8 ms
same, with an UndoManager 4.2–4.6 ms 5.5–7.4 ms 5.0–7.1 ms
Undo of those 1,000 chars 17.5–19.0 ms 0.62–0.69 ms 0.52–0.74 ms
64k: full-range mark / retreat / restore 0.07–0.08 / 0.11–0.13 / 0.02 ms 0.16–0.18 / 0.28–0.29 / 0.11–0.13 ms 0.16–0.17 / 0.32–0.35 / 0.12–0.13 ms
text-repeated-mark-tail-retreat, 1k / 8k 0.23 / 0.36 ms 0.78 / 7.77 ms 0.71 / 8.25 ms
B4 apply 246–255 ms 267–326 ms 265–277 ms
B4 update export 72–76 ms, 1,153,540 B 10–13 ms, 274,574 B 10–11 ms, 274,574 B
B4 snapshot export / import 114–121 / 126–127 ms, 309,780 B 31–45 / 70–114 ms, 206,553 B 31–32 / 73–80 ms, 206,553 B
Heap after B4 apply 106.9 MB 91.7 MB 91.7 MB

bench:complexity at 8k: median now/review head 0.99. Only the repeated-mark entries grow: O(nested anchors), as in Rust, and accepted in review. Mark apply/retreat/restore stay flat at 0.1–0.35 ms across sizes; the gap to main is a constant per anchor.

Typing inside bold is load-sensitive. In this batch it measured 5.3–5.8 ms vs 2.8 ms on main; in an earlier same-load A/B, 2.4–2.5 ms vs 2.9–4.3 ms. The review measured 2.9–4.4 ms vs 1.9 ms and accepted the cost.

Both runs are recorded in context/loro-js-performance.md.

Merge order: #1139 → #1136 → this PR (#1135 is merged). The SequenceElementSet duplicate with #1126 comes in through #1136.

main now has #1126, which adds the same `SequenceElementSet` this branch
added; keep #1126's copy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
zxch3n and others added 4 commits September 29, 2026 14:58
…-richtext-anchors

Integrates #1126–#1128: `_swapState` also swaps the anchor registry and
keeps `_styleIndex` replaceable with `TextStyleIndex<TextStyle>`; the
shallow export passes #1128's retained-container set and this branch's
style redaction to `#buildStateStore`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e moot

#1126 checked snapshot-hydrated styled Text before every full replay
because loro.js did not count the style anchors, so a replay of Rust rich
text could differ from its snapshot state. With the anchors in the
sequence, styled Text replays like plain Text; the unreplayable fallback
stays for snapshots whose state disagrees with their own history
(context/loro-js-performance.md).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ow export

With #1126 on main, a checkout keeps a 0.2 snapshot's state while a fork
or shallow export replays its history (Rust's reading); the test pins the
fork difference from Rust.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…chtext-anchors

# Conflicts:
#	loro-js/README.md
#	loro-js/tests/legacy-data.test.ts
@github-actions

Copy link
Copy Markdown
Contributor

WASM Size Report

  • Original size: 3277.59 KB
  • Gzipped size: 1089.33 KB
  • Brotli size: 762.83 KB

zxch3n and others added 6 commits September 29, 2026 15:18
#canTransitionRecords walked the whole sequence once per insert or mark
op, so checking out across scattered edits was O(ops * elements): 668 ms
at 8k characters with 2k edits on main, 5 ms now. Adds
text-scattered-edits-checkout to bench:complexity.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#canTransitionRecords walked the whole sequence once per insert op, so
checking out across scattered edits was O(ops * elements): 668 ms at 8k
characters with 2k edits on main, 4 ms now. Adds
text-scattered-edits-checkout to bench:complexity.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…chtext-anchors

# Conflicts:
#	loro-js/src/runtime/document.ts
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

zxch3n commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

Merged #1136 (which now includes main ce6498b5 with #1126–#1128). Head is now 6bb9f476.

Integration with #1126–#1128

#1126's unreplayable-text mechanism, narrowed

  • Measured: with the anchors, Rust- and loro.js-written histories replay to their snapshot state. I counted zero fallbacks across the whole test suite, including fix(loro.js): correct checkout after a snapshot import #1126's random checkouts on Rust rich-text histories and its deleted-mark case.

  • Where it still fires:

    • snapshots whose state disagrees with their own history (0.2 snapshots);
    • loro.js-exported shallow snapshots hit by the known retained-range gap.

    Both happen in plain and rich text alike.

  • Removed: the styled-Text check before every full replay and fork (#checkSnapshotSequences, #hasStyleHistory, #markedTexts). It existed only because anchors used to shift positions. The now-unused previousVersion parameter of #rebuildFromHistory is gone too.

  • Kept: the generic fallback.

  • No outcome change: differential seeds with snapshotCheckout on give the same pass/fail sets before and after the narrowing (rich 61/19, plain 65/15). All failures are "shallow snapshot from js: checkout".

  • 0.2 snapshots:

    • a checkout keeps their state (pinned by a loro.js-only test; Rust shows neither reading after the round trip);
    • a fork replays history (Rust's reading);
    • README and context/loro-js-performance.md are updated.

Performance (Node 22, alternating main / #1136 / this PR, 1-minute load 6–9):

main this PR
Rust styled Text from a snapshot (16k chars, 200 marks, 2k edits): first checkout / back / fork 36–37 / 3.1–3.3 / 30 ms, wrong text 24.7–25.0 / 4.1–4.3 / 12 ms, matches Rust
Without #1136's transition-check fix — 350–410 ms per checkout
B4 apply 238–240 ms 256–278 ms
B4 update export, bytes / snapshot bytes / heap 72–73 ms, 1,153,540 B / 309,780 B / 107.0 MB 9 ms, 274,574 B / 206,553 B / 91.8 MB
Type 1,000 chars inside a 64k bold range 2.8–3.1 ms 4.0–7.4 ms
64k full-range mark / retreat / restore 0.06–0.08 / 0.11 / 0.02 ms 0.16 / 0.22–0.24 / 0.12 ms
text-repeated-mark-tail-{retreat,restore} 8k 0.50 / 0.26 ms 6.06 / 8.50 ms (accepted: O(nested anchors), as in Rust)

New finding: constant cost of the zero-width counters on plain Text. They are four extra fields per treap node, updated on every edit.

  • 8,000 subscribed middle inserts take 15.4–15.8 ms vs 14.3–14.4 ms on fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136.
  • text-subscribed-batch, history-commit and history-update-batch-import are 12–20% slower at 8k (three alternating runs).
  • Nothing grows with size.
  • Moving the counters into a sidecar that only anchored sequences allocate, like the line-break totals, would remove it. I haven't done that here. It's recorded in context/loro-js-performance.md.

Validation

  • pnpm check: 25 files, 412 tests (+1 Rust-only skip).
  • 300 rich-text + 150 plain-text differential seeds × 120 steps with per-op checkouts against the main ce6498b5 WASM build.
  • cargo test -p loro --test loro_js_interop: 13 passed.

zxch3n and others added 6 commits September 29, 2026 17:33
Cursors move for side 1 and end-of-text, not astral text; reading the final
state with 1.0 needs the snapshot to be the final version; a shallow export
switches a 0.2-snapshot document to Rust's reading.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…chtext-anchors

# Conflicts:
#	loro-js/README.md
#	loro-js/tests/legacy-data.test.ts
…plays

After a snapshot import and a checkout back past a mark, attach, checkout,
and diff threw `duplicate sequence id` when the forward records could not
move incrementally (a movable-list set, move, or delete): the replay
inserted the mark's anchors again although the state already held them.
`hasMaterializedSequenceInsertions` now counts anchor ops, and `diff`
rebuilds its `from` state in that case too.

The differential harness now edits a movable list, checks out three
versions of every imported full snapshot, diffs, and attaches; seed 19
fails without this fix. It also derived its second checkout roll from
bits that repeated the first. Details: context/loro-js-richtext-anchors.md
and context/loro-js-rust-differential.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Undoing a before- or both-expand mark put the restoring mark's start anchor
after the undone one (`mark`'s insert rule), so text typed at the old start
went between the two anchors and kept the undone value; Rust leaves it
unstyled. `_undoStyle` now starts the restoring mark right before the undone
style's start anchor. Against Rust, 159 of 160 expand/prior-style/typing
cases now give the same text (3 differed before); see
context/loro-js-richtext-anchors.md for the remaining one.

The differential harness now requires only the loro.js diff to reach its
target and logs Rust's (seed 162 at 120 steps shows a Rust diff that
disagrees with its checkout).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ndent

The second checkout version came from `imul(roll, 0x9e3779b1)`, whose low
bits repeat the roll's (the multiplier is 1 modulo 16), so it usually picked
the first version again. It now uses the product's high bits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…chtext-anchors

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

zxch3n commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

Second review, addressed. The heads are listed at the end.

Checks: pnpm --dir loro-js check with the Rust reference required gives 25 files, 422 tests + 1 Rust-only skip. The differential suite passes 300 rich + 150 plain seeds × 80 steps and 200 + 100 × 120 steps.

New, not from this PR: Rust's own diff between concurrent versions can disagree with its checkout (seed 162 at 120 steps), so the suite checks only loro.js's diff against the target.

Heads: #1136 at 84431ba and #1137 at 97d9264 (#1136 merged in with a normal merge and push).

🤖 Generated with Claude Code

zxch3n and others added 2 commits September 29, 2026 18:03
The scope of #1137 was narrowed to N1 and the docs, so this backs out the
`_undoStyle` change of d1663c9 and its tests; the known gap is recorded in
context/loro-js-richtext-anchors.md. The differential harness keeps the
Rust-diff tolerance from that commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

zxch3n commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

Scope narrowed: this PR fixes only N1 (1856d8c) and D1/D2 (docs, plus the measured snapshot-plus-updates example in 8cf87d8: 0.2 shows [a]cXd, 1.0 and Rust show [bX]cd).

The mark-undo boundary fix is backed out (b822913) and listed under known gaps as a follow-up; d1663c9 has the fix. N2 is #1165; N3 and N4 are under known gaps. pnpm --dir loro-js check with the Rust reference: 414 tests + 1 Rust-only skip.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zxch3n
zxch3n changed the base branch from fix/loro-js-plain-text-rust-parity to main September 29, 2026 10:20
@zxch3n
zxch3n merged commit c0ff125 into main Sep 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant