Repository navigation
fix(loro.js): keep rich-text style anchors in the text sequence - #1137
Conversation
A seeded differential suite now runs the same Text edits, syncs, checkouts, snapshots, cursors, undo, and revertTo in loro.js and loro-crdt (nodejs WASM) and compares every step; it skips when no WASM build exists. It found these divergences, fixed here: Fugue placement after a sibling subtree, deletes applied by recorded ID instead of position, doubly deleted text restored twice by checkout events, and cursor semantics. Ops are also written in Rust's shape (merged inserts, last delete run first). See context/loro-js-rust-differential.md and context/loro-js-performance.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Rust writes every Text op position as an entity index that counts style anchors. loro.js kept styles beside the text, so once a document had a mark, each runtime applied the other's later edits at different places (#1133). Anchors are now zero-width elements of the Text sequence: remote marks insert them with Fugue, local edits follow Rust's insert/delete/mark position rules, styles cover the elements between their anchors, and snapshots, checkout, events, and undo use them. The differential suite now runs with marks. See context/loro-js-richtext-anchors.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tree The indexed Fugue path jumped from the last direct child of `originLeft` to the origin-right element. Rust's scan stops at the first concurrent element whose origin is further left, so the insert could land after that element. Replicas then disagreed depending on import order, and all of them could disagree with loro-crdt. Check whether the interval's last element descends from `originLeft` and otherwise binary-search the boundary. The descent test skips implicit runs through per-peer sorted explicit counters. Rationale and cost: context/loro-js-performance.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An element right after its ID predecessor has it as origin-left, so only run starts can end a sibling's subtree (context/loro-js-performance.md). Importing a concurrent character after a 512k-character run drops from ~9.5 ms to ~0.2 ms. Adds text-concurrent-insert-after-long-run to bench:complexity. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Decision and rationale: context/loro-js-rust-differential.md ("Data
written by loro.js 0.2"). The changeset becomes major; README "Upgrading
from 0.2" lists what changes and how to migrate. Fixtures written by
loro.js 0.2.1 pin how loro.js and Rust read its updates, snapshots, and
cursors.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…t events - Insert positions hydrate a lazily loaded Text first; its anchors decide where text goes. - Shallow snapshots null the values of styles with no text left, like Rust's redact_dead_style_values (context/shallow-snapshot-style-redaction.md). - A transition that only toggles a start anchor starts the text's event baseline, so it no longer reports the whole text as inserted. - The differential suite checks out per-op versions, two in a row. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Visits two versions per checkout, including versions right after a random op, so transitions between historical versions and mid-change versions are compared with Rust. README "Upgrading from 0.2" now reflects what a document loaded from a 0.2 snapshot does after its history is used. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The decision and its reasons are in context/loro-js-richtext-anchors.md
("Data written by loro.js 0.2"). The changeset becomes major and lists
the breaking changes; README "Upgrading from 0.2" covers rich text; a
0.2.1 fixture pins how loro.js and Rust read its rich-text ops.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Review follow-up (head Runtime fixes (B3–B5), each with a regression test that runs in loro.js and in
Compatibility (B1/B2), decided as option A: loro.js reads all data with Rust's semantics, no version marker.
Performance (Node 22, fresh processes, order reversed in round 2, 1-minute load 5–25; ranges are the two rounds):
The accepted costs (typing inside bold, repeated-mark retreat/restore, O(nested anchors) as in Rust) are recorded in Validation
Not changed here: UndoManager parity with remote edits, the detach/attach import gap, and Rust's own shallow issue (seed 5048) — separate items per the review. |
Both branches fixed the same misordering after the last sibling. Keep #1139's binary search over origin-left descent and drop this branch's run-start walk (SequenceIndex.nextPhysicalIdRunStart); the measured trade-off is in context/loro-js-performance.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Follow-up: merged the updated #1136, which now uses #1139's Fugue fix instead of its own run walk (reasons and measurements in the #1136 comment). Head Validation:
Benchmark rerun (Node 22, fresh processes, order reversed in round 2; waited 120 s for 1-minute load < 8, round 1 ran at 7.4–8.1 and round 2 at 10–17):
Typing inside bold is load-sensitive. In this batch it measured 5.3–5.8 ms vs 2.8 ms on Both runs are recorded in Merge order: #1139 → #1136 → this PR (#1135 is merged). The |
…-richtext-anchors Integrates #1126–#1128: `_swapState` also swaps the anchor registry and keeps `_styleIndex` replaceable with `TextStyleIndex<TextStyle>`; the shallow export passes #1128's retained-container set and this branch's style redaction to `#buildStateStore`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e moot #1126 checked snapshot-hydrated styled Text before every full replay because loro.js did not count the style anchors, so a replay of Rust rich text could differ from its snapshot state. With the anchors in the sequence, styled Text replays like plain Text; the unreplayable fallback stays for snapshots whose state disagrees with their own history (context/loro-js-performance.md). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ow export With #1126 on main, a checkout keeps a 0.2 snapshot's state while a fork or shallow export replays its history (Rust's reading); the test pins the fork difference from Rust. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…chtext-anchors # Conflicts: # loro-js/README.md # loro-js/tests/legacy-data.test.ts
WASM Size Report
|
#canTransitionRecords walked the whole sequence once per insert or mark op, so checking out across scattered edits was O(ops * elements): 668 ms at 8k characters with 2k edits on main, 5 ms now. Adds text-scattered-edits-checkout to bench:complexity. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#canTransitionRecords walked the whole sequence once per insert op, so checking out across scattered edits was O(ops * elements): 668 ms at 8k characters with 2k edits on main, 4 ms now. Adds text-scattered-edits-checkout to bench:complexity. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…chtext-anchors # Conflicts: # loro-js/src/runtime/document.ts
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Merged #1136 (which now includes
#1126's unreplayable-text mechanism, narrowed
Performance (Node 22, alternating
New finding: constant cost of the zero-width counters on plain Text. They are four extra fields per treap node, updated on every edit.
Validation
|
Cursors move for side 1 and end-of-text, not astral text; reading the final state with 1.0 needs the snapshot to be the final version; a shallow export switches a 0.2-snapshot document to Rust's reading. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…chtext-anchors # Conflicts: # loro-js/README.md # loro-js/tests/legacy-data.test.ts
…plays After a snapshot import and a checkout back past a mark, attach, checkout, and diff threw `duplicate sequence id` when the forward records could not move incrementally (a movable-list set, move, or delete): the replay inserted the mark's anchors again although the state already held them. `hasMaterializedSequenceInsertions` now counts anchor ops, and `diff` rebuilds its `from` state in that case too. The differential harness now edits a movable list, checks out three versions of every imported full snapshot, diffs, and attaches; seed 19 fails without this fix. It also derived its second checkout roll from bits that repeated the first. Details: context/loro-js-richtext-anchors.md and context/loro-js-rust-differential.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Undoing a before- or both-expand mark put the restoring mark's start anchor after the undone one (`mark`'s insert rule), so text typed at the old start went between the two anchors and kept the undone value; Rust leaves it unstyled. `_undoStyle` now starts the restoring mark right before the undone style's start anchor. Against Rust, 159 of 160 expand/prior-style/typing cases now give the same text (3 differed before); see context/loro-js-richtext-anchors.md for the remaining one. The differential harness now requires only the loro.js diff to reach its target and logs Rust's (seed 162 at 120 steps shows a Rust diff that disagrees with its checkout). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ndent The second checkout version came from `imul(roll, 0x9e3779b1)`, whose low bits repeat the roll's (the multiplier is 1 modulo 16), so it usually picked the first version again. It now uses the product's high bits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…chtext-anchors Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Second review, addressed. The heads are listed at the end.
Checks: New, not from this PR: Rust's own Heads: #1136 at 84431ba and #1137 at 97d9264 (#1136 merged in with a normal merge and push). 🤖 Generated with Claude Code |
The scope of #1137 was narrowed to N1 and the docs, so this backs out the `_undoStyle` change of d1663c9 and its tests; the known gap is recorded in context/loro-js-richtext-anchors.md. The differential harness keeps the Rust-diff tolerance from that commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Scope narrowed: this PR fixes only N1 (1856d8c) and D1/D2 (docs, plus the measured snapshot-plus-updates example in 8cf87d8: 0.2 shows The mark-undo boundary fix is backed out (b822913) and listed under known gaps as a follow-up; d1663c9 has the fix. N2 is #1165; N3 and N4 are under known gaps. 🤖 Generated with Claude Code |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Summary
loro.js now keeps rich-text style anchors in the text sequence, like Rust, so every Text op position (insert, delete, mark start/end; binary and JSON) means the same thing in both runtimes. This fixes the rich-text divergence analyzed in #1133:
"abcd", mark[0,2), insertXat 3, delete 1 — loro.js importing Rust's updateacXdacdXacXdacXdbXcdacXd"a😀bc", mark[0,3), unmark[0,1), delete(3,1) — Rust importing loro.js JSONa😀ca😀bca😀c"abc"(v1), mark all + appendd, Rust checks out v1 of loro.js's historyabcabdabcBreaking (major changeset): loro.js reads rich-text history written by 0.2 with Rust's positions and no version marker (decision: option A). In a document edited only with loro.js 0.2, every op after a mark can apply elsewhere (the second row: 0.2 read its own ops as
acXd, Rust and now loro.js readbXcd). README "Upgrading from 0.2" gives measured readings (a 0.2 snapshot keeps its state, but its history reads like Rust; updates imported after it and its ownshallow-snapshotexport switch the document to Rust's reading) and migration paths (reading the final state with 0.2 is the safe one; reading with 1.0 right after importing a 0.2 snapshot is only valid when the snapshot is the final version); the decision is incontext/loro-js-richtext-anchors.md.For each repro, loro.js now writes Rust's JSON ops (
tests/richtext-anchors.test.ts, against Rust-written fixtures), and Rust reads loro.js's updates and snapshots, including checkout (crates/loro/tests/loro_js_interop.rs). Ops are not identical to Rust's for every call (known gaps: delete merging,applyDeltaunmark order, undo); the state is.Design
Full write-up:
context/loro-js-richtext-anchors.md.TextElement(value: "",anchor: { style, isEnd }); both anchors share aTextStyle.SequenceIndexcounts zero-width elements per subtree, so Unicode ↔ entity ↔ UTF-16/UTF-8 conversions stay O(log n). Without anchors every path short-circuits to the old one.start, end anchor atend + 1), likeRichtextDiffCalculator::apply_crdt_op_to_tracker.TextHandler:find_best_insert_posfor inserts and both mark ends; delete runs split at anchors and written last run first; redundant marks/unmarks skipped;start >= endthrows;applyDeltaoverrides inherited styles like Rust'sapply_delta.TextStyleIndex, but membership now comes from the anchors: a style covers everything physically between its anchors (hidden and later-inserted elements included). New elements take the styles shared by their physical neighbors. A style applies once its end anchor is in the version. Mark events are emitted with the end anchor for the text whose value changed.TextStyleIndex.transitions.encode_snapshot_fast), and contiguous text merged into one span.mark/unmarkkeep their signatures but now throw whenstart >= end(as Rust), and the_-prefixedLoroTextmembers that were visible in the type declarations (_insertVisible,_applyMark,_styleRuns,_validateInsertPosition,_unicodePosition,_detachedStyleCounter) are gone; the changeset lists both as breaking. The encoding is unchanged; loro.js now reads and writes it as Rust does. Also covered after review: lazily hydrated nested Text (positions hydrate first), shallow-snapshot redaction of dead style values (Rust'sredact_dead_style_values), and checkout events at versions with a mark's start but not its end.Validation
pnpm --dir loro-js checkon the merge withmainvia fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136: 25 files, 414 tests (+1 Rust-only skip) incl. fix(loro.js): correct checkout after a snapshot import #1126's snapshot-checkout suite; format, lint, typecheck, build. Regression tests for the review's B3–B5 run in loro.js andloro-crdtand fail without their fixes;tests/legacy-data.test.tspins how both readloro.js@0.2.1fixtures.maince6498b5, with per-op two-version checkouts: 300 rich-text + 150 plain-text seeds × 120 steps pass (13001+, 14001+) (all expand modes, nested/overlapping styles, emoji/UTF-8, cross-runtime binary and JSON sync, checkout, snapshots, shallow snapshots, cursors, events, undo,revertTo).snapshotCheckouton). Seed 19 of the default 40 fails without the fix (duplicate sequence id); with it, 300 rich + 150 plain seeds × 80 steps and 200 + 100 × 120 steps pass.pnpm --dir loro-js checkwith the Rust reference required: 25 files, 414 tests + 1 Rust-only skip.cargo test -p loro --test loro_js_interop: 13 passed onmaince6498b5.Integration with #1126–#1128 (now on
main)LoroText._swapStatealso swaps the anchor registry (#anchorStarts);_styleIndexstays replaceable as aTextStyleIndex<TextStyle>;TextStatedropsdetachedStyleCounter.#buildStateStore(frontiers, retained, redaction)takes fix(loro.js): drop containers deleted before the shallow root, keep tree node metadata #1128's retained-container set and this PR's style redaction.REPLAY_POSITION_GAPis no longer inmain'ssnapshot-checkout.test.ts; nothing to remove.#checkSnapshotSequences,#hasStyleHistory,#markedTexts) existed because loro.js did not count the anchors, so it is removed. The generic fallback stays: a completed replay that differs from the snapshot state still keeps that state. With the anchors, Rust- and loro.js-written histories replay to their snapshot state (0 fallbacks in the whole suite); the fallback now fires only for snapshots whose state disagrees with their history (0.2 snapshots, and loro.js shallow exports with the known retained-range gap), plain and rich alike. Differential seeds withsnapshotCheckouton gave the same pass/fail sets before and after the narrowing.Performance
Node 22, alternating with
mainand #1136 at 1-minute load 6–9 (September 29):main238–240); update export 9 ms, 274,574 B (main72–73 ms, 1,153,540 B); snapshot 206,553 B (main309,780 B); heap 91.8 MB (main107.0 MB).main35.9–37.2 / 3.1–3.3 / 29.8–30.5 ms and wrong text. Without the transition-check fix from fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136 these checkouts took 350–410 ms here.bench:complexitystays flat from 1k to 8k excepttext-repeated-mark-tail-{retreat,restore}(0.46 → 6.06 ms and 0.49 → 8.50 ms;main0.12–0.50 ms), O(nested anchors) as in Rust and accepted in review.text-subscribed-batch,history-commit,history-update-batch-import12–20% slower at 8k than fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136. A sidecar like the line-break totals would remove it (not done here).main2.8–3.1 ms).Fixed after the second review
attach/checkout/diffthrewduplicate sequence idwhen the forward records could not move incrementally (a MovableList set/move/delete). The replay re-inserted anchors the state already held.hasMaterializedSequenceInsertionsnow countstext-mark/text-mark-end, anddiffrebuilds itsfromstate in that case (this also fixes the same throw with plainListinserts, pre-existing onmain). Regression:tests/richtext-anchors.test.ts(5→3→attach,4→3→7,diff, both runtimes) plus the suite's seed 19.Known gaps
The scope is narrowed to N1 and the docs; the items below are follow-ups.
before/bothmark, text typed at its old start still gets the undone value, because the restoring mark's start anchor goes after the undone one; Rust leaves the text unstyled. A fix (d1663c9, 159/160 cases equal to Rust) was backed out of this PR (b822913) for a follow-up.shallowCheckout, off). loro.js uses the requested frontiers as the root, while Rust moves it back to a critical version (#calculateShallowStart). The suite samples a few versions per export (6/40 rich and 5/20 plain seeds at 120 steps). The review's exhaustive check found most multi-peer histories affected (14,276/17,724 non-critical-root versions here, 0 with a critical root). It is pre-existing, and fix(loro.js): root shallow snapshots at the latest critical version #1144 roots shallow snapshots at the latest critical version.#mergeTrailingTextDeletes), andapplyDeltawrites the unmarks for inherited styles in a different order (#insertOverrides). The state is the same; fix(loro.js): record absent-key map deletes and zero counter increments like loro-crdt #1145 works on local op parity.unclosed style mark,container_wrapper.rs:198); loro.js writes that state in Rust's layout, so a loro.jsfork/forkAtsnapshot of such a version hits it too. Rust issue: Snapshot of a version with a mark's start but not its end panics on read ("unclosed style mark") #1165. Only op-level frontiers reach such versions.main(test(loro.js): add a Rust differential suite for MovableList #1132); the suite does not compare the list, and leaves out moves and deletes because Rust panics (movable_list_state.rsconsistency check) checking out a loro.js snapshot of a list the runtimes already disagree on.diffbetween concurrent versions can disagree with its checkout (a style value on the wrong side; seed 162 at 120 steps). The suite requires only the loro.js diff to reach its target.UndoManagerunder remote edits (see fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136).Merge order / rebase plan
main.mainafter fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite #1136 merges.#1126–#1128 are on
mainand merged in (see Integration above). #1131 touchesdocument.tsbut not Text positions (resolved in an earlier trial merge). The MovableList rework in progress elsewhere touchessequence-index.ts; the zero-width counters here are additive.🤖 Generated with Claude Code