Skip to content

fix(loro.js): stop a concurrent insert at the end of its origin's subtree - #1139

Closed
zxch3n wants to merge 1 commit into
mainfrom
fix/loro-js-fugue-origin-subtree
Closed

zxch3n wants to merge 1 commit into
mainfrom
fix/loro-js-fugue-origin-subtree

Conversation

@zxch3n

@zxch3n zxch3n commented Sep 27, 2026

Copy link
Copy Markdown
Member

Summary

loro.js Text/List replicas could converge to different orders, and all of them could differ from loro-crdt, when an insert was concurrent with a run typed after the same character and with another concurrent element whose origin was further left.

Minimal repro (Rust gives xaaaabn for every import order; loro.js on main gives xaaaanb):

p1.getText("t").insert(0, "x"); p1.commit();
p2.import(p1.export({ mode: "update" }));
p1.getText("t").insert(1, "aaaa"); p1.commit();   // run under x
p4.getText("t").insert(0, "n"); p4.commit();      // concurrent root
p2.getText("t").insert(1, "b"); p2.commit();      // concurrent with the run

A 4-peer List variant makes one replica disagree with the others (AXBYN vs AXYBN).

Rationale

indexedFugueInsertion only looks at the direct children of originLeft. Sibling subtrees are contiguous, so the gap between two children belongs to the earlier one. After the last child, though, it set the insert position to the origin-right element. The interval can also hold concurrent elements whose origin is left of originLeft (the concurrent root above). Rust's scan in CrdtRope::insert breaks before them; loro.js jumped past them.

The fix keeps the index path. After the last child it checks whether the interval's last element descends from originLeft. If it does not, it binary-searches the boundary (descent is monotone because subtrees are contiguous). The descent test walks origin-left links but jumps over each implicit run through a new per-peer sorted list of explicit (non-consecutive) element counters in FugueOriginIndex. It costs O(explicit links · log n), like Rust's span-based scan, not one probe per scalar. The scan path already matched Rust and is unchanged.

Validation

  • New tests in loro-js/tests/fugue-index.test.ts: the text repro under three import orders, a 4-peer List/Text repro, and a probe-count bound for a 16k concurrent run (fails on main). The expected values come from the same scenarios run on loro-crdt (WASM build of this main).
  • Differential fuzz (harness in a separate PR), 500 seeds of Map/List/Text/Counter, replay/checkout oracles: peer-vs-replay and Rust-replay failures drop from 14 to 5 on this branch alone. The remaining 5 are the detached-import delete bug, fixed in its own PR.
  • pnpm --dir loro-js format:check, lint, typecheck, test (251 passed), build.
  • bench:complexity (1k–8k), 5 alternating rounds vs main, loadavg 15–16: 284 items, no item both >1.5× and >0.3 ms slower. Total 1044 ms on main, 1008 ms here.

🤖 Generated with Claude Code

…tree

The indexed Fugue path jumped from the last direct child of `originLeft`
to the origin-right element. Rust's scan stops at the first concurrent
element whose origin is further left, so the insert could land after
that element. Replicas then disagreed depending on import order, and all
of them could disagree with loro-crdt.

Check whether the interval's last element descends from `originLeft` and
otherwise binary-search the boundary. The descent test skips implicit
runs through per-peer sorted explicit counters. Rationale and cost:
context/loro-js-performance.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

WASM Size Report

  • Original size: 3210.11 KB
  • Gzipped size: 1062.59 KB
  • Brotli size: 744.13 KB

zxch3n added a commit that referenced this pull request Sep 29, 2026
…ntial suite (#1136)

* fix(loro.js): converge plain-text edits with Rust

A seeded differential suite now runs the same Text edits, syncs, checkouts,
snapshots, cursors, undo, and revertTo in loro.js and loro-crdt (nodejs WASM)
and compares every step; it skips when no WASM build exists.

It found these divergences, fixed here: Fugue placement after a sibling
subtree, deletes applied by recorded ID instead of position, doubly deleted
text restored twice by checkout events, and cursor semantics. Ops are also
written in Rust's shape (merged inserts, last delete run first).

See context/loro-js-rust-differential.md and context/loro-js-performance.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(loro.js): stop a concurrent insert at the end of its origin's subtree

The indexed Fugue path jumped from the last direct child of `originLeft`
to the origin-right element. Rust's scan stops at the first concurrent
element whose origin is further left, so the insert could land after
that element. Replicas then disagreed depending on import order, and all
of them could disagree with loro-crdt.

Check whether the interval's last element descends from `originLeft` and
otherwise binary-search the boundary. The descent test skips implicit
runs through per-peer sorted explicit counters. Rationale and cost:
context/loro-js-performance.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(loro.js): jump between ID runs when finding a Fugue subtree end

An element right after its ID predecessor has it as origin-left, so only
run starts can end a sibling's subtree (context/loro-js-performance.md).
Importing a concurrent character after a 512k-character run drops from
~9.5 ms to ~0.2 ms. Adds text-concurrent-insert-after-long-run to
bench:complexity.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): read 0.2 data like Rust and document the upgrade

Decision and rationale: context/loro-js-rust-differential.md ("Data
written by loro.js 0.2"). The changeset becomes major; README "Upgrading
from 0.2" lists what changes and how to migrate. Fixtures written by
loro.js 0.2.1 pin how loro.js and Rust read its updates, snapshots, and
cursors.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(loro.js): check out per-op versions in the differential suite

Visits two versions per checkout, including versions right after a random
op, so transitions between historical versions and mid-change versions
are compared with Rust. README "Upgrading from 0.2" now reflects what a
document loaded from a 0.2 snapshot does after its history is used.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): describe per-op checkouts in the differential suite

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): record Fugue subtree-end measurements

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): record Fugue measurements after adopting #1139

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): describe 0.2 snapshots after checkout, fork, and shallow export

With #1126 on main, a checkout keeps a 0.2 snapshot's state while a fork
or shallow export replays its history (Rust's reading); the test pins the
fork difference from Rust.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(loro.js): check a transition's sequence elements once per container

#canTransitionRecords walked the whole sequence once per insert op, so
checking out across scattered edits was O(ops * elements): 668 ms at 8k
characters with 2k edits on main, 4 ms now. Adds
text-scattered-edits-checkout to bench:complexity.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): record the transition check measurements

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): correct the 0.2 cursor and migration notes

Cursors move for side 1 and end-of-text, not astral text; reading the final
state with 1.0 needs the snapshot to be the final version; a shallow export
switches a 0.2-snapshot document to Rust's reading.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(loro.js): make the differential checkout's second version independent

The second checkout version came from `imul(roll, 0x9e3779b1)`, whose low
bits repeat the roll's (the multiplier is 1 modulo 16), so it usually picked
the first version again. It now uses the product's high bits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

zxch3n commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

Closing: covered by #1136 (e74d64dc), which fixed the same Fugue bug (fugueSubtreeEnd stops the insert at the end of the last sibling's subtree).

Checked on main at c0ff1255: the text repro (xaaaabn) and the 4-peer List repro (AXYBN) match loro-crdt for every import order, and all 5 tests from this PR's tests/fugue-index.test.ts pass on main. An insert concurrent with a 16k/131k run under the same origin takes the same ~0.05–0.4 ms on main as on this branch, so there is no performance reason to keep the alternative implementation.

@zxch3n zxch3n closed this Sep 29, 2026
pull Bot pushed a commit to Superoldman96/loro that referenced this pull request Sep 29, 2026
…-dev#1137)

* fix(loro.js): converge plain-text edits with Rust

A seeded differential suite now runs the same Text edits, syncs, checkouts,
snapshots, cursors, undo, and revertTo in loro.js and loro-crdt (nodejs WASM)
and compares every step; it skips when no WASM build exists.

It found these divergences, fixed here: Fugue placement after a sibling
subtree, deletes applied by recorded ID instead of position, doubly deleted
text restored twice by checkout events, and cursor semantics. Ops are also
written in Rust's shape (merged inserts, last delete run first).

See context/loro-js-rust-differential.md and context/loro-js-performance.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(loro.js): keep rich-text style anchors in the text sequence

Rust writes every Text op position as an entity index that counts style
anchors. loro.js kept styles beside the text, so once a document had a mark,
each runtime applied the other's later edits at different places (loro-dev#1133).

Anchors are now zero-width elements of the Text sequence: remote marks insert
them with Fugue, local edits follow Rust's insert/delete/mark position rules,
styles cover the elements between their anchors, and snapshots, checkout,
events, and undo use them. The differential suite now runs with marks.

See context/loro-js-richtext-anchors.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(loro.js): stop a concurrent insert at the end of its origin's subtree

The indexed Fugue path jumped from the last direct child of `originLeft`
to the origin-right element. Rust's scan stops at the first concurrent
element whose origin is further left, so the insert could land after
that element. Replicas then disagreed depending on import order, and all
of them could disagree with loro-crdt.

Check whether the interval's last element descends from `originLeft` and
otherwise binary-search the boundary. The descent test skips implicit
runs through per-peer sorted explicit counters. Rationale and cost:
context/loro-js-performance.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(loro.js): jump between ID runs when finding a Fugue subtree end

An element right after its ID predecessor has it as origin-left, so only
run starts can end a sibling's subtree (context/loro-js-performance.md).
Importing a concurrent character after a 512k-character run drops from
~9.5 ms to ~0.2 ms. Adds text-concurrent-insert-after-long-run to
bench:complexity.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): read 0.2 data like Rust and document the upgrade

Decision and rationale: context/loro-js-rust-differential.md ("Data
written by loro.js 0.2"). The changeset becomes major; README "Upgrading
from 0.2" lists what changes and how to migrate. Fixtures written by
loro.js 0.2.1 pin how loro.js and Rust read its updates, snapshots, and
cursors.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(loro.js): hydrate lazy text, redact dead styles, and keep checkout events

- Insert positions hydrate a lazily loaded Text first; its anchors decide
  where text goes.
- Shallow snapshots null the values of styles with no text left, like
  Rust's redact_dead_style_values (context/shallow-snapshot-style-redaction.md).
- A transition that only toggles a start anchor starts the text's event
  baseline, so it no longer reports the whole text as inserted.
- The differential suite checks out per-op versions, two in a row.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test(loro.js): check out per-op versions in the differential suite

Visits two versions per checkout, including versions right after a random
op, so transitions between historical versions and mid-change versions
are compared with Rust. README "Upgrading from 0.2" now reflects what a
document loaded from a 0.2 snapshot does after its history is used.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): read 0.2 rich text with Rust's positions

The decision and its reasons are in context/loro-js-richtext-anchors.md
("Data written by loro.js 0.2"). The changeset becomes major and lists
the breaking changes; README "Upgrading from 0.2" covers rich text; a
0.2.1 fixture pins how loro.js and Rust read its rich-text ops.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): describe per-op checkouts in the differential suite

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): record Fugue subtree-end measurements

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): record the accepted rich-text anchor costs

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): record Fugue measurements after adopting loro-dev#1139

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): add the rerun rich-text measurements

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor(loro.js): drop the styled-text replay check that anchors made moot

loro-dev#1126 checked snapshot-hydrated styled Text before every full replay
because loro.js did not count the style anchors, so a replay of Rust rich
text could differ from its snapshot state. With the anchors in the
sequence, styled Text replays like plain Text; the unreplayable fallback
stays for snapshots whose state disagrees with their own history
(context/loro-js-performance.md).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): describe 0.2 snapshots after checkout, fork, and shallow export

With loro-dev#1126 on main, a checkout keeps a 0.2 snapshot's state while a fork
or shallow export replays its history (Rust's reading); the test pins the
fork difference from Rust.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(loro.js): check a transition's sequence elements once per container

#canTransitionRecords walked the whole sequence once per insert or mark
op, so checking out across scattered edits was O(ops * elements): 668 ms
at 8k characters with 2k edits on main, 5 ms now. Adds
text-scattered-edits-checkout to bench:complexity.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf(loro.js): check a transition's sequence elements once per container

#canTransitionRecords walked the whole sequence once per insert op, so
checking out across scattered edits was O(ops * elements): 668 ms at 8k
characters with 2k edits on main, 4 ms now. Adds
text-scattered-edits-checkout to bench:complexity.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): record the transition check measurements

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): record anchor-model constant costs and rich-text checkout

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs(loro.js): correct the 0.2 cursor and migration notes

Cursors move for side 1 and end-of-text, not astral text; reading the final
state with 1.0 needs the snapshot to be the final version; a shallow export
switches a 0.2-snapshot document to Rust's reading.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(loro.js): don't re-insert hidden style anchors when a checkout replays

After a snapshot import and a checkout back past a mark, attach, checkout,
and diff threw `duplicate sequence id` when the forward records could not
move incrementally (a movable-list set, move, or delete): the replay
inserted the mark's anchors again although the state already held them.
`hasMaterializedSequenceInsertions` now counts anchor ops, and `diff`
rebuilds its `from` state in that case too.

The differential harness now edits a movable list, checks out three
versions of every imported full snapshot, diffs, and attaches; seed 19
fails without this fix. It also derived its second checkout roll from
bits that repeated the first. Details: context/loro-js-richtext-anchors.md
and context/loro-js-rust-differential.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(loro.js): text typed where an undone mark started has no style

Undoing a before- or both-expand mark put the restoring mark's start anchor
after the undone one (`mark`'s insert rule), so text typed at the old start
went between the two anchors and kept the undone value; Rust leaves it
unstyled. `_undoStyle` now starts the restoring mark right before the undone
style's start anchor. Against Rust, 159 of 160 expand/prior-style/typing
cases now give the same text (3 differed before); see
context/loro-js-richtext-anchors.md for the remaining one.

The differential harness now requires only the loro.js diff to reach its
target and logs Rust's (seed 162 at 120 steps shows a Rust diff that
disagrees with its checkout).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(loro.js): make the differential checkout's second version independent

The second checkout version came from `imul(roll, 0x9e3779b1)`, whose low
bits repeat the roll's (the multiplier is 1 modulo 16), so it usually picked
the first version again. It now uses the product's high bits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* revert(loro.js): leave the mark-undo boundary fix for a follow-up

The scope of loro-dev#1137 was narrowed to N1 and the docs, so this backs out the
`_undoStyle` change of d1663c9 and its tests; the known gap is recorded in
context/loro-js-richtext-anchors.md. The differential harness keeps the
Rust-diff tolerance from that commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(loro.js): show how a 0.2 snapshot plus later updates reads in 1.0

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant