fix(loro.js): converge plain-text edits with Rust, add a Rust differential suite

.changeset/loro-js-fugue-origin-subtree.md

@@ -0,0 +1,5 @@
+---
+"loro.js": patch
+---
+
+Fix a `loro.js` Text/List convergence bug with concurrent inserts. When an insert was concurrent with a run typed after the same character and with another concurrent element whose origin was further left, `loro.js` put the insert after that element instead of at the end of the run. Replicas could disagree with each other (depending on the order in which they received the updates) and with `loro-crdt`. `loro.js` now places these inserts where Rust does.

.changeset/loro-js-text-interop.md

@@ -0,0 +1,13 @@
+---
+"loro.js": major
+---
+
+Text now follows Rust (`loro-crdt`) everywhere, including when reading documents written by earlier loro.js versions. This breaks documents edited only with loro.js that have concurrent text edits or stored cursors: read "Upgrading from 0.2" in the README before upgrading.
+
+- **Breaking:** concurrent text inserts are ordered like Rust (the fix is shared with loro-dev/loro#1139). Earlier versions could place an insert after a sibling's subtree behind later, unrelated concurrent text, so a history with such inserts can now read differently (for example `béa9` instead of `bé9a`). Earlier loro.js replicas that merged such edits in different orders, and documents shared with `loro-crdt` peers, had already diverged; they now agree with Rust.
+- **Breaking:** text cursors follow Rust: a cursor reports its target's own offset, so a cursor encoded by an earlier version with side 1, or at the end of the text, resolves one character earlier (the end cursor of `abc` at 2 instead of 3). Other cursors resolve as before. The encoded origin is now the Unicode position, as in Rust. A cursor at the end or in empty text has no target, `getCursorPos` returns the target's own offset for every side, and a deleted target reports the length of the text before it with side `-1`. Get new cursors after upgrading.
+- An imported delete is applied by its position, as Rust does, instead of by its recorded `start_id` (Rust's WASM build up to 1.16.3 can record a wrong one around astral characters, loro-dev/loro#1149).
+- `checkout`, `diff`, and `revertTo` no longer restore a character twice when two concurrent deletes removed it.
+- Consecutive inserts in one transaction are merged into one op, and a delete that spans several ID runs writes the last run first, as Rust does.
+
+Upgrading: updates and JSON updates written by an earlier version are read like Rust. A snapshot written by an earlier version keeps its current text, but its history reads like Rust, so a document loaded from it must not stay in use (exported updates, replicas loaded from updates, and checkouts can show other content). Exporting a shallow snapshot from such a document switches the document itself to Rust's reading. Migrate every replica the same way: read the final state with the old version and build a new document from it (reading it with 1.0 right after importing an old snapshot only works when that snapshot is the final version, with no updates stored after it), or import the old updates everywhere and accept Rust's reading. See "Upgrading from 0.2" in the README.

AGENTS.md

@@ -38,6 +38,8 @@ Loro is a Rust CRDT workspace with JS/WASM packaging and a MoonBit codec.
   [skills/loro/SKILL.md](skills/loro/SKILL.md).
 - Pure TypeScript runtime indexes, complexity contracts, benchmarks, and remaining gaps:
   [context/loro-js-performance.md](context/loro-js-performance.md).
+- loro.js vs Rust randomized Text differential tests and the Rust behaviors they
+  work around: [context/loro-js-rust-differential.md](context/loro-js-rust-differential.md).
 - WASM panic/OOM reporting channels, `__wbindgen_start` glue invariant, and
   trap-testing recipes:
   [context/wasm-error-reporting.md](context/wasm-error-reporting.md).

context/CONTEXT-GAPS.md

@@ -5,4 +5,3 @@ fix the docs in that change.
 
 Format:
 `YYYY-MM-DD | <question an agent would ask> | <answer + file anchors> | why it was hard | suggested home`
-2026-09-21 | How do I test that loro.js output imports into Rust/loro-crdt? | `loro-js/scripts/rewrite-rust-fixture.mjs` (`pnpm -C loro-js fixtures:rewrite`) writes `loro-js/tests/fixtures/rust/*.ts.blob`, consumed by `crates/loro/tests/loro_js_interop.rs`; loro.js state encoders live in `loro-js/src/runtime/document.ts` (`#containerState`) and must follow `docs/encoding-container-states.md` (tree sibling order: loro-dev/loro#1088) | root AGENTS only links loro.js performance context; the interop path is mentioned only in `loro-js/README.md` | a `loro-js/AGENTS.md` or a loro.js interop context article

context/loro-js-performance.md

@@ -103,7 +103,32 @@ JavaScript constant factor.
   keep their single-child edge implicit, and `SequenceIndex` can skip an entire
   future ID run while finding the next causally included element. Ordinary local
   edits keep the smaller unindexed path. MovableList continues to use the scan
-  because moves break the origin-tree physical preorder.
+  because moves break the origin-tree physical preorder. Sibling subtrees are
+  contiguous, so the gap between two direct children belongs to the earlier
+  child. After the last child the interval can also hold concurrent elements
+  whose origin is left of `originLeft`; Rust's scan stops before them. The index
+  therefore checks whether the interval's last element descends from
+  `originLeft` and otherwise binary-searches the boundary. The descent test
+  walks origin-left links but jumps over each implicit run through per-peer
+  sorted counters of explicit (non-consecutive) elements, so it costs
+  O(explicit links · log n), like Rust's span-based scan, instead of one probe
+  per scalar in a long concurrent run (loro-dev/loro#1139). A walk over the
+  physical ID runs of the last sibling's subtree finds the same boundary in
+  O((runs + 1) log n) and was measured against it (September 28, Node 22,
+  1-minute load 8–17): with the B4 trace as the sibling subtree the binary
+  search takes 7.3 ms versus 11.1 ms, but on a deep chain of explicit links
+  (two positions typed alternately, 64k elements) 60.6 ms versus 15.5 ms,
+  because each of its O(log n) probes walks the chain. Realistic traces favor
+  the binary search. With a warm origin index, importing one concurrent
+  character after a 512k-character typed run takes 0.28–0.31 ms (0.29–0.30 ms
+  on `main`, which misorders other cases; a per-scalar walk took 9.7–10.7 ms),
+  and `text-concurrent-insert-after-long-run` stays at 0.18–0.30 ms from 64k to
+  512k characters (September 28, Node 22, interleaved, 1-minute load 7–17).
+- An imported Text delete is resolved by its position in the op's causal view,
+  like Rust's tracker (`LoroText._deleteTargets`): O(log n + runs) through
+  `visibleIdRuns` or the cached causal view. The recorded `start_id` is only a
+  fallback, because Rust's WASM build can record one that is off by the UTF-16
+  length of astral text. Local deletes skip the lookup.
 - Merging adjacent changes appends only the new operations and key-table entries
   to the retained record. The cached operation length, peer end, frontier set,
   operation indexes, and subscriber update slice are updated incrementally, so
@@ -228,6 +253,14 @@ JavaScript constant factor.
   that throws restores its previous version and state (`#transitionTo`, which
   also prepares inside its `try`), and `diff` restores the current state with a
   full rebuild when it or its move back throws.
+- Before a transition, `#canTransitionRecords` checks that each sequence still
+  holds the elements that the crossed insert operations name. It collects the
+  runs per container and calls `containsIdRuns` once per container, reading IDs
+  without building element views: O(elements + runs log runs). One call per
+  operation made a checkout O(operations × elements): across 2k scattered
+  inserts in an 8k text it took 698 ms on `main` and takes 4.2 ms now
+  (`text-scattered-edits-checkout`, 13 → 698 ms from 1k to 8k before, 1.0 →
+  4.2 ms now; September 29, Node 22, 1-minute load 6–9).
 - First checkout after importing a 262,144-operation single-peer Text snapshot
   takes about 57 ms (medians of 5 alternating runs on a loaded Apple M5 Pro),
   versus about 148 ms for the earlier whole-document replay; 65,536 operations

context/loro-js-rust-differential.md

@@ -0,0 +1,98 @@
+# loro-js Differential Tests Against Rust
+
+Verified against code 2026-09-28.
+
+`loro-js/tests/richtext-differential.test.ts` runs random multi-peer Text
+scenarios in loro.js and in Rust (`loro-crdt`, nodejs WASM build) side by side
+and compares them after every step. It is the check for "does loro.js mean the
+same thing as Rust" questions about Text positions, deletes, concurrency,
+checkout, snapshots, cursors, and events.
+
+## Running it
+
+1. Build the reference: `pnpm -C crates/loro-wasm build-dev` (or
+   `build-release`). The tests load `crates/loro-wasm/nodejs/index.js`, or the
+   file named by `LORO_WASM_NODEJS`. Only the nodejs target is needed.
+2. `pnpm --dir loro-js test` runs the suite; it is skipped when no build exists.
+   Set `LORO_REQUIRE_WASM_REFERENCE=1` to fail instead of skipping.
+3. Scale with `LORO_DIFF_SEEDS` (default 40), `LORO_DIFF_STEPS` (default 80),
+   and `LORO_DIFF_FIRST_SEED`. 40 × 80 takes a few seconds.
+
+CI runs the loro.js suite with the reference required (`pnpm test-loro-js`, part
+of `test-all` after `release-wasm`), so a Rust change that alters Text behavior
+also shows up here.
+
+## What a scenario does
+
+`tests/support/richtext-differential.ts` generates plain-data actions from a
+seed (`generateActions`), so a failure replays exactly. Each peer exists once
+per runtime with the same peer ID. Actions:
+
+- `insert`/`delete` at UTF-16 or UTF-8 boundaries (BMP, CJK, astral, and
+  multi-scalar emoji), `mark`/`unmark` with before/after/both/none styles;
+- `sync` by binary or JSON updates, within a runtime or crossed (Rust imports
+  the loro.js peer's updates and loro.js imports Rust's), so each side reads
+  the other's positions;
+- `checkout` to two versions in a row, then back to the latest: each is a
+  recorded commit version or the version right after a random op, which can
+  split a change or an op (a mark with only its start anchor, for example), so
+  transitions also run between historical and mid-change versions;
+- full and shallow snapshot export from either runtime, imported into both;
+- cursor creation, then cursor resolution after every later step;
+- `revertTo` and undo/redo.
+
+After each step it compares the delta (normalized: merged runs, `null`
+attributes dropped), UTF-16/Unicode/UTF-8 lengths, cursor positions, and the
+atoms of the JSON history (every inserted scalar and mark with its entity
+position; Rust groups atoms into ops depending on its encoding state). loro.js
+events must rebuild the loro.js state; Rust events are not checked (see below).
+
+`revertTo` must produce the same state on forks of both runtimes, but its ops
+may differ, so the replica runs it in one runtime and the other imports the
+change. Undo/redo run in loro.js and Rust imports the result: loro.js's
+`UndoManager` does not transform against remote edits like Rust's, so exact undo
+parity is only an option (`undoParity`).
+
+## Shrinking a failure
+
+`shrinkActions(rust, actions, options)` removes actions while the scenario
+still fails with the same first-line message (numbers ignored). A throwaway
+Vitest file that calls `generateActions`, `runActions`, and `shrinkActions`
+and prints the result is the quickest loop; paste the shrunk action list into a
+regression test.
+
+## Rust behaviors the harness works around
+
+WASM builds before loro-dev/loro#1135 had bugs of their own. The harness still
+tolerates them, so it also runs against older reference builds:
+
+- A subscribed transaction that merges deletes of astral text could panic with
+  "Op/hint length mismatch" (`change_to_diff` in
+  `crates/loro-internal/src/txn.rs`). The harness subscribes only to loro.js
+  documents.
+- `get_text_entity_ranges` advanced the recorded delete `start_id` by the UTF-16
+  length of astral text, so Rust could write a delete whose `start_id` names the
+  wrong elements; such deletes stay in existing histories (loro-dev/loro#1149).
+  Rust applies deletes by position, and so does loro.js
+  (`LoroText._deleteTargets`). Rust resolves a deleted cursor target through the
+  recorded IDs, so the harness skips cursors Rust cannot resolve.
+- Rust's shallow import labels placeholders with the recorded delete IDs, which
+  can diverge from its own full-history import when they are wrong. The harness
+  compares loro.js with the source document instead.
+
+Snapshot-import checkout (`snapshotCheckout`) is off by default until loro.js
+can check out a document imported from a snapshot (loro-dev/loro#1126).
+
+## Data written by loro.js 0.2
+
+loro.js 0.2 ordered some concurrent text inserts differently from Rust (a
+concurrent insert after a sibling's subtree, fixed by loro-dev/loro#1139) and resolved
+side-1 and end-of-text cursors one character after their target. Decision (2026-09-28): later versions read all
+data with Rust's semantics and add no version marker. The same bytes must mean
+the same thing in both runtimes, documents shared with `loro-crdt` peers had
+already diverged, and a marker would need a format change that Rust does not
+have. The cost falls on documents edited only with 0.2: their history can read
+differently, and a replica loaded from a 0.2 snapshot keeps the 0.2 text while
+one loaded from updates follows Rust. `README.md` ("Upgrading from 0.2")
+describes the migration paths; `tests/legacy-data.test.ts` pins the readings
+with fixtures written by `scripts/write-legacy-fixtures.mjs`.

loro-js/AGENTS.md

@@ -0,0 +1,35 @@
+# loro.js
+
+Pure TypeScript implementation of Loro (`loro-js/src`). It must read and write
+the same binary and JSON data as Rust (`loro-crdt`) and give the same results.
+
+## Context
+
+- Index structures, complexity contracts, and benchmarks:
+  [context/loro-js-performance.md](../context/loro-js-performance.md).
+- Randomized Text comparison with Rust (build the nodejs WASM package first):
+  [context/loro-js-rust-differential.md](../context/loro-js-rust-differential.md).
+- Encoding formats: [context/internal-encoding.md](../context/internal-encoding.md)
+  and `docs/encoding*.md`.
+
+## Interop fixtures
+
+- Rust output read by loro.js: `tests/fixtures/rust/*.blob` and `*.json`.
+- loro.js output read by Rust: `pnpm fixtures:rewrite` runs
+  `scripts/rewrite-rust-fixture.mjs`, which writes `tests/fixtures/rust/*.ts.blob`;
+  `crates/loro/tests/loro_js_interop.rs` imports them. Container state encoders
+  live in `src/runtime/document.ts` (`#containerState`) and must follow
+  `docs/encoding-container-states.md`.
+- loro.js 0.2 output read by later versions: `scripts/write-legacy-fixtures.mjs`
+  writes `tests/fixtures/loro-js-0.2/` with a 0.2 build; `tests/legacy-data.test.ts`
+  pins that loro.js and Rust read it the same way. Later versions read 0.2 data
+  with Rust's semantics, with no version marker (README "Upgrading from 0.2"; the
+  decision is in [context/loro-js-rust-differential.md](../context/loro-js-rust-differential.md)).
+
+## Commands
+
+From `loro-js/`: `pnpm test`, `pnpm typecheck`, `pnpm lint`, `pnpm format:check`,
+`pnpm build`, or `pnpm check` for all of them. From the root, `pnpm test-loro-js`
+runs the tests and fails if the Rust WASM reference is missing (CI runs it after
+`release-wasm`). Benchmarks: `pnpm bench:b4`,
+`pnpm bench:complexity -- 1000,2000,4000,8000`.

loro-js/CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md

loro-js/README.md

@@ -94,6 +94,9 @@ The following paths are checked against the Rust implementation:
 - JSON update import/export and the postcard Awareness/EphemeralStore protocols.
 - Rust-produced fixtures imported by TypeScript and TypeScript-produced fixtures
   imported by Rust.
+- Randomized multi-peer text edits run side by side with the `loro-crdt` WASM build
+  (`tests/richtext-differential.test.ts`): positions, deletes, concurrency, checkout,
+  snapshots, cursors, events, and `revertTo`.
 
 This is not yet a claim of complete behavioral equivalence with every `loro-crdt`
 edge case. Important current limits are:
@@ -118,6 +121,55 @@ Treat the package as experimental when data can be produced by untrusted or olde
 clients. Keep a `loro-crdt` interoperability test for the formats and operations your
 application depends on.
 
+## Upgrading from 0.2
+
+1.0 reads every document the way Rust (`loro-crdt`) does, including documents
+written by 0.2. The data has no version marker, so a document edited only with 0.2
+can read differently after the upgrade:
+
+- **Concurrent text inserts.** 0.2 could order an insert after a sibling's subtree
+  differently from Rust. 1.0 orders it like Rust, so such a history can give other
+  text (`bé9a` in 0.2, `béa9` in 1.0 and Rust).
+- **Cursors.** 1.0 resolves cursors like Rust: a cursor reports its target's own
+  offset. A cursor encoded by 0.2 with side 1, or at the end of the text, therefore
+  resolves one character earlier, whatever the characters are (on `abc`, the end
+  cursor is at 3 in 0.2 and 2 in 1.0; a side-1 cursor on `b` at 2 and 1). Other
+  cursors resolve as before. Get new cursors from the upgraded document.
+
+Documents also edited by `loro-crdt` peers already disagreed with Rust in these cases;
+1.0 makes them agree.
+
+How 1.0 reads data written by 0.2 (measured on the example above):
+
+| 0.2 data              | 1.0 reads                                                               |
+| --------------------- | ----------------------------------------------------------------------- |
+| Updates, JSON updates | Rust's reading: `béa9`                                                  |
+| Full/shallow snapshot | The 0.2 text (`bé9a`) as current state, but the history reads like Rust |
+
+A document loaded from a 0.2 snapshot therefore holds a current state that its own
+history disagrees with. Its current state is safe to read right after the import,
+but do not keep using the document: exporting it as updates gives `béa9`, a
+replica loaded from updates holds other text and stays different after new edits
+(`béQ9aZ` and `béQa9Z`), older versions it checks out can be approximate, and a
+fork can rebuild it from its history, which switches it to Rust's reading.
+Exporting a shallow snapshot from it (`export({ mode: "shallow-snapshot" })`) does
+the same to the document itself: its current state becomes Rust's reading (`bé9a`
+turns into `béa9`). Rust can even show text that matches neither reading after a
+checkout there and back.
+
+**Migrate every replica the same way.** Recommended paths:
+
+1. **Keep the content, drop the history (safest).** Read the final state with 0.2,
+   after checking out the latest version: `doc.toJSON()`, or `text.toDelta()` for
+   rich text. Build one new document from it in 1.0 and share that document with
+   every replica. Do not import old updates or snapshots into it. Reading the state
+   with 1.0 right after importing a 0.2 snapshot gives the same result only when
+   that snapshot is the final version: if updates were stored after it, 1.0 (like
+   Rust) reads those updates the Rust way.
+2. **Keep the history, accept Rust's reading.** Import the 0.2 updates (not
+   snapshots) on every replica, and check the content, because it can differ from
+   what 0.2 showed.
+
 ## Development checks
 
 From this directory: