Skip to content

fix(loro.js): reject invalid MovableList move/set refs (port Rust #1125) - #1168

Merged
zxch3n merged 40 commits into
mainfrom
fix/loro-js-movable-list-invalid-ops
Sep 29, 2026
Merged

zxch3n merged 40 commits into
mainfrom
fix/loro-js-movable-list-invalid-ops

Conversation

@zxch3n

@zxch3n zxch3n commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Summary

Last PR of the MovableList stack, based on #1167 (review only the commits after it). It ports the merged Rust #1125 to loro.js: a MovableList move or set that cannot mean anything is rejected, and the whole import or batch rolls back. A move or set of a deleted element stays valid and keeps its concurrent CRDT meaning.

#validateMovableListRefs in document.ts mirrors OpLog::validate_movable_list_elem_refs_since. It runs on every imported change after integration: FastUpdates, non-initializing snapshots and importBatch. Like Rust, it skips the snapshot that seeds an empty document. It rejects a Move/Set whose element:

  • is unknown, or is not a movable-list-insert into the same list;
  • is outside the op's causal history;
  • on a shallow document, is older than the root but missing from the root state (shallow_root_has_movable_list_elem), that is, it was deleted before the root.

MovableListState.applyMove already rejects a from/to outside the op-index range of the op's causal view before it mutates anything. With #1167, that rejection also rolls the import back.

Complexity:

  • Each check maps (peer, lamport) to an op ID by binary search over the peer's indexed changes: O(log changes).
  • While a snapshot's history is still deferred, only the overlay is indexed. A miss there means the element comes from the snapshot base. It is accepted without decoding history when the list's hydrated state holds it and the op has seen the whole base; a regression test uses an undecodable snapshot history to prove it.
  • The shallow-root element set is built once per root store and list.

Validation

  • pnpm -C loro-js check: passes.
  • loro-js/tests/movable-list-invalid-ops.test.ts ports crates/loro/tests/movable_list_invalid_ops.rs (12 tests):
    • the deleted-element move/set, identical on every import path;
    • unknown, cross-container and out-of-history elements, and out-of-range indices, each on a plain doc and on one with a concurrent edit;
    • a detached import, and importBatch rollback;
    • shallow-root elements, and a snapshot-loaded doc;
    • a move that must not decode snapshot history.
  • The same file runs against a WASM build of current main (with fix: reject malformed movable list move/set ops on import instead of aborting #1125) through a module alias. 11 of the 12 tests pass unchanged apart from the error pattern; the twelfth needs loro.js's codec. The two agree on values, which imports are rejected, the rollback, and the document staying usable. Error texts differ: Rust says "Decode error" / "movable list diff retains N items but state only has M", loro.js says "causal history" / "out of range".
  • The differential suites at 588aa35d (later commits only change docs), with a Rust reference built from current main:

Known limitations

Rust rules that loro.js does not copy; both are edge cases, listed in context/loro-js-movable-list.md:

  • Rust rejects any List/MovableList/Text op position ≥ 1,073,741,822 when it decodes ops, on every path. loro.js rejects such a move only when it applies it, so a detached import records it until attach fails.
  • Rust bounds-checks the composed diff of an import, so an out-of-bounds move that a later op in the same import cancels is accepted. loro.js checks each op and rejects the import.

Merge order

  1. test(loro.js): add a Rust differential suite for MovableList #1132, the differential suite.
  2. fix(loro.js): model MovableList like Rust (positions and LWW elements) #1166, the MovableList model.
  3. fix(loro.js): make import and importBatch atomic #1167, atomic import/importBatch (this PR's base).
  4. This PR.

🤖 Generated with Claude Code

zxch3n and others added 30 commits September 28, 2026 02:07
Drive the Rust WASM build and loro.js with the same random multi-peer
edits, crossing engines on every sync, and require equal values, versions,
per-element metadata and events. Profiles that diverge on the current
MovableList model are recorded as known divergences. Design and findings:
context/loro-js-movable-list.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Event deltas that differ only in shape are checked by effect; reverts whose
op order Rust leaves to hash order are checked by value and replicated from
Rust; peers that both engines reject are retired; peer IDs are never reused;
undo stays unchained; the minimizer keeps those rules. See
context/loro-js-movable-list.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…undo unchained

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…se diff shape differs

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…allow root

Both follow Rust issues listed in context/loro-js-movable-list.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ust reference

Load Rust through tests/support/rust-reference.ts like the other differential
suites, so it runs in `pnpm test` (and in CI's test-loro-js) and skips without
a WASM build. Drop the separate workflow and scripts. insert/delete now
converges on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MovableList keeps a Fugue sequence of list items and elements whose
position and value are last-writer-wins registers, as Rust does. Moves of
concurrently deleted elements, deletes after moves, snapshots (list item,
element and last-set IDs, dead items), metadata getters, version
transitions, revertTo/applyDiff (Rust's apply_delta), undo and events now
match Rust. Snapshot-hydrated documents replay history before concurrent
imports, and a few shallow-snapshot incompatibilities are fixed. The
differential suite passes every profile. Design and remaining gaps:
context/loro-js-movable-list.md.

Rebuilt on current main: MovableList joins #1126's per-container snapshot
completion (its snapshot state names Rust IDs), applyDiff/revertTo/undo use
main's port of Rust #1138's apply_delta on the new model, and undo
transforms only over changes the UndoManager did not make.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ules

Decoding deferred snapshot history dropped pending changes. Shallow imports
now check the unknown part of each change like is_before_shallow_root, and an
import that only adds pending changes creates their root containers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main's diff() reports such a child by its ops, including map keys set and
deleted again, as Rust does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…acker

A causal view per concurrent op made importing concurrent moves O(n^2).
Like Rust's Tracker::checkout, keep a tracker version and a per-position
delta metric that moves incrementally between op versions; see
context/loro-js-movable-list.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A failed import no longer leaves history, pending changes, version,
containers, state or events half applied. In-place history changes are
journaled and undone, state is rebuilt only when it had changed, events are
emitted after success.
Details: context/import-batch-atomicity.md ("loro.js").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…heir history

Port of #1125; see context/loro-js-movable-list.md (Validation).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lues

A MovableList replaced by a plain value made the comparison enumerate the
container and hit its guarded element sequence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Rust-model MovableList passes seeds 0-599 of the #1141 fuzz with all
oracles in lockstep with Rust; main failed 144 of seeds 0-299.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y_delta

Without moved children the delta was walked in order, which gave the ops
other IDs than Rust's and later deletes by ID missed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…diff()

A child created in the range whose keys were all set and deleted again is
listed by Rust and omitted by loro.js, also on main and for any parent; see
context/loro-js-movable-list.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s-movable-list-model

# Conflicts:
#	context/loro-js-movable-list.md
@github-actions

Copy link
Copy Markdown
Contributor

WASM Size Report

  • Original size: 3277.59 KB
  • Gzipped size: 1089.33 KB
  • Brotli size: 762.83 KB

zxch3n and others added 10 commits September 30, 2026 02:47
The performance context still described main's MovableList snapshot special
case and #1163 as open. Measurements are in context/loro-js-performance.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
canTransition walked the position tree once per insert op, which made
checkout, diff and revertTo O(insert ops * positions). Collect the runs and
check them in one containsIdRuns call, as the List branch does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The changeset named a test:differential script that does not exist. Rust's
diff() can list a sequence with an empty delta where loro.js omits it
(context/loro-js-movable-list.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s-movable-list-model

# Conflicts:
#	context/loro-js-movable-list.md
…r import

Each import copied the whole snapshot-sequence table for rollback, so its cost
grew with the number of snapshot-hydrated containers. Record each write in the
import journal instead (context/import-batch-atomicity.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zxch3n
zxch3n changed the base branch from fix/loro-js-atomic-import to main September 29, 2026 19:44
@zxch3n
zxch3n merged commit 6e3e3c0 into main Sep 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant