Skip to content

fix(loro.js): make import and importBatch atomic - #1167

Merged
zxch3n merged 33 commits into
mainfrom
fix/loro-js-atomic-import
Sep 29, 2026
Merged

zxch3n merged 33 commits into
mainfrom
fix/loro-js-atomic-import

Conversation

@zxch3n

@zxch3n zxch3n commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Summary

Third PR of the MovableList stack, based on #1166 (the MovableList model PR; review only the commits after it). It makes loro.js import, importBatch and importJsonUpdates all-or-nothing, matching the Rust contract in context/import-batch-atomicity.md. A failed import leaves no trace in history, pending changes, version, frontiers, containers, state, events, subscriptions or UndoManager, and the document stays usable.

LoroDoc.#atomically in document.ts:

  • While an import runs, every in-place change to history structures pushes an undo closure onto #importUndo: records, per-peer arrays and ends, merged records, Map/Tree op indexes, created containers and roots, and snapshot-sequence entries.
  • Fields the import replaces wholesale are saved by reference or copied: frontiers, pending changes, deferred snapshot history/state, shallow root and its entry index, and counters.
  • On failure the closures run in reverse and the saved fields come back, and the per-container record index is invalidated.
  • The state is rebuilt from the restored history only if the import had already changed container state, so the rebuild happens only on the failure path.
  • A snapshot container that the import first completed from its own history (fix(loro.js): model MovableList like Rust (positions and LWW elements) #1166) keeps that replay, which equals its snapshot state, and gets its snapshot entry back.
  • A pristine document skips the journal; a failure empties it again.
  • Events are emitted only after the whole import succeeded, and a multi-blob importBatch succeeds or fails as a whole.

A checkout/attach that fails already restores the previous version on main (#transitionTo, from #1126). This PR only adds a test for it.

Rationale

Before this, an import that failed midway could:

  • leave history integrated but state half-applied;
  • emit partial events;
  • push a partial undo item.

The next PR rejects invalid MovableList moves/sets the way the merged Rust #1125 does, and a rejection is only useful if nothing of the import remains. Rust gives the same guarantee through import_batch rollback. The "loro.js" section of context/import-batch-atomicity.md now describes the TS side.

Validation

Known limitations

  • Rust applies an update whose changes partly depend on history before a shallow root partially, then returns ImportUpdatesThatDependsOnOutdatedVersion. loro.js rejects the whole blob.

Merge order

  1. test(loro.js): add a Rust differential suite for MovableList #1132, the differential suite.
  2. fix(loro.js): model MovableList like Rust (positions and LWW elements) #1166, the MovableList model (this PR's base).
  3. This PR.
  4. fix(loro.js): reject invalid MovableList move/set refs (port Rust #1125) #1168, MovableList move/set validation (stacked on this one).

🤖 Generated with Claude Code

zxch3n and others added 25 commits September 28, 2026 02:07
Drive the Rust WASM build and loro.js with the same random multi-peer
edits, crossing engines on every sync, and require equal values, versions,
per-element metadata and events. Profiles that diverge on the current
MovableList model are recorded as known divergences. Design and findings:
context/loro-js-movable-list.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Event deltas that differ only in shape are checked by effect; reverts whose
op order Rust leaves to hash order are checked by value and replicated from
Rust; peers that both engines reject are retired; peer IDs are never reused;
undo stays unchained; the minimizer keeps those rules. See
context/loro-js-movable-list.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…undo unchained

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…se diff shape differs

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…allow root

Both follow Rust issues listed in context/loro-js-movable-list.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ust reference

Load Rust through tests/support/rust-reference.ts like the other differential
suites, so it runs in `pnpm test` (and in CI's test-loro-js) and skips without
a WASM build. Drop the separate workflow and scripts. insert/delete now
converges on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MovableList keeps a Fugue sequence of list items and elements whose
position and value are last-writer-wins registers, as Rust does. Moves of
concurrently deleted elements, deletes after moves, snapshots (list item,
element and last-set IDs, dead items), metadata getters, version
transitions, revertTo/applyDiff (Rust's apply_delta), undo and events now
match Rust. Snapshot-hydrated documents replay history before concurrent
imports, and a few shallow-snapshot incompatibilities are fixed. The
differential suite passes every profile. Design and remaining gaps:
context/loro-js-movable-list.md.

Rebuilt on current main: MovableList joins #1126's per-container snapshot
completion (its snapshot state names Rust IDs), applyDiff/revertTo/undo use
main's port of Rust #1138's apply_delta on the new model, and undo
transforms only over changes the UndoManager did not make.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ules

Decoding deferred snapshot history dropped pending changes. Shallow imports
now check the unknown part of each change like is_before_shallow_root, and an
import that only adds pending changes creates their root containers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main's diff() reports such a child by its ops, including map keys set and
deleted again, as Rust does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…acker

A causal view per concurrent op made importing concurrent moves O(n^2).
Like Rust's Tracker::checkout, keep a tracker version and a per-position
delta metric that moves incrementally between op versions; see
context/loro-js-movable-list.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A failed import no longer leaves history, pending changes, version,
containers, state or events half applied. In-place history changes are
journaled and undone, state is rebuilt only when it had changed, events are
emitted after success.
Details: context/import-batch-atomicity.md ("loro.js").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lues

A MovableList replaced by a plain value made the comparison enumerate the
container and hit its guarded element sequence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Rust-model MovableList passes seeds 0-599 of the #1141 fuzz with all
oracles in lockstep with Rust; main failed 144 of seeds 0-299.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y_delta

Without moved children the delta was walked in order, which gave the ops
other IDs than Rust's and later deletes by ID missed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…diff()

A child created in the range whose keys were all set and deleted again is
listed by Rust and omitted by loro.js, also on main and for any parent; see
context/loro-js-movable-list.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s-movable-list-model

# Conflicts:
#	context/loro-js-movable-list.md
@github-actions

Copy link
Copy Markdown
Contributor

WASM Size Report

  • Original size: 3277.59 KB
  • Gzipped size: 1089.33 KB
  • Brotli size: 762.83 KB

The performance context still described main's MovableList snapshot special
case and #1163 as open. Measurements are in context/loro-js-performance.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zxch3n and others added 7 commits September 30, 2026 02:47
canTransition walked the position tree once per insert op, which made
checkout, diff and revertTo O(insert ops * positions). Collect the runs and
check them in one containsIdRuns call, as the List branch does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The changeset named a test:differential script that does not exist. Rust's
diff() can list a sequence with an empty delta where loro.js omits it
(context/loro-js-movable-list.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s-movable-list-model

# Conflicts:
#	context/loro-js-movable-list.md
…r import

Each import copied the whole snapshot-sequence table for rollback, so its cost
grew with the number of snapshot-hydrated containers. Record each write in the
import journal instead (context/import-batch-atomicity.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zxch3n
zxch3n changed the base branch from fix/loro-js-movable-list-model to main September 29, 2026 19:43
@zxch3n
zxch3n merged commit 124b70b into main Sep 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant