Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/apply-diff-atomic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"loro-crdt": patch
---

`applyDiff` and `revertTo` are now all or nothing: when they throw (for example on an out-of-range position, a diff made for another version of the document, or a diff type that doesn't match its container), the document, its history and its subscribers are left as they were. Uncommitted edits made before the call are kept, still uncommitted. Applying a diff that names a tree parent the document doesn't have now throws instead of trapping.
5 changes: 5 additions & 0 deletions .changeset/empty-root-replay.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"loro-crdt": patch
---

A root container touched by a document's history is now part of the value after importing that history, even when its ops add up to an empty value (for example text inserted and then deleted). Previously the writing peer showed `{"t": ""}` while a replay of the same history showed `{}`.
5 changes: 5 additions & 0 deletions .changeset/export-updates-in-range-spans.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"loro-crdt": patch
---

`export({ mode: "updates-in-range" })` no longer panics (a `RuntimeError: unreachable` trap in WASM) when the spans hold several non-contiguous or overlapping ranges of one peer; all the ranges are exported.
5 changes: 5 additions & 0 deletions .changeset/failed-import-state-rollback.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"loro-crdt": patch
---

After an import failed and was rolled back, containers created later could show state left over from containers the failed import had touched (for example a new root map showing a tree node's metadata), read as deleted, or trip an internal assertion when the same bad update was imported again. A failed import now leaves the document's containers unchanged.
5 changes: 5 additions & 0 deletions .changeset/import-reused-peer-id.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"loro-crdt": patch
---

Importing updates or a snapshot that reuse op ids the document already has for different content (two clients that shared a peer id) now throws a "ID ... has been used" error and leaves the document unchanged and usable. Previously the import trapped the WASM instance (`RuntimeError: unreachable`) or silently scrambled text. Imports with a change that skips counters of its peer also throw instead of trapping.
5 changes: 5 additions & 0 deletions .changeset/subscription-drop-deadlock.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"loro-crdt": patch
---

Fix a deadlock when a subscription is dropped and its callback owned another subscription or an `UndoManager` on the same document: subscriber callbacks are now dropped after the subscriber set's lock is released.
5 changes: 5 additions & 0 deletions .changeset/tree-revival-create-event.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"loro-crdt": patch
---

Fix missing tree events on import. When an imported move brought a node back out of a deleted subtree, only the node got a `create` event, not the nodes below it, so a tree rebuilt from events lost them. Importing a move of a node under a deleted node also emitted no `delete` event in some cases. This happened when the import had no concurrency with the local document, for example on a peer that only receives updates.
5 changes: 5 additions & 0 deletions .changeset/unclosed-style-mark-snapshot.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"loro-crdt": patch
---

Fix a panic (a `RuntimeError: unreachable` trap in WASM) when reading a document forked or snapshotted at a version that holds a mark's start but not its end ("unclosed style mark"). Such a start anchor is now decoded like `checkout` keeps it: it styles nothing until its end arrives. A shallow snapshot of a document whose history ends at such a start no longer fails with `FrontiersNotFound`.
5 changes: 5 additions & 0 deletions .changeset/unknown-child-map-event.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"loro-crdt": patch
---

A map event that sets a key to a container of a type unknown to this version (created by a newer `loro-crdt`) is now delivered with its other keys; only the unknown child's entry is left out. Previously the whole map event was dropped, so edits to known keys in the same event never reached subscribers.
13 changes: 13 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,14 @@ Loro is a Rust CRDT workspace with JS/WASM packaging and a MoonBit codec.
[context/mergeable-containers.md](context/mergeable-containers.md).
- `import_batch` force-detach, batch-wide rollback scope, and the never-exit-detached
invariant: [context/import-batch-atomicity.md](context/import-batch-atomicity.md).
- Which root containers are visible (history-touched vs handle-acquired), and
why replays materialize empty roots:
[context/root-container-visibility.md](context/root-container-visibility.md).
- Imports that reuse local op ids (a shared peer id): the known-history check,
what counts as equal, and what it cannot catch:
[context/import-peer-id-reuse.md](context/import-peer-id-reuse.md).
- `apply_diff`/`revert_to` rollback of a failed batch, reused op ids, other
threads: [context/apply-diff-atomicity.md](context/apply-diff-atomicity.md).
- Movable-list `Move`/`Set` import validation, and why a move of a deleted element
is applied rather than rejected:
[context/movable-list-op-validation.md](context/movable-list-op-validation.md).
Expand All @@ -34,6 +42,8 @@ Loro is a Rust CRDT workspace with JS/WASM packaging and a MoonBit codec.
and delete `start_id`s: [context/richtext-insert-positions.md](context/richtext-insert-positions.md).
- Tree diff cache transitions and why their lamport window comes from the two
versions, not the replay base: [context/tree-checkout-window.md](context/tree-checkout-window.md).
- Tree event invariants (revived subtrees get `Create`s) and the two event
paths, diff-calculator vs raw ops: [context/tree-events.md](context/tree-events.md).
- loro.js tree diff item order, index counting, apply resolution, and batch dry-run:
[context/loro-js-tree-diff.md](context/loro-js-tree-diff.md).
- User-facing Loro usage, sync, editor integration, and performance guidance:
Expand All @@ -60,6 +70,9 @@ Loro is a Rust CRDT workspace with JS/WASM packaging and a MoonBit codec.
- How the arena learns a container's parent (parsed changes, state KV, the op
log's creator resolver), its locking (including `ChangeStore`'s lock order),
and import rollback: [context/arena-parent-links.md](context/arena-parent-links.md).
- Why a failed import's rollback keeps arena indices (DocState, caches and handlers
are keyed by them) and what it drops instead:
[context/failed-import-arena-indices.md](context/failed-import-arena-indices.md).
- Which deletions `DocState::is_deleted` caches, and how tree/movable-list
children are revived: [context/dead-container-cache.md](context/dead-container-cache.md).
- Movable-list `apply_diff`/undo: delete + re-insert of a child becomes a move,
Expand Down
1 change: 1 addition & 0 deletions context/CONTEXT-GAPS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ fix the docs in that change.

Format:
`YYYY-MM-DD | <question an agent would ask> | <answer + file anchors> | why it was hard | suggested home`
2026-09-30 | Does `fuzz::actor::assert_value_eq` / `Actor::check_tracker` fail on a mismatch? | No: since #953 it only `tracing::warn!`s (`crates/fuzz/src/actor.rs`), so tracker and value checks in the fuzz crate (including `crates/fuzz/tests/checkout_path.rs`, which `context/tree-checkout-window.md` describes as comparing values) fail only when the tracker panics | a randomized repro of loro-dev/loro#1157 passed on the old code | `context/tree-checkout-window.md` Tests section, or make it assert
119 changes: 119 additions & 0 deletions context/apply-diff-atomicity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# `apply_diff` / `revert_to` Atomicity

Verified against code 2026-09-30 (loro-dev/loro#1154).

`LoroDoc::apply_diff` and `revert_to` (and loro-crdt `applyDiff` / `revertTo`)
are all or nothing. When they return `Err`, the doc's state, history, version
and subscribers are as before the call. Undo/redo (`undo_internal_with`) still
calls `LoroDoc::_apply_diff` directly and keeps what applied.

## Why a rollback, not a pre-check

`_apply_diff` (`crates/loro-internal/src/loro.rs`) turns each entry of a batch
into local ops. Entries fail for many reasons: positions out of range
(`OutOfBound`), a diff made for another version of the doc, a diff type that
doesn't match the container, a missing container, a tree parent the doc
doesn't have. Checking the whole batch first would mean predicting what every
handler's `apply_diff` does while the loop recreates containers and remaps ids
(the unknown-container pre-check in
[unknown-containers.md](unknown-containers.md) already needs that for one
error). Local ops can instead be undone, because nothing outside the doc has
seen them until the transaction commits.

A stale tree diff used to panic on a parent the doc doesn't have
(`TreeHandler::create_at_with_target_for_apply_diff` unwrapped
`is_node_deleted`); it returns that error now, and the batch is rolled back.

## How it works

`LoroDoc::apply_diff_all_or_nothing` runs the batch in the current transaction,
after any uncommitted edits, which must survive a rollback. Committing them first would have been simpler, but it
turns N `apply_diff` calls before one `commit` into N commits (N events, N undo
steps).

1. `begin_apply_diff_scope` marks the transaction
(`Transaction::begin_rollback_scope`: the next counter and lamport, the
first-appearance flag) and starts `DocState::begin_local_rollback`.
2. On success the marks are dropped. The ops stay in the transaction as before.
3. On failure, `rollback_apply_diff_scope`:
- `Transaction::roll_back_scope` takes the transaction's ops out and keeps
the ones before the batch (slicing an op the batch's first op was merged
into), restores the event hints the batch touched (their count and last
hint, recorded on first touch, since the batch's first hint may have been
merged into it) and the counters;
- resets the DAG's local version to before the transaction
(`AppDag::discard_pending_txn`, with the DAG frontiers the transaction
saved when it began), opens an import rollback scope, and inserts all the
transaction's ops as a (non-local) change;
- checks the state out from the batch's last op back to the op before the
batch (the middle of that change) with a fresh `DiffCalculator::new(false)`
(the doc's persistent one never sees the change), with the event recorder
taken out so nothing is recorded;
- rolls the op log back with `OpLog::rollback_import_keeping_arena` (change
store, DAG, pending changes, history cache) and applies the kept ops to the
DAG's version again (`update_version_on_new_local_op`);
- finishes with `DocState::finish_local_rollback`.

The transaction goes on: its options, `on_commit` and kept ops are untouched,
and the next commit reports only the kept ops.

The checkout costs a multiple of the batch (measured 2x for a text batch, 6x
for a list and movable-list batch); freeing the history cache makes the next
checkout rebuild it. Only the failure path pays either. A successful call pays
two more transaction and state locks (about 60 ns) and a hash lookup per op.

## Reused op ids

The next local ops reuse the counters and lamports of the discarded ops, so
anything keyed by them must not survive:

- Container states: the checkout restores every container the ops touched.
`InnerStore` records the containers that got new (not loaded) state during
the scope (`created_journal`), and `finish_local_rollback` removes them, so
the store is as before, including which roots `get_deep_value` shows. A
container created by the batch gets fresh state when an op creates it again.
- Counters: the checkout subtracts the increments, which may not give back the
same float, so their old values are recorded on the first op and set back.
- Parent links: `SharedArena::forget_parents_of_discarded_ops` drops the links
of normal containers whose id is at or after the batch's first op. Without a
link such a container reads as deleted without being cached as deleted
(`dead-container-cache.md`), and the next op that creates it links it again.
The registrations stay: other code may hold their indices.
- The arena is not rolled back. The checkout may register old containers and
create state for them; freeing those indices would leave state entries behind
(loro-dev/loro#1164). The discarded ops' values stay allocated, so the next
op's values are not adjacent to the previous op's and the two are stored as
separate ops instead of one merged op.
- `dead_containers_cache` and `alive_containers_cache` are cleared.

## Other threads

The handlers lock the transaction per op, so another thread can edit or commit
while a batch runs. Rolling back would then drop that thread's edits, so the
batch is left partly applied (the behavior before #1154) and a warning is
logged:

- another thread's op in the scoped transaction sets `foreign_ops` (checked in
`Transaction::apply_local_op` by a thread-local address);
- a commit, import, checkout or export on another thread commits the scoped
transaction, so the transaction found at failure is not the scoped one;
- a batch that starts while another thread's batch runs in the same
transaction gets no scope.

Scope ids also key the `DocState` journal, so a stale scope cannot end a newer
one's journal.

## Tests

- `crates/loro/tests/apply_diff_atomicity.rs`: the #1154 repro, type mismatch,
missing container, detached editing, shallow docs, events, `UndoManager`,
pending edits (still uncommitted, their commit message, ops and event hints
the batch merged into, the single event of the next commit), counters,
containers and tree nodes created by the failed batch and created again,
twins that make the same edits without the failed call (random stale
`doc.diff(a, b)` batches, including mergeable children, and a fixed case for
existing containers), `revert_to`, and another thread editing during failing
batches.
`LORO_APPLY_DIFF_ATOMICITY_SEEDS` runs more than the default 120 seeds.
- `crates/loro-wasm/tests/apply_diff_atomicity.test.ts`: `applyDiff` throws and
leaves the doc, events and undo stack unchanged.
40 changes: 23 additions & 17 deletions context/arena-parent-links.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Arena Parent Links

Verified against code 2026-09-29.
Verified against code 2026-09-30.

`SharedArena` (`crates/loro-internal/src/arena.rs`) stores each container's
parent. Liveness (`DocState::is_deleted`), paths (`DocState::get_path`,
Expand Down Expand Up @@ -130,9 +130,13 @@ that writes the KV store, or a live container would read as deleted. Measured

A failed import can parse old change blocks while it computes its diff
(including through the creator resolver) or validates movable-list ops
(`OpLog::resolve_movable_list_elem`), which registers containers and allocates
their values. The arena rollback (`SharedArena::rollback`) drops registrations
and values made during the import, so `ChangeStore::rollback_import`, after
(`OpLog::resolve_movable_list_elem`), which registers containers with their
parent links and allocates their values. The arena rollback
(`SharedArena::rollback`) keeps every container index, but drops the parent
links of normal containers registered during the import and truncates the
values and text allocated during it (see
[failed-import-arena-indices.md](failed-import-arena-indices.md) for why indices
are kept). So `ChangeStore::rollback_import`, after
truncating and evicting blocks (see
[movable-list-op-validation.md](movable-list-op-validation.md)), drops the
parsed changes of the kept blocks that were parsed since the import began; the
Expand All @@ -142,7 +146,7 @@ block. A block parsed before the checkpoint can only refer to what was there
then, so it keeps its parsed changes (`SharedArenaRollback::keeps`). Dropping
them all made every failed import an O(blocks) pass and the next history read a
full reparse. Blocks without bytes were built from changes inserted before the
import, so their containers were registered then. Before 2026-09-28 the parsed
import, so their containers were registered and linked then. Before 2026-09-28 the parsed
ops kept indices and value slices past the truncated arena: exporting the
history hit `unreachable!` in the JSON encoder, and a later checkout panicked on
a missing value (loro-dev/loro#1161).
Expand All @@ -164,18 +168,18 @@ and runs under the state lock:
`Absent`, so a resolver that reached it before the swap does not register the
discarded history.
- Under the state lock: `OpLog::rollback_import`, `rollback_arena`, and
`reset_to_empty_for_failed_snapshot_import` take `&DocState` as proof. A
state-locked query (`has_container`, `is_deleted`, `get_path_to_container`)
can register containers through the resolver and then use their indices
before it returns; the rollback must not free them in between. Without the
state lock, a loom model hit `get_depth` on a freed index.

Only the state-locked queries are covered. A handler created on another thread
during the import keeps its index, and `DocState` is not rolled back: its store
and dead-container cache keep entries at the freed indices, and the next
container registered at one inherits them. `get_path_to_container` answers
`None` if its registration is already gone. See loro-dev/loro#1164
(pre-existing).
`reset_to_empty_for_failed_snapshot_import` take `&mut DocState`, as proof
and to clear the dead-container cache. A state-locked query (`has_container`,
`is_deleted`, `get_path_to_container`) can register containers through the
resolver and then walk their links before it returns; the rollback must not
drop them in between. Without the state lock, a loom model hit `get_depth` on
an index the rollback freed (it freed indices before loro-dev/loro#1164).

A handler created on another thread during the import keeps its index and its
container, and so do `DocState`'s store entries and caches. If the rollback
dropped the link of a container that a thread is using without the state lock,
the next lookup resolves it again; `get_path_to_container` answers `None` if
the link is gone at that moment.

## Testing pitfall

Expand All @@ -192,6 +196,8 @@ the real path.
thread stress of queries against history readers
(`UNREGISTERED_PARENT_THREAD_TRIALS`), and a random comparison of every tree
meta against a full-history import (`UNREGISTERED_PARENT_SEEDS=0..300`).
- `crates/loro/tests/failed_import_state_rollback.rs`: state, handlers, and
retries after a failed import (loro-dev/loro#1164).
- `crates/loro/tests/unregistered_container_parent.rs`
`queries_race_with_failing_imports`: queries against imports that fail at
decode or depend on history before a shallow root, swept across the import's
Expand Down
7 changes: 6 additions & 1 deletion context/dead-container-cache.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Dead Container Cache

Verified against code 2026-09-28.
Verified against code 2026-09-30.

`DocState::is_deleted` (`crates/loro-internal/src/state/dead_containers_cache.rs`)
answers whether a container is unreachable from a root: it walks the arena
Expand Down Expand Up @@ -49,6 +49,11 @@ walk and assert that no cached entry on the chain contradicts the result.
- `DocState::apply_local_op` clears `revivable` before a tree `Move`. A local
op cannot address a deleted movable-list element.
- Resets replace the cache.
- A failed import's rollback clears both sets
(`DocState::forget_parent_link_caches_after_failed_import`, called by the
`OpLog` rollback methods): an answer found through a parent link the
rollback drops is not true of the kept history. See
[failed-import-arena-indices.md](failed-import-arena-indices.md).

`clear_revivable` is O(1) when the set is empty, so documents that never query
deleted containers pay nothing.
Expand Down
Loading
Loading