fix: rich-text insert positions near style anchors, delete IDs, and delete events

.changeset/richtext-wasm-positions.md

@@ -0,0 +1,9 @@
+---
+"loro-crdt": patch
+---
+
+Fix three rich-text bugs that a randomized comparison with loro.js found:
+
+- Text inserted next to a style anchor could land on the wrong side of it, depending on an internal cursor cache (native Rust and WASM): for example, after an `unmark` that changed nothing, typing right after a mark that does not expand after put the new text inside the mark.
+- In the WASM build, a delete spanning text from separate inserts could record a `start_id` naming the wrong characters when an astral character (for example, an emoji) ended one of the inserts. Imports apply deletes by position, but shallow snapshot imports and cursors on deleted text use the recorded IDs, so they could disagree with a full import. This fixes new ops only; deletes already in a history keep their IDs (loro-dev/loro#1149).
+- In the WASM build, committing a transaction that merged backspaces over astral characters panicked with "Op/hint length mismatch" when the document had a subscriber or an `UndoManager`.

AGENTS.md

@@ -27,6 +27,8 @@ Loro is a Rust CRDT workspace with JS/WASM packaging and a MoonBit codec.
 - Shallow-snapshot dead-style redaction (null-only, both-expand exclusion,
   root-whitelisted latest state):
   [context/shallow-snapshot-style-redaction.md](context/shallow-snapshot-style-redaction.md).
+- Rich-text insert positions next to style anchors, the insert cursor cache,
+  and delete `start_id`s: [context/richtext-insert-positions.md](context/richtext-insert-positions.md).
 - User-facing Loro usage, sync, editor integration, and performance guidance:
   [skills/loro/SKILL.md](skills/loro/SKILL.md).
 - Pure TypeScript runtime indexes, complexity contracts, benchmarks, and remaining gaps:

context/richtext-insert-positions.md

@@ -0,0 +1,67 @@
+# Rich-Text Insert Positions and Delete IDs
+
+Verified against code 2026-09-28.
+
+## Where local text goes next to style anchors
+
+`RichtextState` (`crates/loro-internal/src/container/richtext/richtext_state.rs`)
+keeps every style's start and end anchors in the text. An anchor has entity
+length 1 and length 0 in bytes, Unicode, UTF-16, and event indexes, so one user
+position can name several entity positions. `find_best_insert_pos` picks one:
+
+1. Scan the anchors after the previous character from left to right. Stop at
+   the first anchor that new text stays in front of
+   (`insert_stays_before_anchor`): the start of an unmark (null or false
+   value), or an anchor that prefers inserts before it
+   (`TextStyleInfoFlag::prefer_insert_before`: a start anchor that does not
+   expand before, an end anchor that expands after).
+2. Walk back over the scanned anchors while they prefer inserts before them.
+
+`TextHandler::insert` and both ends of `TextHandler::mark` go through
+`get_entity_index_for_text_insert`, which applies this rule. loro.js follows
+the same rule from loro-dev/loro#1137 on (`LoroText.#insertPosition`).
+
+## The insert cursor cache
+
+`get_entity_index_for_text_insert` first tries the cursor left by the previous
+lookup or insert (`try_get_cache_or_clean`), so consecutive typing skips the
+tree query. The cache is only used where the anchor rules cannot move the
+insert (`anchor_rules_may_move_insert`):
+
+- the text has no style;
+- the cursor is inside a text chunk, between two text chunks, or at the start
+  or end of the text;
+- the cursor is at the end of a text chunk and the next element is an anchor
+  that new text stays in front of. The scan stops at that anchor, so the answer
+  is the cursor itself; this is typing at the end of a bold range.
+
+A cursor on an anchor, or at the start of a text chunk right after an anchor,
+falls back to `find_best_insert_pos`. Typing continues from the end of the new
+text, which is usually followed by text or by an anchor the text stays in front
+of, so typically only the first keystroke next to an anchor pays for the query.
+Before loro-dev/loro#1135 the cache was trusted at
+every chunk boundary, so the side of an anchor that text landed on depended on
+the previous lookup (a no-op `unmark` could move the next insert into a style).
+
+- Test: `richtext_state/insert_pos_cache_test.rs` compares every cached answer
+  with a cold lookup for all five `PosType`s on random documents (all expand
+  types, overlapping styles, null/false values, adjacent and empty anchor
+  pairs, astral/ZWJ/flag text). `LORO_INSERT_CACHE_SEEDS=<n>` runs more seeds.
+- Benchmark: `crates/loro/examples/richtext_typing_bench.rs` (typing next to
+  and away from anchors; see its header for how to run it).
+
+## Delete op IDs
+
+A text delete op records its position and the ID of its first deleted character
+(`start_id`). `get_text_entity_ranges` splits a local delete into runs that are
+contiguous in both entity position and ID, and the handler writes them last run
+first. Up to `loro-crdt` 1.16.3 the WASM build tested ID contiguity with the
+UTF-16 length, so after an astral character a delete could record a `start_id`
+that names other characters (fixed in loro-dev/loro#1135).
+
+Replaying a delete by position (`Tracker::delete`) gives the right text, but
+some readers trust `start_id`: placeholders from a shallow snapshot take their
+real IDs from it (`CrdtRope::delete`), so a shallow import can end with other
+text than a full import, and a cursor on a deleted character that no delete op
+names does not resolve. Histories that already contain such deletes are tracked
+in loro-dev/loro#1149.

crates/loro-internal/AGENTS.md

@@ -27,6 +27,9 @@ over graceful degradation.
 - `src/handler.rs`: typed container handlers, local operation creation, and
   `MapHandler::ensure_mergeable_*`.
 - `src/diff_calc/`: diff calculation when moving between versions.
+- `src/container/richtext/`: text state with style anchors. Where local inserts
+  go next to anchors, the insert cursor cache, and delete `start_id`s:
+  [../../context/richtext-insert-positions.md](../../context/richtext-insert-positions.md).
 - `docs/diff_calc.md`: design notes for diff calculation.
 - `docs/critical-version-spec.md`: specification and proof skeleton for
   replay-base selection (Eg-walker-aligned terminology; defines critical

crates/loro-internal/src/container/richtext/richtext_state.rs

@@ -1408,6 +1408,14 @@ mod query {
     }
 }
 
+/// Whether new text inserted right in front of this anchor stays in front of
+/// it: the start of an unmark (case 1 of `find_best_insert_pos`), or an anchor
+/// that prefers inserts before it (case 2).
+fn insert_stays_before_anchor(style: &StyleOp, anchor_type: AnchorType) -> bool {
+    (anchor_type == AnchorType::Start && (style.value.is_null() || style.value.is_false()))
+        || style.info.prefer_insert_before(anchor_type)
+}
+
 impl RichtextState {
     pub(crate) fn from_chunks<I: Iterator<Item = impl Into<RichtextStateChunk>>>(i: I) -> Self {
         Self {
@@ -1428,7 +1436,12 @@ impl RichtextState {
                 return Ok((0, None));
             }
 
-            if let Some(c) = self.try_get_cache_or_clean(pos, pos_type) {
+            // A cached cursor next to a style anchor would skip the anchor
+            // rules of `find_best_insert_pos`.
+            let cached = self
+                .try_get_cache_or_clean(pos, pos_type)
+                .filter(|c| !self.has_styles() || !self.anchor_rules_may_move_insert(*c));
+            if let Some(c) = cached {
                 let entity_index = self.get_cache_entity_index().unwrap();
                 Ok((entity_index + c.offset, Some(c)))
             } else {
@@ -1460,6 +1473,40 @@ impl RichtextState {
         result
     }
 
+    /// Whether the anchor rules of `find_best_insert_pos` could put an insert
+    /// somewhere other than `cursor`. They cannot inside a text chunk, between
+    /// two text chunks, or at the end of a text chunk followed by an anchor
+    /// that new text stays in front of (the rules stop at that anchor).
+    fn anchor_rules_may_move_insert(&self, cursor: Cursor) -> bool {
+        let Some(elem) = self.tree.get_elem(cursor.leaf) else {
+            return true;
+        };
+        if !matches!(elem, RichtextStateChunk::Text { .. }) {
+            return true;
+        }
+        if cursor.offset == 0 {
+            return self.tree.prev_elem(cursor).is_some_and(|prev| {
+                matches!(
+                    self.tree.get_elem(prev.leaf),
+                    Some(RichtextStateChunk::Style { .. })
+                )
+            });
+        }
+        if cursor.offset < elem.rle_len() {
+            return false;
+        }
+        match self
+            .tree
+            .next_elem(cursor)
+            .and_then(|next| self.tree.get_elem(next.leaf))
+        {
+            Some(RichtextStateChunk::Style { style, anchor_type }) => {
+                !insert_stays_before_anchor(style, *anchor_type)
+            }
+            _ => false,
+        }
+    }
+
     pub(crate) fn has_styles(&self) -> bool {
         self.style_ranges
             .as_ref()
@@ -1892,14 +1939,8 @@ impl RichtextState {
             };
 
             visited.push((style, anchor_type, iter, entity_index));
-            if anchor_type == AnchorType::Start && (style.value.is_null() || style.value.is_false())
-            {
-                // case 1. should be before this anchor
-                break;
-            }
-
-            if style.info.prefer_insert_before(anchor_type) {
-                // case 2.
+            if insert_stays_before_anchor(style, anchor_type) {
+                // case 1 or 2.
                 break;
             }
 
@@ -2016,9 +2057,11 @@ impl RichtextState {
                         let event_len = s.entity_range_to_event_range(start..end).len();
                         let id = s.id().inc(start as i32);
                         match ans.last_mut() {
+                            // Every entity of a range is text, so its IDs advance by
+                            // the entity length (not the UTF-16 event length).
                             Some(last)
                                 if last.entity_end == entity_index
-                                    && last.id_start.inc(last.event_len as i32) == id =>
+                                    && last.id_start.inc(last.entity_len() as i32) == id =>
                             {
                                 last.entity_end += len;
                                 last.event_len += event_len;
@@ -3077,6 +3120,9 @@ mod converter {
     }
 }
 
+#[cfg(test)]
+mod insert_pos_cache_test;
+
 #[cfg(test)]
 mod test {
     use append_only_bytes::AppendOnlyBytes;

crates/loro-internal/src/container/richtext/richtext_state/insert_pos_cache_test.rs

@@ -0,0 +1,172 @@
+//! `get_entity_index_for_text_insert` can start from the cursor cached by an
+//! earlier lookup. That shortcut must never move an insert across a style
+//! anchor, so these tests compare every cached answer with a cold lookup
+//! (`find_best_insert_pos`) on documents full of anchors.
+
+use loro_common::LoroValue;
+use rand::{rngs::StdRng, seq::SliceRandom, Rng, SeedableRng};
+
+use super::{PosType, RichtextState};
+use crate::{
+    container::richtext::{
+        config::{StyleConfig, StyleConfigMap},
+        ExpandType,
+    },
+    handler::{HandlerTrait, TextHandler},
+    LoroDoc,
+};
+
+/// One key per expand type.
+const STYLES: [(&str, ExpandType); 4] = [
+    ("after", ExpandType::After),
+    ("before", ExpandType::Before),
+    ("both", ExpandType::Both),
+    ("none", ExpandType::None),
+];
+
+/// BMP, astral, ZWJ sequences, flags, and modifiers.
+const SNIPPETS: [&str; 9] = ["a", "bc", "é", "中", "😀", "𝒳y", "👍🏽", "👨‍👩‍👧", "🇨🇳"];
+
+const POS_TYPES: [PosType; 5] = [
+    PosType::Bytes,
+    PosType::Unicode,
+    PosType::Utf16,
+    PosType::Event,
+    PosType::Entity,
+];
+
+const MAX_LEN: usize = 40;
+
+fn random_edit(rng: &mut StdRng, text: &TextHandler) {
+    let len = text.len_unicode();
+    match rng.gen_range(0..10) {
+        0..=3 if len < MAX_LEN => {
+            let s = SNIPPETS.choose(rng).unwrap();
+            text.insert(rng.gen_range(0..=len), s, PosType::Unicode)
+                .unwrap();
+        }
+        0..=4 if len > 0 => {
+            let pos = rng.gen_range(0..len);
+            let del = rng.gen_range(1..=(len - pos).min(4));
+            text.delete(pos, del, PosType::Unicode).unwrap();
+        }
+        _ if len > 0 => {
+            let start = rng.gen_range(0..len);
+            let end = rng.gen_range(start + 1..=len);
+            let (key, _) = *STYLES.choose(rng).unwrap();
+            let value = match rng.gen_range(0..5) {
+                0 => LoroValue::Null,
+                1 => false.into(),
+                2 => "v".into(),
+                _ => true.into(),
+            };
+            if rng.gen_bool(0.2) {
+                text.unmark(start, end, key, PosType::Unicode).unwrap();
+            } else {
+                text.mark(start, end, key, value, PosType::Unicode).unwrap();
+            }
+            if rng.gen_bool(0.2) {
+                // Leaves the anchors of the style next to each other.
+                text.delete(start, end - start, PosType::Unicode).unwrap();
+            }
+        }
+        _ => text.insert(0, "a", PosType::Unicode).unwrap(),
+    }
+}
+
+fn boundaries(text: &str, pos_type: PosType, entity_len: usize) -> Vec<usize> {
+    let unit_len = |c: char| match pos_type {
+        PosType::Bytes => c.len_utf8(),
+        PosType::Unicode => 1,
+        PosType::Utf16 => c.len_utf16(),
+        PosType::Event if cfg!(feature = "wasm") => c.len_utf16(),
+        PosType::Event => 1,
+        PosType::Entity => unreachable!(),
+    };
+    if pos_type == PosType::Entity {
+        return (0..=entity_len).collect();
+    }
+    let mut ans = vec![0];
+    for c in text.chars() {
+        ans.push(ans.last().unwrap() + unit_len(c));
+    }
+    ans
+}
+
+fn cold_lookup(state: &RichtextState, pos: usize, pos_type: PosType) -> usize {
+    let mut cold = state.clone();
+    cold.clear_cache();
+    cold.get_entity_index_for_text_insert(pos, pos_type)
+        .unwrap()
+        .0
+}
+
+fn assert_cache_matches_cold_lookup(state: &mut RichtextState, text: &str, context: &str) {
+    for pos_type in POS_TYPES {
+        let positions = boundaries(text, pos_type, state.len_entity());
+        // Starting from the cursor the last edit left behind.
+        for &pos in &positions {
+            let expected = cold_lookup(state, pos, pos_type);
+            let (actual, _) = state
+                .clone()
+                .get_entity_index_for_text_insert(pos, pos_type)
+                .unwrap();
+            assert_eq!(
+                actual, expected,
+                "{context}: {pos_type:?} {pos} from the edit's cursor in {text:?}"
+            );
+        }
+        // Starting from the cursor of the previous position, as when typing.
+        let mut warm = state.clone();
+        for &pos in &positions {
+            let expected = cold_lookup(&warm, pos, pos_type);
+            let (actual, cursor) = warm
+                .get_entity_index_for_text_insert(pos, pos_type)
+                .unwrap();
+            assert_eq!(
+                actual, expected,
+                "{context}: {pos_type:?} {pos} from the previous position in {text:?}"
+            );
+            if let Some(cursor) = cursor {
+                assert_eq!(
+                    warm.get_index_from_cursor(cursor, PosType::Entity),
+                    Some(actual)
+                );
+            }
+        }
+    }
+}
+
+#[test]
+fn cached_insert_position_matches_cold_lookup_near_style_anchors() {
+    let seeds: u64 = std::env::var("LORO_INSERT_CACHE_SEEDS")
+        .ok()
+        .and_then(|s| s.parse().ok())
+        .unwrap_or(12);
+    for seed in 0..seeds {
+        let mut rng = StdRng::seed_from_u64(seed);
+        let doc = LoroDoc::new_auto_commit();
+        let mut styles = StyleConfigMap::new();
+        for (key, expand) in STYLES {
+            styles.insert(key.into(), StyleConfig::new().expand(expand));
+        }
+        doc.config_text_style(styles);
+        let text = doc.get_text("text");
+        for step in 0..60 {
+            random_edit(&mut rng, &text);
+            if rng.gen_bool(0.3) {
+                doc.commit_then_renew();
+            }
+            let value = text.to_string();
+            // The clone keeps the cursor cache of the document's state.
+            let mut state = doc.app_state().lock().with_state_mut(text.idx(), |s| {
+                s.as_richtext_state_mut().unwrap().inner_state_mut().clone()
+            });
+            assert_cache_matches_cold_lookup(
+                &mut state,
+                &value,
+                &format!("seed {seed} step {step}"),
+            );
+        }
+    }
+}

crates/loro-internal/src/state/richtext_state.rs

@@ -991,6 +991,11 @@ impl RichtextState {
         self.state.get_mut().len_unicode()
     }
 
+    #[cfg(test)]
+    pub(crate) fn inner_state_mut(&mut self) -> &mut InnerState {
+        self.state.get_mut()
+    }
+
     #[inline]
     pub(crate) fn get_entity_index_for_text_insert(
         &mut self,