fix: reduce overlapping import costs

.changeset/fix-import-overlap-cost.md

@@ -0,0 +1,9 @@
+---
+"loro-crdt": patch
+---
+
+Reduce known-history comparison costs for snapshot-loaded documents while preserving peer-id reuse detection. Snapshot updates allocate only new history in the document arena and avoid cloning discarded changes.
+
+Avoid unnecessary rollback scopes for detached Text imports while retaining validation of pending movable-list operations.
+
+Keep import rollback decisions consistent across concurrent attach/detach, and fall back to ordinary history parsing when a cold-block comparison is ineligible.

context/arena-parent-links.md

@@ -1,6 +1,6 @@
 # Arena Parent Links
 
-Verified against code 2026-09-30.
+Verified against code 2026-10-01.
 
 `SharedArena` (`crates/loro-internal/src/arena.rs`) stores each container's
 parent. Liveness (`DocState::is_deleted`), paths (`DocState::get_path`,
@@ -112,7 +112,7 @@ that return `bool` or `Option` (`is_deleted`, `has_container`, `get_path`), so
 it has no `Err` to return, and a panic there unwinds under the locks and traps
 the WASM instance. Until 2026-09-30 it panicked.
 
-Every reader of the change store that fails to decode or parse a block records
+Every ordinary history reader that fails to decode or parse a block records
 it in `ChangeStore::parse_failures` (a leaf lock; the first failure is kept) and
 answers "no such change", as the readers other than the resolver always did.
 The resolver answers `CreatorOp::Corrupt`, which the arena treats like `Absent`
@@ -142,8 +142,12 @@ Limits:
   from the change store and panics with "unparsed vv don't match with change
   store" (`loro_dag.rs`, `ensure_lazy_load_node`).
 - A read that returns before any failure was recorded is not undone: an
-  `import` or `checkout` that is the first to read the block finishes on the
-  partial history. Only `export` checks again at the end.
+  `import` or `checkout` using a general history reader can finish on partial
+  history when it is the first to read the block. Only `export` checks again at
+  the end. The direct cold Text comparison in `known_history.rs` falls back to
+  the ordinary reader on a decoding/eligibility error. Its stricter op-length
+  and change-boundary checks never record a `parse_failures` entry themselves;
+  only a failure of the ordinary parser declares local history unparsable.
 - Local edits still succeed after a failure was recorded, but they cannot be
   exported from this document any more. What can be salvaged is the current
   state (`get_deep_value`).
@@ -182,6 +186,20 @@ A vv shortcut for IDs beyond the history would have to be exact for every path
 that writes the KV store, or a live container would read as deleted. Measured
 2026-09-28 (loro-dev/loro#1159).
 
+## Snapshot overlap decoding
+
+`ChangeStore::decode_snapshot_for_updates` uses a temporary arena for unmatched
+incoming blocks, rather than allocating a second known prefix in the document
+arena. Known-history comparison resolves container IDs across the two arenas;
+only checked, trimmed new ops are converted into document indices/value slices.
+`register_container_and_parent_link` runs on those converted changes before they
+leave the decoder. Temporary indices and parent links never escape into the
+document. A rejected comparison follows the existing arena rollback path.
+
+Cold Text-insert-only local blocks can also be compared directly from bytes
+without registering anything in the document arena. Other blocks still use the
+normal lazy reader. See [import-peer-id-reuse.md](import-peer-id-reuse.md).
+
 ## Import rollback
 
 A failed import can parse old change blocks while it computes its diff

context/benchmarks/import-overlap-cost-fat1.csv

@@ -0,0 +1,82 @@
+scenario,n,rep,version,ms,heap_kb,delta_kb
+overlap_snap,8000,0,old,0.268,0,0
+overlap_snap,8000,0,main,1.071,0,0
+overlap_snap,8000,0,fixed,0.594,0,0
+overlap_snap,8000,1,old,0.247,0,0
+overlap_snap,8000,1,main,1.095,0,0
+overlap_snap,8000,1,fixed,0.584,0,0
+overlap_snap,8000,2,old,0.266,0,0
+overlap_snap,8000,2,main,1.058,0,0
+overlap_snap,8000,2,fixed,0.595,0,0
+overlap_snap,32000,0,old,1.056,0,0
+overlap_snap,32000,0,main,4.379,0,0
+overlap_snap,32000,0,fixed,2.36,0,0
+overlap_snap,32000,1,old,1.049,0,0
+overlap_snap,32000,1,main,4.438,0,0
+overlap_snap,32000,1,fixed,2.308,0,0
+overlap_snap,32000,2,old,1.066,0,0
+overlap_snap,32000,2,main,4.461,0,0
+overlap_snap,32000,2,fixed,2.422,0,0
+overlap_snap,64000,0,old,2.209,0,0
+overlap_snap,64000,0,main,9.024,0,0
+overlap_snap,64000,0,fixed,4.867,0,0
+overlap_snap,64000,1,old,2.125,0,0
+overlap_snap,64000,1,main,9.219,0,0
+overlap_snap,64000,1,fixed,4.75,0,0
+overlap_snap,64000,2,old,2.17,0,0
+overlap_snap,64000,2,main,9.08,0,0
+overlap_snap,64000,2,fixed,4.828,0,0
+overlap_partial,8000,0,old,0.249,0,0
+overlap_partial,8000,0,main,0.678,0,0
+overlap_partial,8000,0,fixed,0.445,0,0
+overlap_partial,8000,1,old,0.247,0,0
+overlap_partial,8000,1,main,0.673,0,0
+overlap_partial,8000,1,fixed,0.444,0,0
+overlap_partial,8000,2,old,0.247,0,0
+overlap_partial,8000,2,main,0.672,0,0
+overlap_partial,8000,2,fixed,0.45,0,0
+overlap_partial,32000,0,old,1.028,0,0
+overlap_partial,32000,0,main,2.735,0,0
+overlap_partial,32000,0,fixed,1.735,0,0
+overlap_partial,32000,1,old,1.043,0,0
+overlap_partial,32000,1,main,2.694,0,0
+overlap_partial,32000,1,fixed,1.666,0,0
+overlap_partial,32000,2,old,1.13,0,0
+overlap_partial,32000,2,main,2.725,0,0
+overlap_partial,32000,2,fixed,1.667,0,0
+overlap_partial,64000,0,old,2.082,0,0
+overlap_partial,64000,0,main,5.479,0,0
+overlap_partial,64000,0,fixed,3.372,0,0
+overlap_partial,64000,1,old,2.197,0,0
+overlap_partial,64000,1,main,5.506,0,0
+overlap_partial,64000,1,fixed,3.339,0,0
+overlap_partial,64000,2,old,2.181,0,0
+overlap_partial,64000,2,main,5.421,0,0
+overlap_partial,64000,2,fixed,3.512,0,0
+snap_plus,8000,0,old,0.793,0,354
+snap_plus,8000,0,main,1.731,0,1508
+snap_plus,8000,0,fixed,0.364,0,334
+snap_plus,8000,1,old,0.801,0,353
+snap_plus,8000,1,main,1.745,0,1507
+snap_plus,8000,1,fixed,0.356,0,334
+snap_plus,8000,2,old,0.807,0,353
+snap_plus,8000,2,main,1.746,0,1508
+snap_plus,8000,2,fixed,0.361,0,334
+snap_plus,32000,0,old,3.274,0,1604
+snap_plus,32000,0,main,7.571,0,6673
+snap_plus,32000,0,fixed,1.137,0,1571
+snap_plus,32000,1,old,3.274,0,1605
+snap_plus,32000,1,main,7.692,0,6673
+snap_plus,32000,1,fixed,1.171,0,1571
+snap_plus,32000,2,old,3.296,0,1604
+snap_plus,32000,2,main,7.555,0,6673
+snap_plus,32000,2,fixed,1.12,0,1570
+snap_plus,64000,0,old,6.568,0,3211
+snap_plus,64000,0,main,15.173,0,13346
+snap_plus,64000,0,fixed,2.283,0,3142
+snap_plus,64000,1,old,6.454,0,3211
+snap_plus,64000,1,main,15.927,0,13346
+snap_plus,64000,1,fixed,2.213,0,3143
+snap_plus,64000,2,old,6.492,0,3211
+snap_plus,64000,2,main,15.34,0,13347
+snap_plus,64000,2,fixed,2.236,0,3142

context/benchmarks/import-overlap-cost-fat64.csv

@@ -0,0 +1,244 @@
+scenario,n,rep,version,ms,heap_kb,delta_kb
+overlap_snap,8000,0,old,1.757,0,0
+overlap_snap,8000,0,main,3.095,0,0
+overlap_snap,8000,0,fixed,2.295,0,0
+overlap_snap,8000,1,old,1.742,0,0
+overlap_snap,8000,1,main,3.136,0,0
+overlap_snap,8000,1,fixed,2.328,0,0
+overlap_snap,8000,2,old,1.812,0,0
+overlap_snap,8000,2,main,3.066,0,0
+overlap_snap,8000,2,fixed,2.338,0,0
+overlap_snap,32000,0,old,7.377,0,0
+overlap_snap,32000,0,main,13.0,0,0
+overlap_snap,32000,0,fixed,9.504,0,0
+overlap_snap,32000,1,old,7.318,0,0
+overlap_snap,32000,1,main,12.734,0,0
+overlap_snap,32000,1,fixed,9.317,0,0
+overlap_snap,32000,2,old,7.389,0,0
+overlap_snap,32000,2,main,12.803,0,0
+overlap_snap,32000,2,fixed,9.68,0,0
+overlap_snap,64000,0,old,15.539,0,0
+overlap_snap,64000,0,main,26.87,0,0
+overlap_snap,64000,0,fixed,19.33,0,0
+overlap_snap,64000,1,old,15.462,0,0
+overlap_snap,64000,1,main,26.566,0,0
+overlap_snap,64000,1,fixed,19.493,0,0
+overlap_snap,64000,2,old,15.123,0,0
+overlap_snap,64000,2,main,26.516,0,0
+overlap_snap,64000,2,fixed,19.44,0,0
+overlap_partial,8000,0,old,1.548,0,0
+overlap_partial,8000,0,main,2.24,0,0
+overlap_partial,8000,0,fixed,1.79,0,0
+overlap_partial,8000,1,old,1.612,0,0
+overlap_partial,8000,1,main,2.249,0,0
+overlap_partial,8000,1,fixed,1.852,0,0
+overlap_partial,8000,2,old,1.568,0,0
+overlap_partial,8000,2,main,2.26,0,0
+overlap_partial,8000,2,fixed,1.869,0,0
+overlap_partial,32000,0,old,6.546,0,0
+overlap_partial,32000,0,main,9.233,0,0
+overlap_partial,32000,0,fixed,7.506,0,0
+overlap_partial,32000,1,old,6.509,0,0
+overlap_partial,32000,1,main,9.33,0,0
+overlap_partial,32000,1,fixed,7.676,0,0
+overlap_partial,32000,2,old,6.605,0,0
+overlap_partial,32000,2,main,9.203,0,0
+overlap_partial,32000,2,fixed,7.682,0,0
+overlap_partial,64000,0,old,13.401,0,0
+overlap_partial,64000,0,main,19.404,0,0
+overlap_partial,64000,0,fixed,15.429,0,0
+overlap_partial,64000,1,old,13.599,0,0
+overlap_partial,64000,1,main,19.139,0,0
+overlap_partial,64000,1,fixed,15.51,0,0
+overlap_partial,64000,2,old,13.391,0,0
+overlap_partial,64000,2,main,18.976,0,0
+overlap_partial,64000,2,fixed,15.316,0,0
+overlap_mem,8000,0,old,0.394,0,0
+overlap_mem,8000,0,main,0.398,0,0
+overlap_mem,8000,0,fixed,0.4,0,0
+overlap_mem,8000,1,old,0.387,0,0
+overlap_mem,8000,1,main,0.387,0,0
+overlap_mem,8000,1,fixed,0.393,0,0
+overlap_mem,8000,2,old,0.385,0,0
+overlap_mem,8000,2,main,0.397,0,0
+overlap_mem,8000,2,fixed,0.385,0,0
+overlap_mem,32000,0,old,1.568,0,0
+overlap_mem,32000,0,main,1.567,0,0
+overlap_mem,32000,0,fixed,1.578,0,0
+overlap_mem,32000,1,old,1.543,0,0
+overlap_mem,32000,1,main,1.565,0,0
+overlap_mem,32000,1,fixed,1.567,0,0
+overlap_mem,32000,2,old,1.549,0,0
+overlap_mem,32000,2,main,1.606,0,0
+overlap_mem,32000,2,fixed,1.553,0,0
+overlap_mem,64000,0,old,3.112,0,0
+overlap_mem,64000,0,main,3.125,0,0
+overlap_mem,64000,0,fixed,3.108,0,0
+overlap_mem,64000,1,old,3.028,0,0
+overlap_mem,64000,1,main,3.118,0,0
+overlap_mem,64000,1,fixed,3.082,0,0
+overlap_mem,64000,2,old,3.107,0,0
+overlap_mem,64000,2,main,3.152,0,0
+overlap_mem,64000,2,fixed,3.082,0,0
+snap_plus,8000,0,old,2.473,0,2164
+snap_plus,8000,0,main,3.867,0,4598
+snap_plus,8000,0,fixed,1.597,0,1604
+snap_plus,8000,1,old,2.501,0,2165
+snap_plus,8000,1,main,3.947,0,4598
+snap_plus,8000,1,fixed,1.603,0,1605
+snap_plus,8000,2,old,2.474,0,2164
+snap_plus,8000,2,main,3.925,0,4598
+snap_plus,8000,2,fixed,1.592,0,1604
+snap_plus,32000,0,old,10.024,0,8682
+snap_plus,32000,0,main,16.703,0,19192
+snap_plus,32000,0,fixed,6.526,0,6452
+snap_plus,32000,1,old,10.229,0,8682
+snap_plus,32000,1,main,16.324,0,19192
+snap_plus,32000,1,fixed,6.411,0,6453
+snap_plus,32000,2,old,10.213,0,8681
+snap_plus,32000,2,main,16.615,0,19192
+snap_plus,32000,2,fixed,6.475,0,6452
+snap_plus,64000,0,old,20.766,0,21282
+snap_plus,64000,0,main,34.489,0,41202
+snap_plus,64000,0,fixed,13.397,0,16827
+snap_plus,64000,1,old,20.615,0,21283
+snap_plus,64000,1,main,34.882,0,41229
+snap_plus,64000,1,fixed,13.568,0,16827
+snap_plus,64000,2,old,20.481,0,21273
+snap_plus,64000,2,main,34.627,0,41193
+snap_plus,64000,2,fixed,13.282,0,16832
+snap_mem,8000,0,old,1.2105000000000001,7727,5382
+snap_mem,8000,0,main,1.911,13190,10839
+snap_mem,8000,0,fixed,0.203,3955,1604
+snap_mem,8000,1,old,1.1484999999999999,7727,5382
+snap_mem,8000,1,main,1.958,13189,10838
+snap_mem,8000,1,fixed,0.20400000000000001,3956,1605
+snap_mem,8000,2,old,1.138,7727,5382
+snap_mem,8000,2,main,1.919,13190,10839
+snap_mem,8000,2,fixed,0.2025,3956,1606
+snap_mem,32000,0,old,4.887,30590,21548
+snap_mem,32000,0,main,7.8675,52443,43383
+snap_mem,32000,0,fixed,0.71,15516,6456
+snap_mem,32000,1,old,4.8255,30589,21547
+snap_mem,32000,1,main,7.9475,52445,43385
+snap_mem,32000,1,fixed,0.722,15514,6454
+snap_mem,32000,2,old,4.7844999999999995,30587,21545
+snap_mem,32000,2,main,7.8875,52444,43384
+snap_mem,32000,2,fixed,0.7264999999999999,15515,6455
+snap_mem,64000,0,old,9.486,64813,47011
+snap_mem,64000,0,main,15.8765,108979,91143
+snap_mem,64000,0,fixed,1.589,34664,16828
+snap_mem,64000,1,old,9.5715,64808,47006
+snap_mem,64000,1,main,16.019,108932,91096
+snap_mem,64000,1,fixed,1.512,34665,16829
+snap_mem,64000,2,old,9.811,64809,47007
+snap_mem,64000,2,main,15.917000000000002,108973,91137
+snap_mem,64000,2,fixed,1.5805,34663,16827
+detached,8000,0,old,7.117,0,0
+detached,8000,0,main,8.931,0,0
+detached,8000,0,fixed,7.216,0,0
+detached,8000,1,old,6.865,0,0
+detached,8000,1,main,8.897,0,0
+detached,8000,1,fixed,6.954,0,0
+detached,8000,2,old,6.831,0,0
+detached,8000,2,main,8.863,0,0
+detached,8000,2,fixed,6.909,0,0
+detached,32000,0,old,27.968,0,0
+detached,32000,0,main,35.847,0,0
+detached,32000,0,fixed,27.75,0,0
+detached,32000,1,old,28.11,0,0
+detached,32000,1,main,35.118,0,0
+detached,32000,1,fixed,28.886,0,0
+detached,32000,2,old,28.873,0,0
+detached,32000,2,main,35.053,0,0
+detached,32000,2,fixed,28.423,0,0
+detached,64000,0,old,57.046,0,0
+detached,64000,0,main,73.145,0,0
+detached,64000,0,fixed,59.277,0,0
+detached,64000,1,old,57.146,0,0
+detached,64000,1,main,71.227,0,0
+detached,64000,1,fixed,57.281,0,0
+detached,64000,2,old,56.329,0,0
+detached,64000,2,main,72.767,0,0
+detached,64000,2,fixed,56.579,0,0
+mlist_batch,8000,0,old,7.456,0,0
+mlist_batch,8000,0,main,8.909,0,0
+mlist_batch,8000,0,fixed,9.053,0,0
+mlist_batch,8000,1,old,7.46,0,0
+mlist_batch,8000,1,main,9.06,0,0
+mlist_batch,8000,1,fixed,8.993,0,0
+mlist_batch,8000,2,old,7.563,0,0
+mlist_batch,8000,2,main,8.9,0,0
+mlist_batch,8000,2,fixed,8.982,0,0
+mlist_batch,32000,0,old,29.524,0,0
+mlist_batch,32000,0,main,38.558,0,0
+mlist_batch,32000,0,fixed,39.529,0,0
+mlist_batch,32000,1,old,29.442,0,0
+mlist_batch,32000,1,main,36.899,0,0
+mlist_batch,32000,1,fixed,37.298,0,0
+mlist_batch,32000,2,old,29.867,0,0
+mlist_batch,32000,2,main,37.508,0,0
+mlist_batch,32000,2,fixed,37.358,0,0
+mlist_batch,64000,0,old,79.806,0,0
+mlist_batch,64000,0,main,96.3,0,0
+mlist_batch,64000,0,fixed,95.375,0,0
+mlist_batch,64000,1,old,79.049,0,0
+mlist_batch,64000,1,main,100.955,0,0
+mlist_batch,64000,1,fixed,96.268,0,0
+mlist_batch,64000,2,old,83.69,0,0
+mlist_batch,64000,2,main,96.798,0,0
+mlist_batch,64000,2,fixed,96.028,0,0
+text_stream,8000,0,old,18.094,0,0
+text_stream,8000,0,main,20.952,0,0
+text_stream,8000,0,fixed,20.439,0,0
+text_stream,8000,1,old,18.364,0,0
+text_stream,8000,1,main,20.088,0,0
+text_stream,8000,1,fixed,20.701,0,0
+text_stream,8000,2,old,18.2,0,0
+text_stream,8000,2,main,21.394,0,0
+text_stream,8000,2,fixed,20.529,0,0
+text_stream,32000,0,old,76.572,0,0
+text_stream,32000,0,main,84.157,0,0
+text_stream,32000,0,fixed,84.151,0,0
+text_stream,32000,1,old,71.38,0,0
+text_stream,32000,1,main,85.664,0,0
+text_stream,32000,1,fixed,82.378,0,0
+text_stream,32000,2,old,73.324,0,0
+text_stream,32000,2,main,86.208,0,0
+text_stream,32000,2,fixed,81.682,0,0
+text_stream,64000,0,old,149.703,0,0
+text_stream,64000,0,main,172.67,0,0
+text_stream,64000,0,fixed,168.454,0,0
+text_stream,64000,1,old,151.261,0,0
+text_stream,64000,1,main,165.534,0,0
+text_stream,64000,1,fixed,166.158,0,0
+text_stream,64000,2,old,149.038,0,0
+text_stream,64000,2,main,162.693,0,0
+text_stream,64000,2,fixed,164.711,0,0
+batch,8000,0,old,29.818,0,0
+batch,8000,0,main,29.709,0,0
+batch,8000,0,fixed,29.698,0,0
+batch,8000,1,old,29.492,0,0
+batch,8000,1,main,30.685,0,0
+batch,8000,1,fixed,29.245,0,0
+batch,8000,2,old,30.727,0,0
+batch,8000,2,main,29.681,0,0
+batch,8000,2,fixed,29.886,0,0
+batch,32000,0,old,124.071,0,0
+batch,32000,0,main,127.269,0,0
+batch,32000,0,fixed,124.147,0,0
+batch,32000,1,old,122.213,0,0
+batch,32000,1,main,123.015,0,0
+batch,32000,1,fixed,126.228,0,0
+batch,32000,2,old,125.755,0,0
+batch,32000,2,main,129.023,0,0
+batch,32000,2,fixed,125.134,0,0
+batch,64000,0,old,260.088,0,0
+batch,64000,0,main,255.777,0,0
+batch,64000,0,fixed,255.857,0,0
+batch,64000,1,old,257.137,0,0
+batch,64000,1,main,256.648,0,0
+batch,64000,1,fixed,255.895,0,0
+batch,64000,2,old,254.521,0,0
+batch,64000,2,main,255.13,0,0
+batch,64000,2,fixed,255.577,0,0