Skip to content

Empty root container shows up in a doc's value but not after replaying its own history #1156

Closed

Description

@zxch3n

Summary

A root container whose ops add up to an empty value is visible in the live doc ("t": ""), but a fresh doc that imports that doc's own all_updates does not show it ({}). The two docs have the same history and different get_deep_value() / toJSON() output.

This came up in the adversarial review of #1125: 4 of 1600 seeds of the reviewer's random workload ended with a peer that differs from a fresh replay of its own history. Not caused by #1125, and not fixed there. I rebuilt a probe on main (source below); every mismatch it finds is this class.

Minimal repro (main 1eb14b3b)

use loro::{ExportMode, LoroDoc, ToJson};

let a = LoroDoc::new();
a.set_peer_id(1).unwrap();
a.get_text("t").insert(0, "ab").unwrap();
a.commit();
a.get_text("t").delete(0, 2).unwrap();
a.commit();

let replay = LoroDoc::new();
replay.import(&a.export(ExportMode::all_updates()).unwrap()).unwrap();

assert_eq!(a.get_deep_value().to_json_value(), serde_json::json!({"t": ""}));
assert_eq!(replay.get_deep_value().to_json_value(), serde_json::json!({})); // differs

The same happens after an UndoManager undo of the insert, when a remote peer receives the insert and the delete in separate imports, and for a doc loaded from a snapshot of a: all show {"t": ""}, and all replay to {}.

The same sequence on a root map (insert a key, then delete it) is consistent: {"m": {}} both live and replayed. With longer random histories (undo/redo, snapshot restarts, import_batch), root maps also mismatch: e.g. live {"l": [...], "m": {}, "t": "abab"} against replay {"l": [...], "t": "abab"} (probe seeds 426, 509, 591, 748 with 120 steps). I have not minimized the map variant yet.

Expected

Whether an empty root container is part of the value should depend on the history only, so a peer and a fresh replay of its history agree.

Probe used to find these (release build, replay_probe 0 800 120; 12/800 seeds on main)
//! Temporary: search for peers whose state differs from a fresh replay of their own history.
use loro::{ExportMode, LoroDoc, LoroMap, LoroText, ToJson, UndoManager, ValueOrContainer};
use rand::{rngs::StdRng, Rng, SeedableRng};

struct Peer {
    doc: LoroDoc,
    undo: UndoManager,
    shallow: bool,
}

fn new_peer(doc: LoroDoc, shallow: bool) -> Peer {
    let undo = UndoManager::new(&doc);
    Peer { doc, undo, shallow }
}

fn step(rng: &mut StdRng, peers: &mut Vec<Peer>, i: usize, log: &mut Vec<String>) {
    let n = peers.len();
    let p = rng.gen_range(0..n);
    let q = (p + rng.gen_range(1..n)) % n;
    let kind = rng.gen_range(0..16);
    log.push(format!("step {i} kind {kind} p{p} q{q}"));
    let d = &peers[p].doc;
    let l = d.get_movable_list("l");
    let len = l.len();
    match kind {
        0 | 1 => { l.insert(rng.gen_range(0..=len), i as i64).unwrap(); }
        2 if len > 1 => { l.mov(rng.gen_range(0..len), rng.gen_range(0..len)).unwrap(); }
        3 if len > 0 => { l.set(rng.gen_range(0..len), i as i64).unwrap(); }
        4 if len > 0 => { l.delete(rng.gen_range(0..len), 1).unwrap(); }
        5 => {
            if rng.gen_bool(0.5) {
                let m = l.insert_container(rng.gen_range(0..=len), LoroMap::new()).unwrap();
                m.insert("v", i as i64).unwrap();
            } else {
                let t = l.insert_container(rng.gen_range(0..=len), LoroText::new()).unwrap();
                t.insert(0, "hi").unwrap();
            }
        }
        6 if len > 0 => {
            if let Some(ValueOrContainer::Container(c)) = l.get(rng.gen_range(0..len)) {
                match c {
                    loro::Container::Map(m) => { m.insert("v", i as i64).unwrap(); }
                    loro::Container::Text(t) => { let tl = t.len_unicode(); t.insert(rng.gen_range(0..=tl), "x").unwrap(); }
                    _ => {}
                }
            }
        }
        7 => { let t = d.get_text("t"); let tl = t.len_unicode(); t.insert(rng.gen_range(0..=tl), "ab").unwrap(); }
        8 => { d.get_map("m").insert(&format!("k{}", rng.gen_range(0..4)), i as i64).unwrap(); }
        9 => { let _ = peers[p].undo.undo(); }
        10 => { let _ = peers[p].undo.redo(); }
        11 | 12 => {
            let u = peers[p].doc.export(ExportMode::updates(&peers[q].doc.oplog_vv())).unwrap();
            let r = peers[q].doc.import(&u);
            log.push(format!("  sync {p}->{q} {:?}", r.map(|_| ())));
        }
        13 => {
            let s = peers[p].doc.export(ExportMode::Snapshot).unwrap();
            let r = peers[q].doc.import(&s);
            log.push(format!("  snapshot {p}->{q} {:?}", r.map(|_| ())));
        }
        14 => {
            let peer = peers[q].doc.peer_id();
            let shallow = rng.gen_bool(0.5);
            let bytes = if shallow {
                peers[q].doc.export(ExportMode::shallow_snapshot(&peers[q].doc.oplog_frontiers())).unwrap()
            } else {
                peers[q].doc.export(ExportMode::Snapshot).unwrap()
            };
            let doc = LoroDoc::new();
            doc.import(&bytes).unwrap();
            doc.set_peer_id(peer).unwrap();
            let was_shallow = peers[q].shallow;
            peers[q] = new_peer(doc, shallow || was_shallow);
            log.push(format!("  restart {q} shallow={shallow}"));
        }
        15 => {
            let r = (q + 1) % n;
            let a = peers[p].doc.export(ExportMode::updates(&peers[q].doc.oplog_vv())).unwrap();
            let b = peers[r].doc.export(ExportMode::updates(&peers[q].doc.oplog_vv())).unwrap();
            let res = peers[q].doc.import_batch(&[b, a]);
            log.push(format!("  batch {p},{r}->{q} {:?}", res.map(|_| ())));
        }
        _ => {}
    }
    peers[p].doc.commit();
}

fn run(seed: u64, steps: usize) -> Option<Vec<String>> {
    let mut rng = StdRng::seed_from_u64(seed);
    let n = rng.gen_range(3..=5);
    let mut peers: Vec<Peer> = (0..n)
        .map(|i| { let d = LoroDoc::new(); d.set_peer_id(i as u64 + 1).unwrap(); new_peer(d, false) })
        .collect();
    let mut log = vec![format!("seed {seed} peers {n}")];
    for i in 0..steps {
        step(&mut rng, &mut peers, i, &mut log);
    }
    let mut bad = false;
    for (i, p) in peers.iter().enumerate() {
        p.doc.commit();
        if p.shallow || p.doc.is_shallow() { continue; }
        let replay = LoroDoc::new();
        replay.import(&p.doc.export(ExportMode::all_updates()).unwrap()).unwrap();
        if replay.get_deep_value() != p.doc.get_deep_value() {
            log.push(format!("MISMATCH peer {i}: state={} replay={}",
                p.doc.get_deep_value().to_json_value(), replay.get_deep_value().to_json_value()));
            bad = true;
        }
    }
    bad.then_some(log)
}

fn main() {
    let args: Vec<String> = std::env::args().collect();
    let (from, to, steps): (u64, u64, usize) = (args[1].parse().unwrap(), args[2].parse().unwrap(), args[3].parse().unwrap());
    let mut found = 0;
    for seed in from..to {
        let r = std::panic::catch_unwind(|| run(seed, steps));
        match r {
            Ok(Some(log)) => { found += 1; println!("FOUND seed {seed}"); std::fs::write(format!("/tmp/m1125-replay-{seed}.log"), log.join("\n")).unwrap(); }
            Ok(None) => {}
            Err(_) => println!("PANIC seed {seed}"),
        }
    }
    println!("done found={found}");
}

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions