crateline

A small append-only record store with staged commits.

A store is one directory. Records are written through a transaction that stages its parts privately, drains them into a write-ahead log, and only then publishes a commit marker. Opening a store sweeps whatever a previous writer left behind, so a directory abandoned mid-commit is reconciled before the caller sees it.

from crateline import open_store

store = open_store("/var/lib/example")
txn = store.begin()
txn.stage(b"first")
store.commit(txn)
assert store.records() == (b"first",)

Layout

Module	Responsibility
crateline/__init__.py	The public open_store / Store facade.
crateline/codec.py	Frames a record: magic, epoch stamp, length, CRC-32.
crateline/wal.py	Numbered segment files; append and durable-prefix replay.
crateline/txn.py	Staged transactions and the commit marker protocol.
crateline/sweep.py	Startup reclamation of staging directories.
crateline/compat.py	Reads the retired CRL0 segment format.

Durability rules

• A record is durable once its frame is in a segment and a commit marker vouches for the transaction that wrote it. The marker, not the presence of frames, is the authority on whether a transaction committed.
• Replay stops at the first torn frame. A writer that died mid-append leaves a partial frame, and everything before it is still readable.
• Reclaiming a staging directory can only lose records that were never committed. Sweep never touches segment files.

Tests

python -m pytest -q

Every test is filesystem-only and uses a temporary directory; nothing reaches the network or depends on wall-clock time.