Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,8 @@ add_library(
protocol_storage
STATIC
src/storage/sqlite_connection.cpp
src/storage/sqlite_block_store_v1.cpp
src/storage/sqlite_history_replay_v1.cpp
src/storage/sqlite_ledger.cpp
src/storage/sqlite_schema_v1.cpp
)
Expand Down Expand Up @@ -233,12 +235,22 @@ add_executable(
storage_sqlite_ledger_tests
tests/storage/sqlite_ledger_test.cpp
)
add_executable(
storage_sqlite_history_tests
tests/storage/sqlite_history_test.cpp
)
target_link_libraries(
storage_sqlite_ledger_tests
PRIVATE
protocol_storage
protocol_stack_sqlite
)
target_link_libraries(
storage_sqlite_history_tests
PRIVATE
protocol_storage
protocol_stack_sqlite
)

set(
PROTOCOL_STACK_TARGETS
Expand All @@ -257,6 +269,7 @@ set(
kernel_differential_runner
sqlite_dependency_tests
storage_sqlite_ledger_tests
storage_sqlite_history_tests
)
if(PROTOCOL_STACK_ENABLE_FUZZING)
add_library(
Expand Down Expand Up @@ -443,6 +456,13 @@ add_test(
"${PROJECT_SOURCE_DIR}/test-vectors/ledger-transition-v1.txt"
"${CMAKE_CURRENT_BINARY_DIR}/storage-sqlite-ledger-test"
)
add_test(
NAME storage-sqlite-history
COMMAND
storage_sqlite_history_tests
"${PROJECT_SOURCE_DIR}/test-vectors/ledger-transition-v1.txt"
"${CMAKE_CURRENT_BINARY_DIR}/storage-sqlite-history-test"
)
set_tests_properties(
protocol-primitives-python
ledger-transition-python
Expand Down
44 changes: 31 additions & 13 deletions docs/architecture/sqlite-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,29 @@ remain authoritative.

## Implemented outcome

`create_sqlite_ledger` now creates a brand-new database at height zero and
commits the caller-configured canonical genesis state. `open_sqlite_ledger`
reopens only that closed-world height-zero form. It rejects any database with
block, admitted-transaction, or snapshot rows until replay and recovery are
implemented.
`create_sqlite_ledger` creates a brand-new database at height zero and commits
the caller-configured canonical genesis state. `SQLiteLedger::apply_block`
copy-constructs an independent kernel candidate, applies the complete ordered
raw-input block, persists the resulting materialized state and canonical
history in one transaction, commits, and publishes the candidate through a
non-throwing pointer swap. A kernel block rejection never starts a storage
transaction.

`open_sqlite_ledger` performs full genesis replay of every retained block
before publishing a live ledger. It admits only the stored 200-byte journal
rows, in explicit height and ordinal order, and compares every replayed
transaction ID, receipt, root, application header, and block ID with the
stored canonical output. The replay head must then exactly equal metadata,
materialized accounts, the fee pool, and the public ledger root. Snapshot rows
remain refused until independent snapshot recovery is implemented.

The public header exposes no SQLite handle or SQL type. `SQLiteLedger` is
move-constructible but not copyable or assignable. It owns the live
`protocol::v1::Ledger`, serialized connection, normalized path, exact canonical
genesis bytes, and cached state root. `read_head` returns owned state and root
values while holding the adapter mutex; callers receive no borrowed view.
`apply_block` returns one of the exact kernel `BlockCommit`, the deterministic
kernel `BlockError`, or an operational `SQLiteLedgerError`.

## Trusted creation input

Expand Down Expand Up @@ -76,10 +88,17 @@ rowid, and foreign-key metadata; unknown or modified objects are refused.

Opening requires a single successful `integrity_check` result and an empty
`foreign_key_check`. It exact-compares the persisted canonical genesis with
the independently trusted caller value, bounds account loading by the trusted
genesis account count, reconstructs an owned state, and calls
the independently trusted caller value, replays contiguous block and admitted
transaction rows from genesis, bounds materialized account loading by the
verified replay account count, reconstructs an owned state, and calls
`restore_ledger` with caller-derived immutable parameters and the stored root.
The result must exactly equal the independently loaded genesis state and root.
The result must exactly equal the independently replayed state and root.

Admission failures are absent from the journal. Empty and entirely unadmitted
blocks still have a block row and advance height. Duplicate admitted
transactions retain separate contiguous ordinals. Database projections never
replace canonical transaction, receipt, header, or identifier bytes as the
replay authority.

## Error and lifetime behavior

Expand All @@ -96,8 +115,7 @@ before the completed adapter is returned.

## Remaining issue 11 work

The current boundary deliberately cannot apply or replay a block. The next
vertical result is one atomic durable block commit followed by validated clean
reopen and full genesis replay. Snapshot recovery, portable export/import,
fault injection around commit phases, long restart sequences, and final issue
closure follow that working block path.
The ordinary durable commit and full-genesis-replay path is implemented.
Snapshot recovery, portable export/import, automatic reopen after an ambiguous
commit result, fault injection around every commit phase, long seeded restart
sequences, and final issue closure remain.
40 changes: 30 additions & 10 deletions docs/project/current-state.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ Last updated: 2026-07-24

M1 — Sovereign Devnet Alpha. The deterministic in-memory ledger kernel is
merged and verified. The owning storage adapter can now durably create and
validate a height-zero ledger. Atomic block persistence, replay, snapshots,
and recovery remain the active roadmap slice.
validate a height-zero ledger, atomically persist a complete block, and reopen
the identical head through full genesis replay. Multi-block restart coverage,
snapshots, fault injection, and recovery remain the active roadmap slice.

## Verified facts

Expand Down Expand Up @@ -171,9 +172,17 @@ and recovery remain the active roadmap slice.
Reopening never creates or changes journal mode and publishes no ledger
unless integrity, foreign keys, exact schema, caller-trusted genesis,
materialized state, and root all agree.
- The current adapter is deliberately closed at height zero. Any block,
admitted-transaction, or snapshot row is rejected until complete replay and
recovery validation are implemented.
- `SQLiteLedger::apply_block` applies ordered raw inputs to an independent
ledger candidate, writes changed and created accounts, exact admitted
transaction bytes and kernel outputs, the block row, and head metadata in
one SQLite transaction, durably commits, and publishes through a
non-throwing owning-pointer swap. Kernel block rejection never opens a
storage transaction.
- Opening validates caller-trusted genesis before history, then full-replays
contiguous block and admitted-transaction rows in explicit height and
ordinal order. Every replayed transaction ID, receipt, root, application
header, and block ID must equal storage before the replay head is compared
exactly with materialized state and metadata. Snapshot rows remain refused.
- The owner prefers one active delivery branch, cleanup of obsolete
branches/worktrees/build trees at phase boundaries, focused checks while
iterating, one required completion matrix, and runnable vertical outcomes
Expand Down Expand Up @@ -203,16 +212,27 @@ and recovery remain the active roadmap slice.
wrong genesis, materialized-state and root corruption, immutable-fee
projection corruption, foreign-key damage, truncated files, and refusal of
unvalidated history.
- The frozen 15-input ledger block now passes through durable storage with the
exact kernel `BlockCommit`, 11 admitted journal rows, three omitted
admission failures, unchanged head on a rejected repeated height, clean
close, full genesis replay, and an identical owned head.
- Replay rejection coverage proves wrong-genesis precedence at nonzero height,
missing admitted ordinals, altered transaction and block identifiers, and
materialized state divergence are refused before publication.
- Clean completion verification passes 16/16 CTest tests in GCC debug, GCC
ASan+UBSan, and Clang debug, and 19/19 in Clang ASan+UBSan including the
three existing kernel fuzz smoke tests. Leak detection is disabled only for
sanitizer completion runs because the managed execution sandbox traces
processes and LeakSanitizer refuses to run under `ptrace`.

## Exact next action

Continue issue #11:

> Implement one end-to-end durable block: apply ordered raw inputs to an
> independent ledger candidate, atomically persist materialized state, exact
> admitted bytes and kernel outputs, commit, publish with a non-throwing
> ownership transfer, then cleanly reopen by full genesis replay to the
> identical head.
> Add a deterministic multi-block and repeated-restart storage harness covering
> empty and entirely unadmitted blocks, duplicate admitted transactions,
> continued commits from reopened heads, exact replay after each restart, and
> materialized-state agreement throughout.

## Open autonomous decisions

Expand Down
10 changes: 9 additions & 1 deletion include/protocol/storage/sqlite_ledger.hpp
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#pragma once

#include "protocol/v1/types.hpp"
#include "protocol/v1/ledger.hpp"

#include <cstdint>
#include <filesystem>
Expand Down Expand Up @@ -33,6 +33,11 @@ struct LedgerHead {

struct SQLiteLedgerResult;

using SQLiteBlockResult = std::variant<
protocol::v1::BlockCommit,
protocol::v1::BlockError,
SQLiteLedgerError>;

class SQLiteLedger {
public:
~SQLiteLedger() noexcept;
Expand All @@ -43,6 +48,9 @@ class SQLiteLedger {
SQLiteLedger& operator=(SQLiteLedger&&) = delete;

LedgerHead read_head() const;
SQLiteBlockResult apply_block(
std::uint64_t height,
std::span<const protocol::v1::Bytes> raw_transactions);

private:
struct Impl;
Expand Down
Loading
Loading