From 3837cf1a04f3b649fb386745fdf5133375ca8131 Mon Sep 17 00:00:00 2001 From: jvaught01 Date: Fri, 10 Jul 2026 23:10:02 -0700 Subject: [PATCH 1/3] Data viewer: read-only db.inspect() + a /viewer route over the demo store MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds db.inspect(version?) to the core surface (Design §13): a read-only, format-agnostic projection of the otherwise-private manifest — version, committedAt, formatVersion, and per-table chunks with row counts and primary-key/partition zone-map min/max. Deliberately a derived shape, not the raw Manifest, so on-store format evolution can't break the contract. Reads manifest metadata only (lighter than export()), returns no blob URLs, and resolves any retained past version exactly as asOf does. Builds the read-only web UI from the §14 roadmap in the test-lab app: - app/api/inspect + app/api/viewer-rows (paged rows with asOf time travel and pruned-vs-scanned stats; table/sort identifiers whitelisted against the code-first schema, so no user values reach the SQL string) - app/viewer: table browser (paged, click-to-sort), a version scrubber (time travel), and a chunk/zone-map internals panel; linked from the home header Verified: tsc clean, lint clean, next build green, api-smoke 35/35 against the real Blob store (5 new inspect assertions), and the routes exercised live against the demo store incl. past-version time travel and the injection guard. Co-Authored-By: Claude Opus 4.8 (1M context) --- LARVA-DESIGN.md | 5 +- app/api/inspect/route.ts | 34 +++ app/api/viewer-rows/route.ts | 73 ++++++ app/page.tsx | 6 + app/viewer/page.tsx | 417 ++++++++++++++++++++++++++++++++++ packages/larvadb/src/db.ts | 71 ++++++ packages/larvadb/src/index.ts | 11 +- scripts/api-smoke.ts | 20 ++ 8 files changed, 635 insertions(+), 2 deletions(-) create mode 100644 app/api/inspect/route.ts create mode 100644 app/api/viewer-rows/route.ts create mode 100644 app/viewer/page.tsx diff --git a/LARVA-DESIGN.md b/LARVA-DESIGN.md index 241f3e5..df20dfe 100644 --- a/LARVA-DESIGN.md +++ b/LARVA-DESIGN.md @@ -294,8 +294,11 @@ await db.rollbackTo(version); // recovery in one line await db.export({ format: "sqlite" }); // the escape hatch await db.vacuum(); // reclaim storage outside retention await db.upgrade(); // one-way flip to format 3 (the commit log) +await db.inspect(version?); // read-only physical layout: chunks + zone maps ``` +`inspect()` is the one read-only accessor on the surface, and it earns its place the same way `export()` does: it is a *derived*, format-agnostic projection of the manifest (version, `committedAt`, `formatVersion`, and per-table chunk lists with each chunk's row count and primary-key/partition zone-map min-max), never the raw `Manifest` — so on-store format evolution cannot break its contract. It reads manifest metadata only (strictly lighter than `export`, which also reads every chunk), returns no blob URLs (the private-data invariant holds), and resolves any retained past version exactly as `asOf` does. It exists so a store can be *seen* — it is the primitive under the data viewer (§14) and any store-debugging tool — and it is the deliberate exception to "resist growing the surface": one read-only method, no new write path. + The same surface ships as a shell command — the package's `larva` bin (`npx larva sql | export | upgrade | rollback | vacuum | version`, credentials auto-loaded from `.env.local`). One surface, two doors; the CLI is validated end to end by `scripts/cli-smoke.ts`. ## 14. Roadmap and open questions @@ -309,7 +312,7 @@ v1 ships: the storage engine and commit protocol, the SQL subset of Section 7, c **The SQL dialect roadmap (decided July 2026) — complete.** Four former exclusions, admitted or re-judged against the Section 7 bar. Three shipped in 2.5: additive `ALTER TABLE … ADD COLUMN` (schema-only commit; absent keys read as `NULL` at every read path; additive schema drift auto-migrates at connect — Section 8; renames and drops still wait for a migration design that respects time travel), joins of three or more tables and self-joins (left-deep hash joins in the existing in-memory algebra; distinct alias per occurrence), and uncorrelated subqueries (inner-query-first evaluation rewriting the node into literals, exactly as planned — the result feeds the outer plan and `IN` lists participate in zone-map pruning; correlated subqueries stay out, per Section 7). The fourth, **secondary index blobs**, shipped in 2.6 exactly as sketched — an index is just another immutable blob committed in the same atomic swap, and keying entries by immutable chunk id made it safe without a format bump (Section 8). -Still deferred, unordered: a columnar chunk format; S3/R2/GCS/Azure adapters (the S3 contract harness already exists); a tiny read-only web UI for browsing tables (non-engineers love to *see* their data); an optional Edge Config read-accelerator for flag-shaped tables. +Still deferred, unordered: a columnar chunk format; S3/R2/GCS/Azure adapters (the S3 contract harness already exists); an optional Edge Config read-accelerator for flag-shaped tables. The **read-only web UI** for browsing tables (non-engineers love to *see* their data) is underway in the test-lab app: a `/viewer` route over the demo store that pairs a paged table browser with a version scrubber (time travel via `asOf`) and a chunk/zone-map internals panel — the last of these is why `db.inspect()` (§13) was added, since a store's physical layout lives in the otherwise-private manifest. **The v2 write path.** The v1 ceiling — every logical commit is one CAS on one global manifest — is a protocol choice, not an object-storage limit, and there is a ladder of known techniques for raising it if multi-process operational workloads (several deployments writing steadily, unique-constraint-bearing inserts, sequence assignment) become a target. In rough order of leverage per cost: **sequence range leasing** (shipped — `t.sequence()`, Section 8); **composite unique constraints** (shipped — `defineSchema` `uniques`, Section 8); **an ordered commit log** (shipped — format 3, Section 6: create-only numbered entries, checkpointed manifest, cheap rebases, explicit `upgrade()`); **lease-elected cross-process group commit** (committed — see above; object-storage CAS doubles as a lease primitive; the leaseholder batches every process's transactions into shared log slots — group commit across processes with no server anyone deploys, safe even under split leadership because the log stays the arbiter; needs the intent queues below as its transport, so the two ship together); **a two-tier write API** (committed — see above; constraint-free appends go to per-writer intent prefixes — zero contention, one-PUT-RTT durability, an LSM memtable made of blobs folded in by a lease-elected compactor; constraint-bearing writes go through the ordered log and wait for a deterministic slot verdict, Calvin-style); and **sharded contention domains** (a two-level manifest so unrelated table groups stop sharing one CAS). Each rung ships independently; the format-versioning guard (Section 5) is what makes adopting any of them a loud, upgradeable break instead of silent corruption. The physics no rung changes: sync-commit latency floors at one PUT round-trip, because durability cannot be faked. diff --git a/app/api/inspect/route.ts b/app/api/inspect/route.ts new file mode 100644 index 0000000..322f931 --- /dev/null +++ b/app/api/inspect/route.ts @@ -0,0 +1,34 @@ +import { NextRequest, NextResponse } from "next/server"; +import { SqlError } from "@larva-db/core"; +import { demoDb } from "@/app/lib/demo"; + +export const maxDuration = 60; + +/** + * Read-only physical layout of the demo store at one version (db.inspect, + * Design §13): per-table chunk lists with zone-map min/max. Powers the + * viewer's internals panel and version scrubber. `?version=N` time-travels; + * `currentVersion` is always returned so the scrubber knows its upper bound. + */ +export async function GET(req: NextRequest) { + const versionParam = req.nextUrl.searchParams.get("version"); + try { + const db = await demoDb(); + let version: number | undefined; + if (versionParam !== null) { + version = Number(versionParam); + if (!Number.isInteger(version) || version < 0) { + return NextResponse.json({ error: { code: "BAD_VERSION", message: "version must be a non-negative integer" } }, { status: 400 }); + } + } + const currentVersion = await db.currentVersion(); + const layout = await db.inspect(version); + return NextResponse.json({ ...layout, currentVersion }); + } catch (err) { + if (err instanceof SqlError) { + const status = err.code === "VERSION_NOT_FOUND" ? 404 : 400; + return NextResponse.json({ error: { code: err.code, message: err.message } }, { status }); + } + return NextResponse.json({ error: { code: "INTERNAL", message: err instanceof Error ? err.message : String(err) } }, { status: 500 }); + } +} diff --git a/app/api/viewer-rows/route.ts b/app/api/viewer-rows/route.ts new file mode 100644 index 0000000..2a03aea --- /dev/null +++ b/app/api/viewer-rows/route.ts @@ -0,0 +1,73 @@ +import { NextRequest, NextResponse } from "next/server"; +import { SqlError } from "@larva-db/core"; +import { demoDb, demoSchema } from "@/app/lib/demo"; + +export const maxDuration = 60; +const DEFAULT_PAGE = 50; +const MAX_PAGE = 200; + +/** + * A page of rows from one demo table, optionally at a past version (time + * travel via asOf). Read-only. Table and sort column are whitelisted against + * the code-first schema, and page bounds are coerced to integers, so the only + * thing interpolated into SQL are validated identifiers — no user values reach + * the query string (and the parser rejects statement stacking regardless). + * Returns the fetch's pruning stats (chunks scanned vs. total) so the UI can + * show the zone maps doing their job. + */ +export async function GET(req: NextRequest) { + const p = req.nextUrl.searchParams; + const table = p.get("table") ?? ""; + const spec = demoSchema[table]; + if (!spec) { + return NextResponse.json( + { error: { code: "UNKNOWN_TABLE", message: `pick a table: ${Object.keys(demoSchema).join(", ")}` } }, + { status: 400 }, + ); + } + const columns = Object.keys(spec.columns); + const orderByParam = p.get("orderBy"); + const orderBy = orderByParam && columns.includes(orderByParam) ? orderByParam : spec.primaryKey; + const dir = p.get("dir") === "desc" ? "DESC" : "ASC"; + const limit = Math.min(MAX_PAGE, Math.max(1, Math.trunc(Number(p.get("limit"))) || DEFAULT_PAGE)); + const offset = Math.max(0, Math.trunc(Number(p.get("offset"))) || 0); + const versionParam = p.get("version"); + + try { + const db = await demoDb(); + const currentVersion = await db.currentVersion(); + const wantsPast = versionParam !== null && Number(versionParam) !== currentVersion; + if (versionParam !== null && !Number.isInteger(Number(versionParam))) { + return NextResponse.json({ error: { code: "BAD_VERSION", message: "version must be an integer" } }, { status: 400 }); + } + const version = wantsPast ? Number(versionParam) : currentVersion; + const src = wantsPast ? await db.asOf(version) : db; + + const rows = await src.query(`SELECT * FROM ${table} ORDER BY ${orderBy} ${dir} LIMIT ${limit} OFFSET ${offset}`); + const stats = db.lastQueryStats; // capture before the COUNT query overwrites it + const [{ n }] = await src.query<{ n: number }>(`SELECT COUNT(*) AS n FROM ${table}`); + + return NextResponse.json({ + table, + columns, + primaryKey: spec.primaryKey, + partitionColumn: spec.partitionColumn ?? null, + types: Object.fromEntries(columns.map((c) => [c, spec.columns[c].type])), + rows, + total: Number(n), + limit, + offset, + orderBy, + dir: dir.toLowerCase(), + version, + isCurrent: !wantsPast, + stats, + }); + } catch (err) { + if (err instanceof SqlError) { + const status = err.code === "VERSION_NOT_FOUND" ? 404 : 400; + return NextResponse.json({ error: { code: err.code, message: err.message } }, { status }); + } + return NextResponse.json({ error: { code: "INTERNAL", message: err instanceof Error ? err.message : String(err) } }, { status: 500 }); + } +} diff --git a/app/page.tsx b/app/page.tsx index 4abf177..7da1565 100644 --- a/app/page.tsx +++ b/app/page.tsx @@ -57,6 +57,12 @@ export default function Home() {

Larva / test lab

+ + data viewer + ; +} +interface RowsResponse { + table: string; + columns: string[]; + primaryKey: string; + partitionColumn: string | null; + types: Record; + rows: Record[]; + total: number; + limit: number; + offset: number; + orderBy: string; + dir: "asc" | "desc"; + version: number; + isCurrent: boolean; + stats: { chunksTotal: number; chunksFetched: number }; +} +interface ErrBody { + code: string; + message: string; +} + +const PAGE = 50; + +export default function Viewer() { + const [layout, setLayout] = useState(null); + const [head, setHead] = useState(null); // live current version + const [version, setVersion] = useState(null); // committed selection + const [scrub, setScrub] = useState(null); // live slider position + const [table, setTable] = useState(null); + const [data, setData] = useState(null); + const [offset, setOffset] = useState(0); + const [orderBy, setOrderBy] = useState(null); + const [dir, setDir] = useState<"asc" | "desc">("asc"); + const [error, setError] = useState(null); + const [loadingRows, setLoadingRows] = useState(false); + + // --- inspect: layout for the selected version (and, on first load, the head) --- + useEffect(() => { + let cancelled = false; + (async () => { + const res = await fetch(`/api/inspect${version === null ? "" : `?version=${version}`}`); + const body = (await res.json()) as Inspection | { error: ErrBody }; + if (cancelled) return; + if ("error" in body) { + setError(body.error); + return; + } + setError(null); + setLayout(body); + setHead(body.currentVersion); + setVersion((v) => (v === null ? body.version : v)); + setScrub((s) => (s === null ? body.version : s)); + setTable((t) => (t === null ? Object.keys(body.tables)[0] ?? null : t)); + })(); + return () => { + cancelled = true; + }; + }, [version]); + + // --- rows: a page of the selected table at the selected version --- + useEffect(() => { + if (table === null || version === null) return; + let cancelled = false; + (async () => { + setLoadingRows(true); + const params = new URLSearchParams({ table, version: String(version), limit: String(PAGE), offset: String(offset) }); + if (orderBy) params.set("orderBy", orderBy); + params.set("dir", dir); + try { + const res = await fetch(`/api/viewer-rows?${params}`); + const body = (await res.json()) as RowsResponse | { error: ErrBody }; + if (cancelled) return; + if ("error" in body) { + setError(body.error); + setData(null); + } else { + setError(null); + setData(body); + } + } finally { + if (!cancelled) setLoadingRows(false); + } + })(); + return () => { + cancelled = true; + }; + }, [table, version, offset, orderBy, dir]); + + const commitScrub = () => { + if (scrub !== null && scrub !== version) { + setVersion(scrub); + setOffset(0); + } + }; + const toLive = () => { + if (head !== null) { + setScrub(head); + setVersion(head); + setOffset(0); + } + }; + const pickTable = (t: string) => { + setTable(t); + setOffset(0); + setOrderBy(null); + setDir("asc"); + }; + const sortBy = (col: string) => { + if (orderBy === col || (orderBy === null && data?.primaryKey === col)) { + setDir((d) => (d === "asc" ? "desc" : "asc")); + } else { + setOrderBy(col); + setDir("asc"); + } + setOffset(0); + }; + + const isPast = layout !== null && head !== null && layout.version !== head; + const tableNames = layout ? Object.keys(layout.tables) : []; + const tv = layout && table ? layout.tables[table] : null; + + return ( +
+
+
+

+ Larva / data viewer +

+ + stress lab + + + docs + +
+

+ Browse the live demo store, scrub back through every commit (time travel is a byproduct + of the architecture — old manifests are complete snapshots), and watch the chunk zone + maps that make queries prune. Read-only: nothing here can write. +

+
+ + {error && ( +
+

+ {error.code} + {error.message} +

+
+ )} + + {/* --- version scrubber --- */} + {layout && head !== null && ( +
+
+ + version {scrub ?? layout.version} + / {head} + + {isPast ? ( + + time-travelling — {new Date(layout.committedAt).toLocaleString()} + + ) : ( + ● live head · {new Date(layout.committedAt).toLocaleString()} + )} + format {layout.formatVersion} · commit log +
+
+ v0 + setScrub(Number(e.target.value))} + onPointerUp={commitScrub} + onKeyUp={commitScrub} + className="accent-series-1 h-1 grow cursor-pointer" + aria-label="database version" + /> + v{head} + +
+
+ )} + + {/* --- table tabs --- */} + {tableNames.length > 0 && ( +
+ {tableNames.map((t) => { + const active = t === table; + return ( + + ); + })} +
+ )} + + {/* --- rows grid --- */} + {data && ( +
+
+ + rows{" "} + + {data.total === 0 ? 0 : data.offset + 1}–{Math.min(data.offset + data.limit, data.total)} + {" "} + of {data.total} + + + scanned {data.stats.chunksFetched} of{" "} + {data.stats.chunksTotal} chunk + {data.stats.chunksTotal === 1 ? "" : "s"} + {data.stats.chunksFetched < data.stats.chunksTotal && " — zone maps pruned the rest"} + + {loadingRows && loading…} + + ordered by {data.orderBy} {data.dir} + +
+
+ + + + {data.columns.map((c) => ( + + ))} + + + + {data.rows.map((row, i) => ( + + {data.columns.map((c) => ( + + ))} + + ))} + {data.rows.length === 0 && ( + + + + )} + +
+ +
+ +
+ no rows at this version +
+
+ {data.total > data.limit && ( +
+ + + page {Math.floor(data.offset / data.limit) + 1} of {Math.max(1, Math.ceil(data.total / data.limit))} + + +
+ )} +
+ )} + + {/* --- chunk / zone-map internals --- */} + {tv && table && ( +
+
+

chunks & zone maps

+ + {tv.chunkCount} chunk{tv.chunkCount === 1 ? "" : "s"} · {tv.rowCount} rows · {table} @ v{layout?.version} + +
+

+ Every chunk is an immutable blob. The manifest stores each one's row count and the + min/max of the primary key{data?.partitionColumn ? ` and the partition column (${data.partitionColumn})` : ""} — + the zone maps a query uses to skip chunks it can't match. +

+
+ + + + + + + + + + + {tv.chunks.map((ch) => ( + + + + + + + ))} + {tv.chunks.length === 0 && ( + + + + )} + +
chunkrows{data?.primaryKey ?? "pk"} min → max + {data?.partitionColumn ? `${data.partitionColumn} min → max` : "partition"} +
+ {ch.id.slice(0, 8)}… + {ch.rows} + {ch.pk ? : —} + + {ch.partition ? : —} +
+ no chunks — this table is empty at v{layout?.version} +
+
+
+ )} + +
+ Reads only. Rows come from db.query (and db.asOf for past versions); + the chunk layout from db.inspect — a read-only projection of the manifest. + Both hit the same demo store as the console and stress lab. +
+
+ ); +} + +function Cell({ value }: { value: Scalar }) { + if (value === null || value === undefined) return NULL; + if (typeof value === "boolean") return {value ? "true" : "false"}; + if (typeof value === "number") return {value}; + return <>{String(value)}; +} + +function cellTitle(value: Scalar): string { + if (value === null || value === undefined) return "NULL"; + return String(value); +} + +function ZoneRange({ min, max }: { min: Scalar; max: Scalar | null }) { + const fmt = (v: Scalar) => (v === null || v === undefined ? "∅" : String(v)); + const a = fmt(min); + const b = fmt(max); + return ( + + {trunc(a)} → {trunc(b)} + + ); +} + +function trunc(s: string): string { + return s.length > 22 ? `${s.slice(0, 10)}…${s.slice(-6)}` : s; +} diff --git a/packages/larvadb/src/db.ts b/packages/larvadb/src/db.ts index 774fbeb..c0cba7e 100644 --- a/packages/larvadb/src/db.ts +++ b/packages/larvadb/src/db.ts @@ -16,6 +16,34 @@ interface TxState { applies: PlanOutcome["apply"][]; } +/** One chunk's physical stats, as surfaced by db.inspect() — a stable + * projection of the manifest's zone map (never the raw ChunkRef). */ +export interface ChunkInspection { + /** ULID chunk id (also its blob pathname stem). Not a secret; not a URL. */ + id: string; + rows: number; + /** Primary-key zone map: the min/max PK in this chunk (drives pruning). */ + pk: { min: Scalar; max: Scalar } | null; + /** Partition-column zone map, when the table declares a partitionBy() column. */ + partition: { min: Scalar; max: Scalar | null } | null; +} + +export interface TableInspection { + rowCount: number; + chunkCount: number; + chunks: ChunkInspection[]; +} + +/** Read-only view of a store's physical layout at one version (db.inspect()). */ +export interface DbInspection { + version: number; + committedAt: string; + formatVersion: number; + /** True when this is the live head, false for a time-travelled past version. */ + isCurrent: boolean; + tables: Record; +} + /** * Statement handle inside db.transaction(). Reads see the transaction's own * uncommitted writes (a virtual manifest); nothing touches the live database @@ -951,6 +979,49 @@ export class LarvaDb { return (await this.proto.snapshot()).manifest.version; } + /** + * Read-only introspection of the store's physical layout at one version + * (Design §13). Projects the manifest — normally private — into a stable, + * format-agnostic view: the schema-declared shape, per-table chunk lists, and + * each chunk's row count and zone-map min/max. This is deliberately a *derived* + * shape, not the raw Manifest, so on-store format evolution never breaks the + * contract. Powers the data viewer (§14) and any store-debugging tool; it is + * strictly lighter than export(), reading manifest metadata and never a chunk. + * + * With no argument, the current version; with a version number, any retained + * past version (same resolution as asOf — throws VERSION_NOT_FOUND if pruned). + */ + async inspect(version?: number): Promise { + await this.ensureReady(); + const current = await this.proto.snapshot(); + let manifest: Manifest | null; + if (version === undefined || version === current.manifest.version) { + manifest = current.manifest; + } else { + manifest = await this.proto.manifestAt(version); + if (!manifest) { + throw new SqlError("VERSION_NOT_FOUND", `version ${version} is not in retained history (current version: ${current.manifest.version})`); + } + } + const tables: Record = {}; + for (const [name, table] of Object.entries(manifest.tables)) { + const chunks = table.chunks.map((c) => ({ + id: c.id, + rows: c.rows, + pk: c.stats ? { min: c.stats.pkMin, max: c.stats.pkMax } : null, + partition: c.stats && c.stats.partMin !== undefined ? { min: c.stats.partMin, max: c.stats.partMax ?? null } : null, + })); + tables[name] = { rowCount: chunks.reduce((n, c) => n + c.rows, 0), chunkCount: chunks.length, chunks }; + } + return { + version: manifest.version, + committedAt: manifest.committedAt, + formatVersion: manifest.formatVersion ?? 1, + isCurrent: manifest.version === current.manifest.version, + tables, + }; + } + /** Delete the entire database (test helper; not part of the Design §13 surface). */ async destroy(): Promise { await this.proto.destroy(); diff --git a/packages/larvadb/src/index.ts b/packages/larvadb/src/index.ts index ab80785..be2cc50 100644 --- a/packages/larvadb/src/index.ts +++ b/packages/larvadb/src/index.ts @@ -1,5 +1,14 @@ /** The larvadb public surface (Design §13). */ -export { larva, LarvaDb, LarvaSnapshot, LarvaTx, type LarvaOptions } from "./db"; +export { + larva, + LarvaDb, + LarvaSnapshot, + LarvaTx, + type LarvaOptions, + type DbInspection, + type TableInspection, + type ChunkInspection, +} from "./db"; export { defineSchema, t, diff --git a/scripts/api-smoke.ts b/scripts/api-smoke.ts index da48c1b..01f19be 100644 --- a/scripts/api-smoke.ts +++ b/scripts/api-smoke.ts @@ -94,6 +94,26 @@ ok( `v=${vAfter}, bounds=(${v2}, ${v2 + WRITERS * TXS}]`, ); +// --- inspect: read-only physical layout (§13) --- +const layout = await db.inspect(); +ok("inspect reports the current version and format", layout.isCurrent && layout.version === (await db.currentVersion()) && layout.formatVersion >= 1, JSON.stringify({ v: layout.version, f: layout.formatVersion })); +const invLayout = layout.tables.inventory; +ok( + "inspect: per-table rowCount matches chunk row sums and the live table", + invLayout.rowCount === invLayout.chunks.reduce((n, c) => n + c.rows, 0) && + invLayout.rowCount === ((await db.sql`SELECT COUNT(*) AS n FROM inventory`)[0].n as number), + JSON.stringify(invLayout), +); +ok("inspect: chunks carry primary-key zone maps", invLayout.chunkCount > 0 && invLayout.chunks.every((c) => c.pk !== null), JSON.stringify(invLayout.chunks)); +const pastLayout = await db.inspect(v0); +ok("inspect(version) time-travels to a past layout", pastLayout.version === v0 && !pastLayout.isCurrent, JSON.stringify({ v: pastLayout.version, cur: pastLayout.isCurrent })); +try { + await db.inspect(999_999); + ok("inspect on an unretained version fails loudly", false); +} catch (err) { + ok("inspect on an unretained version fails loudly", (err as SqlError).code === "VERSION_NOT_FOUND", (err as Error).message); +} + // --- export --- const json = await db.export({ format: "json" }); ok("json export has both tables", json.inventory?.length === 2 && json.orders?.length === 1); From 6ec24405e1e814d0edf342a6bbb8c6021cdbdab8 Mon Sep 17 00:00:00 2001 From: jvaught01 Date: Fri, 10 Jul 2026 23:49:12 -0700 Subject: [PATCH 2/3] Viewer: React Query data fetching + persistent table structure with skeletons MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fetch the viewer's data with @tanstack/react-query (provider scoped to the /viewer route via a route layout). Both queries use placeholderData: keepPreviousData so the table never collapses during a fetch. The table structure is now always on screen: /api/inspect returns each table's schema (columns/types/pk/partition, via a shared app/lib/viewer helper), so the client draws headers and tabs from the schema before any row loads. Row cells and chunk rows fall back to skeleton loaders while the page/version/table they belong to is in flight — pagination keeps the previous page (keepPreviousData), while a table or version switch shows skeletons under the correct new headers (guarded by matching table+version, so stale rows never paint under new columns). Co-Authored-By: Claude Opus 4.8 (1M context) --- app/api/inspect/route.ts | 7 +- app/api/viewer-rows/route.ts | 6 +- app/lib/viewer.ts | 28 ++ app/viewer/layout.tsx | 5 + app/viewer/page.tsx | 585 +++++++++++++++++++---------------- app/viewer/providers.tsx | 27 ++ bun.lock | 7 +- package.json | 1 + 8 files changed, 399 insertions(+), 267 deletions(-) create mode 100644 app/lib/viewer.ts create mode 100644 app/viewer/layout.tsx create mode 100644 app/viewer/providers.tsx diff --git a/app/api/inspect/route.ts b/app/api/inspect/route.ts index 322f931..3f5dc1d 100644 --- a/app/api/inspect/route.ts +++ b/app/api/inspect/route.ts @@ -1,6 +1,7 @@ import { NextRequest, NextResponse } from "next/server"; import { SqlError } from "@larva-db/core"; import { demoDb } from "@/app/lib/demo"; +import { schemaMeta } from "@/app/lib/viewer"; export const maxDuration = 60; @@ -8,7 +9,9 @@ export const maxDuration = 60; * Read-only physical layout of the demo store at one version (db.inspect, * Design §13): per-table chunk lists with zone-map min/max. Powers the * viewer's internals panel and version scrubber. `?version=N` time-travels; - * `currentVersion` is always returned so the scrubber knows its upper bound. + * `currentVersion` is always returned so the scrubber knows its upper bound, + * and `schema` (every table's columns) so the client can draw the table + * structure before a single row loads. */ export async function GET(req: NextRequest) { const versionParam = req.nextUrl.searchParams.get("version"); @@ -23,7 +26,7 @@ export async function GET(req: NextRequest) { } const currentVersion = await db.currentVersion(); const layout = await db.inspect(version); - return NextResponse.json({ ...layout, currentVersion }); + return NextResponse.json({ ...layout, currentVersion, schema: schemaMeta() }); } catch (err) { if (err instanceof SqlError) { const status = err.code === "VERSION_NOT_FOUND" ? 404 : 400; diff --git a/app/api/viewer-rows/route.ts b/app/api/viewer-rows/route.ts index 2a03aea..8b615c3 100644 --- a/app/api/viewer-rows/route.ts +++ b/app/api/viewer-rows/route.ts @@ -1,6 +1,7 @@ import { NextRequest, NextResponse } from "next/server"; import { SqlError } from "@larva-db/core"; import { demoDb, demoSchema } from "@/app/lib/demo"; +import { tableMeta } from "@/app/lib/viewer"; export const maxDuration = 60; const DEFAULT_PAGE = 50; @@ -49,10 +50,7 @@ export async function GET(req: NextRequest) { return NextResponse.json({ table, - columns, - primaryKey: spec.primaryKey, - partitionColumn: spec.partitionColumn ?? null, - types: Object.fromEntries(columns.map((c) => [c, spec.columns[c].type])), + ...tableMeta(table)!, rows, total: Number(n), limit, diff --git a/app/lib/viewer.ts b/app/lib/viewer.ts new file mode 100644 index 0000000..532b6c3 --- /dev/null +++ b/app/lib/viewer.ts @@ -0,0 +1,28 @@ +import { demoSchema } from "@/app/lib/demo"; + +/** The client-facing description of one table's shape — enough to render the + * full table structure (headers, pk/partition badges) before any row loads. */ +export interface TableMeta { + columns: string[]; + types: Record; + primaryKey: string; + partitionColumn: string | null; +} + +export function tableMeta(table: string): TableMeta | null { + const spec = demoSchema[table]; + if (!spec) return null; + const columns = Object.keys(spec.columns); + return { + columns, + types: Object.fromEntries(columns.map((c) => [c, spec.columns[c].type])), + primaryKey: spec.primaryKey, + partitionColumn: spec.partitionColumn ?? null, + }; +} + +/** Every table's shape, keyed by name — shipped with the inspect response so the + * viewer can draw the skeleton table immediately. */ +export function schemaMeta(): Record { + return Object.fromEntries(Object.keys(demoSchema).map((t) => [t, tableMeta(t)!])); +} diff --git a/app/viewer/layout.tsx b/app/viewer/layout.tsx new file mode 100644 index 0000000..c7138df --- /dev/null +++ b/app/viewer/layout.tsx @@ -0,0 +1,5 @@ +import { Providers } from "./providers"; + +export default function ViewerLayout({ children }: { children: React.ReactNode }) { + return {children}; +} diff --git a/app/viewer/page.tsx b/app/viewer/page.tsx index 7db119a..5241cb0 100644 --- a/app/viewer/page.tsx +++ b/app/viewer/page.tsx @@ -1,7 +1,8 @@ "use client"; import Link from "next/link"; -import { useEffect, useState } from "react"; +import { useState } from "react"; +import { keepPreviousData, useQuery } from "@tanstack/react-query"; import type { Scalar } from "@larva-db/core"; interface ChunkView { @@ -15,6 +16,12 @@ interface TableView { chunkCount: number; chunks: ChunkView[]; } +interface TableMeta { + columns: string[]; + types: Record; + primaryKey: string; + partitionColumn: string | null; +} interface Inspection { version: number; committedAt: string; @@ -22,6 +29,7 @@ interface Inspection { isCurrent: boolean; currentVersion: number; tables: Record; + schema: Record; } interface RowsResponse { table: string; @@ -39,99 +47,96 @@ interface RowsResponse { isCurrent: boolean; stats: { chunksTotal: number; chunksFetched: number }; } -interface ErrBody { + +const PAGE = 50; +const SKELETON_ROWS = 8; + +class ApiError extends Error { code: string; - message: string; + constructor(code: string, message: string) { + super(message); + this.code = code; + } } -const PAGE = 50; +async function getJSON(url: string): Promise { + const res = await fetch(url); + const body = await res.json(); + if (body && typeof body === "object" && "error" in body) { + throw new ApiError(body.error.code, body.error.message); + } + return body as T; +} export default function Viewer() { - const [layout, setLayout] = useState(null); - const [head, setHead] = useState(null); // live current version - const [version, setVersion] = useState(null); // committed selection - const [scrub, setScrub] = useState(null); // live slider position - const [table, setTable] = useState(null); - const [data, setData] = useState(null); + // UI selections. `null` means "follow the default" (head version / first table), + // so the derived values below stay correct even before the first fetch lands — + // no effects, no setState-in-effect. + const [userVersion, setUserVersion] = useState(null); + const [scrub, setScrub] = useState(null); + const [userTable, setUserTable] = useState(null); const [offset, setOffset] = useState(0); const [orderBy, setOrderBy] = useState(null); const [dir, setDir] = useState<"asc" | "desc">("asc"); - const [error, setError] = useState(null); - const [loadingRows, setLoadingRows] = useState(false); - // --- inspect: layout for the selected version (and, on first load, the head) --- - useEffect(() => { - let cancelled = false; - (async () => { - const res = await fetch(`/api/inspect${version === null ? "" : `?version=${version}`}`); - const body = (await res.json()) as Inspection | { error: ErrBody }; - if (cancelled) return; - if ("error" in body) { - setError(body.error); - return; - } - setError(null); - setLayout(body); - setHead(body.currentVersion); - setVersion((v) => (v === null ? body.version : v)); - setScrub((s) => (s === null ? body.version : s)); - setTable((t) => (t === null ? Object.keys(body.tables)[0] ?? null : t)); - })(); - return () => { - cancelled = true; - }; - }, [version]); + const inspectQuery = useQuery({ + queryKey: ["inspect", userVersion], + queryFn: () => getJSON(`/api/inspect${userVersion === null ? "" : `?version=${userVersion}`}`), + placeholderData: keepPreviousData, + }); + + const inspect = inspectQuery.data; + const head = inspect?.currentVersion ?? null; + const version = userVersion ?? head; // the version being viewed + const schema = inspect?.schema ?? {}; + const tableNames = Object.keys(schema); + const table = userTable ?? tableNames[0] ?? null; + const meta = table ? schema[table] ?? null : null; - // --- rows: a page of the selected table at the selected version --- - useEffect(() => { - if (table === null || version === null) return; - let cancelled = false; - (async () => { - setLoadingRows(true); - const params = new URLSearchParams({ table, version: String(version), limit: String(PAGE), offset: String(offset) }); + const rowsQuery = useQuery({ + queryKey: ["rows", table, version, offset, orderBy, dir], + queryFn: () => { + const params = new URLSearchParams({ table: table!, version: String(version), limit: String(PAGE), offset: String(offset), dir }); if (orderBy) params.set("orderBy", orderBy); - params.set("dir", dir); - try { - const res = await fetch(`/api/viewer-rows?${params}`); - const body = (await res.json()) as RowsResponse | { error: ErrBody }; - if (cancelled) return; - if ("error" in body) { - setError(body.error); - setData(null); - } else { - setError(null); - setData(body); - } - } finally { - if (!cancelled) setLoadingRows(false); - } - })(); - return () => { - cancelled = true; - }; - }, [table, version, offset, orderBy, dir]); + return getJSON(`/api/viewer-rows?${params}`); + }, + enabled: table !== null && version !== null, + placeholderData: keepPreviousData, + }); + + const rows = rowsQuery.data; + // Only trust row data that belongs to the table+version currently selected. + // On a table/version switch the previous data lingers (keepPreviousData) but + // must not be painted under the new headers — show the skeleton instead. + // Within a table+version (pagination) it matches and stays for a smooth swap. + const rowsMatch = rows != null && rows.table === table && rows.version === version; + const tv = inspect && table ? inspect.tables[table] ?? null : null; + const chunksMatch = inspect != null && inspect.version === version; + + const error = (inspectQuery.error ?? rowsQuery.error) as ApiError | null; + const isPast = version !== null && head !== null && version !== head; + const sliderValue = scrub ?? version ?? 0; const commitScrub = () => { if (scrub !== null && scrub !== version) { - setVersion(scrub); + setUserVersion(scrub); setOffset(0); } }; const toLive = () => { - if (head !== null) { - setScrub(head); - setVersion(head); - setOffset(0); - } + setUserVersion(null); + setScrub(null); + setOffset(0); }; const pickTable = (t: string) => { - setTable(t); + setUserTable(t); setOffset(0); setOrderBy(null); setDir("asc"); }; const sortBy = (col: string) => { - if (orderBy === col || (orderBy === null && data?.primaryKey === col)) { + const activeCol = orderBy ?? meta?.primaryKey; + if (activeCol === col) { setDir((d) => (d === "asc" ? "desc" : "asc")); } else { setOrderBy(col); @@ -140,10 +145,6 @@ export default function Viewer() { setOffset(0); }; - const isPast = layout !== null && head !== null && layout.version !== head; - const tableNames = layout ? Object.keys(layout.tables) : []; - const tv = layout && table ? layout.tables[table] : null; - return (
@@ -175,220 +176,284 @@ export default function Viewer() { )} {/* --- version scrubber --- */} - {layout && head !== null && ( -
-
- - version {scrub ?? layout.version} - / {head} - - {isPast ? ( - - time-travelling — {new Date(layout.committedAt).toLocaleString()} +
+ {inspect && head !== null ? ( + <> +
+ + version {sliderValue} + / {head} - ) : ( - ● live head · {new Date(layout.committedAt).toLocaleString()} - )} - format {layout.formatVersion} · commit log -
+ {isPast ? ( + + time-travelling — {new Date(inspect.committedAt).toLocaleString()} + + ) : ( + ● live head · {new Date(inspect.committedAt).toLocaleString()} + )} + format {inspect.formatVersion} · commit log +
+
+ v0 + setScrub(Number(e.target.value))} + onPointerUp={commitScrub} + onKeyUp={commitScrub} + className="accent-series-1 h-1 grow cursor-pointer" + aria-label="database version" + /> + v{head} + +
+ + ) : (
- v0 - setScrub(Number(e.target.value))} - onPointerUp={commitScrub} - onKeyUp={commitScrub} - className="accent-series-1 h-1 grow cursor-pointer" - aria-label="database version" - /> - v{head} - + + +
-
- )} + )} + {/* --- table tabs --- */} - {tableNames.length > 0 && ( -
- {tableNames.map((t) => { - const active = t === table; - return ( - - ); - })} -
- )} +
+ {tableNames.length > 0 + ? tableNames.map((t) => { + const active = t === table; + return ( + + ); + }) + : [0, 1, 2].map((i) => )} +
- {/* --- rows grid --- */} - {data && ( -
-
- - rows{" "} - - {data.total === 0 ? 0 : data.offset + 1}–{Math.min(data.offset + data.limit, data.total)} - {" "} - of {data.total} - - - scanned {data.stats.chunksFetched} of{" "} - {data.stats.chunksTotal} chunk - {data.stats.chunksTotal === 1 ? "" : "s"} - {data.stats.chunksFetched < data.stats.chunksTotal && " — zone maps pruned the rest"} - - {loadingRows && loading…} + {/* --- rows grid: structure always present; body skeletons while loading --- */} +
+
+ {rowsMatch ? ( + <> + + rows{" "} + + {rows!.total === 0 ? 0 : rows!.offset + 1}–{Math.min(rows!.offset + rows!.limit, rows!.total)} + {" "} + of {rows!.total} + + + scanned {rows!.stats.chunksFetched} of{" "} + {rows!.stats.chunksTotal} chunk + {rows!.stats.chunksTotal === 1 ? "" : "s"} + {rows!.stats.chunksFetched < rows!.stats.chunksTotal && " — zone maps pruned the rest"} + + + ) : ( + + )} + {rowsQuery.isFetching && loading…} + {meta && ( - ordered by {data.orderBy} {data.dir} + ordered by {rowsMatch ? `${rows!.orderBy} ${rows!.dir}` : `${orderBy ?? meta.primaryKey} ${dir}`} -
-
- - - - {data.columns.map((c) => ( - - ))} - - - - {data.rows.map((row, i) => ( - - {data.columns.map((c) => ( - + )} + +
+
- -
- -
+ + + {meta + ? meta.columns.map((c) => ( + + )) + : SKELETON_COLS.map((i) => ( + ))} - - ))} - {data.rows.length === 0 && ( + + + + {rowsMatch ? ( + rows!.rows.length > 0 ? ( + rows!.rows.map((row, i) => ( + + {rows!.columns.map((c) => ( + + ))} + + )) + ) : ( - - )} - -
+ + +
+ +
+
+ +
+ no rows at this version
+ ) + ) : ( + + )} + + +
+ {rowsMatch && rows!.total > rows!.limit && ( +
+ + + page {Math.floor(rows!.offset / rows!.limit) + 1} of {Math.max(1, Math.ceil(rows!.total / rows!.limit))} + +
- {data.total > data.limit && ( -
- - - page {Math.floor(data.offset / data.limit) + 1} of {Math.max(1, Math.ceil(data.total / data.limit))} - - -
- )} -
- )} + )} +
{/* --- chunk / zone-map internals --- */} - {tv && table && ( -
-
-

chunks & zone maps

- - {tv.chunkCount} chunk{tv.chunkCount === 1 ? "" : "s"} · {tv.rowCount} rows · {table} @ v{layout?.version} - -
-

- Every chunk is an immutable blob. The manifest stores each one's row count and the - min/max of the primary key{data?.partitionColumn ? ` and the partition column (${data.partitionColumn})` : ""} — - the zone maps a query uses to skip chunks it can't match. -

-
- - - - - - - - - - - {tv.chunks.map((ch) => ( - - - - - - - ))} - {tv.chunks.length === 0 && ( +
+
+

chunks & zone maps

+ + {tv && chunksMatch ? ( + <> + {tv.chunkCount} chunk{tv.chunkCount === 1 ? "" : "s"} · {tv.rowCount} rows · {table} @ v{inspect?.version} + + ) : ( + <>loading layout… + )} + +
+

+ Every chunk is an immutable blob. The manifest stores each one's row count and the + min/max of the primary key{meta?.partitionColumn ? ` and the partition column (${meta.partitionColumn})` : ""} — + the zone maps a query uses to skip chunks it can't match. +

+
+
chunkrows{data?.primaryKey ?? "pk"} min → max - {data?.partitionColumn ? `${data.partitionColumn} min → max` : "partition"} -
- {ch.id.slice(0, 8)}… - {ch.rows} - {ch.pk ? : —} - - {ch.partition ? : —} -
+ + + + + + + + + + {tv && chunksMatch ? ( + tv.chunks.length > 0 ? ( + tv.chunks.map((ch) => ( + + + + + + + )) + ) : ( - )} - -
chunkrows{meta?.primaryKey ?? "pk"} min → max + {meta?.partitionColumn ? `${meta.partitionColumn} min → max` : "partition"} +
+ {ch.id.slice(0, 8)}… + {ch.rows} + {ch.pk ? : —} + + {ch.partition ? : —} +
- no chunks — this table is empty at v{layout?.version} + no chunks — this table is empty at v{inspect?.version}
-
-
- )} + ) + ) : ( + + )} + + + +
- Reads only. Rows come from db.query (and db.asOf for past versions); - the chunk layout from db.inspect — a read-only projection of the manifest. - Both hit the same demo store as the console and stress lab. + Reads only, fetched with React Query. Rows come from db.query (and{" "} + db.asOf for past versions); the chunk layout from db.inspect — a + read-only projection of the manifest. Both hit the same demo store as the console and stress lab.
); } +const SKELETON_COLS = [0, 1, 2, 3, 4]; + +function Skeleton({ className = "", style }: { className?: string; style?: React.CSSProperties }) { + return
; +} + +function SkeletonRows({ cols, count = SKELETON_ROWS }: { cols: number; count?: number }) { + return ( + <> + {Array.from({ length: count }, (_, r) => ( + + {Array.from({ length: cols }, (_, c) => ( + + + + ))} + + ))} + + ); +} + +// Deterministic pseudo-random widths so the skeleton looks like data, not a grid. +function skelWidth(row: number, col: number): number { + const widths = [80, 55, 40, 65, 48, 72, 35, 60]; + return widths[(row * 3 + col * 5) % widths.length]; +} + function Cell({ value }: { value: Scalar }) { if (value === null || value === undefined) return NULL; if (typeof value === "boolean") return {value ? "true" : "false"}; diff --git a/app/viewer/providers.tsx b/app/viewer/providers.tsx new file mode 100644 index 0000000..b4ce53c --- /dev/null +++ b/app/viewer/providers.tsx @@ -0,0 +1,27 @@ +"use client"; + +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; +import { useState } from "react"; + +/** + * React Query provider scoped to the /viewer route. The QueryClient is created + * in state so it survives re-renders but is never shared across requests (it is + * a client component — one per browser session). Chunk blobs are immutable and + * the manifest is small, so we keep data fresh for a few seconds and lean on + * keepPreviousData at the call sites to keep the table on screen during fetches. + */ +export function Providers({ children }: { children: React.ReactNode }) { + const [client] = useState( + () => + new QueryClient({ + defaultOptions: { + queries: { + staleTime: 5_000, + retry: false, + refetchOnWindowFocus: false, + }, + }, + }), + ); + return {children}; +} diff --git a/bun.lock b/bun.lock index 12e83c8..875d2bd 100644 --- a/bun.lock +++ b/bun.lock @@ -6,6 +6,7 @@ "name": "larva-starter", "dependencies": { "@larva-db/core": "workspace:*", + "@tanstack/react-query": "^5.101.2", "@vercel/blob": "^2.6.1", "next": "16.2.10", "nextra": "^4.6.1", @@ -29,7 +30,7 @@ }, "packages/larvadb": { "name": "@larva-db/core", - "version": "2.1.0", + "version": "2.6.0", "bin": { "larva": "./dist/cli.js", }, @@ -338,6 +339,10 @@ "@tailwindcss/postcss": ["@tailwindcss/postcss@4.3.2", "", { "dependencies": { "@alloc/quick-lru": "^5.2.0", "@tailwindcss/node": "4.3.2", "@tailwindcss/oxide": "4.3.2", "postcss": "^8.5.15", "tailwindcss": "4.3.2" } }, "sha512-rjVWYCa7Ngbi5AarT6k8TkxUG3Wl1QKzHdIZVsjZSzf36Jmo2IKZt/NHRAwly8oDkbBOH0YTu+CHuf9jPxMc+g=="], + "@tanstack/query-core": ["@tanstack/query-core@5.101.2", "", {}, "sha512-hH5MLoJhF7KaIGd7q3xTXGXvslI+GYlM1Z/35aSHHWaCJWB7XvTSHYuV3eM7tw+aE0mT/xMro4M4Q9rCGHT0lw=="], + + "@tanstack/react-query": ["@tanstack/react-query@5.101.2", "", { "dependencies": { "@tanstack/query-core": "5.101.2" }, "peerDependencies": { "react": "^18 || ^19" } }, "sha512-seDkr6kzGzX1okaaTtZPtgA688CDPlXUz1C6xSg0ESqn04Vuc8tlrYms1s3de+znBqhPVxFRfpAfUf+6XvfPWg=="], + "@tanstack/react-virtual": ["@tanstack/react-virtual@3.14.5", "", { "dependencies": { "@tanstack/virtual-core": "3.17.3" }, "peerDependencies": { "react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0", "react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-4EKRXh7zBLkbKbFmG3AUVkircuHd+7OdT1pocJSepxtfBd3qnrJgJ5rtPkRYyo9fmyVb2+pI2xPy5oYvMLQy6A=="], "@tanstack/virtual-core": ["@tanstack/virtual-core@3.17.3", "", {}, "sha512-8Np/TFELpI0ySuJoVmjvOrQYXH/8sTX0Biv9szhFhY39xOdAAY+smrMxjxOum/ux3eM8MUJQsEJ0/R0UpvC8dw=="], diff --git a/package.json b/package.json index 541a58e..301b97c 100644 --- a/package.json +++ b/package.json @@ -13,6 +13,7 @@ }, "dependencies": { "@larva-db/core": "workspace:*", + "@tanstack/react-query": "^5.101.2", "@vercel/blob": "^2.6.1", "next": "16.2.10", "nextra": "^4.6.1", From 41ebf92fedce8438ba425733400f84d5ad84b9d0 Mon Sep 17 00:00:00 2001 From: Julio Vaught Date: Fri, 10 Jul 2026 23:59:35 -0700 Subject: [PATCH 3/3] =?UTF-8?q?Release=202.7.0=20=E2=80=94=20version=20bum?= =?UTF-8?q?p=20+=20docs=20for=20db.inspect()=20and=20the=20viewer?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 2.6.0 is already on npm; the new public API needs a minor bump or the main-push publish job fails on the duplicate version - api.mdx: inspect() in the surface block + its own section (incl. the tier-A visibility note: durable-but-unfolded appends appear at fold, like any cross-instance reader) - test-lab.mdx: the /viewer walkthrough - CLAUDE.md: inspect added to the §13 surface invariant Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 2 +- README.md | 2 +- bun.lock | 2 +- content/api.mdx | 9 +++++++++ content/test-lab.mdx | 4 ++++ packages/larvadb/package.json | 2 +- 6 files changed, 17 insertions(+), 4 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 3ca91a5..12d6438 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -43,7 +43,7 @@ Larva is a deliberate miniaturization of the Delta Lake / Iceberg pattern on top - The SQL dialect is a closed, documented subset (Design §7). Precise, machine-readable error messages for unsupported SQL are a design feature — agents self-correct from specific errors. Do not quietly accept SQL outside the subset. - `UPDATE`/`DELETE` without `WHERE` requires explicit `{ allowFullTable: true }`. - The parser rejects multiple statements per string (injection stacking vector). -- The public API surface (Design §13) must fit on one screen: `larva()`, `db.sql` tagged template (primary API), `db.query(str, params)`, `db.transaction`, `db.asOf`, `db.rollbackTo`, `db.export`, `db.vacuum`, `db.upgrade`. Resist growing it. +- The public API surface (Design §13) must fit on one screen: `larva()`, `db.sql` tagged template (primary API), `db.query(str, params)`, `db.transaction`, `db.asOf`, `db.rollbackTo`, `db.export`, `db.vacuum`, `db.upgrade`, `db.inspect` (read-only layout projection — the one read accessor). Resist growing it. - On-store format versioning (Design §5): formats 1 (original), 2 (sequences, auto-UUID columns + composite uniques), 3 (ordered commit log), 4 (two-tier writes: intent queues + durable-at-PUT appends). Manifests declare the minimum format their content requires; clients refuse newer formats loudly (`FormatError`). Any future format bump ships with an explicit atomic `upgrade()`; `rollbackTo` preserves the format version. - Export (SQLite/Postgres/JSON/CSV) is a v1 feature, not an afterthought — the escape hatch is part of the product promise. - Only Vercel-specific code should be the storage adapter; the `StorageAdapter` contract is exactly four operations: get, put-with-CAS, delete, list-by-prefix. diff --git a/README.md b/README.md index 1fd43c5..8a040ae 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ [![types](https://img.shields.io/badge/types-included-blue)](packages/larvadb/src/index.ts) [![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE) -**Current release: 2.6.0.** Real SQL (time series, upserts, JSON, multi-table joins, subqueries), atomic transactions, time travel, and a guaranteed exit path to SQLite *or* Postgres. +**Current release: 2.7.0.** Real SQL (time series, upserts, JSON, multi-table joins, subqueries), atomic transactions, time travel, and a guaranteed exit path to SQLite *or* Postgres. ## Sixty seconds to a database diff --git a/bun.lock b/bun.lock index 875d2bd..949bbdf 100644 --- a/bun.lock +++ b/bun.lock @@ -30,7 +30,7 @@ }, "packages/larvadb": { "name": "@larva-db/core", - "version": "2.6.0", + "version": "2.7.0", "bin": { "larva": "./dist/cli.js", }, diff --git a/content/api.mdx b/content/api.mdx index 142e309..6798dc7 100644 --- a/content/api.mdx +++ b/content/api.mdx @@ -22,6 +22,7 @@ await db.rollbackTo(past.version); // restore (itself u await db.export({ format: "postgres" }); // the escape hatch await db.vacuum(); // reclaim storage outside retention await db.upgrade(); // one-way flip to the top format +await db.inspect(); // read-only physical layout: chunks + zone maps ``` ## `larva(options)` @@ -122,6 +123,14 @@ Drops history outside retention (`retainDays: 7`, `retainVersions: 50` — which Flips the store to the top format — [the ordered commit log plus two-tier fast appends](/docs/how-it-works/format-versioning) — one atomic commit, one-way, idempotent. Data, history, and rollback survive; clients older than the format then refuse loudly instead of writing through the wrong protocol. +## `db.inspect(version?)` + +Read-only introspection of the store's **physical layout**: `version`, `committedAt`, `formatVersion`, and per-table chunk lists with each chunk's row count and primary-key/partition zone-map min/max. A *derived*, format-agnostic projection of the otherwise-private manifest (never the raw thing, so on-store format evolution can't break the contract), with no blob URLs — the private-data invariant holds. Reads manifest metadata only, never a chunk — strictly lighter than `export()`. + +With a version number it time-travels to any retained past version, exactly like `asOf` (throws `VERSION_NOT_FOUND` if pruned). One visibility note: [fast-append](/docs/how-it-works/commit-protocol#format-4-two-tier-writes) rows that are durable but not yet folded aren't in any chunk yet, so they don't appear here until the fold — same as for any other cross-instance reader. + +This is what powers [the data viewer](/viewer) — the chunk panel and version scrubber are `inspect()` calls. + ## Errors All machine-readable, by design: diff --git a/content/test-lab.mdx b/content/test-lab.mdx index 0c83520..fd6c2f0 100644 --- a/content/test-lab.mdx +++ b/content/test-lab.mdx @@ -14,6 +14,10 @@ Type any statement in [the dialect](/docs/sql) and run it live. The example chip **Export buttons** produce real files — the Postgres one loads with `psql $DATABASE_URL < larva-demo.sql`. **Reset demo data** re-seeds everything; mangle freely, that's the point. +## The data viewer + +[`/viewer`](/viewer) browses the same store visually: paged, sortable tables; a **version scrubber** that time-travels through every commit (`asOf`, live); and a chunk/zone-map internals panel powered by [`db.inspect()`](/docs/api#dbinspectversion). Scrubbing is itself a demo of the architecture — watch a table grow from one chunk to several, and the format version flip, as history replays. + ## The stress lab The bottom of the home page hammers one database with concurrent writers, then audits the final state for lost updates, duplicates, and version drift — the commit protocol's core promise, tested against the real store on every click. diff --git a/packages/larvadb/package.json b/packages/larvadb/package.json index 41778bb..bfb0182 100644 --- a/packages/larvadb/package.json +++ b/packages/larvadb/package.json @@ -1,6 +1,6 @@ { "name": "@larva-db/core", - "version": "2.6.0", + "version": "2.7.0", "description": "A tiny SQL database that lives inside your Vercel Blob store. Real SQL, atomic transactions, time travel, and a guaranteed exit path when you outgrow it.", "license": "MIT", "type": "module",