diff --git a/CLAUDE.md b/CLAUDE.md index 3ca91a5..12d6438 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -43,7 +43,7 @@ Larva is a deliberate miniaturization of the Delta Lake / Iceberg pattern on top - The SQL dialect is a closed, documented subset (Design §7). Precise, machine-readable error messages for unsupported SQL are a design feature — agents self-correct from specific errors. Do not quietly accept SQL outside the subset. - `UPDATE`/`DELETE` without `WHERE` requires explicit `{ allowFullTable: true }`. - The parser rejects multiple statements per string (injection stacking vector). -- The public API surface (Design §13) must fit on one screen: `larva()`, `db.sql` tagged template (primary API), `db.query(str, params)`, `db.transaction`, `db.asOf`, `db.rollbackTo`, `db.export`, `db.vacuum`, `db.upgrade`. Resist growing it. +- The public API surface (Design §13) must fit on one screen: `larva()`, `db.sql` tagged template (primary API), `db.query(str, params)`, `db.transaction`, `db.asOf`, `db.rollbackTo`, `db.export`, `db.vacuum`, `db.upgrade`, `db.inspect` (read-only layout projection — the one read accessor). Resist growing it. - On-store format versioning (Design §5): formats 1 (original), 2 (sequences, auto-UUID columns + composite uniques), 3 (ordered commit log), 4 (two-tier writes: intent queues + durable-at-PUT appends). Manifests declare the minimum format their content requires; clients refuse newer formats loudly (`FormatError`). Any future format bump ships with an explicit atomic `upgrade()`; `rollbackTo` preserves the format version. - Export (SQLite/Postgres/JSON/CSV) is a v1 feature, not an afterthought — the escape hatch is part of the product promise. - Only Vercel-specific code should be the storage adapter; the `StorageAdapter` contract is exactly four operations: get, put-with-CAS, delete, list-by-prefix. diff --git a/LARVA-DESIGN.md b/LARVA-DESIGN.md index 241f3e5..df20dfe 100644 --- a/LARVA-DESIGN.md +++ b/LARVA-DESIGN.md @@ -294,8 +294,11 @@ await db.rollbackTo(version); // recovery in one line await db.export({ format: "sqlite" }); // the escape hatch await db.vacuum(); // reclaim storage outside retention await db.upgrade(); // one-way flip to format 3 (the commit log) +await db.inspect(version?); // read-only physical layout: chunks + zone maps ``` +`inspect()` is the one read-only accessor on the surface, and it earns its place the same way `export()` does: it is a *derived*, format-agnostic projection of the manifest (version, `committedAt`, `formatVersion`, and per-table chunk lists with each chunk's row count and primary-key/partition zone-map min-max), never the raw `Manifest` — so on-store format evolution cannot break its contract. It reads manifest metadata only (strictly lighter than `export`, which also reads every chunk), returns no blob URLs (the private-data invariant holds), and resolves any retained past version exactly as `asOf` does. It exists so a store can be *seen* — it is the primitive under the data viewer (§14) and any store-debugging tool — and it is the deliberate exception to "resist growing the surface": one read-only method, no new write path. + The same surface ships as a shell command — the package's `larva` bin (`npx larva sql | export | upgrade | rollback | vacuum | version`, credentials auto-loaded from `.env.local`). One surface, two doors; the CLI is validated end to end by `scripts/cli-smoke.ts`. ## 14. Roadmap and open questions @@ -309,7 +312,7 @@ v1 ships: the storage engine and commit protocol, the SQL subset of Section 7, c **The SQL dialect roadmap (decided July 2026) — complete.** Four former exclusions, admitted or re-judged against the Section 7 bar. Three shipped in 2.5: additive `ALTER TABLE … ADD COLUMN` (schema-only commit; absent keys read as `NULL` at every read path; additive schema drift auto-migrates at connect — Section 8; renames and drops still wait for a migration design that respects time travel), joins of three or more tables and self-joins (left-deep hash joins in the existing in-memory algebra; distinct alias per occurrence), and uncorrelated subqueries (inner-query-first evaluation rewriting the node into literals, exactly as planned — the result feeds the outer plan and `IN` lists participate in zone-map pruning; correlated subqueries stay out, per Section 7). The fourth, **secondary index blobs**, shipped in 2.6 exactly as sketched — an index is just another immutable blob committed in the same atomic swap, and keying entries by immutable chunk id made it safe without a format bump (Section 8). -Still deferred, unordered: a columnar chunk format; S3/R2/GCS/Azure adapters (the S3 contract harness already exists); a tiny read-only web UI for browsing tables (non-engineers love to *see* their data); an optional Edge Config read-accelerator for flag-shaped tables. +Still deferred, unordered: a columnar chunk format; S3/R2/GCS/Azure adapters (the S3 contract harness already exists); an optional Edge Config read-accelerator for flag-shaped tables. The **read-only web UI** for browsing tables (non-engineers love to *see* their data) is underway in the test-lab app: a `/viewer` route over the demo store that pairs a paged table browser with a version scrubber (time travel via `asOf`) and a chunk/zone-map internals panel — the last of these is why `db.inspect()` (§13) was added, since a store's physical layout lives in the otherwise-private manifest. **The v2 write path.** The v1 ceiling — every logical commit is one CAS on one global manifest — is a protocol choice, not an object-storage limit, and there is a ladder of known techniques for raising it if multi-process operational workloads (several deployments writing steadily, unique-constraint-bearing inserts, sequence assignment) become a target. In rough order of leverage per cost: **sequence range leasing** (shipped — `t.sequence()`, Section 8); **composite unique constraints** (shipped — `defineSchema` `uniques`, Section 8); **an ordered commit log** (shipped — format 3, Section 6: create-only numbered entries, checkpointed manifest, cheap rebases, explicit `upgrade()`); **lease-elected cross-process group commit** (committed — see above; object-storage CAS doubles as a lease primitive; the leaseholder batches every process's transactions into shared log slots — group commit across processes with no server anyone deploys, safe even under split leadership because the log stays the arbiter; needs the intent queues below as its transport, so the two ship together); **a two-tier write API** (committed — see above; constraint-free appends go to per-writer intent prefixes — zero contention, one-PUT-RTT durability, an LSM memtable made of blobs folded in by a lease-elected compactor; constraint-bearing writes go through the ordered log and wait for a deterministic slot verdict, Calvin-style); and **sharded contention domains** (a two-level manifest so unrelated table groups stop sharing one CAS). Each rung ships independently; the format-versioning guard (Section 5) is what makes adopting any of them a loud, upgradeable break instead of silent corruption. The physics no rung changes: sync-commit latency floors at one PUT round-trip, because durability cannot be faked. diff --git a/README.md b/README.md index 1fd43c5..8a040ae 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ [![types](https://img.shields.io/badge/types-included-blue)](packages/larvadb/src/index.ts) [![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE) -**Current release: 2.6.0.** Real SQL (time series, upserts, JSON, multi-table joins, subqueries), atomic transactions, time travel, and a guaranteed exit path to SQLite *or* Postgres. +**Current release: 2.7.0.** Real SQL (time series, upserts, JSON, multi-table joins, subqueries), atomic transactions, time travel, and a guaranteed exit path to SQLite *or* Postgres. ## Sixty seconds to a database diff --git a/app/api/inspect/route.ts b/app/api/inspect/route.ts new file mode 100644 index 0000000..3f5dc1d --- /dev/null +++ b/app/api/inspect/route.ts @@ -0,0 +1,37 @@ +import { NextRequest, NextResponse } from "next/server"; +import { SqlError } from "@larva-db/core"; +import { demoDb } from "@/app/lib/demo"; +import { schemaMeta } from "@/app/lib/viewer"; + +export const maxDuration = 60; + +/** + * Read-only physical layout of the demo store at one version (db.inspect, + * Design §13): per-table chunk lists with zone-map min/max. Powers the + * viewer's internals panel and version scrubber. `?version=N` time-travels; + * `currentVersion` is always returned so the scrubber knows its upper bound, + * and `schema` (every table's columns) so the client can draw the table + * structure before a single row loads. + */ +export async function GET(req: NextRequest) { + const versionParam = req.nextUrl.searchParams.get("version"); + try { + const db = await demoDb(); + let version: number | undefined; + if (versionParam !== null) { + version = Number(versionParam); + if (!Number.isInteger(version) || version < 0) { + return NextResponse.json({ error: { code: "BAD_VERSION", message: "version must be a non-negative integer" } }, { status: 400 }); + } + } + const currentVersion = await db.currentVersion(); + const layout = await db.inspect(version); + return NextResponse.json({ ...layout, currentVersion, schema: schemaMeta() }); + } catch (err) { + if (err instanceof SqlError) { + const status = err.code === "VERSION_NOT_FOUND" ? 404 : 400; + return NextResponse.json({ error: { code: err.code, message: err.message } }, { status }); + } + return NextResponse.json({ error: { code: "INTERNAL", message: err instanceof Error ? err.message : String(err) } }, { status: 500 }); + } +} diff --git a/app/api/viewer-rows/route.ts b/app/api/viewer-rows/route.ts new file mode 100644 index 0000000..8b615c3 --- /dev/null +++ b/app/api/viewer-rows/route.ts @@ -0,0 +1,71 @@ +import { NextRequest, NextResponse } from "next/server"; +import { SqlError } from "@larva-db/core"; +import { demoDb, demoSchema } from "@/app/lib/demo"; +import { tableMeta } from "@/app/lib/viewer"; + +export const maxDuration = 60; +const DEFAULT_PAGE = 50; +const MAX_PAGE = 200; + +/** + * A page of rows from one demo table, optionally at a past version (time + * travel via asOf). Read-only. Table and sort column are whitelisted against + * the code-first schema, and page bounds are coerced to integers, so the only + * thing interpolated into SQL are validated identifiers — no user values reach + * the query string (and the parser rejects statement stacking regardless). + * Returns the fetch's pruning stats (chunks scanned vs. total) so the UI can + * show the zone maps doing their job. + */ +export async function GET(req: NextRequest) { + const p = req.nextUrl.searchParams; + const table = p.get("table") ?? ""; + const spec = demoSchema[table]; + if (!spec) { + return NextResponse.json( + { error: { code: "UNKNOWN_TABLE", message: `pick a table: ${Object.keys(demoSchema).join(", ")}` } }, + { status: 400 }, + ); + } + const columns = Object.keys(spec.columns); + const orderByParam = p.get("orderBy"); + const orderBy = orderByParam && columns.includes(orderByParam) ? orderByParam : spec.primaryKey; + const dir = p.get("dir") === "desc" ? "DESC" : "ASC"; + const limit = Math.min(MAX_PAGE, Math.max(1, Math.trunc(Number(p.get("limit"))) || DEFAULT_PAGE)); + const offset = Math.max(0, Math.trunc(Number(p.get("offset"))) || 0); + const versionParam = p.get("version"); + + try { + const db = await demoDb(); + const currentVersion = await db.currentVersion(); + const wantsPast = versionParam !== null && Number(versionParam) !== currentVersion; + if (versionParam !== null && !Number.isInteger(Number(versionParam))) { + return NextResponse.json({ error: { code: "BAD_VERSION", message: "version must be an integer" } }, { status: 400 }); + } + const version = wantsPast ? Number(versionParam) : currentVersion; + const src = wantsPast ? await db.asOf(version) : db; + + const rows = await src.query(`SELECT * FROM ${table} ORDER BY ${orderBy} ${dir} LIMIT ${limit} OFFSET ${offset}`); + const stats = db.lastQueryStats; // capture before the COUNT query overwrites it + const [{ n }] = await src.query<{ n: number }>(`SELECT COUNT(*) AS n FROM ${table}`); + + return NextResponse.json({ + table, + ...tableMeta(table)!, + rows, + total: Number(n), + limit, + offset, + orderBy, + dir: dir.toLowerCase(), + version, + isCurrent: !wantsPast, + stats, + }); + } catch (err) { + if (err instanceof SqlError) { + const status = err.code === "VERSION_NOT_FOUND" ? 404 : 400; + return NextResponse.json({ error: { code: err.code, message: err.message } }, { status }); + } + return NextResponse.json({ error: { code: "INTERNAL", message: err instanceof Error ? err.message : String(err) } }, { status: 500 }); + } +} diff --git a/app/lib/viewer.ts b/app/lib/viewer.ts new file mode 100644 index 0000000..532b6c3 --- /dev/null +++ b/app/lib/viewer.ts @@ -0,0 +1,28 @@ +import { demoSchema } from "@/app/lib/demo"; + +/** The client-facing description of one table's shape — enough to render the + * full table structure (headers, pk/partition badges) before any row loads. */ +export interface TableMeta { + columns: string[]; + types: Record; + primaryKey: string; + partitionColumn: string | null; +} + +export function tableMeta(table: string): TableMeta | null { + const spec = demoSchema[table]; + if (!spec) return null; + const columns = Object.keys(spec.columns); + return { + columns, + types: Object.fromEntries(columns.map((c) => [c, spec.columns[c].type])), + primaryKey: spec.primaryKey, + partitionColumn: spec.partitionColumn ?? null, + }; +} + +/** Every table's shape, keyed by name — shipped with the inspect response so the + * viewer can draw the skeleton table immediately. */ +export function schemaMeta(): Record { + return Object.fromEntries(Object.keys(demoSchema).map((t) => [t, tableMeta(t)!])); +} diff --git a/app/page.tsx b/app/page.tsx index 4abf177..7da1565 100644 --- a/app/page.tsx +++ b/app/page.tsx @@ -57,6 +57,12 @@ export default function Home() {

Larva / test lab

+ + data viewer + {children}; +} diff --git a/app/viewer/page.tsx b/app/viewer/page.tsx new file mode 100644 index 0000000..5241cb0 --- /dev/null +++ b/app/viewer/page.tsx @@ -0,0 +1,482 @@ +"use client"; + +import Link from "next/link"; +import { useState } from "react"; +import { keepPreviousData, useQuery } from "@tanstack/react-query"; +import type { Scalar } from "@larva-db/core"; + +interface ChunkView { + id: string; + rows: number; + pk: { min: Scalar; max: Scalar } | null; + partition: { min: Scalar; max: Scalar | null } | null; +} +interface TableView { + rowCount: number; + chunkCount: number; + chunks: ChunkView[]; +} +interface TableMeta { + columns: string[]; + types: Record; + primaryKey: string; + partitionColumn: string | null; +} +interface Inspection { + version: number; + committedAt: string; + formatVersion: number; + isCurrent: boolean; + currentVersion: number; + tables: Record; + schema: Record; +} +interface RowsResponse { + table: string; + columns: string[]; + primaryKey: string; + partitionColumn: string | null; + types: Record; + rows: Record[]; + total: number; + limit: number; + offset: number; + orderBy: string; + dir: "asc" | "desc"; + version: number; + isCurrent: boolean; + stats: { chunksTotal: number; chunksFetched: number }; +} + +const PAGE = 50; +const SKELETON_ROWS = 8; + +class ApiError extends Error { + code: string; + constructor(code: string, message: string) { + super(message); + this.code = code; + } +} + +async function getJSON(url: string): Promise { + const res = await fetch(url); + const body = await res.json(); + if (body && typeof body === "object" && "error" in body) { + throw new ApiError(body.error.code, body.error.message); + } + return body as T; +} + +export default function Viewer() { + // UI selections. `null` means "follow the default" (head version / first table), + // so the derived values below stay correct even before the first fetch lands — + // no effects, no setState-in-effect. + const [userVersion, setUserVersion] = useState(null); + const [scrub, setScrub] = useState(null); + const [userTable, setUserTable] = useState(null); + const [offset, setOffset] = useState(0); + const [orderBy, setOrderBy] = useState(null); + const [dir, setDir] = useState<"asc" | "desc">("asc"); + + const inspectQuery = useQuery({ + queryKey: ["inspect", userVersion], + queryFn: () => getJSON(`/api/inspect${userVersion === null ? "" : `?version=${userVersion}`}`), + placeholderData: keepPreviousData, + }); + + const inspect = inspectQuery.data; + const head = inspect?.currentVersion ?? null; + const version = userVersion ?? head; // the version being viewed + const schema = inspect?.schema ?? {}; + const tableNames = Object.keys(schema); + const table = userTable ?? tableNames[0] ?? null; + const meta = table ? schema[table] ?? null : null; + + const rowsQuery = useQuery({ + queryKey: ["rows", table, version, offset, orderBy, dir], + queryFn: () => { + const params = new URLSearchParams({ table: table!, version: String(version), limit: String(PAGE), offset: String(offset), dir }); + if (orderBy) params.set("orderBy", orderBy); + return getJSON(`/api/viewer-rows?${params}`); + }, + enabled: table !== null && version !== null, + placeholderData: keepPreviousData, + }); + + const rows = rowsQuery.data; + // Only trust row data that belongs to the table+version currently selected. + // On a table/version switch the previous data lingers (keepPreviousData) but + // must not be painted under the new headers — show the skeleton instead. + // Within a table+version (pagination) it matches and stays for a smooth swap. + const rowsMatch = rows != null && rows.table === table && rows.version === version; + const tv = inspect && table ? inspect.tables[table] ?? null : null; + const chunksMatch = inspect != null && inspect.version === version; + + const error = (inspectQuery.error ?? rowsQuery.error) as ApiError | null; + const isPast = version !== null && head !== null && version !== head; + const sliderValue = scrub ?? version ?? 0; + + const commitScrub = () => { + if (scrub !== null && scrub !== version) { + setUserVersion(scrub); + setOffset(0); + } + }; + const toLive = () => { + setUserVersion(null); + setScrub(null); + setOffset(0); + }; + const pickTable = (t: string) => { + setUserTable(t); + setOffset(0); + setOrderBy(null); + setDir("asc"); + }; + const sortBy = (col: string) => { + const activeCol = orderBy ?? meta?.primaryKey; + if (activeCol === col) { + setDir((d) => (d === "asc" ? "desc" : "asc")); + } else { + setOrderBy(col); + setDir("asc"); + } + setOffset(0); + }; + + return ( +
+
+
+

+ Larva / data viewer +

+ + stress lab + + + docs + +
+

+ Browse the live demo store, scrub back through every commit (time travel is a byproduct + of the architecture — old manifests are complete snapshots), and watch the chunk zone + maps that make queries prune. Read-only: nothing here can write. +

+
+ + {error && ( +
+

+ {error.code} + {error.message} +

+
+ )} + + {/* --- version scrubber --- */} +
+ {inspect && head !== null ? ( + <> +
+ + version {sliderValue} + / {head} + + {isPast ? ( + + time-travelling — {new Date(inspect.committedAt).toLocaleString()} + + ) : ( + ● live head · {new Date(inspect.committedAt).toLocaleString()} + )} + format {inspect.formatVersion} · commit log +
+
+ v0 + setScrub(Number(e.target.value))} + onPointerUp={commitScrub} + onKeyUp={commitScrub} + className="accent-series-1 h-1 grow cursor-pointer" + aria-label="database version" + /> + v{head} + +
+ + ) : ( +
+ + + +
+ )} +
+ + {/* --- table tabs --- */} +
+ {tableNames.length > 0 + ? tableNames.map((t) => { + const active = t === table; + return ( + + ); + }) + : [0, 1, 2].map((i) => )} +
+ + {/* --- rows grid: structure always present; body skeletons while loading --- */} +
+
+ {rowsMatch ? ( + <> + + rows{" "} + + {rows!.total === 0 ? 0 : rows!.offset + 1}–{Math.min(rows!.offset + rows!.limit, rows!.total)} + {" "} + of {rows!.total} + + + scanned {rows!.stats.chunksFetched} of{" "} + {rows!.stats.chunksTotal} chunk + {rows!.stats.chunksTotal === 1 ? "" : "s"} + {rows!.stats.chunksFetched < rows!.stats.chunksTotal && " — zone maps pruned the rest"} + + + ) : ( + + )} + {rowsQuery.isFetching && loading…} + {meta && ( + + ordered by {rowsMatch ? `${rows!.orderBy} ${rows!.dir}` : `${orderBy ?? meta.primaryKey} ${dir}`} + + )} +
+
+ + + + {meta + ? meta.columns.map((c) => ( + + )) + : SKELETON_COLS.map((i) => ( + + ))} + + + + {rowsMatch ? ( + rows!.rows.length > 0 ? ( + rows!.rows.map((row, i) => ( + + {rows!.columns.map((c) => ( + + ))} + + )) + ) : ( + + + + ) + ) : ( + + )} + +
+ + +
+ +
+
+ +
+ no rows at this version +
+
+ {rowsMatch && rows!.total > rows!.limit && ( +
+ + + page {Math.floor(rows!.offset / rows!.limit) + 1} of {Math.max(1, Math.ceil(rows!.total / rows!.limit))} + + +
+ )} +
+ + {/* --- chunk / zone-map internals --- */} +
+
+

chunks & zone maps

+ + {tv && chunksMatch ? ( + <> + {tv.chunkCount} chunk{tv.chunkCount === 1 ? "" : "s"} · {tv.rowCount} rows · {table} @ v{inspect?.version} + + ) : ( + <>loading layout… + )} + +
+

+ Every chunk is an immutable blob. The manifest stores each one's row count and the + min/max of the primary key{meta?.partitionColumn ? ` and the partition column (${meta.partitionColumn})` : ""} — + the zone maps a query uses to skip chunks it can't match. +

+
+ + + + + + + + + + + {tv && chunksMatch ? ( + tv.chunks.length > 0 ? ( + tv.chunks.map((ch) => ( + + + + + + + )) + ) : ( + + + + ) + ) : ( + + )} + +
chunkrows{meta?.primaryKey ?? "pk"} min → max + {meta?.partitionColumn ? `${meta.partitionColumn} min → max` : "partition"} +
+ {ch.id.slice(0, 8)}… + {ch.rows} + {ch.pk ? : —} + + {ch.partition ? : —} +
+ no chunks — this table is empty at v{inspect?.version} +
+
+
+ +
+ Reads only, fetched with React Query. Rows come from db.query (and{" "} + db.asOf for past versions); the chunk layout from db.inspect — a + read-only projection of the manifest. Both hit the same demo store as the console and stress lab. +
+
+ ); +} + +const SKELETON_COLS = [0, 1, 2, 3, 4]; + +function Skeleton({ className = "", style }: { className?: string; style?: React.CSSProperties }) { + return
; +} + +function SkeletonRows({ cols, count = SKELETON_ROWS }: { cols: number; count?: number }) { + return ( + <> + {Array.from({ length: count }, (_, r) => ( + + {Array.from({ length: cols }, (_, c) => ( + + + + ))} + + ))} + + ); +} + +// Deterministic pseudo-random widths so the skeleton looks like data, not a grid. +function skelWidth(row: number, col: number): number { + const widths = [80, 55, 40, 65, 48, 72, 35, 60]; + return widths[(row * 3 + col * 5) % widths.length]; +} + +function Cell({ value }: { value: Scalar }) { + if (value === null || value === undefined) return NULL; + if (typeof value === "boolean") return {value ? "true" : "false"}; + if (typeof value === "number") return {value}; + return <>{String(value)}; +} + +function cellTitle(value: Scalar): string { + if (value === null || value === undefined) return "NULL"; + return String(value); +} + +function ZoneRange({ min, max }: { min: Scalar; max: Scalar | null }) { + const fmt = (v: Scalar) => (v === null || v === undefined ? "∅" : String(v)); + const a = fmt(min); + const b = fmt(max); + return ( + + {trunc(a)} → {trunc(b)} + + ); +} + +function trunc(s: string): string { + return s.length > 22 ? `${s.slice(0, 10)}…${s.slice(-6)}` : s; +} diff --git a/app/viewer/providers.tsx b/app/viewer/providers.tsx new file mode 100644 index 0000000..b4ce53c --- /dev/null +++ b/app/viewer/providers.tsx @@ -0,0 +1,27 @@ +"use client"; + +import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; +import { useState } from "react"; + +/** + * React Query provider scoped to the /viewer route. The QueryClient is created + * in state so it survives re-renders but is never shared across requests (it is + * a client component — one per browser session). Chunk blobs are immutable and + * the manifest is small, so we keep data fresh for a few seconds and lean on + * keepPreviousData at the call sites to keep the table on screen during fetches. + */ +export function Providers({ children }: { children: React.ReactNode }) { + const [client] = useState( + () => + new QueryClient({ + defaultOptions: { + queries: { + staleTime: 5_000, + retry: false, + refetchOnWindowFocus: false, + }, + }, + }), + ); + return {children}; +} diff --git a/bun.lock b/bun.lock index 12e83c8..949bbdf 100644 --- a/bun.lock +++ b/bun.lock @@ -6,6 +6,7 @@ "name": "larva-starter", "dependencies": { "@larva-db/core": "workspace:*", + "@tanstack/react-query": "^5.101.2", "@vercel/blob": "^2.6.1", "next": "16.2.10", "nextra": "^4.6.1", @@ -29,7 +30,7 @@ }, "packages/larvadb": { "name": "@larva-db/core", - "version": "2.1.0", + "version": "2.7.0", "bin": { "larva": "./dist/cli.js", }, @@ -338,6 +339,10 @@ "@tailwindcss/postcss": ["@tailwindcss/postcss@4.3.2", "", { "dependencies": { "@alloc/quick-lru": "^5.2.0", "@tailwindcss/node": "4.3.2", "@tailwindcss/oxide": "4.3.2", "postcss": "^8.5.15", "tailwindcss": "4.3.2" } }, "sha512-rjVWYCa7Ngbi5AarT6k8TkxUG3Wl1QKzHdIZVsjZSzf36Jmo2IKZt/NHRAwly8oDkbBOH0YTu+CHuf9jPxMc+g=="], + "@tanstack/query-core": ["@tanstack/query-core@5.101.2", "", {}, "sha512-hH5MLoJhF7KaIGd7q3xTXGXvslI+GYlM1Z/35aSHHWaCJWB7XvTSHYuV3eM7tw+aE0mT/xMro4M4Q9rCGHT0lw=="], + + "@tanstack/react-query": ["@tanstack/react-query@5.101.2", "", { "dependencies": { "@tanstack/query-core": "5.101.2" }, "peerDependencies": { "react": "^18 || ^19" } }, "sha512-seDkr6kzGzX1okaaTtZPtgA688CDPlXUz1C6xSg0ESqn04Vuc8tlrYms1s3de+znBqhPVxFRfpAfUf+6XvfPWg=="], + "@tanstack/react-virtual": ["@tanstack/react-virtual@3.14.5", "", { "dependencies": { "@tanstack/virtual-core": "3.17.3" }, "peerDependencies": { "react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0", "react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-4EKRXh7zBLkbKbFmG3AUVkircuHd+7OdT1pocJSepxtfBd3qnrJgJ5rtPkRYyo9fmyVb2+pI2xPy5oYvMLQy6A=="], "@tanstack/virtual-core": ["@tanstack/virtual-core@3.17.3", "", {}, "sha512-8Np/TFELpI0ySuJoVmjvOrQYXH/8sTX0Biv9szhFhY39xOdAAY+smrMxjxOum/ux3eM8MUJQsEJ0/R0UpvC8dw=="], diff --git a/content/api.mdx b/content/api.mdx index 142e309..6798dc7 100644 --- a/content/api.mdx +++ b/content/api.mdx @@ -22,6 +22,7 @@ await db.rollbackTo(past.version); // restore (itself u await db.export({ format: "postgres" }); // the escape hatch await db.vacuum(); // reclaim storage outside retention await db.upgrade(); // one-way flip to the top format +await db.inspect(); // read-only physical layout: chunks + zone maps ``` ## `larva(options)` @@ -122,6 +123,14 @@ Drops history outside retention (`retainDays: 7`, `retainVersions: 50` — which Flips the store to the top format — [the ordered commit log plus two-tier fast appends](/docs/how-it-works/format-versioning) — one atomic commit, one-way, idempotent. Data, history, and rollback survive; clients older than the format then refuse loudly instead of writing through the wrong protocol. +## `db.inspect(version?)` + +Read-only introspection of the store's **physical layout**: `version`, `committedAt`, `formatVersion`, and per-table chunk lists with each chunk's row count and primary-key/partition zone-map min/max. A *derived*, format-agnostic projection of the otherwise-private manifest (never the raw thing, so on-store format evolution can't break the contract), with no blob URLs — the private-data invariant holds. Reads manifest metadata only, never a chunk — strictly lighter than `export()`. + +With a version number it time-travels to any retained past version, exactly like `asOf` (throws `VERSION_NOT_FOUND` if pruned). One visibility note: [fast-append](/docs/how-it-works/commit-protocol#format-4-two-tier-writes) rows that are durable but not yet folded aren't in any chunk yet, so they don't appear here until the fold — same as for any other cross-instance reader. + +This is what powers [the data viewer](/viewer) — the chunk panel and version scrubber are `inspect()` calls. + ## Errors All machine-readable, by design: diff --git a/content/test-lab.mdx b/content/test-lab.mdx index 0c83520..fd6c2f0 100644 --- a/content/test-lab.mdx +++ b/content/test-lab.mdx @@ -14,6 +14,10 @@ Type any statement in [the dialect](/docs/sql) and run it live. The example chip **Export buttons** produce real files — the Postgres one loads with `psql $DATABASE_URL < larva-demo.sql`. **Reset demo data** re-seeds everything; mangle freely, that's the point. +## The data viewer + +[`/viewer`](/viewer) browses the same store visually: paged, sortable tables; a **version scrubber** that time-travels through every commit (`asOf`, live); and a chunk/zone-map internals panel powered by [`db.inspect()`](/docs/api#dbinspectversion). Scrubbing is itself a demo of the architecture — watch a table grow from one chunk to several, and the format version flip, as history replays. + ## The stress lab The bottom of the home page hammers one database with concurrent writers, then audits the final state for lost updates, duplicates, and version drift — the commit protocol's core promise, tested against the real store on every click. diff --git a/package.json b/package.json index 541a58e..301b97c 100644 --- a/package.json +++ b/package.json @@ -13,6 +13,7 @@ }, "dependencies": { "@larva-db/core": "workspace:*", + "@tanstack/react-query": "^5.101.2", "@vercel/blob": "^2.6.1", "next": "16.2.10", "nextra": "^4.6.1", diff --git a/packages/larvadb/package.json b/packages/larvadb/package.json index 41778bb..bfb0182 100644 --- a/packages/larvadb/package.json +++ b/packages/larvadb/package.json @@ -1,6 +1,6 @@ { "name": "@larva-db/core", - "version": "2.6.0", + "version": "2.7.0", "description": "A tiny SQL database that lives inside your Vercel Blob store. Real SQL, atomic transactions, time travel, and a guaranteed exit path when you outgrow it.", "license": "MIT", "type": "module", diff --git a/packages/larvadb/src/db.ts b/packages/larvadb/src/db.ts index 774fbeb..c0cba7e 100644 --- a/packages/larvadb/src/db.ts +++ b/packages/larvadb/src/db.ts @@ -16,6 +16,34 @@ interface TxState { applies: PlanOutcome["apply"][]; } +/** One chunk's physical stats, as surfaced by db.inspect() — a stable + * projection of the manifest's zone map (never the raw ChunkRef). */ +export interface ChunkInspection { + /** ULID chunk id (also its blob pathname stem). Not a secret; not a URL. */ + id: string; + rows: number; + /** Primary-key zone map: the min/max PK in this chunk (drives pruning). */ + pk: { min: Scalar; max: Scalar } | null; + /** Partition-column zone map, when the table declares a partitionBy() column. */ + partition: { min: Scalar; max: Scalar | null } | null; +} + +export interface TableInspection { + rowCount: number; + chunkCount: number; + chunks: ChunkInspection[]; +} + +/** Read-only view of a store's physical layout at one version (db.inspect()). */ +export interface DbInspection { + version: number; + committedAt: string; + formatVersion: number; + /** True when this is the live head, false for a time-travelled past version. */ + isCurrent: boolean; + tables: Record; +} + /** * Statement handle inside db.transaction(). Reads see the transaction's own * uncommitted writes (a virtual manifest); nothing touches the live database @@ -951,6 +979,49 @@ export class LarvaDb { return (await this.proto.snapshot()).manifest.version; } + /** + * Read-only introspection of the store's physical layout at one version + * (Design §13). Projects the manifest — normally private — into a stable, + * format-agnostic view: the schema-declared shape, per-table chunk lists, and + * each chunk's row count and zone-map min/max. This is deliberately a *derived* + * shape, not the raw Manifest, so on-store format evolution never breaks the + * contract. Powers the data viewer (§14) and any store-debugging tool; it is + * strictly lighter than export(), reading manifest metadata and never a chunk. + * + * With no argument, the current version; with a version number, any retained + * past version (same resolution as asOf — throws VERSION_NOT_FOUND if pruned). + */ + async inspect(version?: number): Promise { + await this.ensureReady(); + const current = await this.proto.snapshot(); + let manifest: Manifest | null; + if (version === undefined || version === current.manifest.version) { + manifest = current.manifest; + } else { + manifest = await this.proto.manifestAt(version); + if (!manifest) { + throw new SqlError("VERSION_NOT_FOUND", `version ${version} is not in retained history (current version: ${current.manifest.version})`); + } + } + const tables: Record = {}; + for (const [name, table] of Object.entries(manifest.tables)) { + const chunks = table.chunks.map((c) => ({ + id: c.id, + rows: c.rows, + pk: c.stats ? { min: c.stats.pkMin, max: c.stats.pkMax } : null, + partition: c.stats && c.stats.partMin !== undefined ? { min: c.stats.partMin, max: c.stats.partMax ?? null } : null, + })); + tables[name] = { rowCount: chunks.reduce((n, c) => n + c.rows, 0), chunkCount: chunks.length, chunks }; + } + return { + version: manifest.version, + committedAt: manifest.committedAt, + formatVersion: manifest.formatVersion ?? 1, + isCurrent: manifest.version === current.manifest.version, + tables, + }; + } + /** Delete the entire database (test helper; not part of the Design §13 surface). */ async destroy(): Promise { await this.proto.destroy(); diff --git a/packages/larvadb/src/index.ts b/packages/larvadb/src/index.ts index ab80785..be2cc50 100644 --- a/packages/larvadb/src/index.ts +++ b/packages/larvadb/src/index.ts @@ -1,5 +1,14 @@ /** The larvadb public surface (Design §13). */ -export { larva, LarvaDb, LarvaSnapshot, LarvaTx, type LarvaOptions } from "./db"; +export { + larva, + LarvaDb, + LarvaSnapshot, + LarvaTx, + type LarvaOptions, + type DbInspection, + type TableInspection, + type ChunkInspection, +} from "./db"; export { defineSchema, t, diff --git a/scripts/api-smoke.ts b/scripts/api-smoke.ts index da48c1b..01f19be 100644 --- a/scripts/api-smoke.ts +++ b/scripts/api-smoke.ts @@ -94,6 +94,26 @@ ok( `v=${vAfter}, bounds=(${v2}, ${v2 + WRITERS * TXS}]`, ); +// --- inspect: read-only physical layout (§13) --- +const layout = await db.inspect(); +ok("inspect reports the current version and format", layout.isCurrent && layout.version === (await db.currentVersion()) && layout.formatVersion >= 1, JSON.stringify({ v: layout.version, f: layout.formatVersion })); +const invLayout = layout.tables.inventory; +ok( + "inspect: per-table rowCount matches chunk row sums and the live table", + invLayout.rowCount === invLayout.chunks.reduce((n, c) => n + c.rows, 0) && + invLayout.rowCount === ((await db.sql`SELECT COUNT(*) AS n FROM inventory`)[0].n as number), + JSON.stringify(invLayout), +); +ok("inspect: chunks carry primary-key zone maps", invLayout.chunkCount > 0 && invLayout.chunks.every((c) => c.pk !== null), JSON.stringify(invLayout.chunks)); +const pastLayout = await db.inspect(v0); +ok("inspect(version) time-travels to a past layout", pastLayout.version === v0 && !pastLayout.isCurrent, JSON.stringify({ v: pastLayout.version, cur: pastLayout.isCurrent })); +try { + await db.inspect(999_999); + ok("inspect on an unretained version fails loudly", false); +} catch (err) { + ok("inspect on an unretained version fails loudly", (err as SqlError).code === "VERSION_NOT_FOUND", (err as Error).message); +} + // --- export --- const json = await db.export({ format: "json" }); ok("json export has both tables", json.inventory?.length === 2 && json.orders?.length === 1);