Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
ea2eeb6
fix(engine): reclaim dead tuples, fail-stop on partial apply, auto-ch…
ChromaBeast Oct 28, 2026
c774fb3
feat(engine): conditional writes and scan limit, plus post-V1 roadmap…
ChromaBeast Oct 28, 2026
069adff
feat(engine): values up to 2 MiB, page-slot reuse, and WAL frame cap
claude Oct 28, 2026
259eff7
fix(engine): verify a chain's CRC before freeing it; count WAL framin…
claude Oct 28, 2026
d8c1280
Merge pull request #1 from ChromaBeast/ccr-c5e72350-yy7e7c
ChromaBeast Oct 28, 2026
e815ba8
feat(auth): per-method gRPC role authorization and API token expiry
ChromaBeast Nov 12, 2026
7ed1da2
feat(bench): add beastbench YCSB-style load generator and benchmarkin…
ChromaBeast Nov 12, 2026
c9d5113
feat(obs): Prometheus /metrics endpoint with gRPC/HTTP request metrics
ChromaBeast Nov 12, 2026
7ccd824
perf(wal,engine): group commit - concurrent writers share one fsync
ChromaBeast Nov 12, 2026
65051a7
fix(storage): enforce write-ahead barrier before dirty page writes
ChromaBeast Nov 12, 2026
a8d1f96
Merge worktree-agent-a85706995f3f27667
ChromaBeast Nov 12, 2026
0699a5b
Merge worktree-agent-abfdd378c2ca5a1cf
ChromaBeast Nov 12, 2026
a64f2cb
Merge worktree-agent-ac87d15ba95064563 (Prometheus metrics)
ChromaBeast Nov 12, 2026
cf7ed6c
docs: document group commit, WAL barrier, gRPC roles, metrics, beastb…
ChromaBeast Nov 12, 2026
06b662e
docs: add SECURITY.md with private vulnerability reporting policy
ChromaBeast Nov 12, 2026
2cac2c9
refactor: split wal.go, buffer_pool.go and beastbench runner.go under…
ChromaBeast Nov 12, 2026
ade5d0a
docs: replace hard 200-line limit with a one-responsibility-per-file …
ChromaBeast Nov 12, 2026
c6b851e
docs: update phase checklists to the file-modularity guideline
ChromaBeast Nov 12, 2026
43c1875
Merge pull request #2 from ChromaBeast/claude/db-security-usability-p…
ChromaBeast Nov 12, 2026
a52bb00
test(metrics): only require disk-free gauge where the platform report…
ChromaBeast Nov 12, 2026
d466719
perf(engine): let the first writer lead its own commit group
ChromaBeast Nov 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
os: [ubuntu-latest, windows-latest, macos-latest]

steps:
- name: Checkout Code
Expand Down
26 changes: 26 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# BeastDB: instructions for coding agents

Read by Codex and other agents that use `AGENTS.md`. Claude Code reads it through `CLAUDE.md`. Antigravity also reads `GEMINI.md`, which carries the same rules; keep the two in step.

## File modularity (guideline, not a hard limit)

- One responsibility per file. About **400 lines** is the point to stop and ask "does this file do more than one thing?" If yes, split along that seam (for example sync vs. lifecycle, load vs. run). If no, leave it alone.
- Never split a cohesive file, or merge small files, just to satisfy a line count.
- Exempt: generated code (`*.pb.go`), test files, JSON and raw data.
- Do not add CI or lint checks that fail on line count. A non-exempt file past about 500 lines should say why in the change description.

## Engine principles

- **WAL before page:** a data page must never reach the data file before the WAL records that describe it are durable. The buffer pool enforces this with the WAL barrier (`WAL.EnsureDurable`); keep every dirty-page write on that path.
- **Zero-allocation hot paths:** query, hashing, parsing and buffer-pinning paths avoid heap allocations; prefer byte slices and `sync.Pool`.
- **Mechanical sympathy:** align structs to avoid false sharing, prefer flat memory layouts, respect 4 KB page alignment.

## Scope and honesty

- BeastDB is a general-purpose, domain-agnostic database. Do not hard-code schema or domain rules from downstream applications; client-specific partition metadata belongs in `partitions.json` or `-partition-config`.
- Do not claim performance relative to other databases without an equivalent-workload benchmark (see `docs/benchmarking.md`). Describe planned features as planned.
- Report vulnerabilities privately; see `SECURITY.md`.

## Before you finish

Run `go build ./... && go vet ./... && go test ./... -race`. `TestEngineSystemWorkloadBenchmark` has a fixed 1000 ops/s floor and can fail when the machine is loaded; re-run it alone before treating a failure as a regression.
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@AGENTS.md
20 changes: 14 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,11 @@ Thank you for your interest in contributing to BeastDB! We welcome contributions

## 🏛️ Engineering Standards

### 1. Strict File Modularity (< 200 LoC)
- **Constraint:** Every source file (excluding auto-generated `.pb.go` code) must strictly remain readable and **under 200 Lines of Code (LoC)**.
- **Decomposition:** Large subsystems (B+ Tree, Buffer Pool, Hash Table, WAL) must be decomposed into focused, single-responsibility files and companion helpers.
### 1. File Modularity (guideline, not a hard limit)
- **Principle:** One responsibility per file. Treat about **400 lines** as a prompt to ask "does this file do more than one thing?" If it does, split along that seam. If it doesn't, leave it as is. Never split, or merge, files only to hit a number.
- **Exempt:** generated code (`*.pb.go`), test files, and data files (JSON and similar).
- **Review, not CI:** there is no automated line-count gate. A non-exempt file past about 500 lines should say in the pull request why it stays one file.
- **Decomposition:** Large subsystems (B+ Tree, Buffer Pool, Hash Table, WAL) should be organized into focused, single-responsibility files and companion helpers.

### 2. Zero-Allocation Hot Paths
- Query execution, hash calculation, parsing, and buffer pinning paths must not trigger unnecessary heap allocations.
Expand All @@ -21,6 +23,12 @@ Thank you for your interest in contributing to BeastDB! We welcome contributions

---

## 🔒 Security Issues

Do not report vulnerabilities in public issues or pull requests. Follow [SECURITY.md](SECURITY.md).

---

## 🧪 Verification & Testing

Before submitting a pull request, ensure all validations pass:
Expand All @@ -32,9 +40,9 @@ go test -v -count=1 ./...
# 2. Run benchmarks to verify zero allocations on hot paths
go test -bench=. -benchmem ./...

# 3. Enforce the < 200 LoC modularity limit
Get-ChildItem -Recurse -Filter "*.go" | Where-Object { $_.FullName -notmatch "\.pb\.go" } | ForEach-Object {
# 3. (Optional) List large non-test, non-generated files to review. Informational only.
Get-ChildItem -Recurse -Filter "*.go" | Where-Object { $_.Name -notmatch "\.pb\.go$|_test\.go$" } | ForEach-Object {
$lines = (Get-Content $_.FullName).Count
if ($lines -ge 200) { Write-Error "$lines lines in $($_.Name)" }
if ($lines -ge 400) { Write-Host "$lines lines in $($_.FullName): does it do more than one thing?" }
}
```
9 changes: 6 additions & 3 deletions GEMINI.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,12 @@ These rules govern all interactions, coding, and architecture in this project.

---

## 1. File Modularity & Readability (< 200 LoC)
* **Strict File Length Limit:** In any project, if a file is not JSON or raw data, it must strictly remain readable and **< 200 Lines of Code (LoC)**.
* **Component Decomposition:** Decompose complex systems (B+ trees, buffer pools, hash tables, page managers) into small, single-responsibility files, structs, and reusable components.
## 1. File Modularity & Readability (guideline, not a hard limit)
* **One responsibility per file.** Organize files by what they do. File length is a smell, not a rule: roughly **400 lines** is the point to stop and ask "does this file do more than one thing?" If yes, split it along that seam (for example sync vs. lifecycle, load vs. run). If no, leave it alone.
* **Never split just to hit a number.** Do not cut a cohesive file into arbitrary pieces, move code without a clear responsibility boundary, or merge small files back together to satisfy a count.
* **Exemptions:** generated code (`*.pb.go`), test files, and JSON/raw data have no size expectation.
* **No automated gate.** Do not add CI or lint checks that fail on line count. Past about 500 lines, mention the reason in the change description or review.
* **Component Decomposition:** Decompose complex systems (B+ trees, buffer pools, hash tables, page managers) into focused structs and reusable components.

---

Expand Down
9 changes: 6 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@
</picture>
</p>

<h3 align="center">High-Performance, Crash-Resilient Distributed Database — Built from Scratch in Pure Go</h3>
<h3 align="center">Crash-Safe Embedded OLTP Database with Leader-Follower Replication — Built from Scratch in Pure Go</h3>

<p align="center">
<a href="https://go.dev/"><img src="https://img.shields.io/badge/Go-1.26+-00ADD8?style=flat&logo=go" alt="Go Version" /></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License" /></a>
<a href="ROADMAP.md"><img src="https://img.shields.io/badge/Hot%20Paths-0%20allocs%2Fop-brightgreen" alt="Zero Allocs" /></a>
<img src="https://img.shields.io/badge/Tests-Passing-success" alt="Tests" />
<a href=".github/workflows/ci.yml"><img src="https://github.com/ChromaBeast/beastdb/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
<img src="https://img.shields.io/badge/Studio-Embedded-blueviolet" alt="Studio" />
</p>

Expand All @@ -27,7 +27,7 @@ It ships as a **single static binary** that includes a full gRPC service, WAL-ba

| Concern | How BeastDB addresses it |
|---|---|
| **Predictable latency** | B+ Tree with `O(log N)` point reads at 180 ns, 0 allocs — no compaction spikes |
| **Predictable latency** | B+ Tree with `O(log N)` cached point reads at 180 ns, 0 allocs — no compaction spikes |
| **Crash safety** | 23-byte ARIES WAL frames with CRC32 torn-write detection; full replay on boot |
| **Operational simplicity** | Single binary, `go run ./cmd/server -dev` for local dev, `docker compose up` for prod |
| **Developer ergonomics** | Embedded Studio web UI, `beastctl` CLI, partition-aware key builder, fixture seeding |
Expand Down Expand Up @@ -159,6 +159,9 @@ BeastDB is optimized for **read-heavy OLTP** — deterministic point reads and o

- [**V1 Semantics & Guarantees**](docs/v1-semantics.md) — Durability acknowledgment, limits, and replica reads.
- [**System Benchmarks**](docs/benchmarks.md) — Reproducible throughput, tail latencies, and cache-eviction profiles.
- [**Benchmarking Guide**](docs/benchmarking.md) — `beastbench` YCSB-style workloads and how to compare fairly with other databases.
- [**Security Policy**](SECURITY.md) — How to report a vulnerability privately, scope, and safe deployment checklist.
- [**Upgrade Spec**](docs/upgrade-spec.md) — Security, usability and performance work packages and what is still planned.
- [**Operator Guide**](docs/runbooks/operator-guide.md) — TLS configuration, cluster roles, manual failover, and `/healthz` probes.
- [**Disaster Recovery**](docs/runbooks/disaster-recovery.md) — Physical backup drills, checksum verification, and RTO/RPO SLA.
- [**Client Integration**](docs/clients.md) — Protobuf contracts for Go, Python, TypeScript, and Rust.
Expand Down
4 changes: 4 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,10 @@ Order is dependency-based, not a calendar promise. Initial planning range for on
7. [x] Operational probes: public `/healthz` and `/readyz` endpoints with [operator guide](docs/runbooks/operator-guide.md) and [disaster recovery runbook](docs/runbooks/disaster-recovery.md).
8. [x] Studio pagination in Table/Gallery views, whitespace-safe status filtering, and reproducible system benchmarks ([docs/benchmarks.md](docs/benchmarks.md)).

## Beyond V1 gates

See the [roadmap plan](docs/roadmap-plan.md) for post-gate phases and the [upgrade spec](docs/upgrade-spec.md) for the current security, usability and performance work. Shipped so far: conditional writes (`ABSENT`/`PRESENT`/`VALUE_EQUALS` on Put, Delete and BatchWrite), limit-based scan pagination, values up to 2 MiB with 8 MiB batches ([ADR 0001](docs/adr/0001-large-values.md)), WAL group commit with a write-ahead buffer-pool barrier, per-method gRPC role authorization with token expiry, a Prometheus `/metrics` endpoint, and the `beastbench` YCSB-style load generator ([benchmarking guide](docs/benchmarking.md)).

## References

- [PostgreSQL WAL archiving and recovery](https://www.postgresql.org/docs/17/continuous-archiving.html) — backup/restore design reference, not a BeastDB feature claim.
Expand Down
62 changes: 62 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Security Policy

## Reporting a vulnerability

**Please do not open a public issue, pull request or discussion for a security problem.**

Report it privately through GitHub:

1. Go to the repository's **Security** tab.
2. Choose **Report a vulnerability** (GitHub private vulnerability reporting).

Include what you can of:

- the affected version or commit and how you run BeastDB (flags, TLS, roles, replication);
- steps to reproduce, or a proof of concept;
- the impact you observed (for example unauthorised read or write, data corruption, denial of service);
- any suggested fix.

If the **Report a vulnerability** button is missing, the maintainers have not enabled it yet. Open a public issue that says only "security contact needed" and contains no technical details, and a maintainer will arrange a private channel.

### What to expect

BeastDB is maintained by a small team, so these are goals, not guarantees:

- acknowledgement of your report within about 5 working days;
- an assessment and a plan within about 14 days;
- a fix or documented mitigation as quickly as the severity allows, with coordinated public disclosure after a fix is available. We will credit you if you want to be credited.

## Supported versions

BeastDB has not reached a tagged V1 release (see [ROADMAP.md](ROADMAP.md)). Until then, security fixes are made on the `main` branch only. Once V1 is released, this section will list the supported release lines.

## Scope

In scope:

- authentication and authorization (gRPC bearer tokens and roles, Studio sessions and cookies, the `/metrics` endpoint);
- transport security (TLS and mTLS for clients and replicas);
- the storage engine, WAL, recovery, backup and restore paths, where a flaw could lose or corrupt acknowledged data or expose data to the wrong caller;
- the replication stream;
- the container image and default configuration shipped in this repository.

Out of scope:

- findings that require the attacker to already have admin credentials, local root access, or write access to the data directory;
- running with `--dev` or `--insecure-auth` on a reachable network (these flags are documented as for local, non-production use);
- denial of service that needs a very large volume of traffic;
- missing hardening that is already listed as planned in [docs/upgrade-spec.md](docs/upgrade-spec.md) (for example encryption at rest). Reports on how a planned item could be exploited today are still welcome.

## Deploying safely

The [Operator Guide](docs/runbooks/operator-guide.md) is the reference. In short:

- set a strong `BEASTDB_ADMIN_PASSWORD`; the server refuses the default password on non-loopback listeners and when `BEASTDB_ENV=production`;
- enable TLS (and mTLS for replicas) and keep gRPC and Studio off public interfaces;
- give each application its own API token with the smallest role it needs (`viewer`, `writer`, `admin`, `replication`) and set an expiry where you can;
- protect `/metrics` with a dedicated `BEASTDB_METRICS_TOKEN` and scrape it over TLS;
- keep backups off the host and test restores.

## Safe harbour

We will not pursue action against researchers who make a good-faith effort to follow this policy: test only against systems you own or have permission to test, avoid accessing or modifying other people's data, do not degrade service for others, and give us reasonable time to fix an issue before disclosing it.
Loading