RaiBench

Rai means lightning in Japanese - RaiBench is a web framework speed benchmark.

RaiBench is a simple, automated benchmark built to compare web framework performance when starting a new backend project under low-budget server constraints.

Every framework is paired with a database, ORM or query builder, input validation, and JSON serialization—all running inside Docker containers constrained to 1 vCPU and 256MB of RAM.

View live benchmark results: https://raibench.pages.dev

---

Why RaiBench?

I built RaiBench to answer a practical question for myself before starting a new project: which web stack delivers the best performance under a tight server budget (1 vCPU, 256MB RAM) when paired with a real database?

The focus here is entirely on realistic, low-cost server conditions:

• Real database workloads: Every test executes actual database queries (POST /users inserts and GET /users/{id} reads) against a pre-seeded PostgreSQL instance.
• Low-budget resource limits: Each framework container is capped at 1.0 vCPU and 256MB RAM to reflect affordable VPS tiers or serverless container limits.
• Real-world stacks: Compares practical setups like Go + Gin + sqlc, Node + Express + Drizzle, Bun + Hono + Drizzle, Deno + Fresh + Drizzle, and Python + FastAPI + SQLModel.
• Automated orchestration: A single Go command boots isolated containers, runs a 3-stage k6 load test, collects CPU/RAM usage every 500ms, and exports the metrics to a React dashboard.

---

Architecture

                     ┌──────────────────────────────────────┐
                     │          RaiBench Orchestrator       │
                     └──────────────────┬───────────────────┘
                                        │
             ┌──────────────────────────┼──────────────────────────┐
             ▼                          ▼                          ▼
   ┌───────────────────┐      ┌───────────────────┐      ┌───────────────────┐
   │ Docker Containers │      │   k6 Load Agent   │      │ React Dashboard   │
   │  - 1.0 CPU Limit  │      │  - Health Check   │      │  - RPS & Latency  │
   │  - 256MB RAM      │◄────  - Write (/users)   │      │  - CPU & Memory   │
   │  - PostgreSQL DB  │      │  - Read (/users)  │      │  - Interactive JS │
   └───────────────────┘      └───────────────────┘      └───────────────────┘

The system consists of five main components:

1. Database Infrastructure (db-init.sql, docker-compose.yml): Runs PostgreSQL 18 on a dedicated Docker network (raibench-net) and seeds the database with 1,000 user records.
2. Framework Target Suite (frameworks/): Implements identical API endpoints across Go, Node.js, Bun, Deno, and Python runtimes using popular frameworks and ORMs.
3. Load Testing Agent (benchmarks/): Uses k6 to run three distinct test stages:
   • Stage 1 (Baseline): /health endpoint under 100 constant virtual users (VUs) for 10 seconds.
   • Stage 2 (Write): POST /users database inserts under 50 VUs for 10 seconds.
   • Stage 3 (Read): GET /users/{id} database reads under 100 VUs for 10 seconds.
4. Orchestrator CLI (orchestrator/): A Go program that handles container lifecycle management, health polling, connection warm-up, k6 execution, resource profiling, and JSON report generation (dashboard/public/results.json).
5. Analytics Dashboard (dashboard/): A React + Vite frontend that visualizes requests per second, latency percentiles (P90/P95), and CPU/memory curves using Chart.js.

---

Prerequisites

• Docker & Docker Compose (for running database, framework targets, and k6)
• Go 1.26+ (to run the orchestrator)
• Node.js 24+ & Bun (to run the frontend dashboard)

Toolchain versions can be managed with mise using the included mise.toml.

---

Reference Hardware Specs

Recorded baseline benchmark runs were captured on the following environment:

Component	Specification
OS	Ubuntu 24.04.4 LTS
CPU	Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz (4 cores / 8 threads)
RAM	24 GB
Container Engine	Docker / Docker Compose

---

Getting Started

1. Start the Database

docker compose up -d db

This starts PostgreSQL and seeds the users table with initial mock data.

2. Run the Benchmarks

go run ./orchestrator/main.go

The CLI will build images, spin up target containers sequentially, run load tests, collect telemetry, and update dashboard/public/results.json.

3. Launch the Dashboard

cd dashboard
bun install
bun run dev

Open http://localhost:5173 in your browser to view the interactive charts.

---

Metrics Tracked

• Throughput: Successful HTTP requests completed per second (RPS).
• Latency: Average, Median, Min, Max, P90, and P95 latency distributions.
• Errors: Non-2xx HTTP responses, timeouts, and validation failures.
• Resource Usage: Live CPU percentage and memory utilization (MiB) sampled at 500ms intervals.

---

Adding a Framework

To add another framework or runtime to the benchmark suite:

1. Create a directory inside frameworks/your-framework-name/.
2. Implement three endpoints on port 3000:
   • GET /health -> Returns HTTP 200 with {"status": "ok"}
   • POST /users -> Accepts JSON {"name": "...", "email": "..."}, inserts into Postgres, returns HTTP 201 with the created record.
   • GET /users/:id -> Fetches user by ID from Postgres, returns HTTP 200 or 404.
3. Add a Dockerfile exposed on port 3000 that connects via the DATABASE_URL env variable.
4. Register your framework in orchestrator/main.go:

   {
       ID:           "language-framework-orm",
       WebFramework: "Framework Name",
       Path:         "frameworks/language-framework-orm",
       Language:     "Language",
       ORM:          "ORM/QueryBuilder",
   }

5. Re-run go run ./orchestrator/main.go.