Skip to content

Commit b84d4bb

Browse files
dkijaniaclaude
dkijania
and
claude
authored
P0: Add security & deployment hardening guide (#168) (#186)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 64d21b1 commit b84d4bb

5 files changed

Lines changed: 167 additions & 10 deletions

File tree

‎README.md‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -110,19 +110,18 @@ From 1.0.0 the GraphQL schema, HTTP endpoints, and configuration are a versioned
110110

111111
## Hardware requirements
112112

113-
The bottleneck is the Postgres database, not this server. Listing multiple hosts
114-
in `PG_CONN` gives failover, not read fan-out; put a load balancer or managed
115-
reader endpoint in front of read replicas when you need to spread query load. A
116-
recent benchmark on a 12-core / 32 GB box (API + Postgres co-located) sustained
117-
~800 req/s with p99 latency of 39 ms. Use `npm run benchmark` to size your own
118-
deployment.
113+
The bottleneck is the Postgres database, not this server. A recent benchmark on a 12-core / 32 GB box (API + Postgres co-located) sustained ~800 req/s with p99 latency of 39 ms. Use `npm run benchmark` to size your own deployment.
114+
115+
Listing multiple hosts in `PG_CONN` (`postgres://host1:5432,host2:5432/archive`) buys **redundancy, not read throughput**: the client connects to the first host and only moves to the next one when that connection fails, so it fails over rather than spreading queries across replicas. To actually scale reads, load-balance in front of Postgres (PgBouncer, HAProxy, or a managed reader endpoint) and point `PG_CONN` at that.
119116

120117
For SLOs, capacity guidance, what to monitor, and incident response, see the [operations runbook](./docs/runbook.md).
121118

122119
## Deployment
123120

124121
Reference Kubernetes and production Docker Compose manifests — with liveness/readiness probes, resource limits, autoscaling, and a hardened pod security context — live in [`deploy/`](./deploy/). Read [`docs/security.md`](./docs/security.md) for the deployment contract (TLS gateway, read-only DB role, private Postgres).
125122

123+
The API is a public, read-only service and is meant to run **behind a TLS-terminating gateway**, against a **read-only** Postgres role. Before exposing it publicly, read [`docs/security.md`](./docs/security.md) — it covers the security model, network architecture, the built-in abuse protections, and a deployment checklist.
124+
126125
## Contributing
127126

128127
- AI coding agents: read [`AGENTS.md`](./AGENTS.md) first.

‎deploy/README.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -43,9 +43,10 @@ kubectl apply -f deploy/kubernetes.yaml
4343

4444
Put a TLS-terminating Ingress/gateway in front and set **`TRUST_PROXY` to the
4545
number of hops** it adds. The gateway must set `X-Forwarded-For`, but the API
46-
ignores that header while `TRUST_PROXY=0` (the safe default for a directly
47-
exposed server), so leaving it unset behind an ingress collapses every client
48-
into a single rate-limit bucket. See [`docs/security.md`](../docs/security.md).
46+
does not assume a safe default hop count: while `TRUST_PROXY` is unset, rate
47+
limiting is disabled with a startup warning. Use `TRUST_PROXY=0` only for a
48+
directly exposed server; behind an ingress, set the real hop count so clients do
49+
not collapse into one bucket. See [`docs/security.md`](../docs/security.md).
4950

5051
## Docker Compose — [`docker-compose.prod.yml`](./docker-compose.prod.yml)
5152

‎docs/getting-started.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -318,6 +318,7 @@ Replace `B62...` with the address of the zkApp whose events you want.
318318

319319
## Where to go next
320320

321+
- [Security & deployment hardening](./security.md) — read before exposing the API publicly
321322
- [Schema reference](../schema.graphql) — the full GraphQL surface
322323
- [Mina archive node docs](https://docs.minaprotocol.com/node-operators/archive-node) — what an archive node is and how to run one
323324
- [`AGENTS.md`](../AGENTS.md) — orientation for AI coding agents working in this repo

‎docs/runbook.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ Recovery semantics to expect:
9292
| ---------------------------------------- | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
9393
| `/readiness` 503, `/healthcheck` 200 | Postgres unreachable | check DB health/network; pods recover automatically when it returns |
9494
| p99 latency climbing, `in_flight` rising | slow/expensive queries or DB CPU | check DB load; review slow queries; confirm `PG_STATEMENT_TIMEOUT` is set; known consumers give up after 20s, so treat that as the practical latency ceiling |
95-
| Many 429s, across unrelated clients | `TRUST_PROXY` unset or too low behind a gateway, so clients share buckets | set `TRUST_PROXY` to the exact trusted-proxy hop count for the topology (GCP external ALB commonly needs `2`); the app logs a warning on startup when it sees `X-Forwarded-For` with `TRUST_PROXY=0` |
95+
| Many 429s, across unrelated clients | `TRUST_PROXY=0` or the wrong hop count behind a gateway, so clients share buckets | set `TRUST_PROXY` to the exact trusted-proxy hop count for the topology (GCP external ALB commonly needs `2`); the app logs a warning on startup when it sees `X-Forwarded-For` with `TRUST_PROXY=0` |
9696
| Many 429s, one client | a client over the rate limit, or limits too low | confirm the gateway sets `X-Forwarded-For`; adjust `RATE_LIMIT_MAX` |
9797
| Connection-pool exhaustion errors | `PG_MAX_CONNECTIONS` × replicas > DB capacity | lower pool size or raise DB `max_connections` |
9898
| Memory growth / OOM kills | heavy result sets or a leak | lower `BLOCK_RANGE_SIZE`; inspect heap metrics; cap container memory |

‎docs/security.md‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
# Security & Deployment Hardening
2+
3+
This document describes the intended security posture of the Archive Node API
4+
and how to deploy it safely on a public network. It complements the
5+
[setup guide](./getting-started.md); read that first for installation and the
6+
full [configuration reference](./getting-started.md#configuration).
7+
8+
## Security model
9+
10+
The Archive Node API is a **public, read-only** GraphQL service over an existing
11+
archive-node Postgres database. It exposes already-public on-chain data (blocks,
12+
events, actions, transactions) and **never writes** to the database or the chain.
13+
14+
Consequences of that model:
15+
16+
- **No application-level authentication.** The API is meant to be openly
17+
queryable, the same way a block explorer's read API is. Access control, if you
18+
need it, is enforced at the gateway in front of the service (see below) — not
19+
in the app.
20+
- **The data is not secret; availability is the asset to protect.** The main
21+
threat is abuse that degrades the service or the backing Postgres for everyone.
22+
The hardening below is aimed at that.
23+
24+
> If you require per-caller authentication or quotas, terminate it at the
25+
> gateway (API keys, JWT, or mTLS). The application is intentionally kept simple
26+
> and unauthenticated; gating is an operator concern.
27+
28+
## Network architecture
29+
30+
Run the API behind a **TLS-terminating reverse proxy or load balancer**. The
31+
application itself speaks plain HTTP on `PORT` and does not terminate TLS.
32+
33+
```
34+
┌─────────────────────────┐
35+
client ───▶│ TLS gateway / LB │ (HTTPS, X-Forwarded-For,
36+
(HTTPS) │ nginx / Envoy / ALB │ request-size limits, optional auth)
37+
└────────────┬────────────┘
38+
│ HTTP (private network)
39+
┌────────────▼────────────┐
40+
│ Archive Node API │ (this service, :8080)
41+
└────────────┬────────────┘
42+
│ TCP (private network)
43+
┌────────────▼────────────┐
44+
│ Postgres (archive DB) │ read replicas, not publicly reachable
45+
└─────────────────────────┘
46+
```
47+
48+
Requirements:
49+
50+
- **TLS at the gateway.** Never expose the plain-HTTP app port to the internet.
51+
- **Set `X-Forwarded-For` and `TRUST_PROXY` together (from 1.0.0).** A gateway
52+
should append `X-Forwarded-For`, and the API derives the rate-limit client
53+
from that header only as far as `TRUST_PROXY` allows: it names how many proxy
54+
hops sit in front of the API, and the client is read that many entries from the
55+
_right_ of the header — the part your own proxies appended. `TRUST_PROXY` has
56+
no default; while it is unset, rate limiting is disabled with a startup
57+
warning. Use `TRUST_PROXY=0` only for a directly exposed server. Behind a
58+
gateway, set the real hop count for your topology. A generic single reverse
59+
proxy is often `1`; a GCP external Application Load Balancer commonly needs
60+
`2` because it appends two `X-Forwarded-For` entries. Too low collapses
61+
clients onto a proxy address; too high can trust caller-prepended entries.
62+
- **Keep Postgres private.** The database must not be reachable from the public
63+
internet — only from the API instances.
64+
65+
## Built-in protections
66+
67+
The service ships with abuse controls that are safe by default and tunable via
68+
the [configuration](./getting-started.md#configuration):
69+
70+
| Protection | Default | Purpose |
71+
| --------------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------ |
72+
| Per-IP **rate limiting** | on once `TRUST_PROXY` is set | Bounds request volume per client; disabled with a startup warning while `TRUST_PROXY` is unset |
73+
| GraphQL **query-cost limits** (depth / aliases / tokens / cost) | on | Rejects expensive/abusive query shapes before execution |
74+
| Postgres **statement timeout** & pool limits | on | Caps how long/much a single query can consume |
75+
| **CORS** | same-origin only | Cross-origin browser access is opt-in — see the caveat below before locking it down |
76+
| **Introspection** | off | Schema introspection disabled unless explicitly enabled |
77+
| Field-suggestion blocking | on | Hides `Did you mean ...?` suggestions while preserving GraphQL validation text |
78+
79+
> **These controls arrive in 1.0.0.** On `0.0.x` releases they are absent or
80+
> default-open, or have older env parsing — notably `CORS_ORIGIN` defaults to
81+
> `*` there, so cross-origin access is wide open rather than opt-in.
82+
> Introspection disabling already exists on `0.0.x`, but any non-empty
83+
> `ENABLE_INTROSPECTION` value, including `false`, enables it. Check your
84+
> running version before relying on any row above.
85+
86+
Tune these to your traffic; see the configuration table for the exact
87+
environment variables and defaults.
88+
89+
### CORS and browser clients
90+
91+
Cross-origin browser clients **cannot reach this API unless their web origin is
92+
allowlisted in `CORS_ORIGIN`** — and when they fail, they fail silently from the
93+
server's point of view: the browser blocks the response and the server logs stay
94+
clean. This catches people out, so decide deliberately:
95+
96+
- **A genuinely public read API** that any browser may call — including the
97+
[mina-explorer](https://github.com/o1-labs/mina-explorer) and third-party
98+
dashboards — wants `CORS_ORIGIN=*`. That is the correct setting here, not a
99+
lapse in hardening: the data is already public, and CORS is not an access
100+
control (it constrains browsers, not `curl` or a server-side client).
101+
- **A deployment with a known, fixed set of front-ends** wants those origins
102+
listed explicitly. This only limits which _browser pages_ may read responses;
103+
it does not restrict anyone else.
104+
105+
## Least-privilege database access
106+
107+
The API only ever issues `SELECT`s. Give it a **read-only** Postgres role rather
108+
than a superuser or the archive ingest user:
109+
110+
```sql
111+
-- one-time setup on the archive database
112+
CREATE ROLE archive_api_ro LOGIN PASSWORD 'change-me';
113+
GRANT CONNECT ON DATABASE archive TO archive_api_ro;
114+
GRANT USAGE ON SCHEMA public TO archive_api_ro;
115+
GRANT SELECT ON ALL TABLES IN SCHEMA public TO archive_api_ro;
116+
-- so the role can also read tables added by future archive migrations
117+
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO archive_api_ro;
118+
```
119+
120+
Then point `PG_CONN` at `archive_api_ro`. Even in the event of a query-layer bug,
121+
the credentials cannot modify or delete data.
122+
123+
## Operational practices
124+
125+
- **Secrets:** pass `PG_CONN` (and any gateway secrets) via environment / a
126+
secret manager, never bake them into the image. Prefer SSL to Postgres
127+
(`?sslmode=require`) when the DB is on a managed provider.
128+
- **Keep `ENABLE_GRAPHIQL` / `ENABLE_INTROSPECTION` off in production** unless you
129+
intentionally want a public playground.
130+
- **Updates:** track and apply dependency and base-image security updates
131+
(supply-chain scanning is part of the [production-readiness work](https://github.com/o1-labs/Archive-Node-API/issues/163)).
132+
133+
## Deployment checklist
134+
135+
- [ ] TLS terminated at a gateway; plain-HTTP app port not publicly exposed
136+
- [ ] `TRUST_PROXY` explicitly set: `0` only for direct exposure, or the real
137+
hop count behind a gateway; rate limiting is disabled until this is set
138+
- [ ] Gateway sets `X-Forwarded-For`
139+
- [ ] Postgres reachable only from the API, not the public internet
140+
- [ ] API uses a **read-only** Postgres role
141+
- [ ] Rate-limit and query-cost limits reviewed for your expected traffic
142+
- [ ] `CORS_ORIGIN` matches your clients: `*` for a public API any browser may
143+
call, or an explicit allowlist if your front-ends are known and fixed —
144+
leaving it unset blocks all cross-origin browser clients
145+
- [ ] `ENABLE_GRAPHIQL` and `ENABLE_INTROSPECTION` off (unless intentionally
146+
public) — on `0.0.x`, leave them unset; any non-empty value, including
147+
`false`, enables introspection
148+
- [ ] Secrets injected via env / secret manager; SSL to Postgres where applicable
149+
150+
## Scope
151+
152+
This document covers deploying _this service_ securely. It does not cover
153+
securing the upstream Mina archive node or its Postgres ingest pipeline. Broader
154+
production-readiness work (observability, readiness probes, supply-chain
155+
scanning, runbooks) is tracked in the
156+
[production-readiness epic](https://github.com/o1-labs/Archive-Node-API/issues/163).

0 commit comments

Comments
 (0)