Skip to content

architecture.md

Latest commit

 

History

History
368 lines (311 loc) · 19.3 KB

architecture.md

File metadata and controls

368 lines (311 loc) · 19.3 KB
Raw

Architecture - Guard Proxy

High-Level Overview

Guard Proxy is a self-hosted reverse proxy WAF. The M1 stack runs HAProxy, Coraza SPOA with OWASP CRS, the FastAPI backend, the React frontend, and PostgreSQL through Docker Compose.

graph TB
    C[Clients] -->|HTTP on :8080| H[HAProxy]
    H -.->|SPOE request inspection| CS[Coraza SPOA + OWASP CRS]
    CS -.->|allow / deny / error metadata| H
    H -->|allowed requests| BE[FastAPI Backend]

    FE[React Admin UI] -->|REST API| BE
    BE --> DB[(PostgreSQL)]
    BE -.->|writes generated runtime config| RV[(guard_proxy_runtime)]
    RV -.->|read-only mount| H
    RV -.->|read-only rule overrides| CS
    CS -.->|polls /runtime/current| CS
    CS -->|JSON audit events| AF[(coraza_audit volume)]
    AF -->|tailed by| LS[Log Shipper sidecar]
    LS -->|POST /logs/ingest| BE
Loading

Request Flow

sequenceDiagram
    Client->>HAProxy: HTTP request
    HAProxy->>Coraza: SPOE request metadata
    Coraza->>Coraza: Evaluate request-phase CRS rules
    alt WAF inspection fails
        HAProxy->>Client: 503 Service Unavailable
    else CRS blocks request
        Coraza->>HAProxy: deny decision + anomaly metadata
        HAProxy->>Client: 403 Forbidden
    else Request allowed
        Coraza->>HAProxy: allow decision + anomaly metadata
        HAProxy->>Backend: Forward request
        Backend->>Client: Response
    end
Loading

M1 focuses on request inspection. HAProxy sends method, path, query, headers, and body data to Coraza SPOA over SPOE. Coraza evaluates the request against the configured OWASP CRS bundle and returns transaction variables such as the action and anomaly score. HAProxy blocks deny decisions with 403. If SPOE inspection is unavailable or returns an error, HAProxy fails closed with 503 Service Unavailable, X-WAF-Status: degraded, and a machine-readable degraded reason header.

Components

Component Role Location
HAProxy Reference reverse proxy, host routing, SPOE filter, WAF enforcement, degraded-mode handling configs/haproxy/
Coraza SPOA + OWASP CRS Request-phase WAF inspection, CRS anomaly scoring, JSON audit log configs/coraza/, docker/coraza.Dockerfile
Log Shipper sidecar Tails Coraza's JSON audit file and ships each event to POST /logs/ingest with exponential backoff src/log-shipper/
FastAPI Backend Control-plane API for auth, vhosts, policies, rule overrides, logs, and health/readiness probes src/backend/
React Frontend Admin panel SPA built with React, TypeScript, Vite, Tailwind CSS, and pnpm src/frontend/
PostgreSQL Docker Compose database for backend state docker/docker-compose.yml
Docker Compose Stack Local full-stack orchestration, health checks, networks, logs, and persistent volumes docker/

Data Flow

Request Processing

  1. Client sends an HTTP request to HAProxy.
  2. HAProxy rejects unknown hosts with 421 before WAF inspection.
  3. HAProxy sends request-phase metadata to Coraza SPOA through SPOE.
  4. Coraza evaluates OWASP CRS rules and returns allow/deny metadata.
  5. HAProxy returns 403 for denied traffic, 503 for WAF inspection failures (coraza-unavailable or spoe-processing-error), or forwards allowed requests to the FastAPI backend.

Policy Management

  1. Admin users manage vhosts, policies, and rule overrides through the React UI and FastAPI API.
  2. FastAPI persists control-plane state in PostgreSQL.
  3. FastAPI writes generated runtime config into the shared guard_proxy_runtime volume.
  4. HAProxy reads the active generated haproxy.cfg from the volume and reloads through its Runtime API socket when POST /config/apply succeeds.
  5. Coraza reads the active generated rule-overrides.conf from the same volume after CRS rules are loaded. The Coraza container runs a supervisor that polls the /runtime/current symlink; when the backend atomically swaps it, the supervisor restarts coraza-spoa automatically — no Docker socket access required.

Runtime Event Ingestion

  1. Coraza writes one JSON audit event per newline to /var/log/coraza/audit.log on the coraza_audit Docker named volume (SecAuditEngine RelevantOnly — only transactions where at least one rule fired produce an entry).
  2. The log-shipper sidecar (src/log-shipper/) tails the audit file from a persisted byte-offset checkpoint. For each complete line it maps the Coraza JSON to a LogIngestRequest payload (see configs/coraza/README.md for the field mapping) and POSTs it to /logs/ingest with X-Guard-Proxy-Ingest-Secret.
  3. The offset is only advanced and persisted after a 2xx response or a deliberate skip (parse error or 4xx rejection). Transient failures (5xx, network errors, 429) trigger exponential backoff without advancing the offset — so a backend outage stalls the pipeline rather than dropping events. The coraza_audit file itself is the durable buffer.
  4. Idempotency is guaranteed via producer_event_id = Coraza transaction.id; the backend returns 200 for a duplicate rather than creating a second row.
  5. FastAPI validates and normalizes payloads into the persisted Log model. During ingest, the backend resolves the current vhost row by domain and stores nullable vhost_id and policy_id snapshots on the log row. GET /logs policy filtering uses the stored logs.policy_id, so results reflect the policy assignment captured at event-ingest time, not a later vhost reassignment.
  6. GET /logs exposes stored events for the admin panel log viewer.

Log Retention

LOG_RETENTION_DAYS (default 30) controls how long Log rows are kept. A daily APScheduler job (purge_old_logs in src/backend/app/services/scheduler.py) deletes rows whose event_at is older than the threshold and logs the deleted count at INFO. Admins can also trigger an immediate cleanup via POST /logs/cleanup. Both paths share purge_logs_older_than in src/backend/app/services/log_retention.py.

Limitation — one log row per request, even when multiple rules fire. A single Coraza transaction can trigger several CRS rules at once (e.g. an SQL injection probe matching 942100, 942190, 942270, and 942360 simultaneously). The Log model has no per-rule child table, so the shipper picks the first rule-bearing message as rule_id/rule_message (_primary_rule_data in src/log-shipper/app/mapping.py); the request still produces exactly one ingested log row, and anomaly_score reflects the combined score from all matched rules, not just the first one. The other matched rules are not lost — they remain in raw_context (the full Coraza event JSON) for manual inspection in the log detail view, but are not independently queryable or filterable. See src/log-shipper/tests/test_mapping.py::test_multi_rule_request_collapses_to_one_ingest_payload and ::test_multi_rule_request_preserves_all_matched_rules_in_raw_context for the tested behavior. A per-rule breakdown is deferred post-MVP.

GeoIP Country Filtering

A policy can restrict a vhost by client country (issue #175) with geoip_mode (off / allowlist / blocklist) and geoip_countries (a list of ISO 3166-1 alpha-2 codes). The mechanism is intentionally native to HAProxy:

  1. app.services.geoip_service.download_database() downloads the free, no-registration, no-license-key country-level MMDB published by ip66.dev from GEOIP_DATABASE_URL. GeoIP data is provided by ip66.dev and licensed under CC BY 4.0. The download is a conditional GET using the ETag/Last-Modified validators persisted from the previous run; a 304 Not Modified response is a no-op.
  2. generate_map_file() converts the MMDB into a plain-text HAProxy map file (CIDR <space> ISO-country-code per line), written atomically (os.replace()) to <runtime_root>/geoip/country.map on the shared guard_proxy_runtime volume — the same volume HAProxy reads at /etc/haproxy/generated. Because ip66.dev splits blocks by ASN as well as by country, adjacent networks are grouped by (country_code, ip_version) and merged with ipaddress.collapse_addresses() before writing, cutting roughly 1.3M raw networks down to ~920k lines (~18 MB). Merging is done per run of consecutive same-country networks as the reader streams them, not by buffering everything first: only networks adjacent in address space can merge, so the result is identical while peak memory stays around 70 MB instead of ~780 MB.
  3. The generated haproxy.cfg uses HAProxy's native map_ip() fetch against that file to resolve src to a country in var(txn.geoip_country), then denies with 403 when the resolved (or absent) country violates the configured mode. No Lua and no HAProxy MMDB module are required — the map lookup is table-driven and fully supported by stock HAProxy, and haproxy -c never fails because of a missing map: a non-empty stub map is always written first if no real database has been downloaded yet. Country codes are emitted as repeated same-name ACLs of 50 codes each, which HAProxy ORs together — HAProxy truncates any config line after 64 words, so a single-line list of every ISO code yields a fatally invalid config. Only ACME challenges are exempt from the deny rules (they have their own local backend_acme); /health deliberately is not, because it is a host-independent path match routed to the customer origin and exempting it would make it a trivial full bypass of country filtering. Loading the ~920k-entry map costs HAProxy roughly 280 MB RSS (measured on haproxy:3.0-alpine), which is the main operational cost of this feature. Budget for double that on the host: a reload runs the old and new processes side by side, so the peak is ~560 MB. The map_ip() lookup itself is emitted once per frontend and so runs for all traffic, including vhosts with geoip_mode = off; a radix lookup is cheap, but the resident map is not.
  4. A daily APScheduler job (refresh_geoip_database, interval controlled by GEOIP_REFRESH_INTERVAL_DAYS, matching ip66.dev's daily rebuild cadence) re-downloads the MMDB and regenerates the map. POST /geoip/refresh (admin only) triggers the same pipeline on-demand. Both paths reload HAProxy only when the generated map actually changed, so a no-op refresh does not cause an unnecessary reload.

Fail-open by default. Unlike the SPOE WAF inspection path above — which fails closed with 503 because an unavailable Coraza SPOA is a security control failure that must stop traffic — GeoIP filtering fails open (allows the request) by default when the map is missing, the database has never been downloaded, or an IP cannot be resolved to a country. A stale or absent geolocation database is not a security control failure in the same sense; it must never take a site offline on its own. This default is controlled by GEOIP_FAIL_OPEN (default true); set GEOIP_FAIL_OPEN=false to fail closed instead, at the cost of blocking traffic whenever geolocation data is unavailable. Note this setting is read when the config is generated, not at request time — it is baked into the rendered haproxy.cfg, so changing the environment variable takes effect only after a full config re-apply, not on a plain HAProxy reload.

Not every address resolves to a country. Around 306k networks in the upstream database carry no country at all, and a further ~90k are labelled EU — a Europe-wide allocation that is not an ISO 3166-1 country. Those are left unresolved, so with the default fail-open they are allowed even in allowlist mode: an allowlist of PL alone still admits every EU-labelled network. Set GEOIP_FAIL_OPEN=false if that is unacceptable. The one code that is rewritten is UK, which the database uses for a handful of networks that ISO 3166-1 spells GB; without that alias, blocking GB would silently miss them.

Authentication & Rate Limiting

The FastAPI backend issues short-lived JWT access tokens (30 min, HS256) and long-lived refresh tokens (7 days) stored in an HttpOnly cookie. The refresh cookie path defaults to / so it is sent through proxied API prefixes such as /api/v1/auth/refresh, while remaining unavailable to JavaScript. The frontend keeps no long-lived secret in memory or localStorage.

Brute-force protection

POST /auth/login and POST /auth/refresh are rate-limited to 5 requests per minute per client IP using slowapi. Exceeding the limit returns:

HTTP/1.1 429 Too Many Requests
Retry-After: 60

The limit is enforced in-process (in-memory via limits.MemoryStorage), which is appropriate for a single-uvicorn-process deployment.

Client IP resolution

HAProxy sits in front of the backend. The frontend config includes:

http-request set-header X-Forwarded-For %[src]

This overwrites any client-supplied X-Forwarded-For value with the real source IP before the request reaches the backend, preventing header-spoofing bypass of the rate limit. The backend's key function reads the first (and only) XFF entry and falls back to the socket peer when the header is absent (direct connections in development and test).

Timing-attack mitigation

The login handler always runs bcrypt.verify against a precomputed dummy hash when the requested email does not exist, keeping response time consistent and preventing user-enumeration through timing.

CORS policy

CORS is restricted to the methods and headers the panel actually uses:

  • allow_methods: GET, POST, PATCH, DELETE, OPTIONS
  • allow_headers: Authorization, Content-Type
  • allow_credentials: true (required for the HttpOnly refresh cookie)
  • allow_origins: driven by settings.cors_origins (env var CORS_ORIGINS, CSV or JSON array); the built-in default lists only local dev origins (localhost/127.0.0.1 on ports 3000, 5173, 5174) and must be overridden with the production panel origin(s) via environment configuration before deployment.

Deployment

Health and Readiness Probes

The backend exposes two probe endpoints, both unauthenticated:

Endpoint Type Returns Checks
GET /health Liveness 200 always None — proves the process is alive
GET /ready Readiness 200 / 503 DB connectivity (SELECT 1), runtime config volume writable

/ready returns a JSON body with per-check status so operators can tell which dependency is down:

// 200 — all clear
{"status": "ready", "checks": {"database": {"status": "ok"}, "runtime_config": {"status": "ok"}}}

// 503 — one or more dependencies unavailable
{"status": "not ready", "checks": {"database": {"status": "ok"}, "runtime_config": {"status": "error", "detail": "/var/lib/guard-proxy/generated is not a writable directory"}}}

The Docker Compose healthcheck for the backend service targets /ready. Almost every other service in the stack (frontend, haproxy, coraza, log-shipper) depends on backend with condition: service_healthy, so /ready must pass before dependents start. This prevents dependents from launching while the database is still initialising or the config volume is unavailable.

/health is retained as a lightweight liveness probe for orchestrators that separately track process aliveness (e.g. a future Kubernetes livenessProbe).

Development (Docker Compose)

The implemented M1 stack lives in docker/docker-compose.yml.

services:
  haproxy:      # host port 8080 -> container port 80
  coraza:       # internal port 9000 (SPOE)
  log-shipper:  # tails coraza_audit volume, ships to backend
  backend:      # internal port 8000 (FastAPI)
  frontend:     # host port 3000 -> Vite port 5173
  postgres:     # internal port 5432

Prepare docker/.env from docker/.env.example, then use make run for the normal stack or make dev for HAProxy -d output and Coraza debug logging. The end-to-end smoke test is benchmarks/smoke/e2e.sh; it starts the stack, waits for healthy services, checks a benign request, checks that a SQL injection request is blocked, and tears the stack down.

Shared Runtime Volume

Generated runtime artifacts live in the Docker named volume guard_proxy_runtime. The backend mounts it read-write at /var/lib/guard-proxy/generated, HAProxy mounts the same volume at /etc/haproxy/generated, and Coraza mounts it read-only at /runtime.

The active release is selected through the current symlink:

/runtime/current/
  haproxy.cfg           # generated HAProxy config
  crs-setup.conf        # generated CRS setup snapshot
  rule-overrides.conf   # generated CRS rule removals

The backend container starts as root only long enough to create and seed Coraza's generated rule override include, assign the runtime volume to the non-root app user, and then drops privileges before running migrations and Uvicorn. HAProxy copies the checked-in reference config into the same seed release when no generated haproxy.cfg exists yet.

The Coraza container image is built on alpine:3.19 with tini as PID 1. A shell supervisor (coraza-supervisor.sh) drops to the non-root coraza user, starts coraza-spoa as a child process, and polls /runtime/current once per second. Coraza writes JSON audit events to /var/log/coraza/audit.log on the coraza_audit named volume, which is mounted read-only by the log-shipper sidecar. The earlier inotify-based supervisor looked simpler, but it did not reliably observe the backend's atomic current symlink replacement through the read-only runtime volume mount while Coraza kept using the previous loaded rules. Polling the symlink target is intentionally less clever but directly tests the state Coraza includes. When the target changes, the supervisor restarts coraza-spoa — picking up the new rule-overrides.conf without any external signal or Docker socket access. If the child process exits, the supervisor exits non-zero so Compose can restart the container. Note that this is a full process restart, not a hot-reload: port 9000 is briefly unavailable (~sub-second) during the restart, causing HAProxy SPOE to return an error for any request that lands in that window. This is acceptable for a manual rule-apply operation.

Key Decisions

See notes/decisions/ for Architecture Decision Records:

  • ADR-001 - FastAPI over Flask/Django
  • ADR-002 - PostgreSQL + SQLite dev
  • ADR-003 - React + TypeScript
  • ADR-004 - Docker Compose deployment
  • ADR-006 - Synchronous SQLAlchemy for MVP
  • ADR-007 - Coraza SPOA integration approach
  • ADR-008 - Log shipper: custom Python sidecar over Vector/Fluent Bit

M0-08 — Rate limiting (slowapi, in-memory): The auth endpoints are guarded by a 5/minute per-IP limit via slowapi with in-memory storage. A Redis-backed distributed limiter was considered but is not needed for a single-process deployment; the simpler in-process approach avoids an external dependency.