Skip to content

architecture.md

Latest commit

 

History

History
134 lines (115 loc) · 6.22 KB

architecture.md

File metadata and controls

134 lines (115 loc) · 6.22 KB
Raw

Architecture

How Forge fits together. This is the map from the concepts to the running system — the services, the packages behind them, and how a spec becomes a merged, tracked change. For standing it up, see Getting started and Self-hosting. The full platform specification lives in FORGE_SPEC.md.

Design principles

  • Self-hosted first. Every component ships in one hardened Docker Compose stack (or a Helm chart) that you own end to end. No hosted control plane is required.
  • Postgres-backed substrate. Board, workflow state, specs, knowledge vectors, and audit all live in one Postgres (with pgvector) — one database to back up, one source of truth.
  • Spec-driven. Work is gated on an approved spec, not a free-form prompt.
  • Contracts at the boundaries. Frozen Pydantic DTOs and Protocols (forge_contracts) define the seams between packages, so implementations swap without breaking callers.

Runtime topology

The Compose/Helm stack runs these services:

Service Package(s) Responsibility
web @forge/web (Next.js 16) The UI — board, specs, runs, approvals, admin
api forge_api (FastAPI) + CLI HTTP API, auth, spec engine, orchestration entrypoints
worker forge_worker (Celery) Async execution: agent runs, knowledge indexing, syncs
mcp-gateway forge_mcp_gateway Manages MCP tool-source clients for agents
db Postgres + pgvector Board, workflow FSM, specs, vectors, audit — the substrate
redis Redis Celery broker/result backend, rate-limit + cache state
minio MinIO (S3 API) Object storage (artifacts, blobs)
caddy Caddy TLS-terminating edge proxy in front of web + api

make dev builds and starts all of them, runs Alembic migrations, and seeds a demo workspace.

Package map

The Python workspace is managed by uv; the web app is a separate pnpm workspace. Business logic lives in packages/* behind contracts, and the apps/* services compose them.

apps/
  api/            forge_api            FastAPI backend + CLI
  worker/         forge_worker         Celery workers
  mcp-gateway/    forge_mcp_gateway    MCP client manager
  web/            @forge/web           Next.js 16 frontend
packages/
  contracts/      forge_contracts      Frozen Pydantic DTOs + Protocols (the seams)
  db/             forge_db             SQLAlchemy models + Alembic migrations
  spec-engine/    forge_spec           Spec validation + implementation gate
  workflow-engine/forge_workflow       Postgres FSM workflow layer
  agent-runtime/  forge_agent          LangGraph plan/execute/verify + sandboxes
  multi-agent-coordinator/ forge_coordinator  Fan-out / supervision
  board-core/     forge_board          Native project board
  knowledge-core/ forge_knowledge      Hybrid retrieval (pgvector + FTS + RRF + rerank)
  integration-sdk/forge_integrations   Integrations incl. PM adapters
  mcp-sdk/        forge_mcp            MCP gateway SDK
  policy-sdk/     forge_policy         .forge/policy.yaml engine
  skill-sdk/      forge_skill          Skill profiles
  evaluation/     forge_eval           Eval harness + release-readiness gate
  auth-sdk/       forge_auth           Auth, secrets vault, BYOK, SSO/SCIM
  authz-sdk/      forge_authz          Multi-team RBAC
  approval-sdk/   forge_approval       Human approval gates
  marketplace-sdk/forge_marketplace    Integration marketplace
  observability/  forge_obs            Telemetry + cost metrics
  deploy-core/    forge_deploy         Deployment gates

How a change flows

 1. Author spec (manifest.yaml)         forge_spec
        │  implementation gate: approved?
        ▼
 2. Workflow drives plan→execute→verify  forge_workflow (Postgres FSM)
        │
        ▼
 3. Agent run in a sandbox               forge_agent (LangGraph) + sandbox provider
        │  grounded by hybrid retrieval  forge_knowledge (pgvector + FTS + RRF)
        │  tools via MCP gateway          forge_mcp_gateway
        │  constrained by policy          forge_policy
        ▼
 4. Opens a pull request                 GitHub App integration
        │
        ▼
 5. Sensitive actions pause for approval  forge_approval
        │
        ▼
 6. Tracked on the board + audited        forge_board + audit log + forge_obs (cost)

Every hop is recorded: the run-trace viewer replays step 3, the cost metrics attribute the model spend, and the append-only audit log records the decisions in steps 4–6.

Data & security substrate

  • Secrets are envelope-encrypted in a per-workspace vault (forge_auth): a versioned KEK wraps per-workspace DEKs with workspace-id AAD binding, so a blob is useless outside its workspace. BYOK model keys use the same path.
  • AuthN/Z: API keys (peppered), agent tokens with a TTL, SAML SSO + SCIM provisioning, and multi-team RBAC (forge_authz).
  • Network hardening: outbound allowlist, SSRF protections, request-body limits, and rate limiting are configured through FORGE_* env vars.
  • Supply chain: digest-pinned images, per-image SBOMs, and a wired enforcement-matrix regression suite. See the threat model and security policy.

Deployment options

Path Use Reference
Docker Compose (dev) Local evaluation make dev
Docker Compose (prod) Single-host, hardened, digest-pinned docker-compose.md
Kubernetes / Helm Clustered kubernetes.md
Infrastructure as Code Provisioned cloud (Hetzner + Cloudflare + Fly) iac.md

Tech stack

  • Backend: Python 3.14, FastAPI, Pydantic v2, SQLAlchemy 2.x, Alembic
  • Agents / workflow: LangGraph, Postgres FSM, Temporal, Redis + Celery
  • Knowledge / RAG: pgvector (cosine) + Postgres full-text, RRF fusion (k=60), reranker
  • Frontend: Next.js 16, React 19, TypeScript, Tailwind CSS v4, shadcn/ui, TanStack Query/Table
  • Infra: Docker Compose, Caddy, MinIO
  • Tooling: uv + Ruff + mypy (Python), pnpm (Node)