|
| 1 | +# Run a Cluster on S3 |
| 2 | + |
| 3 | +This guide takes a cluster from a local config directory to a server that boots |
| 4 | +**config-free from an object-storage bucket** — the bucket is the whole |
| 5 | +deployment artifact. For the full control-plane reference, see |
| 6 | +[operating a cluster](../clusters/index.md) and |
| 7 | +[cluster config](../clusters/config.md). |
| 8 | + |
| 9 | +## 1. Declare the cluster |
| 10 | + |
| 11 | +Lay out a config directory. The one S3-specific line is `storage:` — it puts the |
| 12 | +state ledger, catalog, and graph data on the bucket instead of in the folder: |
| 13 | + |
| 14 | +``` |
| 15 | +company-brain/ |
| 16 | +├── cluster.yaml |
| 17 | +├── people.pg |
| 18 | +├── queries/ |
| 19 | +│ └── people.gq |
| 20 | +└── base.policy.yaml |
| 21 | +``` |
| 22 | + |
| 23 | +```yaml |
| 24 | +# cluster.yaml |
| 25 | +version: 1 |
| 26 | +storage: s3://my-bucket/clusters/company-brain # the deployment lives here |
| 27 | +metadata: |
| 28 | + name: company-brain |
| 29 | +graphs: |
| 30 | + knowledge: |
| 31 | + schema: people.pg |
| 32 | + queries: queries/ |
| 33 | +policies: |
| 34 | + base: |
| 35 | + file: base.policy.yaml |
| 36 | + applies_to: [knowledge] |
| 37 | +``` |
| 38 | +
|
| 39 | +Set the S3 credentials in the environment (for a non-AWS S3-compatible store such |
| 40 | +as MinIO or RustFS, also set `AWS_ENDPOINT_URL_S3`): |
| 41 | + |
| 42 | +```bash |
| 43 | +export AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... AWS_REGION=us-east-1 |
| 44 | +# export AWS_ENDPOINT_URL_S3=https://... # non-AWS S3-compatible stores |
| 45 | +``` |
| 46 | + |
| 47 | +## 2. Validate, plan, apply |
| 48 | + |
| 49 | +`apply` is the only command that changes the world; `plan` previews it: |
| 50 | + |
| 51 | +```bash |
| 52 | +omnigraph cluster validate --config company-brain # parse + typecheck |
| 53 | +omnigraph cluster import --config company-brain # create the state ledger |
| 54 | +omnigraph cluster plan --config company-brain # preview the diff |
| 55 | +omnigraph cluster apply --config company-brain # converge onto the bucket |
| 56 | +``` |
| 57 | + |
| 58 | +`apply` creates the graph at the derived root |
| 59 | +(`s3://my-bucket/clusters/company-brain/graphs/knowledge.omni`), applies its |
| 60 | +schema, and publishes the query and policy into the content-addressed catalog. |
| 61 | +`converged: true` means there is nothing left to do — re-running `apply` is always |
| 62 | +safe. |
| 63 | + |
| 64 | +## 3. Load data |
| 65 | + |
| 66 | +The control plane manages *definitions*; rows go through the normal data plane. |
| 67 | +Address the graph by its storage URI (the derived `graphs/<id>.omni` root): |
| 68 | + |
| 69 | +```bash |
| 70 | +omnigraph load --data seed.jsonl --mode overwrite \ |
| 71 | + s3://my-bucket/clusters/company-brain/graphs/knowledge.omni |
| 72 | +``` |
| 73 | + |
| 74 | +## 4. Serve config-free from the bucket |
| 75 | + |
| 76 | +A serving host needs only the storage-root URI and credentials — no checkout of |
| 77 | +the config repo: |
| 78 | + |
| 79 | +```bash |
| 80 | +OMNIGRAPH_SERVER_BEARER_TOKENS_JSON='{"act-reader":"s3cret"}' \ |
| 81 | + omnigraph-server --cluster s3://my-bucket/clusters/company-brain --bind 0.0.0.0:8080 |
| 82 | +``` |
| 83 | + |
| 84 | +The server boots from the **applied revision** recorded in the ledger — never from |
| 85 | +config that was merely written. Roll out a change by `apply`-ing again, then |
| 86 | +restarting replicas. |
| 87 | + |
| 88 | +## 5. Maintain it |
| 89 | + |
| 90 | +Storage maintenance runs out-of-band, addressed by cluster + graph name (it |
| 91 | +resolves the graph's storage URI from the served state): |
| 92 | + |
| 93 | +```bash |
| 94 | +omnigraph optimize --cluster company-brain --cluster-graph knowledge |
| 95 | +omnigraph cleanup --cluster company-brain --cluster-graph knowledge --keep 10 --confirm |
| 96 | +``` |
| 97 | + |
| 98 | +See [maintenance](../operations/maintenance.md) for what each command does. |
0 commit comments