Repository navigation
feat: central vault ingot unlocks against #37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
alanshaw
wants to merge
1
commit into
main
Choose a base branch
from
ash/feat/central-openbao
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file modified
BIN
+195 Bytes
(100%)
snapshots/3-piri-postgres-s3/volumes/hilt-postgres-data.tar.gz
Binary file not shown.
Binary file not shown.
Binary file modified
BIN
+2.8 KB
(100%)
snapshots/3-piri-postgres-s3/volumes/ingot-openbao-data.tar.gz
Binary file not shown.
Binary file modified
BIN
+5 Bytes
(100%)
snapshots/3-piri-postgres-s3/volumes/ingot-openbao-init.tar.gz
Binary file not shown.
Binary file modified
BIN
+672 Bytes
(100%)
snapshots/3-piri-postgres-s3/volumes/ingot-postgres-data.tar.gz
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file modified
BIN
+55 Bytes
(100%)
snapshots/3-piri-postgres-s3/volumes/piri-minio-data.tar.gz
Binary file not shown.
Binary file modified
BIN
-43 Bytes
(100%)
snapshots/3-piri-postgres-s3/volumes/piri-postgres-data.tar.gz
Binary file not shown.
Binary file modified
BIN
+502 Bytes
(100%)
snapshots/3-piri-postgres-s3/volumes/plc-postgres-data.tar.gz
Binary file not shown.
Binary file modified
BIN
+73 Bytes
(100%)
snapshots/3-piri-postgres-s3/volumes/swarf-postgres-data.tar.gz
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| # Central OpenBao (appliance seal root) | ||
|
|
||
| The central OpenBao of the | ||
| [regional security and key management RFC](https://github.com/fil-one/RFC/pull/21). | ||
| Each appliance runs its own OpenBao ([ingot-openbao](../ingot/) here) whose | ||
| storage is sealed by a transit key held on this instance. At boot the | ||
| appliance presents its seal token, central unwraps the appliance's barrier | ||
| key, and the appliance unseals. Revoking that token here is the RFC's | ||
| startup-kill lever: the appliance never starts again, and everything on its | ||
| disk stays unreadable. | ||
|
|
||
| This instance is deliberately separate from [hilt-vault](../hilt/). It is | ||
| the one Fil One endpoint that operator-controlled hardware talks to, so it | ||
| holds nothing else: no tenant keys, no Hilt state. | ||
|
|
||
| ## Services | ||
|
|
||
| - **central-openbao** — `openbao/openbao:2.6` in server mode, raft storage | ||
| on `central-openbao-data`. | ||
| - **central-openbao-init** — one-shot: initializes and unseals | ||
| `central-openbao`, then provisions the `seal/` transit engine, the | ||
| appliance seal key, policy, and token (`init.sh`). | ||
|
|
||
| ## Ports | ||
|
|
||
| | Host Port | Container Port | Service | Description | | ||
| |-----------|----------------|---------|-------------| | ||
| | 15210 | 8200 | central-openbao | OpenBao HTTP API | | ||
|
|
||
| ## What init does | ||
|
|
||
| On every `make up`: | ||
|
|
||
| 1. First boot: `bao operator init` with one unseal share. The share and the | ||
| root token are written to the `central-openbao-init` volume (`/init`). | ||
| Dev-only custody; production roots this instance in an HSM or KMS. | ||
| 2. Every boot: unseal if sealed; enable transit at `seal/` if missing; | ||
| create `seal/keys/ingot-openbao` (`aes256-gcm96`, non-derived, | ||
| `exportable=false`) if missing; write the `appliance-ingot-openbao` | ||
| policy (`update` on `seal/encrypt/ingot-openbao` and | ||
| `seal/decrypt/ingot-openbao`, nothing else); recreate the appliance's | ||
| seal token: orphan, periodic (24h, renewed by the appliance's seal), fixed | ||
| id `dev-ingot-openbao-seal-token` (override with | ||
| `INGOT_OPENBAO_SEAL_TOKEN=... make up`; `systems/ingot/compose.yml` reads | ||
| the same variable). | ||
|
|
||
| `make clean` removes both volumes together, which keeps the unseal share and | ||
| the storage it opens in step. | ||
|
|
||
| ## The kill lever | ||
|
|
||
| ```bash | ||
| # Root token for admin actions (dev custody on the init volume; not printed here) | ||
| ROOT=$(docker run --rm -v smelt_central-openbao-init:/init alpine cat /init/root-token) | ||
|
|
||
| # Revoke the appliance: it fails to unseal on its next start | ||
| docker compose exec -e BAO_TOKEN="$ROOT" central-openbao bao token revoke dev-ingot-openbao-seal-token | ||
| docker compose restart ingot-openbao # stays sealed / exits with a seal error | ||
|
|
||
| # Reinstate: re-run the provisioner, then start the appliance again | ||
| docker compose up central-openbao-init | ||
| docker compose up -d ingot-openbao | ||
| ``` | ||
|
|
||
| ## Production notes | ||
|
|
||
| Per the RFC, the production shape adds: TLS on the listener, an appliance | ||
| token bound to the region's egress CIDR (and single-use where the deployment | ||
| permits), an HSM or KMS root for this instance, and an operator for central | ||
| (an open question in the RFC). None of those change the appliance side. | ||
|
|
||
| ## Volumes | ||
|
|
||
| - `central-openbao-data` - raft storage (appliance seal keys) | ||
| - `central-openbao-init` - unseal share + root token (dev-only custody) | ||
|
|
||
| ## Used By | ||
|
|
||
| - ingot-openbao (`seal "transit"` in `systems/ingot/openbao/config.hcl`) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # Central OpenBao System - seal root for the appliance OpenBaos | ||
| # | ||
| # The "central OpenBao" of the regional security RFC. It holds one transit | ||
| # key per appliance under `seal/`; each appliance's local OpenBao | ||
| # (ingot-openbao) is configured with `seal "transit"` against it and presents | ||
| # a scoped token at boot to unwrap its barrier key and unseal. Revoking that | ||
| # token here means the appliance never starts again (the RFC's startup-kill | ||
| # lever). Deliberately separate from hilt-vault: this is the one Fil One | ||
| # endpoint operator-controlled hardware talks to, so it holds nothing else. | ||
| # Used by: ingot-openbao (seal), and central-openbao-init (provisioning). | ||
|
|
||
| services: | ||
| central-openbao: | ||
| image: openbao/openbao:2.6 | ||
| command: ["server"] | ||
| ports: | ||
| - "${SMELT_CENTRAL_OPENBAO_PORT:-15210:8200}" | ||
| environment: | ||
| # For `docker compose exec central-openbao bao ...` (the CLI defaults to https) | ||
| - BAO_ADDR=http://127.0.0.1:8200 | ||
| volumes: | ||
| - ./config.hcl:/openbao/config/config.hcl:ro | ||
| - central-openbao-data:/openbao/file | ||
| # Healthy means listening: a fresh server is uninitialized and a | ||
| # restarted one is sealed until central-openbao-init runs again (which | ||
| # `make up` does, since one-shot services re-run on every up). | ||
| healthcheck: | ||
| test: ["CMD", "wget", "-q", "--spider", "http://127.0.0.1:8200/v1/sys/health?sealedcode=200&uninitcode=200"] | ||
| start_interval: 1s | ||
| interval: 10s | ||
| timeout: 5s | ||
| retries: 5 | ||
| start_period: 10s | ||
| restart: unless-stopped | ||
| networks: | ||
| - forge-network | ||
|
|
||
| # One-shot initializer: `operator init` on first boot (one unseal share + | ||
| # root token on the central-openbao-init volume, dev-only custody), unseal | ||
| # on every boot, then make sure the `seal/` transit engine, the appliance | ||
| # seal key, the appliance policy (encrypt/decrypt on that key only), and the | ||
| # appliance's periodic seal token exist. See init.sh. | ||
| central-openbao-init: | ||
| image: openbao/openbao:2.6 | ||
| user: "0:0" # the image runs as `openbao`; the init volume is root-owned | ||
| entrypoint: ["sh", "/init.sh"] | ||
| environment: | ||
| - BAO_ADDR=http://central-openbao:8200 | ||
| - APPLIANCE_NAME=ingot-openbao | ||
| # Token ingot-openbao presents to unseal; local-dev value, override with | ||
| # INGOT_OPENBAO_SEAL_TOKEN. Must match the BAO_TOKEN ingot-openbao gets | ||
| # in systems/ingot/compose.yml. | ||
| - APPLIANCE_SEAL_TOKEN=${INGOT_OPENBAO_SEAL_TOKEN:-dev-ingot-openbao-seal-token} | ||
| volumes: | ||
| - ./init.sh:/init.sh:ro | ||
| - central-openbao-init:/init | ||
| depends_on: | ||
| central-openbao: | ||
| condition: service_healthy | ||
| restart: "no" | ||
| networks: | ||
| - forge-network | ||
|
|
||
| volumes: | ||
| central-openbao-data: # raft storage: appliance seal keys | ||
| central-openbao-init: # unseal share + root token (dev-only custody) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| # central-openbao server config (local dev). | ||
| # | ||
| # The central OpenBao of the regional security RFC: it holds the transit | ||
| # keys that seal each appliance's local OpenBao (ingot-openbao here). At | ||
| # boot an appliance presents its token, central unwraps the appliance's | ||
| # barrier key, and the appliance unseals; revoking that token at central is | ||
| # the startup-kill lever. This instance holds nothing else: hilt's tenant | ||
| # keys stay in hilt-vault. | ||
| # | ||
| # Production differs in the root of trust for this instance (HSM/KMS unseal | ||
| # instead of a stored share), TLS on the listener, and CIDR-bound appliance | ||
| # tokens. All of those are changes to this file or to init.sh. | ||
|
|
||
| ui = false | ||
|
|
||
| api_addr = "http://central-openbao:8200" | ||
| cluster_addr = "http://central-openbao:8201" | ||
|
|
||
| listener "tcp" { | ||
| address = "0.0.0.0:8200" | ||
| tls_disable = true | ||
| } | ||
|
|
||
| storage "raft" { | ||
| path = "/openbao/file" | ||
| node_id = "central-openbao" | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am wondering if we really need to exercise OpenBao unlocking in the ephemeral dev deployments created by Smelt. It feels like unneeded complexity to me.
The real dev/staging/prod deployment will not use Smelt, so we are gaining very little by adding the central OpenBao here.
The real process for setting up "local OpenBao unsealed by central OpenBao" is a bit complex, see https://github.com/fil-forge/infra-central/blob/main/docs/decisions/2026-08-region-onboarding.md, fil-forge/infra-central#50 and fil-forge/infra-central#49. (Note: I haven't exercised this process yet; the final version may be slightly different if I need to fix any bugs.)
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My proposal: keep it simple in Smelt, let Ingot's OpenBao unseal automatically using a locally stored secret.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It just means it's impossible to test that revoked central access means an appliance can no longer read/serve encrypted data or that adding an appliance to the network in the first instance is blocked by it being authorised by the central vault.
Perhaps that is the useful functionality and tests we should add to this PR to make it worthwhile?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see your point.
I am not sure if this is the right place for testing these two features. In the current direction, Smelt is used for ephemeral deployments only. Long-living deployments will use infra-central & infra-nodes. (Why? The first nudge towards that direction comes from #11 (comment). More importantly, Smelt deploys both the central and the regional services together, while the real deployments need to manage them differently.)
Do we need to periodically test whether "local OpenBao unsealed via central OpenBao" still works as a concept? I don't see much value in that.
Also: Revoking the unsealing key in the central OpenBao is not enough to retire a node - the node can keep operating as long as it does not restart. See https://linear.app/filecoin-foundation/issue/FIL-1090 for more details.
It all depends on how the appliance is configured. In particular, how the node operator configures their OpenBao instance.
Anyhow, I don't feel too strongly about this. If you see value in adding the central vault to Smelt deployments, then go for it.