Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .claude/PRs.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,5 @@ PRs raised through Claude Code.
| [#234](https://github.com/ewitulsk/SuiOptions/pull/234) | [SO-237](https://suioptions.atlassian.net/browse/SO-237) | SO-19 Frontend | Group Vault Carousel by Asset with Vertical Cadence Coverflow |
| [#251](https://github.com/ewitulsk/SuiOptions/pull/251) | [SO-253](https://suioptions.atlassian.net/browse/SO-253) | SO-8 Protocol | Migrate Keeper Realized-Vol to Cached BenchmarkVol (fix Pyth 429 storm) |
| [#265](https://github.com/ewitulsk/SuiOptions/pull/265) | [SO-266](https://suioptions.atlassian.net/browse/SO-266) | SO-19 Frontend | Gate Exercise on Expired Options + Fix Off-Screen Popup Positioning |
| [#268](https://github.com/ewitulsk/SuiOptions/pull/268) | [SO-269](https://suioptions.atlassian.net/browse/SO-269) | — | Sponsor coinWithBalance coin Cleanup in All Gas-Station Templates |
| [#276](https://github.com/ewitulsk/SuiOptions/pull/276) | [SO-275](https://suioptions.atlassian.net/browse/SO-275) | SO-274 Cross-Chain Bridge | Bridge M0+M1 Foundation: Messaging Contracts, Locker, Signer & Relayer |
162 changes: 162 additions & 0 deletions .github/workflows/bridge-enclave.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
name: Bridge Enclave (Nautilus)

# Build + measure the AWS Nitro Enclave image (EIF) for bridge-signer-service,
# and (on manual dispatch) deploy it to ECR. bridge_tickets/07.
#
# Runs on a FREE public-repo arm64 hosted runner (ubuntu-24.04-arm) — the EIF
# must be built natively on arm64 for the c7g.large (Graviton) host; x86+QEMU is
# avoided because emulation jeopardizes PCR reproducibility.
#
# CLEAN SEPARATION from the rest of CI:
# - dedicated file + name; its own concurrency group
# - path-filtered so it ONLY runs on enclave/signer changes (no overlap with
# move-ci.yml `contracts/**` or the deploy-*.yml stack)
# - does NOT call the shared _deploy.yml; its own ECR repo + (later) IAM role
# from the isolated `infra-bridge` terraform root, not the main infra.

on:
workflow_dispatch:
inputs:
deploy:
description: 'Push the image to ECR + publish the EIF after building'
type: boolean
default: false
environment:
description: 'Target environment for deploy'
type: choice
options: [staging, prod]
default: staging
push:
branches: [main, ewitulsk/sui-bridge]
paths: &enclave_paths
- 'rust-backend/bridge-enclave/**'
- 'rust-backend/services/bridge-signer-service/**'
- 'rust-backend/crates/bridge-types/**'
- 'rust-backend/crates/bridge-signer/**'
- '.github/workflows/bridge-enclave.yml'
pull_request:
paths: *enclave_paths

# Serialize per-ref; cancel superseded CI runs (but never a deploy).
concurrency:
group: bridge-enclave-${{ github.ref }}
cancel-in-progress: ${{ github.event_name != 'workflow_dispatch' }}

permissions:
contents: read
id-token: write # OIDC → AWS, only used on deploy

env:
# PCR0 depends on the nitro-cli / bundled-kernel version, NOT just the app
# image — pin it exactly and bump deliberately (a bump changes PCR0).
NITRO_CLI_VERSION: v1.3.1
# ECR repo is created by the infra-bridge terraform root (ticket 07 Phase 5).
ECR_REPO: ${{ vars.BRIDGE_ENCLAVE_ECR_REPO }}

jobs:
build-enclave:
runs-on: ubuntu-24.04-arm # free native arm64 on public repos (GA Aug 2025)
steps:
- uses: actions/checkout@v4

# --- build the reproducible app image (context = rust-backend workspace) ---
- name: Build enclave app image
working-directory: rust-backend
run: |
docker build \
-f bridge-enclave/Dockerfile \
-t bridge-signer-enclave:${{ github.sha }} \
.

# --- install nitro-cli (SMOKE-TEST POINT, ticket 07 Phase 2) ---
# build-enclave is a build/measure step and should not need Nitro hardware
# or the nitro_enclaves driver — but this is the thing to confirm first on
# a hosted runner. If it fails here, move the build-enclave step to a
# self-hosted Graviton runner (restricted to non-fork triggers) per README.
- name: Install nitro-cli (${{ env.NITRO_CLI_VERSION }})
run: |
set -euo pipefail
sudo apt-get update
sudo apt-get install -y --no-install-recommends build-essential llvm-dev libclang-dev
git clone --depth 1 --branch "${NITRO_CLI_VERSION}" \
https://github.com/aws/aws-nitro-enclaves-cli.git /tmp/nitro-cli
# Builds the nitro-cli binary + ships the arch-specific kernel/init blobs
# that build-enclave bundles into the EIF (blobs affect PCR0).
make -C /tmp/nitro-cli nitro-cli
# `make install` (install-tools) installs BOTH binaries; vsock-proxy
# must be built too or install dies on the missing binary.
make -C /tmp/nitro-cli vsock-proxy
# `make install` installs to the LOCAL prefix ./build/install (not
# /usr/bin); its env.sh sets PATH + NITRO_CLI_BLOBS. Source it in
# every step that runs nitro-cli.
sudo make -C /tmp/nitro-cli install
# The rpm/deb packages create this log dir; the from-source local
# install doesn't, and nitro-cli dies (E19) if it can't open it.
sudo mkdir -p /var/log/nitro_enclaves
sudo chown "$(id -u):$(id -g)" /var/log/nitro_enclaves
source /tmp/nitro-cli/build/install/etc/profile.d/nitro-cli-env.sh
nitro-cli --version

# --- build the EIF + capture measurements ---
- name: Build EIF
run: |
source /tmp/nitro-cli/build/install/etc/profile.d/nitro-cli-env.sh
nitro-cli build-enclave \
--docker-uri bridge-signer-enclave:${{ github.sha }} \
--output-file signer.eif \
> measurements.json
cat measurements.json

# --- reproducibility gate: PCR0 must match the approved value ---
- name: Verify PCR0
run: |
set -euo pipefail
PCR0=$(jq -r '.Measurements.PCR0' measurements.json)
EXPECTED=$(tr -d '[:space:]' < rust-backend/bridge-enclave/expected_pcr0.txt)
echo "built PCR0 = $PCR0"
echo "expect PCR0 = $EXPECTED"
if [ "$EXPECTED" = "PENDING" ]; then
echo "::notice::expected_pcr0.txt is PENDING — first-run capture. Commit this PCR0 to enable the drift gate:"
echo "::notice::$PCR0"
elif [ "$PCR0" != "$EXPECTED" ]; then
echo "::error::PCR0 drift — built EIF does not match the approved measurement. The on-chain EnclaveConfig would reject this build."
exit 1
else
echo "PCR0 matches the approved value."
fi

- name: Upload EIF + measurements
uses: actions/upload-artifact@v4
with:
name: signer-eif-${{ github.sha }}
path: |
signer.eif
measurements.json
retention-days: 14

# --- deploy (manual dispatch only) ---
- name: Configure AWS credentials
if: github.event_name == 'workflow_dispatch' && inputs.deploy
uses: aws-actions/configure-aws-credentials@v4
with:
# Dedicated bridge deploy role (infra-bridge root) — the main
# DEPLOY_ROLE_ARN's OIDC trust only covers staging/main refs.
role-to-assume: ${{ vars.BRIDGE_DEPLOY_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}

- name: Login to ECR
if: github.event_name == 'workflow_dispatch' && inputs.deploy
uses: aws-actions/amazon-ecr-login@v2
id: ecr

- name: Push image to ECR
if: github.event_name == 'workflow_dispatch' && inputs.deploy
run: |
set -euo pipefail
REG="${{ steps.ecr.outputs.registry }}"
docker tag bridge-signer-enclave:${{ github.sha }} "$REG/$ECR_REPO:${{ github.sha }}"
docker push "$REG/$ECR_REPO:${{ github.sha }}"
echo "pushed $REG/$ECR_REPO:${{ github.sha }}"
# NOTE: the enclave HOST runs the measured EIF directly (Model B, ship
# the EIF) rather than rebuilding — publish signer.eif to S3 for the
# host here once the infra-bridge bucket exists (ticket 07 Phase 5).
163 changes: 163 additions & 0 deletions BRIDGE_CONTEXT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# Bridge enclave — session context (2026-07-03)

Working state of ticket 07 (`bridge_tickets/07-nautilus-enclave.md`, SO-275) after the
2026-07-03 session: the Nitro host is live, the CI build/deploy pipeline works end-to-end,
and the stock-nautilus validation (Phase 1 step 1) passed — including on-chain attestation
verification against the Phase 3 registry. This doc records every artifact, credential
surface, and gotcha so work can resume cold.

**Phase status:** P2 (CI) ✅ · P3 (registry) ✅ · P5 (terraform) ✅ APPLIED · P1 step 1
(stock validation) ✅ → **next is the real P1 work** (vendor + library-fy + vsock/TLS
egress rework), then P4, P6.

---

## 1. Infrastructure (terraform applied — the host is LIVE)

`rust-backend/infra-bridge/` applied 2026-07-03 with zero impact on existing infra
(plan verified additive-only; `options-prod-host` health-checked before/during/after).

| Resource | Value |
|---|---|
| EC2 instance | `i-0ad5124712c538893` — c7g.large (Graviton, 2 vCPU / 4 GB), AL2023 arm64, `enclave_options` enabled, IMDSv2 |
| IPs | private `10.40.1.142`, public `54.144.200.244` (us-east-1, shared `options-vpc` / `options-public-0`) |
| Access | **SSM only, no SSH**: `aws ssm start-session --target i-0ad5124712c538893 --region us-east-1` |
| ECR repo | `502186568577.dkr.ecr.us-east-1.amazonaws.com/options-bridge-signer-enclave` (immutable tags) |
| Host IAM | role/profile `options-bridge-enclave` (SSM core + scoped ECR pull) |
| CI IAM | role `options-bridge-gh-deploy` (see §2) |
| Security group | `sg-091a8455c990ff511` — egress all; ingress tcp/3000 only via `signer_api_ingress_cidrs` (currently empty) |
| Allocator | 1 vCPU / 1536 MiB (`/etc/nitro_enclaves/allocator.yaml`), nitro-cli 1.4.4 + docker verified active |

**Caveats**
- Terraform state is **LOCAL to Evan's machine** (gitignored). No S3 backend yet — no
other machine can manage these resources. Add a `backend "s3"` block when this stops
being a one-person root.
- The root is deliberately isolated from `rust-backend/infra/` (which has a known
destructive-drift landmine). Blanket `terraform apply` in `infra-bridge/` is safe;
it reads the VPC/subnet via data lookups only.
- `ignore_changes = [ami]` pins the host against AL2023 AMI-release replacement.

## 2. CI pipeline (`.github/workflows/bridge-enclave.yml`) — working

First fully successful deploy run: [28686106945](https://github.com/ewitulsk/SuiOptions/actions/runs/28686106945)
(dispatch with `deploy=true`, env staging). Pushed image
`options-bridge-signer-enclave:8f8a6d8f13f2f36eed154af79cc3eaffc47ea192`
(digest `sha256:f61659dcde586e5f7260b151dd52ca505a4b8571e5fa948b5f4198c8efb2ab28`).

Run it: `gh workflow run bridge-enclave.yml --ref <branch> -f deploy=true -f environment=staging`.
Watch out: a push to enclave paths auto-triggers a build-only run in the same
concurrency group — cancel it before dispatching or the dispatch queues behind it.

**Four fixes were needed (all committed on `ewitulsk/sui-bridge`):**
| Commit | Fix |
|---|---|
| `c8f729d` | `make vsock-proxy` before `make install` — install-tools installs BOTH binaries; only nitro-cli was being built |
| `36853cd` | source nitro-cli's `env.sh` in every step that runs it — from-source install lands in a local prefix (`./build/install`), not `/usr/bin` |
| `f71aa1f` | pre-create `/var/log/nitro_enclaves` — rpm/deb installs create it, from-source doesn't (nitro-cli dies E19) |
| `8f8a6d8` | assume dedicated `BRIDGE_DEPLOY_ROLE_ARN` — the shared `DEPLOY_ROLE_ARN`'s OIDC trust only covers `refs/heads/{staging,main}` and its ECR allowlist lacks the bridge repo |

**Repo variables (set):**
- `BRIDGE_ENCLAVE_ECR_REPO` = `options-bridge-signer-enclave`
- `BRIDGE_DEPLOY_ROLE_ARN` = `arn:aws:iam::502186568577:role/options-bridge-gh-deploy`
— defined in `infra-bridge/iam_ci.tf`; OIDC trust covers `ewitulsk/sui-bridge`,
`staging`, `main` refs; carries the scoped ECR-push policy. The shared
`options-gh-actions-deploy` role was returned to its exact pre-session state.

**Answered smoke-test question (ticket P2):** `nitro-cli build-enclave` DOES work on the
free hosted `ubuntu-24.04-arm` runner — no Nitro hardware or self-hosted Graviton runner
needed for build/measure.

**PCR0 drift gate is deliberately still `PENDING`** (`rust-backend/bridge-enclave/expected_pcr0.txt`):
- the scaffold Dockerfile `COPY . .`s the whole `rust-backend/` workspace, so ANY
workspace change shifts PCR0 (two CI runs → two different PCR0s, as expected);
- it self-documents as not-yet-bit-reproducible (base images by tag, no
`SOURCE_DATE_EPOCH`);
- `expected_pcr0.txt` sits inside the docker context → committing a value is circular.

Arm the gate as part of the P1/P2 reproducibility work, not before.

## 3. Stock-nautilus validation (Phase 1 step 1) — PASSED

Ran upstream `MystenLabs/nautilus` @ **`af7535b9d314f034fa7f9f1f208264540d54cde1`**
(pin this in `FORK_DELTA.md` when vendoring) end-to-end on the box:

1. **Build (on-box):** upstream's reproducible Containerfile/Makefile is **amd64-ONLY**
(x86_64 StageX digests, x86 `bzImage`, hardcoded `--platform linux/amd64`), so
validation used a plain aarch64 Dockerfile (rust:1.90-slim → bookworm-slim, stock
`nautilus-server` weather-example + its `run.sh`) + `nitro-cli build-enclave` on the
host. EIF measurements: PCR0 `25bdd759b0906c3703d0e0dd3a907a6cfc4330a9e20ee025bf13cce7f76ff7eaae691bb8717304b4c968c64b25320485`.
2. **Boot:** `nitro-cli run-enclave --cpu-count 1 --memory 1024` → enclave up (CID 16).
Secrets handshake over vsock 7777 (`{"API_KEY":"dummy-validation-key"}` via socat, as
upstream `expose_enclave.sh` does), parent-side `socat TCP4-LISTEN:3000 ↔ VSOCK`
forward, `GET /` → `Pong!`, `/get_attestation` → 4492-byte COSE_Sign1 doc.
3. **On-chain (testnet, deployer `0xab8d…4865`):** published the Phase 3 package and
verified the real attestation path that Move unit tests couldn't cover:

| Object | ID |
|---|---|
| Package `bridge_enclave` | `0xeda4ddd012c724e1fdcf8c69abdf3d365a6b52448846ccf8098d011e037cc466` |
| `Cap<BRIDGE_SIGNER>` | `0x5c7fae89626c96ad3ffcac5fbea823461196189bef03c2fb6df2ecc111dfd828` |
| `UpgradeCap` | `0xf4fc772617e4d61850f194ce113daeb830cddb7043110b7d8ddfadd4773902e8` |
| `EnclaveConfig` (validation PCRs) | `0x1b00efcab7113f978fb50a12564778df8fe68c974559e69780ead21722e76ec7` |
| Registered `Enclave` (shared) | `0xeb33c0a8f532f29ca91bde92c69bb42291ee892ea1f08dc690db19573b5911ae` |
| `register_enclave` tx | `HnS4TQz1bHutYB3XCFjALHRvG6HdkX3MMZctcVZ9gYuM` |

The chain verified the doc's signature chain to the AWS Nitro root CA, matched PCRs
against the config, stored the enclave's boot-generated ephemeral pubkey, and emitted
`EnclaveRegistered`. Registration PTB shape (from upstream `register_enclave.sh`):

```
sui client ptb \
--assign v "vector[<attestation bytes as u8 array>]" \
--move-call "0x2::nitro_attestation::load_nitro_attestation" v @0x6 \
--assign doc \
--move-call "$PKG::enclave::register_enclave<$PKG::signer::BRIDGE_SIGNER>" @$CONFIG doc
```

**Box state as left:** validation enclave may still be running (dummy key, nothing
sensitive) — `nitro-cli terminate-enclave --all` via SSM to kill. Upstream clone at
`/root/nautilus`, EIF at `/root/nautilus-validation.eif`, measurements at
`/var/tmp/nautilus-measurements.json`, attestation at `/var/tmp/attestation.json`.
The validation `EnclaveConfig`/`Enclave` objects are throwaway — a real config (fresh
PCRs, real name) supersedes them when the signer EIF exists.

**Operational gotchas hit:**
- `nitro-cli build-enclave` needs `NITRO_CLI_ARTIFACTS` (or `HOME`) set → E51 otherwise
(SSM RunShellScript sessions have neither).
- SSM `get-command-invocation` truncates output (~24KB) — pull large files (e.g. the
9KB attestation hex) in chunks.
- SSM shell mangles multi-line scripts passed as parameters — ship them base64-encoded.
- The box has 1 usable parent vCPU; on-box cargo builds are slow (~30 min for the
validation image). Fine for validation; real builds belong in CI.

## 4. What's next (remaining ticket 07 work)

1. **Phase 1 (the crux, ~1–2 wk):**
- Vendor the pinned nautilus subtree into `rust-backend/bridge-enclave/`
(`src/nautilus-server`, vsock forwarder, `allowed_endpoints.yaml`) + `FORK_DELTA.md`
recording `af7535b9…` and deltas. Own `Cargo.lock`, NOT a main-workspace member.
- Library-fy `bridge-signer-service` (expose router + `AppState` as a lib crate);
nautilus-server app pulls it by path dep.
- Egress rework: route `EvmProbe`/`SuiProbe`/`SuiClientBuilder` HTTPS through
vsock→parent-forwarder with **rustls terminating in-enclave** (pinned provider certs).
2. **Phase 2 completion:** arm64 reproducible build (upstream StageX pipeline is
amd64-only — port it or pin our Dockerfile by digest + `SOURCE_DATE_EPOCH`), then
commit the real PCR0 and arm the drift gate.
3. **Phase 4:** ticket-02 `RpcVerifier` in-enclave over the vsock/TLS transport.
4. **Phase 6:** lifecycle scripts + boot flow (enclave boots → attest → operator
registers on-chain → signer flips ready; signer refuses `/sign_requests` until
its key is registered).
5. Module-ize the terraform for N=3 (ticket 09) and add the S3 state backend.

## 5. Session artifact inventory

- **Committed this session:** CI fixes + OIDC role (`c8f729d`, `36853cd`, `f71aa1f`,
`8f8a6d8`), `enclave/Move.lock` + `Published.toml` (testnet publish record), this doc.
- **AWS (all in `infra-bridge` terraform state except noted):** instance, SG, host
role/profile, ECR repo, CI role `options-bridge-gh-deploy` + its inline policy.
Nothing outside this list was modified; `options-gh-actions-deploy` briefly carried a
scoped bridge-ECR policy mid-session and was restored to its original state.
- **Testnet:** the five objects in §3 (throwaway validation config/enclave; package +
caps are real).
- **On the box (ephemeral):** `/root/nautilus`, `/root/nautilus-validation.eif`,
`/var/tmp/{nautilus-*,attestation.json,run.sh,setup.sh}`.
Loading