From ae44392112e9758006763a1679c7341db79042fc Mon Sep 17 00:00:00 2001 From: Diana Borbe Date: Thu, 20 Aug 2026 12:51:19 +0200 Subject: [PATCH 1/3] Add IFT relay to demo Signed-off-by: Diana Borbe --- examples/besu-to-besu/README.md | 215 ++++++++++++++++++- examples/besu-to-besu/docker-compose.yml | 115 ++++++++++- examples/besu-to-besu/lib/chains.sh | 20 +- examples/besu-to-besu/lib/link.sh | 238 ++++++++++++++++++++++ examples/besu-to-besu/link/attestor-a.yml | 23 +++ examples/besu-to-besu/link/attestor-b.yml | 23 +++ examples/besu-to-besu/link/deploy.yml | 28 +++ examples/besu-to-besu/link/ibc.yml | 56 +++++ examples/besu-to-besu/link/kms.yaml | 25 +++ examples/besu-to-besu/setup.sh | 107 ++++++---- 10 files changed, 770 insertions(+), 80 deletions(-) create mode 100644 examples/besu-to-besu/lib/link.sh create mode 100644 examples/besu-to-besu/link/attestor-a.yml create mode 100644 examples/besu-to-besu/link/attestor-b.yml create mode 100644 examples/besu-to-besu/link/deploy.yml create mode 100644 examples/besu-to-besu/link/ibc.yml create mode 100644 examples/besu-to-besu/link/kms.yaml diff --git a/examples/besu-to-besu/README.md b/examples/besu-to-besu/README.md index db93ffed3..815279447 100644 --- a/examples/besu-to-besu/README.md +++ b/examples/besu-to-besu/README.md @@ -4,7 +4,29 @@ Two independent single-validator Besu QBFT chains (A and B), the substrate for a -single IBC pair. +single IBC pair, plus the IBC Link services that relay between them: + +``` +besu-a ◀──┐ ┌──▶ besu-b + │ attestor-a attestor-b │ + └── (41001) (41002) ──┘ + │ │ + ├──── ibc-link ────┤ relayer, no attestation key + │ │ + └────── kms ───────┘ every signing key lives here +``` + +Each attestor is a **standalone external process**: its own container, its own +config, reached by the relayer over gRPC as a `type: remote` attestor. The +relayer never loads an attestor config and cannot produce an attestation itself, +which is the difference from dual mode (one process running the relayer and a +`type: local` attestor together). + +No signing key is on disk in any attestor or relayer container. +[cosmos/kms](https://github.com/cosmos/kms) runs in gRPC-only mode — no chains, +no validators, no privval dial-out — and serves all four keys over its +SignerService. Each service addresses the key it is allowed to use by id +(`type: remote`, `remoteKeyId:`), and the private key never leaves kms. ## Prerequisites @@ -13,19 +35,57 @@ single IBC pair. `cast` (foundry) does the BIP-39 derivation. If it isn't on `PATH`, `cast_cli` falls back to running it inside `$FOUNDRY_IMAGE`, so a host install is optional. +Nothing is built from source. The images are pulled from GHCR and are public, so +no `docker login` is needed: + +| Service | Image | Override | +|----------------------------------|--------------|-------------| +| ibc-link, attestor-a/b, deployer | `ghcr.io/cosmos/ibc:main` | `IBC_IMAGE` | +| kms | `ghcr.io/cosmos/kms:latest` | `KMS_IMAGE` | +| besu-a, besu-b | `hyperledger/besu:25.4.0` | `BESU_IMAGE` | + +`IBC_IMAGE` must be built from a commit that has both the `deploy` command and +the unified top-level `attestors[]` list. Older tags fail late and obscurely — +`unknown command "deploy"` two phases in, or attestors restart-looping on `no +attestations provided`, because `attestor.attestations[]` entries carry no `type` +and no `grpc` and so cannot express a standalone external attestor. If you see +either, check the tag first. Rebuild one with `gh workflow run +ibc-link-build.yml --ref ` — the image is tagged after the ref it runs +on. + +To run local relayer or attestor changes, build the image and point `IBC_IMAGE` +at it: + +```bash +docker build -t ibc-link:local --target target-builder ../../link +IBC_IMAGE=ibc-link:local ./setup.sh +``` + ## Usage Run from `examples/besu-to-besu/`: ```bash -./setup.sh # init + start + wait for RPC (end-to-end) -./setup.sh init # derive keys, render configs into chains/local/ (no containers) -./setup.sh start # docker compose up both chains, wait for RPC -./setup.sh accounts # print the funded addresses and their roles -./setup.sh status # RPC endpoints, chain IDs, block heights +./setup.sh # the demo: init + start + deploy + link ./setup.sh clean # stop containers, remove volumes and chains/local/ ``` +Two commands, on purpose. The five phases always run together: + +| Phase | What it does | +|------------|-------------------------------------------------------------------| +| `init` | derive every key, render the chain configs into `chains/local/` | +| `start` | `docker compose up` both chains, wait for RPC | +| `deploy` | `ibc deploy core` + `client` on each chain (writing `link.env`), then GMP, an IFT token per chain, and the bridge | +| `link` | `docker compose up` kms, both attestors, ibc-link | +| `transfer` | mint IFT on A, send it to B, relay it, assert the balance moved | + +Use `docker compose` directly to poke at a running stack (`ps`, `logs -f +ibc-link`, `exec attestor-a /opt/ibc attestor info attestor-a --home /home/ibc`). + +A bare `./setup.sh` runs all five phases and ends with half a token having +crossed from chain A to chain B. + Verify once the chains are up: ```bash @@ -50,6 +110,108 @@ chain's genesis; index 0 is the deployer, index 1 the validator. Set `A_MNEMONIC` and `B_MNEMONIC` to different phrases if you want the chains to have fully independent account sets instead. +## Signing keys + +The init phase derives four keys from the same mnemonics and writes them to +`chains/local/kms/keys/.hex`, one per `grpc.keys` entry in `link/kms.yaml`. +Every run prints them. + +| kms key id | Source | Used by | Needs gas | +|--------------|----------------------------|------------|-----------| +| `relayer-a` | `A_MNEMONIC` index 2 | ibc-link | yes, on A | +| `relayer-b` | `B_MNEMONIC` index 2 | ibc-link | yes, on B | +| `attestor-a` | `A_MNEMONIC` index 3 | attestor-a | no | +| `attestor-b` | `B_MNEMONIC` index 4 | attestor-b | no | + +Two relayer keys rather than one shared key because `A_MNEMONIC` and +`B_MNEMONIC` are independently overridable: one key would only be funded on +chain A the moment they differ. The attestors sit at *different* indices for the +opposite reason — the two mnemonics default to the same phrase, and a shared +index would give two supposedly independent attestors one address. + +All four are `algorithm: secp256k1eth` in `link/kms.yaml`, not `secp256k1`. Only +that scheme signs a pre-hashed 32-byte digest and returns the 65-byte +recoverable signature `AttestationLightClient.sol` recovers an attestor address +from; link rejects any other scheme at startup. + +The keys are written mode `0644`, unlike the `0600` Besu validator keys, because +the kms image runs unprivileged (uid 10001) and could not otherwise read them +through the bind mount on Linux. Every one is a public BIP-39 test-vector key. + +## Deploying + +The deploy phase puts IBC on both chains through the one-shot `deployer` +compose service — the same `ibc` image with [link/deploy.yml](link/deploy.yml) +mounted, run as `docker compose run --rm deployer`. Running it in a container +means no Go toolchain on the host, and `besu-a` / `besu-b` resolve exactly as +they do for the real services. Per chain: + +``` +ibc keys import ecdsa deployer- # index 0 of that chain's mnemonic +ibc deploy core --chain +ibc deploy client --chain --counterparty-chain \ + --attestors --threshold 1 +``` + +A client on chain A tracks chain B, so it verifies attestations *about* B and +authorizes **B's** attestor — the counterparty's, not its own. `threshold 1` +because each chain has exactly one attestor here. + +The deployer is the one key that cannot live in kms: `ibc deploy` needs the raw +private key and rejects a `type: remote` signer outright. That is why +`link/deploy.yml` is separate from `link/ibc.yml` — it keeps the only local key +in the example out of the relayer's config and out of the relayer's process. + +Deployment writes a manifest per chain to `chains/local/deploy/deployments/`, +and `deploy` then writes `chains/local/link.env`. Only the router addresses are +read back out of the manifests (`.core.router`); the client id is passed to +`deploy client` explicitly, so it is known before anything runs and needs no +parsing. + +All four values are required to start, and each is missed at a different stage: + +| Symptom on startup | Cause | +|-------------------------------------------------|------------------------------| +| `.clientId required` | `*_CLIENT_ID` empty | +| `invalid ics26 router address "" for chain ` | `*_ICS26_ROUTER` empty | +| `no contract code at given address` | router set but not deployed | + +Re-running `./setup.sh` against a live stack is safe: an already-imported +deployer key is skipped, and `ibc deploy` skips steps whose artefacts already +exist, so the same addresses come back out. `./setup.sh clean` resets everything. + +## Relaying an IFT transfer + +The last phase is the end-to-end assertion for everything above it. A balance +only moves on chain B if both attestors signed through kms and the relayer +assembled their attestations into a proof the light client accepted. + +``` +ibc deploy gmp --chain # IFT rides on ICS27-GMP +ibc deploy ift --chain --name --symbol # one token per chain +ibc deploy ift-bridge --chain-a A --ift-a … --chain-b B --ift-b … --client-id … +ibc tx ift mint --chain A --ift … --from deployer-a --to --amount 1e18 +ibc tx ift send --chain A --ift … --from deployer-a --to --amount 5e17 \ + --client-id link-41001-41002 +ibc relayer relay --tx-hash --chain-id A +ibc query ift balance --chain B --ift … --address +``` + +`deploy gmp` is not optional — `deploy ift` refuses without it (`no gmp +deployment recorded for chain `). + +**The relay step is explicit.** `relayer.connections[].autoRelay` exists in the +config schema and validates, but nothing in `link` reads `.Enabled` or +`.Lookback` yet, so a packet sits unrelayed until it is handed to the relayer by +transaction hash. That is why `link/ibc.yml` carries no `autoRelay` block: it +would only imply a behaviour that is not wired up. `relayer relay` runs inside +the `ibc-link` container, since the command dials the relayer's own gRPC and +only its config describes it. + +`tx ift` and `deploy` both need the raw private key, so both run as the +`deployer` service against `link/deploy.yml`. The transfer defaults are +overridable: `IFT_NAME`, `IFT_SYMBOL`, `IFT_MINT_AMOUNT`, `IFT_SEND_AMOUNT`, +`IFT_RELAY_TIMEOUT` (120s), `IFT_POLL_INTERVAL` (3s). ## Ports @@ -61,28 +223,61 @@ Only the host-side mappings differ: | besu-a | 8545 | 8546 | 9545 | | besu-b | 8745 | 8746 | 9745 | +The link services all listen on 3000 internally: + +| Service | gRPC (host) | +|------------|------------:| +| ibc-link | 3000 | +| attestor-a | 3010 | +| attestor-b | 3011 | +| kms | not published | + Every host mapping binds to `127.0.0.1` — the JSON-RPC endpoints are unauthenticated and each node holds the key that signs every block on its chain, so they stay off the LAN. Container-to-container traffic goes over the `besu-besu-net` compose network and does not depend on these mappings. +`kms` is deliberately absent from that table. It serves plaintext gRPC and +performs no caller authentication or authorization at all, so anything that can +reach it can sign with any of the four keys. It stays on the compose network +only; use `docker compose exec kms ...` to inspect it. A real deployment sets +`tls_cert` / `tls_key` in `link/kms.yaml` and puts network controls in front. + ## Layout ``` examples/besu-to-besu/ ├── README.md -├── setup.sh — entrypoint: init | start | accounts | status | clean -├── docker-compose.yml — besu-a + besu-b +├── setup.sh — entrypoint: init | start | link | accounts | +│ signers | status | clean +├── docker-compose.yml — besu-a, besu-b, kms, attestor-a, attestor-b, +│ ibc-link ├── lib/ │ ├── common.sh — logging, prerequisite checks, RPC waiter, │ │ render_template, cast_cli -│ └── chains.sh — derivation, QBFT extraData, rendering, start / -│ wait / status / clean +│ ├── chains.sh — derivation, QBFT extraData, rendering, start / +│ │ wait / status / clean +│ └── link.sh — kms key derivation, deployment, link.env +├── link/ — committed, no secrets. Bind-mounted verbatim: +│ ├── kms.yaml — gRPC-only remote signer, 4 secp256k1eth keys +│ ├── attestor-a.yml — standalone attestor for chain A +│ ├── attestor-b.yml — standalone attestor for chain B +│ ├── ibc.yml — relayer: both attestors as type: remote +│ └── deploy.yml — one-shot deployer, the only local key └── chains/ ├── besu.toml.tmpl — rendered once per chain ├── el-genesis.json.tmpl — ${CHAIN_ID}, ${QBFT_EXTRADATA}, ${GENESIS_ALLOC} └── local/ — generated, gitignored: ├── chains.env — addresses, deployer keys, chain IDs, RPC URLs + ├── link.env — router addresses and client ids, written by + │ the deploy phase + ├── kms/keys/*.hex — the four signing keys kms serves + ├── deploy/keys/ — the two deployer keyfiles + ├── deploy/deployments/ — one deployment manifest per chain ├── A/{besu.toml, el-genesis.json, key} └── B/{besu.toml, el-genesis.json, key} ``` + +The `link/*.yml` placeholders are expanded by link itself (`os.ExpandEnv` at +config load), not by `render_template` — which is why these are plain committed +files rather than `.tmpl` files under `chains/`. diff --git a/examples/besu-to-besu/docker-compose.yml b/examples/besu-to-besu/docker-compose.yml index cfbf22dd5..d043945d1 100644 --- a/examples/besu-to-besu/docker-compose.yml +++ b/examples/besu-to-besu/docker-compose.yml @@ -6,14 +6,20 @@ # known BIP-39 test vectors. Local devnet use only — never on a public, shared, # or production network. # -# Each /evm mount points at chains/local//, which ./setup.sh init -# renders from chains/*.tmpl — besu.toml, el-genesis.json, and the validator -# key derived from the demo mnemonic. Nothing under chains/local/ is committed, -# so `docker compose up` without a prior init has no config to mount. +# +# kms ◀── Sign ──┬── attestor-a ──┐ +# ├── attestor-b ──┤ StateAttestation / PacketAttestation +# └── ibc-link ◀───┘ +x-link-service: &link-service + image: ${IBC_IMAGE:-ghcr.io/cosmos/ibc:main} + env_file: + - path: ./chains/local/link.env + required: false + networks: [ besu-besu-net ] + restart: on-failure services: - # ── Chains ──────────────────────────────────────────────────────────────── besu-a: image: ${BESU_IMAGE:-hyperledger/besu:25.4.0} container_name: besu-a @@ -23,11 +29,6 @@ services: - "--metrics-host=0.0.0.0" - "--metrics-port=9545" - "--Xqbft-enable-early-round-change" - # Published on the loopback interface only. Without the explicit 127.0.0.1 - # these bind to 0.0.0.0 and expose an unauthenticated JSON-RPC endpoint — - # holding keys that can sign as the chain's sole validator — to the whole - # LAN. Container-to-container traffic goes over besu-besu-net and does not - # depend on these mappings. ports: - "127.0.0.1:8545:8545" - "127.0.0.1:8546:8546" @@ -53,7 +54,7 @@ services: - "--metrics-host=0.0.0.0" - "--metrics-port=9545" - "--Xqbft-enable-early-round-change" - ports: # loopback only — see besu-a + ports: - "127.0.0.1:8745:8545" - "127.0.0.1:8746:8546" - "127.0.0.1:9745:9545" @@ -69,10 +70,102 @@ services: retries: 60 start_period: 10s + kms: + image: ${KMS_IMAGE:-ghcr.io/cosmos/kms:latest} + container_name: kms + volumes: + - ./link/kms.yaml:/home/kms/kms.yaml:ro + - ./chains/local/kms/keys:/home/kms/keys:ro + networks: [ besu-besu-net ] + restart: on-failure + healthcheck: + test: [ "CMD", "bash", "-c", "exec 3<>/dev/tcp/localhost/9090" ] + interval: 5s + timeout: 3s + retries: 30 + start_period: 5s + + attestor-a: + <<: *link-service + container_name: attestor-a + command: [ "attestor", "run", "--home", "/home/ibc" ] + volumes: + - ./link/attestor-a.yml:/home/ibc/ibc.yml:ro + ports: + - "127.0.0.1:3010:3000" + depends_on: + besu-a: + condition: service_healthy + kms: + condition: service_healthy + healthcheck: + test: [ "CMD", "/opt/ibc", "attestor", "info", "attestor-a", "--home", "/home/ibc" ] + interval: 5s + timeout: 5s + retries: 30 + start_period: 10s + + attestor-b: + <<: *link-service + container_name: attestor-b + command: [ "attestor", "run", "--home", "/home/ibc" ] + volumes: + - ./link/attestor-b.yml:/home/ibc/ibc.yml:ro + ports: + - "127.0.0.1:3011:3000" + depends_on: + besu-b: + condition: service_healthy + kms: + condition: service_healthy + healthcheck: + test: [ "CMD", "/opt/ibc", "attestor", "info", "attestor-b", "--home", "/home/ibc" ] + interval: 5s + timeout: 5s + retries: 30 + start_period: 10s + + deployer: + image: ${IBC_IMAGE:-ghcr.io/cosmos/ibc:main} + profiles: [ tools ] + user: "${DEPLOY_UID:-65532}:${DEPLOY_GID:-65532}" + volumes: + - ./link/deploy.yml:/home/ibc/ibc.yml:ro + - ./chains/local/deploy/keys:/home/ibc/keys + - ./chains/local/deploy/deployments:/home/ibc/deployments + networks: [ besu-besu-net ] + depends_on: + besu-a: + condition: service_healthy + besu-b: + condition: service_healthy + + ibc-link: + <<: *link-service + container_name: ibc-link + command: [ "relayer", "run", "--home", "/home/ibc" ] + volumes: + - ibc-link-home:/home/ibc + - ./link/ibc.yml:/home/ibc/ibc.yml:ro + ports: + - "127.0.0.1:3000:3000" # loopback only — see besu-a + depends_on: + attestor-a: + condition: service_healthy + attestor-b: + condition: service_healthy + healthcheck: + test: [ "CMD", "nc", "-z", "127.0.0.1", "3000" ] + interval: 5s + timeout: 3s + retries: 30 + start_period: 10s + volumes: besu-a-data: besu-b-data: + ibc-link-home: networks: besu-besu-net: diff --git a/examples/besu-to-besu/lib/chains.sh b/examples/besu-to-besu/lib/chains.sh index 8a0381d4d..f31e2c5f6 100644 --- a/examples/besu-to-besu/lib/chains.sh +++ b/examples/besu-to-besu/lib/chains.sh @@ -256,7 +256,7 @@ _start_chains_env() { chmod 600 "$CHAINS_ENV_FILE" { - echo "# Generated by ./setup.sh init — do not edit, do not commit." + echo "# Generated by ./setup.sh — do not edit, do not commit." echo "#" echo "# DEMO KEYS — LOCAL DEVNET ONLY. The private keys below are derived from" echo "# publicly known BIP-39 test vectors and are worthless by design. Never" @@ -306,7 +306,7 @@ start_chains() { local name for name in "${CHAINS[@]}"; do [[ -f "$LOCAL_DIR/$name/key" ]] \ - || die "$LOCAL_DIR/$name/key is missing — run './setup.sh init' first" + || die "$LOCAL_DIR/$name/key is missing — the init phase did not complete" done log "Bringing up besu-a and besu-b..." docker compose up -d besu-a besu-b @@ -320,22 +320,6 @@ wait_for_chains() { done } -print_accounts() { - local name i index role - for name in "${CHAINS[@]}"; do - derive_chain_accounts "$name" - index=$(_chain_attr "$name" VALIDATOR_INDEX) - log "chain $name — $FUNDED_ACCOUNTS accounts from ${name}_MNEMONIC," \ - "each funded with $(_genesis_balance_eth) on $name only:" - for (( i = 0; i < ${#CHAIN_ACCT_ADDRS[@]}; i++ )); do - role="" - [[ $i -eq 0 ]] && role+=" [deployer]" - [[ $i -eq "$index" ]] && role+=" [validator]" - log " index $i ${CHAIN_ACCT_ADDRS[$i]}${role} privkey=0x${CHAIN_ACCT_KEYS[$i]}" - done - done -} - print_status() { local name block log "Chain status:" diff --git a/examples/besu-to-besu/lib/link.sh b/examples/besu-to-besu/lib/link.sh new file mode 100644 index 000000000..bafa7865b --- /dev/null +++ b/examples/besu-to-besu/lib/link.sh @@ -0,0 +1,238 @@ +#!/usr/bin/env bash + +# SPDX-License-Identifier: Apache-2.0 + +# IBC Link: kms keys, deployment, and service bring-up. +# +KMS_KEYS_DIR="$LOCAL_DIR/kms/keys" +LINK_ENV_FILE="$LOCAL_DIR/link.env" +DEPLOY_DIR="$LOCAL_DIR/deploy" + +LINK_SERVICES=(kms attestor-a attestor-b ibc-link) +LINK_CLIENT_ID="link-$A_CHAIN_ID-$B_CHAIN_ID" + +export DEPLOY_UID="${DEPLOY_UID:-$(id -u)}" +export DEPLOY_GID="${DEPLOY_GID:-$(id -g)}" + +_lc() { printf '%s' "$1" | tr 'A-Z' 'a-z'; } +_other() { [[ "$1" == "A" ]] && printf 'B' || printf 'A'; } + +_key() { + local index account + index=$(_chain_attr "$1" "${2}_INDEX") + [[ "$index" =~ ^[0-9]+$ ]] \ + || die "${1}_${2}_INDEX must be a non-negative integer, got '$index'" + account=$(derive_account "$(_chain_mnemonic "$1")" "$index") \ + || die "could not derive chain $1's $(_lc "$2") key at index $index — check ${1}_MNEMONIC" + printf '%s' "$account" +} + +_addr() { + local pk addr + read -r pk addr <<<"$(_key "$1" "$2")" + printf '%s' "$addr" +} + +_ibc() { + docker compose run --rm -T deployer "$@" --home /home/ibc +} + +init_link() { + mkdir -p "$KMS_KEYS_DIR" + + local name role pk addr file + for name in "${CHAINS[@]}"; do + for role in RELAYER ATTESTOR; do + read -r pk addr <<<"$(_key "$name" "$role")" + file="$KMS_KEYS_DIR/$(_lc "$role")-$(_lc "$name").hex" + printf '%s\n' "$pk" > "$file" + chmod 644 "$file" + done + done + + print_signers +} + +print_signers() { + local name role index addr + log "kms keys (served on kms:9090, referenced by remoteKeyId in link/*.yml):" + for name in "${CHAINS[@]}"; do + for role in RELAYER ATTESTOR; do + index=$(_chain_attr "$name" "${role}_INDEX") + addr=$(_addr "$name" "$role") + log " $(_lc "$role")-$(_lc "$name") $addr (${name}_MNEMONIC index $index)" + [[ "$role" == "RELAYER" && "$index" -ge "$FUNDED_ACCOUNTS" ]] \ + && warn "$addr is outside the funded range (FUNDED_ACCOUNTS=$FUNDED_ACCOUNTS)" \ + "and has no gas on chain $name" + done + done + return 0 +} + +_write_link_env() { + cat > "$LINK_ENV_FILE" </dev/null) + [[ -n "$router" ]] || die "no router address in $file — did 'ibc deploy core' run?" + printf '%s' "$router" +} + +deploy_contracts() { + [[ -d "$KMS_KEYS_DIR" ]] || die "$KMS_KEYS_DIR is missing — the init phase did not complete" + mkdir -p "$DEPLOY_DIR/keys" "$DEPLOY_DIR/deployments" + + local name other pk addr + for name in "${CHAINS[@]}"; do + if [[ ! -f "$DEPLOY_DIR/keys/deployer-$(_lc "$name").json" ]]; then + read -r pk addr <<<"$(_key "$name" DEPLOYER)" + _ibc keys import ecdsa "deployer-$(_lc "$name")" --private-key "0x$pk" >/dev/null \ + || die "[$name] importing the deployer key failed" + log "[$name] imported deployer-$(_lc "$name") ($addr)" + fi + done + + for name in "${CHAINS[@]}"; do + log "[$name] deploying the core IBC routing stack..." + _ibc deploy core --chain "$(_chain_attr "$name" CHAIN_ID)" --yes \ + || die "[$name] 'ibc deploy core' failed" + done + + for name in "${CHAINS[@]}"; do + other=$(_other "$name") + log "[$name] registering $LINK_CLIENT_ID tracking chain $other," \ + "authorizing attestor-$(_lc "$other") $(_addr "$other" ATTESTOR)..." + _ibc deploy client \ + --chain "$(_chain_attr "$name" CHAIN_ID)" \ + --counterparty-chain "$(_chain_attr "$other" CHAIN_ID)" \ + --client-id "$LINK_CLIENT_ID" --counterparty-client-id "$LINK_CLIENT_ID" \ + --attestors "$(_addr "$other" ATTESTOR)" --threshold 1 --yes \ + || die "[$name] 'ibc deploy client' failed" + done + + _write_link_env "$(_router "$A_CHAIN_ID")" "$(_router "$B_CHAIN_ID")" "$LINK_CLIENT_ID" + log "Wrote the deployment into $LINK_ENV_FILE:" + grep -v '^#' "$LINK_ENV_FILE" | while read -r line; do log " $line"; done +} + +# ── IFT transfer ──────────────────────────────────────────────────────────── + +# _token — the IFT address from that chain's manifest. Scoped to the +# "tokens" array: clients[] carries an "address" too, and it comes first. +_token() { + perl -0777 -ne 'print $1 if /"tokens"\s*:\s*\[.*?"address"\s*:\s*"([^"]+)"/s' \ + "$DEPLOY_DIR/deployments/$1.json" 2>/dev/null +} + +# _balance
— IFT balance in base units. +_balance() { + _ibc query ift balance --chain "$1" --ift "$2" --address "$3" 2>/dev/null \ + | perl -0777 -ne 'print $1 if /"balance"\s*:\s*"([^"]+)"/' +} + +# Deploys one IFT token per chain and bridges them over the client registered by +# deploy_contracts. Skipped when both manifests already carry a token, so a +# re-run does not mint a second one. +deploy_ift() { + local name other + if [[ -n "$(_token "$A_CHAIN_ID")" && -n "$(_token "$B_CHAIN_ID")" ]]; then + log "IFT already deployed on both chains" + return 0 + fi + + for name in "${CHAINS[@]}"; do + # IFT rides on ICS27-GMP, and `deploy ift` refuses without it + # ("no gmp deployment recorded for chain "). + log "[$name] deploying the ICS27-GMP app..." + _ibc deploy gmp --chain "$(_chain_attr "$name" CHAIN_ID)" --yes \ + || die "[$name] 'ibc deploy gmp' failed" + + log "[$name] deploying IFT $IFT_SYMBOL..." + _ibc deploy ift --chain "$(_chain_attr "$name" CHAIN_ID)" \ + --name "$IFT_NAME" --symbol "$IFT_SYMBOL" --yes \ + || die "[$name] 'ibc deploy ift' failed" + done + + log "bridging $(_token "$A_CHAIN_ID") <-> $(_token "$B_CHAIN_ID") over $LINK_CLIENT_ID..." + _ibc deploy ift-bridge \ + --chain-a "$A_CHAIN_ID" --ift-a "$(_token "$A_CHAIN_ID")" \ + --chain-b "$B_CHAIN_ID" --ift-b "$(_token "$B_CHAIN_ID")" \ + --client-id "$LINK_CLIENT_ID" --yes \ + || die "'ibc deploy ift-bridge' failed" +} + +# Mint on chain A, send across, and wait for the relayer to deliver it. This is +# the end-to-end assertion for the whole example: the balance on chain B only +# moves if both attestors signed through kms and the relayer assembled their +# attestations into a proof the light client accepted. +relay_ift_transfer() { + local ift_a ift_b sender receiver balance before expected waited=0 + ift_a=$(_token "$A_CHAIN_ID"); ift_b=$(_token "$B_CHAIN_ID") + [[ -n "$ift_a" && -n "$ift_b" ]] || die "no IFT address in the manifests — did 'ibc deploy ift' run?" + sender=$(_addr A DEPLOYER); receiver=$(_addr B DEPLOYER) + + # Wait on a delta, not an absolute balance: a second run starts with the first + # run's tokens already on chain B, and an absolute check would pass before the + # new packet ever landed. bigint because 1e18 is past what bash arithmetic can + # be trusted with once the amounts are overridden. + before=$(_balance "$B_CHAIN_ID" "$ift_b" "$receiver") + [[ -n "$before" ]] || die "could not read the chain B balance of $receiver" + expected=$(perl -Mbigint -e 'print $ARGV[0] + $ARGV[1]' "$before" "$IFT_SEND_AMOUNT") + + log "[A] minting $IFT_MINT_AMOUNT $IFT_SYMBOL to $sender..." + _ibc tx ift mint --chain "$A_CHAIN_ID" --ift "$ift_a" --from deployer-a \ + --to "$sender" --amount "$IFT_MINT_AMOUNT" \ + || die "'ibc tx ift mint' failed" + + log "[A] sending $IFT_SEND_AMOUNT $IFT_SYMBOL to $receiver on chain B over $LINK_CLIENT_ID..." + local hash + hash=$(_ibc tx ift send --chain "$A_CHAIN_ID" --ift "$ift_a" --from deployer-a \ + --client-id "$LINK_CLIENT_ID" --to "$receiver" --amount "$IFT_SEND_AMOUNT" 2>/dev/null \ + | perl -0777 -ne 'print $1 if /"txHash"\s*:\s*"([^"]+)"/') + [[ -n "$hash" ]] || die "'ibc tx ift send' returned no transaction hash" + log " sent in $hash" + + # The relay is explicit. `relayer.connections[].autoRelay` exists in the config + # schema but nothing in link reads it yet, so a packet sits unrelayed until it + # is handed to the relayer by transaction hash. Run inside ibc-link: the + # command dials the relayer's own gRPC, which only its config describes. + log "handing the packet to the relayer..." + docker compose exec -T ibc-link /opt/ibc relayer relay \ + --tx-hash "$hash" --chain-id "$A_CHAIN_ID" --home /home/ibc >/dev/null \ + || die "'ibc relayer relay' failed" + + log "waiting for chain B's balance to go $before -> $expected..." + while (( waited < IFT_RELAY_TIMEOUT )); do + balance=$(_balance "$B_CHAIN_ID" "$ift_b" "$receiver") + if [[ "$balance" == "$expected" ]]; then + log "relayed: $receiver holds $balance $IFT_SYMBOL on chain B" + log " $sender holds $(_balance "$A_CHAIN_ID" "$ift_a" "$sender") $IFT_SYMBOL on chain A" + return 0 + fi + sleep "$IFT_POLL_INTERVAL" + (( waited += IFT_POLL_INTERVAL )) + done + + die "chain B balance is '${balance:-unset}' after ${IFT_RELAY_TIMEOUT}s, expected $expected" \ + "— check 'docker compose logs ibc-link attestor-a attestor-b'" +} + +start_link() { + # Written by deploy_contracts. Without it the attestors exit on + # `invalid ics26 router address` and ibc-link on `.clientId required`. + grep -qE '^A_ICS26_ROUTER=.+' "$LINK_ENV_FILE" 2>/dev/null \ + || die "$LINK_ENV_FILE holds no deployment — the deploy phase did not complete" + + log "Bringing up ${LINK_SERVICES[*]}..." + docker compose up -d "${LINK_SERVICES[@]}" +} diff --git a/examples/besu-to-besu/link/attestor-a.yml b/examples/besu-to-besu/link/attestor-a.yml new file mode 100644 index 000000000..2cbb9c096 --- /dev/null +++ b/examples/besu-to-besu/link/attestor-a.yml @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: Apache-2.0 + +server: + listenAddr: 0.0.0.0:3000 + +signers: + - alias: attestor-key + type: remote + grpc: kms:9090 + remoteKeyId: attestor-a + +chains: + - chainId: "41001" + evm: + rpc: http://besu-a:8545 + ics26Router: "${A_ICS26_ROUTER}" + +attestors: + - name: attestor-a + chainId: "41001" + type: local + signer: attestor-key + finalityOffset: 1 diff --git a/examples/besu-to-besu/link/attestor-b.yml b/examples/besu-to-besu/link/attestor-b.yml new file mode 100644 index 000000000..e37c70797 --- /dev/null +++ b/examples/besu-to-besu/link/attestor-b.yml @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: Apache-2.0 + +server: + listenAddr: 0.0.0.0:3000 + +signers: + - alias: attestor-key + type: remote + grpc: kms:9090 + remoteKeyId: attestor-b + +chains: + - chainId: "41002" + evm: + rpc: http://besu-b:8545 + ics26Router: "${B_ICS26_ROUTER}" + +attestors: + - name: attestor-b + chainId: "41002" + type: local + signer: attestor-key + finalityOffset: 1 diff --git a/examples/besu-to-besu/link/deploy.yml b/examples/besu-to-besu/link/deploy.yml new file mode 100644 index 000000000..28113a2d1 --- /dev/null +++ b/examples/besu-to-besu/link/deploy.yml @@ -0,0 +1,28 @@ +# SPDX-License-Identifier: Apache-2.0 + +server: + listenAddr: 0.0.0.0:3000 + +db: + type: sqlite + url: ibc.db + +chains: + - chainId: "41001" + evm: + rpc: http://besu-a:8545 + deployer: deployer-a + + - chainId: "41002" + evm: + rpc: http://besu-b:8545 + deployer: deployer-b + +signers: + - alias: deployer-a + type: local + file: keys/deployer-a.json + + - alias: deployer-b + type: local + file: keys/deployer-b.json diff --git a/examples/besu-to-besu/link/ibc.yml b/examples/besu-to-besu/link/ibc.yml new file mode 100644 index 000000000..7f4b4264e --- /dev/null +++ b/examples/besu-to-besu/link/ibc.yml @@ -0,0 +1,56 @@ +# SPDX-License-Identifier: Apache-2.0 + +server: + listenAddr: 0.0.0.0:3000 + +db: + type: sqlite + url: ibc.db + +signers: + - alias: relayer-a + type: remote + grpc: kms:9090 + remoteKeyId: relayer-a + + - alias: relayer-b + type: remote + grpc: kms:9090 + remoteKeyId: relayer-b + +chains: + - chainId: "41001" + evm: + rpc: http://besu-a:8545 + ics26Router: "${A_ICS26_ROUTER}" + + - chainId: "41002" + evm: + rpc: http://besu-b:8545 + ics26Router: "${B_ICS26_ROUTER}" + +relayer: + dispatchPollInterval: 3s + connections: + - alias: a-b + # No autoRelay block: the field parses but nothing reads it yet, so + # packets are relayed explicitly with `ibc relayer relay --tx-hash`. + clientA: + chainId: "41001" + signer: relayer-a + clientId: "${A_CLIENT_ID}" + type: attestation + clientB: + chainId: "41002" + signer: relayer-b + clientId: "${B_CLIENT_ID}" + type: attestation + +attestors: + - name: attestor-a + type: remote + grpc: attestor-a:3000 + + - name: attestor-b + type: remote + grpc: attestor-b:3000 diff --git a/examples/besu-to-besu/link/kms.yaml b/examples/besu-to-besu/link/kms.yaml new file mode 100644 index 000000000..01b53e91e --- /dev/null +++ b/examples/besu-to-besu/link/kms.yaml @@ -0,0 +1,25 @@ +# SPDX-License-Identifier: Apache-2.0 + +grpc: + listen: 0.0.0.0:9090 + + keys: + - id: attestor-a + backend: file + algorithm: secp256k1eth + key_file: keys/attestor-a.hex + + - id: attestor-b + backend: file + algorithm: secp256k1eth + key_file: keys/attestor-b.hex + + - id: relayer-a + backend: file + algorithm: secp256k1eth + key_file: keys/relayer-a.hex + + - id: relayer-b + backend: file + algorithm: secp256k1eth + key_file: keys/relayer-b.hex diff --git a/examples/besu-to-besu/setup.sh b/examples/besu-to-besu/setup.sh index 3f81ada5d..7f701b7d7 100755 --- a/examples/besu-to-besu/setup.sh +++ b/examples/besu-to-besu/setup.sh @@ -33,19 +33,19 @@ # # A ◀──IBC──▶ B # -# Every secret is derived from a BIP-39 mnemonic at init time — no private key -# is committed to the repo. `init` derives each chain's validator key, computes -# the matching QBFT genesis extraData, and renders besu.toml / el-genesis.json -# into chains/local//, which is what docker-compose bind-mounts. +# +# Five phases, always run together: +# 1. init derive every key, render the chain configs into chains/local/ +# 2. start docker compose up both chains, wait for RPC +# 3. deploy ibc deploy core + client on each chain (writing +# chains/local/link.env), then an IFT token per chain and the +# bridge between them +# 4. link docker compose up kms, both attestors, ibc-link +# 5. transfer mint IFT on chain A, send it to chain B, and wait for the +# relayer to deliver it — the end-to-end assertion # # Usage: -# ./setup.sh — init + start + wait for RPC (default end-to-end) -# ./setup.sh init — derive validator keys and render chain configs -# into chains/local/. Touches no containers. -# ./setup.sh start — docker compose up both chains, wait for RPC -# (init must have run) -# ./setup.sh accounts — print the derived accounts and their roles -# ./setup.sh status — RPC endpoints, chain IDs, block heights +# ./setup.sh — run all five (the demo) # ./setup.sh clean — stop containers, remove volumes and chains/local/ # # Environment (optional): @@ -54,12 +54,25 @@ # (defaults to the same phrase as A_MNEMONIC, so both # chains share the same account set, including the # validator; override independently for separate sets) -# A_VALIDATOR_INDEX index of chain A's validator within A_MNEMONIC (1) -# B_VALIDATOR_INDEX index of chain B's validator within B_MNEMONIC (1) -# Index 0 of each phrase is that chain's deployer and is -# deliberately not a validator anywhere. +# Each chain draws four keys from its own mnemonic, by index: +# A/B_DEPLOYER_INDEX deploys the contracts (0). The one key kms does not +# hold: `ibc deploy` needs the raw private key. +# A/B_VALIDATOR_INDEX signs that chain's blocks (1). Kept off the deployer so +# no chain's proposer is also the account sending txs — a +# validator is its chain's coinbase and would pocket the +# priority fee back. +# A_RELAYER_INDEX index of the key the relayer signs chain A txs with (2) +# B_RELAYER_INDEX same for chain B (2). Must be inside the funded range — +# this key pays for gas. +# A_ATTESTOR_INDEX index of chain A's attestor key (3) +# B_ATTESTOR_INDEX same for chain B (4) — distinct from A's so the two +# attestors have different addresses even when both +# chains share a mnemonic. Attestors sign attestations +# only and need no balance. # FUNDED_ACCOUNTS how many accounts from each chain's phrase to pre-fund # in its genesis (default 5). init logs every address. +# IBC_IMAGE / KMS_IMAGE image overrides for the link services and the remote +# signer; their defaults live in docker-compose.yml # GENESIS_BALANCE hex wei per funded account (default 1 000 000 ETH) # QBFT_BLOCK_PERIOD_SECONDS / QBFT_EPOCH_LENGTH / # QBFT_REQUEST_TIMEOUT_SECONDS QBFT consensus tunables @@ -90,26 +103,26 @@ export COMPOSE_PROJECT_NAME="${COMPOSE_PROJECT_NAME:-$(basename "$SCRIPT_DIR")}" export BESU_IMAGE="${BESU_IMAGE:-hyperledger/besu:25.4.0}" export FOUNDRY_IMAGE="${FOUNDRY_IMAGE:-ghcr.io/foundry-rs/foundry:latest}" -# Key material. One mnemonic per chain, independently overridable, but -# defaulted to the same phrase so out of the box both chains fund the same -# account set. -# -# The default below is a standard BIP-39 test vector, published in the spec -# itself — demo material, safe to commit precisely because it protects nothing. -# -# Never point either of these at a mnemonic holding real funds, on any network. -# Everything derived from them is written to chains/local// unencrypted -# for Besu to read, and their addresses are printed to the console and the log -# file on every run. export A_MNEMONIC="${A_MNEMONIC:-legal winner thank year wave sausage worth useful legal winner thank yellow}" export B_MNEMONIC="${B_MNEMONIC:-$A_MNEMONIC}" -# Index 0 is the deployer on both chains and index 1 is the validator on both, -# so no chain's block proposer is also the account sending txs. A validator is -# its chain's coinbase and pockets the priority fee, which would otherwise make -# an identical tx cost the deployer less on its own chain than on the other. export A_VALIDATOR_INDEX="${A_VALIDATOR_INDEX:-1}" export B_VALIDATOR_INDEX="${B_VALIDATOR_INDEX:-1}" +export A_DEPLOYER_INDEX="${A_DEPLOYER_INDEX:-0}" +export B_DEPLOYER_INDEX="${B_DEPLOYER_INDEX:-0}" +export A_RELAYER_INDEX="${A_RELAYER_INDEX:-2}" +export B_RELAYER_INDEX="${B_RELAYER_INDEX:-2}" +export A_ATTESTOR_INDEX="${A_ATTESTOR_INDEX:-3}" +export B_ATTESTOR_INDEX="${B_ATTESTOR_INDEX:-4}" export FUNDED_ACCOUNTS="${FUNDED_ACCOUNTS:-5}" + +# The IFT token the transfer phase deploys, mints, and sends. Amounts are in the +# token's base unit (18 decimals), so the defaults are 1.0 minted and 0.5 sent. +export IFT_NAME="${IFT_NAME:-Demo Token}" +export IFT_SYMBOL="${IFT_SYMBOL:-DEMO}" +export IFT_MINT_AMOUNT="${IFT_MINT_AMOUNT:-1000000000000000000}" +export IFT_SEND_AMOUNT="${IFT_SEND_AMOUNT:-500000000000000000}" +export IFT_RELAY_TIMEOUT="${IFT_RELAY_TIMEOUT:-120}" +export IFT_POLL_INTERVAL="${IFT_POLL_INTERVAL:-3}" export GENESIS_BALANCE="${GENESIS_BALANCE:-0xd3c21bcecceda1000000}" # 1e24 wei = 1M ETH # QBFT consensus tunables — substituted into el-genesis.json.tmpl. @@ -117,17 +130,16 @@ export QBFT_BLOCK_PERIOD_SECONDS="${QBFT_BLOCK_PERIOD_SECONDS:-2}" export QBFT_EPOCH_LENGTH="${QBFT_EPOCH_LENGTH:-30000}" export QBFT_REQUEST_TIMEOUT_SECONDS="${QBFT_REQUEST_TIMEOUT_SECONDS:-4}" -# shellcheck source=lib/common.sh source "$LIB_DIR/common.sh" -# shellcheck source=lib/chains.sh source "$LIB_DIR/chains.sh" +source "$LIB_DIR/link.sh" cmd_init() { check_prerequisites - log "--- Phase 1: Derive keys + render chain configs ---" - # init_chains logs each chain's funded alloc set as it renders, so there is - # no print_accounts call here — that would restate the same addresses. + log "--- Phase 1A: Derive keys + render chain configs ---" init_chains + log "--- Phase 1B: Derive kms keys + link.env ---" + init_link } cmd_start() { @@ -137,23 +149,36 @@ cmd_start() { log "Chains are live and producing blocks." } +cmd_deploy() { + run_phase "Phase 3A: Deploy IBC on both chains" deploy_contracts + run_phase "Phase 3B: Deploy the IFT token and bridge" deploy_ift +} + +cmd_link() { + run_phase "Phase 4: Start IBC Link services" start_link + docker compose ps +} + +cmd_transfer() { + run_phase "Phase 5: Relay an IFT transfer A -> B" relay_ift_transfer +} + main() { case "${1:-}" in - init) cmd_init; exit 0 ;; - start) cmd_start; exit 0 ;; - accounts) print_accounts; exit 0 ;; - status) print_status; exit 0 ;; clean) clean; exit 0 ;; "") log "╔══════════════════════════════════════════════════╗" - log "║ besu-to-besu: 2 Besu QBFT chains (A, B) ║" + log "║ besu-to-besu: 2 Besu QBFT chains (A, B) + Link ║" log "╚══════════════════════════════════════════════════╝" cmd_init cmd_start + cmd_deploy + cmd_link + cmd_transfer exit 0 ;; *) - echo "Usage: $0 [init|start|accounts|status|clean]" >&2 + echo "Usage: $0 [clean]" >&2 exit 1 ;; esac From dcb63193a88e1b986a0d4fc007b574c0127338db Mon Sep 17 00:00:00 2001 From: Diana Borbe Date: Thu, 20 Aug 2026 14:52:38 +0200 Subject: [PATCH 2/3] Update Signed-off-by: Diana Borbe --- examples/besu-to-besu/README.md | 299 ++++++++++++++++------- examples/besu-to-besu/docker-compose.yml | 10 +- examples/besu-to-besu/lib/chains.sh | 42 ++-- examples/besu-to-besu/lib/common.sh | 20 ++ examples/besu-to-besu/lib/link.sh | 223 ++++++++++++----- examples/besu-to-besu/link/ibc.yml | 2 - examples/besu-to-besu/setup.sh | 61 ++++- 7 files changed, 463 insertions(+), 194 deletions(-) diff --git a/examples/besu-to-besu/README.md b/examples/besu-to-besu/README.md index 815279447..953176d18 100644 --- a/examples/besu-to-besu/README.md +++ b/examples/besu-to-besu/README.md @@ -2,99 +2,136 @@ # besu-to-besu - -Two independent single-validator Besu QBFT chains (A and B), the substrate for a -single IBC pair, plus the IBC Link services that relay between them: +Two independent single-validator Besu QBFT chains and the IBC Link services that +move a token between them. One command brings up the chains, deploys IBC on +both, and relays a transfer from A to B. ``` -besu-a ◀──┐ ┌──▶ besu-b - │ attestor-a attestor-b │ - └── (41001) (41002) ──┘ - │ │ - ├──── ibc-link ────┤ relayer, no attestation key - │ │ - └────── kms ───────┘ every signing key lives here + chain A (41001) chain B (41002) + + ┌────────────┐ ┌────────────┐ + │ besu-a │ │ besu-b │ + └─────┬──────┘ └─────┬──────┘ + │ watches watches │ + ▼ ▼ + ┌────────────┐ ┌────────────┐ + │ attestor-a │ │ attestor-b │ + └─────┬──────┘ └─────┬──────┘ + │ │ + └──────────▶┌───────────┐◀──────────┘ + │ relayer │ + └───────────┘ ``` -Each attestor is a **standalone external process**: its own container, its own -config, reached by the relayer over gRPC as a `type: remote` attestor. The -relayer never loads an attestor config and cannot produce an attestation itself, -which is the difference from dual mode (one process running the relayer and a -`type: local` attestor together). +Each attestor watches one chain and signs attestations about it, nothing else. +The relayer is the only process wired to everything: both attestors over gRPC, +and an RPC connection to both chains. A transfer from A to B goes: -No signing key is on disk in any attestor or relayer container. -[cosmos/kms](https://github.com/cosmos/kms) runs in gRPC-only mode — no chains, -no validators, no privval dial-out — and serves all four keys over its -SignerService. Each service addresses the key it is allowed to use by id -(`type: remote`, `remoteKeyId:`), and the private key never leaves kms. +1. `ift send` on chain A burns 0.5 DEMO and emits a packet. The send tx is + handed to the relayer by hash. +2. The relayer pulls an attestation over chain A's state from attestor-a. +3. The relayer submits the packet to chain B with that attestation as its proof. + Chain B's light client verifies it against the attestor it authorizes — + attestor-a, the counterparty's, not its own. -## Prerequisites +kms serves all four signing keys over gRPC; no key is on disk in any other +container. -`docker` (with the compose v2 plugin), `curl`, `perl`. +## Quick start -`cast` (foundry) does the BIP-39 derivation. If it isn't on `PATH`, `cast_cli` -falls back to running it inside `$FOUNDRY_IMAGE`, so a host install is optional. +Needs `docker` (with the compose v2 plugin), `curl`, and `perl`. Everything else +is pulled from public GHCR images — nothing is built, no `docker login`. `cast` +(foundry) does the BIP-39 derivation; if it isn't on `PATH`, `cast_cli` runs it +inside `$FOUNDRY_IMAGE` instead, so a host install is optional. -Nothing is built from source. The images are pulled from GHCR and are public, so -no `docker login` is needed: +```bash +cd examples/besu-to-besu +./setup.sh +``` -| Service | Image | Override | -|----------------------------------|--------------|-------------| -| ibc-link, attestor-a/b, deployer | `ghcr.io/cosmos/ibc:main` | `IBC_IMAGE` | -| kms | `ghcr.io/cosmos/kms:latest` | `KMS_IMAGE` | -| besu-a, besu-b | `hyperledger/besu:25.4.0` | `BESU_IMAGE` | +Takes about two minutes on a warm cache, and ends with: + +``` +[14:14:17] [A] minting 1000000000000000000 DEMO to 0x58A57ed9... +[14:14:20] [A] sending 500000000000000000 DEMO to 0x58A57ed9... on chain B over link-41001-41002... +[14:14:22] sent in 0xbc60671b3e081d770e8704f0e6c4c25ba9fe0bfeae0a1d99418dd17dc1a5f146 +[14:14:22] handing the packet to the relayer... +[14:14:22] waiting for chain B's balance to go 0 -> 500000000000000000... +[14:14:38] relayed A -> B: 0x58A57ed9... holds 500000000000000000 DEMO on chain B +[14:14:40] DEMO held by the deployer: chain A 1500000000000000000, chain B 500000000000000000 +``` -`IBC_IMAGE` must be built from a commit that has both the `deploy` command and -the unified top-level `attestors[]` list. Older tags fail late and obscurely — -`unknown command "deploy"` two phases in, or attestors restart-looping on `no -attestations provided`, because `attestor.attestations[]` entries carry no `type` -and no `grpc` and so cannot express a standalone external attestor. If you see -either, check the tag first. Rebuild one with `gh workflow run -ibc-link-build.yml --ref ` — the image is tagged after the ref it runs -on. +That `relayed` line is the whole point: the balance on chain B only moves if +attestor-a signed through kms and the relayer assembled that attestation into a +proof chain B's light client accepted. -To run local relayer or attestor changes, build the image and point `IBC_IMAGE` -at it: +`roundtrip` sends the same tokens straight back afterwards. The return leg needs +no mint — it spends what the first leg delivered — so chain B ends where it +started and chain A ends holding everything it minted: ```bash -docker build -t ibc-link:local --target target-builder ../../link -IBC_IMAGE=ibc-link:local ./setup.sh +./setup.sh roundtrip # phase 5 alone when the stack is already up ``` -## Usage +``` +[14:13:27] relayed A -> B: 0x58A57ed9... holds 500000000000000000 DEMO on chain B +[14:13:29] [B] sending 500000000000000000 DEMO to 0x58A57ed9... on chain A over link-41001-41002... +[14:13:48] relayed B -> A: 0x58A57ed9... holds 1000000000000000000 DEMO on chain A +[14:13:50] DEMO held by the deployer: chain A 1000000000000000000, chain B 0 +``` + +Worth running at least once: A → B only proves attestor-a and chain B's client. +The return leg is what exercises attestor-b and chain A's client, so a one-way +demo leaves half the stack unverified. -Run from `examples/besu-to-besu/`: +`roundtrip` is also the command for iterating on the relay itself. It checks +whether the stack is usable — all four values in `link.env`, both IFT tokens in +the manifests, and all six containers healthy — and if so runs phase 5 alone, in +about 45 seconds instead of two minutes. If any of that is missing or the +containers are stopped, it brings the stack up first. ```bash -./setup.sh # the demo: init + start + deploy + link ./setup.sh clean # stop containers, remove volumes and chains/local/ ``` -Two commands, on purpose. The five phases always run together: +Three commands, on purpose. Re-running any of them against a live stack is safe +and idempotent — every deploy step re-checks on-chain state and reports +`skipped` — so it relays another transfer rather than rebuilding anything. + +Each invocation writes a timestamped log to `logs/`. Use `docker compose` +directly to poke at a running stack: + +```bash +docker compose ps +docker compose logs -f relayer +docker compose exec attestor-a /opt/ibc attestor info attestor-a --home /home/ibc +``` + +## The five phases + +They always run together; the names are internal, not subcommands. | Phase | What it does | -|------------|-------------------------------------------------------------------| +|------------|------------------------------------------------------------------| | `init` | derive every key, render the chain configs into `chains/local/` | | `start` | `docker compose up` both chains, wait for RPC | | `deploy` | `ibc deploy core` + `client` on each chain (writing `link.env`), then GMP, an IFT token per chain, and the bridge | -| `link` | `docker compose up` kms, both attestors, ibc-link | -| `transfer` | mint IFT on A, send it to B, relay it, assert the balance moved | - -Use `docker compose` directly to poke at a running stack (`ps`, `logs -f -ibc-link`, `exec attestor-a /opt/ibc attestor info attestor-a --home /home/ibc`). +| `link` | `docker compose up` kms, both attestors, relayer | +| `transfer` | mint IFT on A, send it to B, relay it, assert the balance moved — and with `roundtrip`, send it back | -A bare `./setup.sh` runs all five phases and ends with half a token having -crossed from chain A to chain B. +## What makes this example different -Verify once the chains are up: - -```bash -curl -s -X POST -H 'Content-Type: application/json' \ - --data '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}' \ - http://localhost:8745 # → 0xa02a (= 41002, chain B) -``` +Each attestor is a **standalone external process**: its own container, its own +config, reached by the relayer over gRPC as a `type: remote` attestor. The +relayer never loads an attestor config and cannot produce an attestation itself, +which is the difference from dual mode (one process running the relayer and a +`type: local` attestor together). -Each invocation writes a timestamped log file to `logs/`. +No signing key is on disk in any attestor or relayer container. +[cosmos/kms](https://github.com/cosmos/kms) runs in gRPC-only mode — no chains, +no validators, no privval dial-out — and serves all four keys over its +SignerService. Each service addresses the key it is allowed to use by id +(`type: remote`, `remoteKeyId:`), and the private key never leaves kms. ## Chains and accounts @@ -110,6 +147,9 @@ chain's genesis; index 0 is the deployer, index 1 the validator. Set `A_MNEMONIC` and `B_MNEMONIC` to different phrases if you want the chains to have fully independent account sets instead. +Every key here is derived from a public BIP-39 test vector and printed in the +logs. Local devnet only — never send real funds to any address this prints. + ## Signing keys The init phase derives four keys from the same mnemonics and writes them to @@ -118,8 +158,8 @@ Every run prints them. | kms key id | Source | Used by | Needs gas | |--------------|----------------------------|------------|-----------| -| `relayer-a` | `A_MNEMONIC` index 2 | ibc-link | yes, on A | -| `relayer-b` | `B_MNEMONIC` index 2 | ibc-link | yes, on B | +| `relayer-a` | `A_MNEMONIC` index 2 | relayer | yes, on A | +| `relayer-b` | `B_MNEMONIC` index 2 | relayer | yes, on B | | `attestor-a` | `A_MNEMONIC` index 3 | attestor-a | no | | `attestor-b` | `B_MNEMONIC` index 4 | attestor-b | no | @@ -136,15 +176,14 @@ from; link rejects any other scheme at startup. The keys are written mode `0644`, unlike the `0600` Besu validator keys, because the kms image runs unprivileged (uid 10001) and could not otherwise read them -through the bind mount on Linux. Every one is a public BIP-39 test-vector key. +through the bind mount on Linux. ## Deploying The deploy phase puts IBC on both chains through the one-shot `deployer` compose service — the same `ibc` image with [link/deploy.yml](link/deploy.yml) -mounted, run as `docker compose run --rm deployer`. Running it in a container -means no Go toolchain on the host, and `besu-a` / `besu-b` resolve exactly as -they do for the real services. Per chain: +mounted. Running it in a container means no Go toolchain on the host, and +`besu-a` / `besu-b` resolve exactly as they do for the real services. Per chain: ``` ibc keys import ecdsa deployer- # index 0 of that chain's mnemonic @@ -168,23 +207,20 @@ read back out of the manifests (`.core.router`); the client id is passed to `deploy client` explicitly, so it is known before anything runs and needs no parsing. -All four values are required to start, and each is missed at a different stage: - -| Symptom on startup | Cause | -|-------------------------------------------------|------------------------------| -| `.clientId required` | `*_CLIENT_ID` empty | -| `invalid ics26 router address "" for chain ` | `*_ICS26_ROUTER` empty | -| `no contract code at given address` | router set but not deployed | +To run an `ibc` command by hand against the same config, pass the uid and gid +`setup.sh` exports — the service writes to host-owned bind mounts, and the +compose default of `65532` cannot on Linux: -Re-running `./setup.sh` against a live stack is safe: an already-imported -deployer key is skipped, and `ibc deploy` skips steps whose artefacts already -exist, so the same addresses come back out. `./setup.sh clean` resets everything. +```bash +DEPLOY_UID=$(id -u) DEPLOY_GID=$(id -g) \ + docker compose run --rm deployer query ift balance --chain 41002 \ + --ift
--address --home /home/ibc +``` ## Relaying an IFT transfer -The last phase is the end-to-end assertion for everything above it. A balance -only moves on chain B if both attestors signed through kms and the relayer -assembled their attestations into a proof the light client accepted. +The last phase is the end-to-end assertion for everything above it, and this is +what it runs: ``` ibc deploy gmp --chain # IFT rides on ICS27-GMP @@ -197,6 +233,8 @@ ibc relayer relay --tx-hash --chain-id A ibc query ift balance --chain B --ift … --address ``` +`roundtrip` then repeats the last three with A and B swapped, and no mint. + `deploy gmp` is not optional — `deploy ift` refuses without it (`no gmp deployment recorded for chain `). @@ -205,29 +243,49 @@ config schema and validates, but nothing in `link` reads `.Enabled` or `.Lookback` yet, so a packet sits unrelayed until it is handed to the relayer by transaction hash. That is why `link/ibc.yml` carries no `autoRelay` block: it would only imply a behaviour that is not wired up. `relayer relay` runs inside -the `ibc-link` container, since the command dials the relayer's own gRPC and -only its config describes it. +the `relayer` container, since the command dials the relayer's own gRPC and only +its config describes it. + +The wait is on a *delta*, not an absolute balance — a second run starts with the +first run's tokens already in place, and an absolute check would pass before the +new packet ever landed. + +Direction is a parameter to the same helper, because the two legs prove +different halves of the stack: A → B rests on attestor-a and chain B's client, +B → A on attestor-b and chain A's client. Both directions were registered back +in the deploy phase — `deploy client` runs on each chain authorizing the +counterparty's attestor, and `deploy ift-bridge` registers a bridge on both +tokens — so the return leg needs no extra setup. `tx ift` and `deploy` both need the raw private key, so both run as the `deployer` service against `link/deploy.yml`. The transfer defaults are overridable: `IFT_NAME`, `IFT_SYMBOL`, `IFT_MINT_AMOUNT`, `IFT_SEND_AMOUNT`, -`IFT_RELAY_TIMEOUT` (120s), `IFT_POLL_INTERVAL` (3s). +`IFT_RELAY_TIMEOUT` (120), `IFT_POLL_INTERVAL` (3). The last two are whole +seconds with no unit suffix — `3`, not `3s`. Changing `IFT_NAME` or +`IFT_SYMBOL` against a stack that already has a token mints a second one rather +than replacing the first; run `./setup.sh clean` first. ## Ports -Both containers use the same internal ports (8545 RPC, 8546 WS, 9545 metrics). -Only the host-side mappings differ: +Both chains use the same internal ports (8545 RPC, 8546 WS, 9545 metrics). Only +the host-side mappings differ: | Service | RPC (host) | WS (host) | Metrics (host) | |---------|-----------:|----------:|---------------:| | besu-a | 8545 | 8546 | 9545 | | besu-b | 8745 | 8746 | 9745 | +```bash +curl -s -X POST -H 'Content-Type: application/json' \ + --data '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}' \ + http://localhost:8745 # → 0xa02a (= 41002, chain B) +``` + The link services all listen on 3000 internally: | Service | gRPC (host) | |------------|------------:| -| ibc-link | 3000 | +| relayer | 3000 | | attestor-a | 3010 | | attestor-b | 3011 | | kms | not published | @@ -243,21 +301,74 @@ reach it can sign with any of the four keys. It stays on the compose network only; use `docker compose exec kms ...` to inspect it. A real deployment sets `tls_cert` / `tls_key` in `link/kms.yaml` and puts network controls in front. +## Configuration + +| Variable | Default | Effect | +|----------|---------|--------| +| `IBC_IMAGE` | `ghcr.io/cosmos/ibc:main` | relayer, attestors, deployer | +| `KMS_IMAGE` | `ghcr.io/cosmos/kms:latest` | the remote signer | +| `BESU_IMAGE` | `hyperledger/besu:25.4.0` | both chains | +| `A_MNEMONIC` / `B_MNEMONIC` | a public test vector | every account on that chain | +| `FUNDED_ACCOUNTS` | `5` | accounts pre-funded in each genesis | +| `GENESIS_BALANCE` | 1 000 000 ETH | hex wei per funded account | +| `IFT_*` | see above | the token and the transfer | +| `QBFT_BLOCK_PERIOD_SECONDS` etc. | `2` | consensus tunables | + +Changing any chain-shaping variable invalidates the on-disk chain data: run +`./setup.sh clean` first, or Besu refuses to start against a genesis that no +longer matches its database. + +To run local relayer or attestor changes, build the image and point `IBC_IMAGE` +at it: + +```bash +docker build -t ibc-link:local --target target-builder ../../link +IBC_IMAGE=ibc-link:local ./setup.sh +``` + +## Troubleshooting + +**`unknown command "deploy"`, or attestors restart-looping on `no attestations +provided`.** `IBC_IMAGE` is too old. It must come from a commit that has both +the `deploy` command and the unified top-level `attestors[]` list — older +`attestor.attestations[]` entries carry no `type` and no `grpc`, so they cannot +express a standalone external attestor. Check the tag first; rebuild one with +`gh workflow run ibc-link-build.yml --ref `, which tags the image after +the ref it runs on. + +**A link service exits at startup.** All four values in `chains/local/link.env` +are required, and each is missed at a different stage: + +| Symptom on startup | Cause | +|--------------------------------------------------|-----------------------------| +| `.clientId required` | `*_CLIENT_ID` empty | +| `invalid ics26 router address "" for chain ` | `*_ICS26_ROUTER` empty | +| `no contract code at given address` | router set but not deployed | + +**The transfer times out.** `docker compose logs relayer attestor-a attestor-b`. +An attestor that cannot reach kms, or a client authorizing the wrong attestor +address, both surface here. + +**Stale state after an upgrade.** `./setup.sh clean` removes containers, volumes, +and `chains/local/`. It cannot remove a volume that a previous version of +`docker-compose.yml` named and this one does not; `docker volume ls` will show +any leftovers under the `besu-to-besu_` prefix. + ## Layout ``` examples/besu-to-besu/ ├── README.md -├── setup.sh — entrypoint: init | start | link | accounts | -│ signers | status | clean +├── setup.sh — entrypoint: the demo, or `clean` ├── docker-compose.yml — besu-a, besu-b, kms, attestor-a, attestor-b, -│ ibc-link +│ relayer, deployer (profile: tools) ├── lib/ -│ ├── common.sh — logging, prerequisite checks, RPC waiter, -│ │ render_template, cast_cli +│ ├── common.sh — logging, prerequisite checks, image pull, +│ │ RPC waiter, render_template, cast_cli │ ├── chains.sh — derivation, QBFT extraData, rendering, start / │ │ wait / status / clean -│ └── link.sh — kms key derivation, deployment, link.env +│ └── link.sh — kms key derivation, deployment, link.env, +│ the IFT transfer ├── link/ — committed, no secrets. Bind-mounted verbatim: │ ├── kms.yaml — gRPC-only remote signer, 4 secp256k1eth keys │ ├── attestor-a.yml — standalone attestor for chain A diff --git a/examples/besu-to-besu/docker-compose.yml b/examples/besu-to-besu/docker-compose.yml index d043945d1..d3daa7dec 100644 --- a/examples/besu-to-besu/docker-compose.yml +++ b/examples/besu-to-besu/docker-compose.yml @@ -9,7 +9,7 @@ # # kms ◀── Sign ──┬── attestor-a ──┐ # ├── attestor-b ──┤ StateAttestation / PacketAttestation -# └── ibc-link ◀───┘ +# └── relayer ◀───┘ x-link-service: &link-service image: ${IBC_IMAGE:-ghcr.io/cosmos/ibc:main} env_file: @@ -140,12 +140,12 @@ services: besu-b: condition: service_healthy - ibc-link: + relayer: <<: *link-service - container_name: ibc-link + container_name: relayer command: [ "relayer", "run", "--home", "/home/ibc" ] volumes: - - ibc-link-home:/home/ibc + - relayer-home:/home/ibc - ./link/ibc.yml:/home/ibc/ibc.yml:ro ports: - "127.0.0.1:3000:3000" # loopback only — see besu-a @@ -165,7 +165,7 @@ services: volumes: besu-a-data: besu-b-data: - ibc-link-home: + relayer-home: networks: besu-besu-net: diff --git a/examples/besu-to-besu/lib/chains.sh b/examples/besu-to-besu/lib/chains.sh index f31e2c5f6..2e0c193d5 100644 --- a/examples/besu-to-besu/lib/chains.sh +++ b/examples/besu-to-besu/lib/chains.sh @@ -38,6 +38,7 @@ B_SERVICE=besu-b; B_CHAIN_ID=41002; B_RPC_PORT=8745; B_WS_PORT=8746 CHAIN_ACCT_ADDRS=() CHAIN_ACCT_KEYS=() _DERIVED_MNEMONIC="" # phrase the arrays above currently hold; cache key +_PRINTED_ALLOC="" # alloc set already spelled out; suppresses a repeat listing # _chain_attr — read A_CHAIN_ID / B_RPC_PORT / … indirectly. _chain_attr() { @@ -223,21 +224,29 @@ _init_chain() { # Spell out the alloc set. These are the only accounts that can pay for gas # on this chain, so a wrong or missing one is the first thing to check when a # tx fails with "insufficient funds". - local i funded role privkey - log "[$name] genesis funds ${#CHAIN_ALLOC_ADDRS[@]} accounts with $(_genesis_balance_eth) each:" - for (( i = 0; i < ${#CHAIN_ALLOC_ADDRS[@]}; i++ )); do - funded="${CHAIN_ALLOC_ADDRS[$i]}" - role="" - [[ "$funded" == "${CHAIN_ACCT_ADDRS[0]}" ]] && role+=" [deployer]" - [[ "$funded" == "$addr" ]] && role+=" [validator]" - if (( i < ${#CHAIN_ACCT_ADDRS[@]} )); then - privkey="${CHAIN_ACCT_KEYS[$i]}" - log " index $i $funded${role} privkey=0x${privkey}" - else - privkey="$pk" - log " index $index $funded${role} privkey=0x${privkey}" - fi - done + local i funded role privkey alloc_key + # Both chains default to one mnemonic and one validator index, so the second + # listing would repeat the first line for line. Print it once and say so. + alloc_key="${CHAIN_ALLOC_ADDRS[*]}|$addr" + if [[ "$alloc_key" == "$_PRINTED_ALLOC" ]]; then + log "[$name] genesis funds the same ${#CHAIN_ALLOC_ADDRS[@]} accounts listed above" + else + log "[$name] genesis funds ${#CHAIN_ALLOC_ADDRS[@]} accounts with $(_genesis_balance_eth) each:" + for (( i = 0; i < ${#CHAIN_ALLOC_ADDRS[@]}; i++ )); do + funded="${CHAIN_ALLOC_ADDRS[$i]}" + role="" + [[ "$funded" == "${CHAIN_ACCT_ADDRS[0]}" ]] && role+=" [deployer]" + [[ "$funded" == "$addr" ]] && role+=" [validator]" + if (( i < ${#CHAIN_ACCT_ADDRS[@]} )); then + privkey="${CHAIN_ACCT_KEYS[$i]}" + log " index $i $funded${role} privkey=0x${privkey}" + else + privkey="$pk" + log " index $index $funded${role} privkey=0x${privkey}" + fi + done + _PRINTED_ALLOC="$alloc_key" + fi _append_chain_env "$name" "$addr" } @@ -309,7 +318,8 @@ start_chains() { || die "$LOCAL_DIR/$name/key is missing — the init phase did not complete" done log "Bringing up besu-a and besu-b..." - docker compose up -d besu-a besu-b + docker compose --progress quiet up -d besu-a besu-b \ + || die "besu-a/besu-b did not start — check 'docker compose logs besu-a besu-b'" } wait_for_chains() { diff --git a/examples/besu-to-besu/lib/common.sh b/examples/besu-to-besu/lib/common.sh index 88c00f47a..dd4f4ecf9 100644 --- a/examples/besu-to-besu/lib/common.sh +++ b/examples/besu-to-besu/lib/common.sh @@ -21,6 +21,26 @@ check_prerequisites() { command -v curl >/dev/null || die "curl is required" command -v cast >/dev/null \ || info "cast not on PATH — falling back to $FOUNDRY_IMAGE for key derivation" + + pull_images +} + +# Pull anything not already local, up front and with docker's progress bars — +# the only place they appear. Every later `compose up` and `compose run` is +# `--progress quiet`, where a first-run pull would look like a hang instead. +# Silent when there is nothing to fetch, so a re-run says nothing at all. +# COMPOSE_PROFILES reaches `deployer`, otherwise skipped as an inactive profile. +pull_images() { + local img missing=() + while read -r img; do + docker image inspect "$img" >/dev/null 2>&1 || missing+=("$img") + done < <(COMPOSE_PROFILES=tools docker compose config --images | sort -u) + + (( ${#missing[@]} )) || return 0 + log "Pulling ${#missing[@]} image(s), first run only:" + printf ' %s\n' "${missing[@]}" + COMPOSE_PROFILES=tools docker compose pull --policy missing \ + || die "could not pull the images — check network access" } # Run `cast`, preferring a host binary and falling back to the pinned foundry diff --git a/examples/besu-to-besu/lib/link.sh b/examples/besu-to-besu/lib/link.sh index bafa7865b..5898fde93 100644 --- a/examples/besu-to-besu/lib/link.sh +++ b/examples/besu-to-besu/lib/link.sh @@ -8,7 +8,7 @@ KMS_KEYS_DIR="$LOCAL_DIR/kms/keys" LINK_ENV_FILE="$LOCAL_DIR/link.env" DEPLOY_DIR="$LOCAL_DIR/deploy" -LINK_SERVICES=(kms attestor-a attestor-b ibc-link) +LINK_SERVICES=(kms attestor-a attestor-b relayer) LINK_CLIENT_ID="link-$A_CHAIN_ID-$B_CHAIN_ID" export DEPLOY_UID="${DEPLOY_UID:-$(id -u)}" @@ -33,12 +33,26 @@ _addr() { printf '%s' "$addr" } +# --progress quiet: this runs a dozen times, and each run would otherwise +# reprint the "Container besu-a Waiting / Healthy" block. Images are pulled by +# check_prerequisites so nothing slow is being hidden. _ibc() { - docker compose run --rm -T deployer "$@" --home /home/ibc + docker compose --progress quiet run --rm -T deployer "$@" --home /home/ibc +} + +# _steps — one indented line per step, in place of the raw +# JSON array `ibc deploy` prints. "executed" on a first run, "skipped" on a +# re-run, which is the idempotency of the deploy path made visible. +_steps() { + printf '%s' "$1" | perl -0777 -ne \ + 'print " $1: $2\n" while /"name"\s*:\s*"([^"]+)"\s*,\s*"action"\s*:\s*"([^"]+)"/gs' } init_link() { mkdir -p "$KMS_KEYS_DIR" + # Traversable by the unprivileged kms container (uid 10001) whatever the + # host umask is — the 0644 on the keys below is not enough on its own. + chmod 755 "$LOCAL_DIR/kms" "$KMS_KEYS_DIR" local name role pk addr file for name in "${CHAINS[@]}"; do @@ -92,7 +106,7 @@ deploy_contracts() { [[ -d "$KMS_KEYS_DIR" ]] || die "$KMS_KEYS_DIR is missing — the init phase did not complete" mkdir -p "$DEPLOY_DIR/keys" "$DEPLOY_DIR/deployments" - local name other pk addr + local name other pk addr out for name in "${CHAINS[@]}"; do if [[ ! -f "$DEPLOY_DIR/keys/deployer-$(_lc "$name").json" ]]; then read -r pk addr <<<"$(_key "$name" DEPLOYER)" @@ -104,135 +118,214 @@ deploy_contracts() { for name in "${CHAINS[@]}"; do log "[$name] deploying the core IBC routing stack..." - _ibc deploy core --chain "$(_chain_attr "$name" CHAIN_ID)" --yes \ + out=$(_ibc deploy core --chain "$(_chain_attr "$name" CHAIN_ID)" --yes) \ || die "[$name] 'ibc deploy core' failed" + _steps "$out" done for name in "${CHAINS[@]}"; do other=$(_other "$name") log "[$name] registering $LINK_CLIENT_ID tracking chain $other," \ "authorizing attestor-$(_lc "$other") $(_addr "$other" ATTESTOR)..." - _ibc deploy client \ + out=$(_ibc deploy client \ --chain "$(_chain_attr "$name" CHAIN_ID)" \ --counterparty-chain "$(_chain_attr "$other" CHAIN_ID)" \ --client-id "$LINK_CLIENT_ID" --counterparty-client-id "$LINK_CLIENT_ID" \ - --attestors "$(_addr "$other" ATTESTOR)" --threshold 1 --yes \ + --attestors "$(_addr "$other" ATTESTOR)" --threshold 1 --yes) \ || die "[$name] 'ibc deploy client' failed" + _steps "$out" done - _write_link_env "$(_router "$A_CHAIN_ID")" "$(_router "$B_CHAIN_ID")" "$LINK_CLIENT_ID" + # Via locals, not inline `$(...)`: `die` in a command substitution only exits + # the subshell, and errexit ignores a failed substitution in argument + # position — inline, a missing router would print the error and then write an + # empty address into link.env. + local a_router b_router + a_router=$(_router "$A_CHAIN_ID") || exit 1 + b_router=$(_router "$B_CHAIN_ID") || exit 1 + _write_link_env "$a_router" "$b_router" "$LINK_CLIENT_ID" log "Wrote the deployment into $LINK_ENV_FILE:" grep -v '^#' "$LINK_ENV_FILE" | while read -r line; do log " $line"; done } # ── IFT transfer ──────────────────────────────────────────────────────────── -# _token — the IFT address from that chain's manifest. Scoped to the -# "tokens" array: clients[] carries an "address" too, and it comes first. +# _token — the address of the $IFT_SYMBOL token in that chain's +# manifest. Anchored on the symbol rather than on the first "address" in the +# file: clients[] carries an "address" too and comes first, and a manifest can +# hold more than one token. "address" follows "symbol" inside the same object. _token() { - perl -0777 -ne 'print $1 if /"tokens"\s*:\s*\[.*?"address"\s*:\s*"([^"]+)"/s' \ + IFT_SYMBOL="$IFT_SYMBOL" perl -0777 -ne \ + 'print $1 if /"symbol"\s*:\s*"\Q$ENV{IFT_SYMBOL}\E".*?"address"\s*:\s*"([^"]+)"/s' \ "$DEPLOY_DIR/deployments/$1.json" 2>/dev/null } -# _balance
— IFT balance in base units. +# _balance
— IFT balance in base units, empty when +# the query fails. Callers check for empty: an unguarded assignment would take +# the pipeline's non-zero status through pipefail into an errexit abort with no +# message, pre-empting their own diagnostics. _balance() { _ibc query ift balance --chain "$1" --ift "$2" --address "$3" 2>/dev/null \ - | perl -0777 -ne 'print $1 if /"balance"\s*:\s*"([^"]+)"/' + | perl -0777 -ne 'print $1 if /"balance"\s*:\s*"([^"]+)"/' \ + || true } # Deploys one IFT token per chain and bridges them over the client registered by -# deploy_contracts. Skipped when both manifests already carry a token, so a -# re-run does not mint a second one. +# deploy_contracts. +# +# No short-circuit here: every step below is idempotent on the link side. `deploy +# ift` skips when a token with this symbol, name, and owner is already on chain, +# and `deploy ift-bridge` re-reads the on-chain bridge, so a re-run after a +# failed bridge retries the bridge instead of reporting the tokens as done and +# leaving the pair unbridged. Changing IFT_NAME or IFT_SYMBOL against a live +# stack mints a second token rather than replacing the first — './setup.sh +# clean' first. deploy_ift() { - local name other - if [[ -n "$(_token "$A_CHAIN_ID")" && -n "$(_token "$B_CHAIN_ID")" ]]; then - log "IFT already deployed on both chains" - return 0 - fi - + local name out for name in "${CHAINS[@]}"; do # IFT rides on ICS27-GMP, and `deploy ift` refuses without it # ("no gmp deployment recorded for chain "). log "[$name] deploying the ICS27-GMP app..." - _ibc deploy gmp --chain "$(_chain_attr "$name" CHAIN_ID)" --yes \ + out=$(_ibc deploy gmp --chain "$(_chain_attr "$name" CHAIN_ID)" --yes) \ || die "[$name] 'ibc deploy gmp' failed" + _steps "$out" log "[$name] deploying IFT $IFT_SYMBOL..." - _ibc deploy ift --chain "$(_chain_attr "$name" CHAIN_ID)" \ - --name "$IFT_NAME" --symbol "$IFT_SYMBOL" --yes \ + out=$(_ibc deploy ift --chain "$(_chain_attr "$name" CHAIN_ID)" \ + --name "$IFT_NAME" --symbol "$IFT_SYMBOL" --yes) \ || die "[$name] 'ibc deploy ift' failed" + _steps "$out" done log "bridging $(_token "$A_CHAIN_ID") <-> $(_token "$B_CHAIN_ID") over $LINK_CLIENT_ID..." - _ibc deploy ift-bridge \ + out=$(_ibc deploy ift-bridge \ --chain-a "$A_CHAIN_ID" --ift-a "$(_token "$A_CHAIN_ID")" \ --chain-b "$B_CHAIN_ID" --ift-b "$(_token "$B_CHAIN_ID")" \ - --client-id "$LINK_CLIENT_ID" --yes \ + --client-id "$LINK_CLIENT_ID" --yes) \ || die "'ibc deploy ift-bridge' failed" + _steps "$out" } -# Mint on chain A, send across, and wait for the relayer to deliver it. This is -# the end-to-end assertion for the whole example: the balance on chain B only -# moves if both attestors signed through kms and the relayer assembled their -# attestations into a proof the light client accepted. -relay_ift_transfer() { - local ift_a ift_b sender receiver balance before expected waited=0 - ift_a=$(_token "$A_CHAIN_ID"); ift_b=$(_token "$B_CHAIN_ID") - [[ -n "$ift_a" && -n "$ift_b" ]] || die "no IFT address in the manifests — did 'ibc deploy ift' run?" - sender=$(_addr A DEPLOYER); receiver=$(_addr B DEPLOYER) +# _leg — move IFT_SEND_AMOUNT from src's deployer to dst's deployer +# and block until it lands. This is the end-to-end assertion for everything the +# earlier phases set up: the balance on dst only moves if *src's* attestor +# signed through kms and the relayer assembled that attestation into a proof +# dst's light client accepted. +# +# Direction is a parameter because the two legs exercise different halves of +# the stack. A -> B proves attestor-a and dst chain B's client; B -> A proves +# attestor-b and chain A's client. A demo that only ever went one way would +# leave attestor-b unverified. +_leg() { + local src="$1" dst="$2" + local src_id dst_id ift_src ift_dst sender receiver + local before expected balance hash waited=0 + + src_id=$(_chain_attr "$src" CHAIN_ID); dst_id=$(_chain_attr "$dst" CHAIN_ID) + ift_src=$(_token "$src_id"); ift_dst=$(_token "$dst_id") + [[ -n "$ift_src" && -n "$ift_dst" ]] \ + || die "no IFT address in the manifests — did 'ibc deploy ift' run?" + sender=$(_addr "$src" DEPLOYER); receiver=$(_addr "$dst" DEPLOYER) # Wait on a delta, not an absolute balance: a second run starts with the first - # run's tokens already on chain B, and an absolute check would pass before the + # run's tokens already in place, and an absolute check would pass before the # new packet ever landed. bigint because 1e18 is past what bash arithmetic can # be trusted with once the amounts are overridden. - before=$(_balance "$B_CHAIN_ID" "$ift_b" "$receiver") - [[ -n "$before" ]] || die "could not read the chain B balance of $receiver" + before=$(_balance "$dst_id" "$ift_dst" "$receiver") + [[ "$before" =~ ^[0-9]+$ ]] \ + || die "could not read the chain $dst balance of $receiver, got '${before:-nothing}'" expected=$(perl -Mbigint -e 'print $ARGV[0] + $ARGV[1]' "$before" "$IFT_SEND_AMOUNT") - log "[A] minting $IFT_MINT_AMOUNT $IFT_SYMBOL to $sender..." - _ibc tx ift mint --chain "$A_CHAIN_ID" --ift "$ift_a" --from deployer-a \ - --to "$sender" --amount "$IFT_MINT_AMOUNT" \ - || die "'ibc tx ift mint' failed" - - log "[A] sending $IFT_SEND_AMOUNT $IFT_SYMBOL to $receiver on chain B over $LINK_CLIENT_ID..." - local hash - hash=$(_ibc tx ift send --chain "$A_CHAIN_ID" --ift "$ift_a" --from deployer-a \ + log "[$src] sending $IFT_SEND_AMOUNT $IFT_SYMBOL to $receiver on chain $dst over $LINK_CLIENT_ID..." + hash=$(_ibc tx ift send --chain "$src_id" --ift "$ift_src" --from "deployer-$(_lc "$src")" \ --client-id "$LINK_CLIENT_ID" --to "$receiver" --amount "$IFT_SEND_AMOUNT" 2>/dev/null \ | perl -0777 -ne 'print $1 if /"txHash"\s*:\s*"([^"]+)"/') - [[ -n "$hash" ]] || die "'ibc tx ift send' returned no transaction hash" - log " sent in $hash" + [[ -n "$hash" ]] || die "'ibc tx ift send' on chain $src returned no transaction hash" + log " sent in $hash" # The relay is explicit. `relayer.connections[].autoRelay` exists in the config # schema but nothing in link reads it yet, so a packet sits unrelayed until it - # is handed to the relayer by transaction hash. Run inside ibc-link: the - # command dials the relayer's own gRPC, which only its config describes. - log "handing the packet to the relayer..." - docker compose exec -T ibc-link /opt/ibc relayer relay \ - --tx-hash "$hash" --chain-id "$A_CHAIN_ID" --home /home/ibc >/dev/null \ - || die "'ibc relayer relay' failed" - - log "waiting for chain B's balance to go $before -> $expected..." + # is handed to the relayer by transaction hash. Run inside the relayer + # container: the command dials the relayer's own gRPC, which only its config + # describes. + log " handing the packet to the relayer..." + docker compose --progress quiet exec -T relayer /opt/ibc relayer relay \ + --tx-hash "$hash" --chain-id "$src_id" --home /home/ibc >/dev/null \ + || die "'ibc relayer relay' failed for the $src -> $dst leg" + + log " waiting for chain $dst's balance to go $before -> $expected..." while (( waited < IFT_RELAY_TIMEOUT )); do - balance=$(_balance "$B_CHAIN_ID" "$ift_b" "$receiver") + # A failed query is a retry, not a fatal: the poll outlives one transient + # RPC hiccup and only the timeout below ends the wait. + balance=$(_balance "$dst_id" "$ift_dst" "$receiver") if [[ "$balance" == "$expected" ]]; then - log "relayed: $receiver holds $balance $IFT_SYMBOL on chain B" - log " $sender holds $(_balance "$A_CHAIN_ID" "$ift_a" "$sender") $IFT_SYMBOL on chain A" + log " relayed $src -> $dst: $receiver holds $balance $IFT_SYMBOL on chain $dst" return 0 fi sleep "$IFT_POLL_INTERVAL" (( waited += IFT_POLL_INTERVAL )) done - die "chain B balance is '${balance:-unset}' after ${IFT_RELAY_TIMEOUT}s, expected $expected" \ - "— check 'docker compose logs ibc-link attestor-a attestor-b'" + die "chain $dst balance is '${balance:-unset}' after ${IFT_RELAY_TIMEOUT}s, expected $expected" \ + "— check 'docker compose logs relayer attestor-a attestor-b'" +} + +# Top up chain A's deployer so there is something to send. +_mint_on_a() { + local ift_a sender + ift_a=$(_token "$A_CHAIN_ID"); sender=$(_addr A DEPLOYER) + log "[A] minting $IFT_MINT_AMOUNT $IFT_SYMBOL to $sender..." + _ibc tx ift mint --chain "$A_CHAIN_ID" --ift "$ift_a" --from deployer-a \ + --to "$sender" --amount "$IFT_MINT_AMOUNT" >/dev/null \ + || die "'ibc tx ift mint' failed" +} + +_print_balances() { + local a b + a=$(_balance "$A_CHAIN_ID" "$(_token "$A_CHAIN_ID")" "$(_addr A DEPLOYER)") + b=$(_balance "$B_CHAIN_ID" "$(_token "$B_CHAIN_ID")" "$(_addr B DEPLOYER)") + log "$IFT_SYMBOL held by the deployer: chain A ${a:-?}, chain B ${b:-?}" +} + +relay_ift_transfer() { + _mint_on_a + _leg A B + _print_balances +} + +# Send the same tokens straight back. The return leg needs no mint — it spends +# what the A -> B leg just delivered — so chain B ends where it started and +# chain A ends holding everything it minted. +relay_ift_roundtrip() { + _mint_on_a + _leg A B + _leg B A + _print_balances +} + +stack_ready() { + local key svc health + + for key in A_ICS26_ROUTER B_ICS26_ROUTER A_CLIENT_ID B_CLIENT_ID; do + grep -qE "^$key=.+" "$LINK_ENV_FILE" 2>/dev/null || return 1 + done + [[ -n "$(_token "$A_CHAIN_ID")" && -n "$(_token "$B_CHAIN_ID")" ]] || return 1 + + health=$(docker compose ps --format '{{.Service}}|{{.Health}}' 2>/dev/null) || return 1 + for svc in besu-a besu-b "${LINK_SERVICES[@]}"; do + grep -Fqx "$svc|healthy" <<<"$health" || return 1 + done } start_link() { - # Written by deploy_contracts. Without it the attestors exit on - # `invalid ics26 router address` and ibc-link on `.clientId required`. - grep -qE '^A_ICS26_ROUTER=.+' "$LINK_ENV_FILE" 2>/dev/null \ - || die "$LINK_ENV_FILE holds no deployment — the deploy phase did not complete" + local key + for key in A_ICS26_ROUTER B_ICS26_ROUTER A_CLIENT_ID B_CLIENT_ID; do + grep -qE "^$key=.+" "$LINK_ENV_FILE" 2>/dev/null \ + || die "$LINK_ENV_FILE has no $key — the deploy phase did not complete" + done log "Bringing up ${LINK_SERVICES[*]}..." - docker compose up -d "${LINK_SERVICES[@]}" + docker compose --progress quiet up -d --wait "${LINK_SERVICES[@]}" \ + || die "the link services did not come up healthy" \ + "— check 'docker compose logs ${LINK_SERVICES[*]}'" } diff --git a/examples/besu-to-besu/link/ibc.yml b/examples/besu-to-besu/link/ibc.yml index 7f4b4264e..596f2f348 100644 --- a/examples/besu-to-besu/link/ibc.yml +++ b/examples/besu-to-besu/link/ibc.yml @@ -33,8 +33,6 @@ relayer: dispatchPollInterval: 3s connections: - alias: a-b - # No autoRelay block: the field parses but nothing reads it yet, so - # packets are relayed explicitly with `ibc relayer relay --tx-hash`. clientA: chainId: "41001" signer: relayer-a diff --git a/examples/besu-to-besu/setup.sh b/examples/besu-to-besu/setup.sh index 7f701b7d7..9046f5a06 100755 --- a/examples/besu-to-besu/setup.sh +++ b/examples/besu-to-besu/setup.sh @@ -40,12 +40,15 @@ # 3. deploy ibc deploy core + client on each chain (writing # chains/local/link.env), then an IFT token per chain and the # bridge between them -# 4. link docker compose up kms, both attestors, ibc-link +# 4. link docker compose up kms, both attestors, relayer # 5. transfer mint IFT on chain A, send it to chain B, and wait for the # relayer to deliver it — the end-to-end assertion # # Usage: # ./setup.sh — run all five (the demo) +# ./setup.sh roundtrip — relay A -> B -> A. The return leg is what +# exercises attestor-b. Skips phases 1-4 when the +# stack is already up and deployed. # ./setup.sh clean — stop containers, remove volumes and chains/local/ # # Environment (optional): @@ -134,6 +137,15 @@ source "$LIB_DIR/common.sh" source "$LIB_DIR/chains.sh" source "$LIB_DIR/link.sh" +# The transfer phase feeds these to `(( ))` and to `sleep`. Checked up front so +# a unit suffix — IFT_POLL_INTERVAL=3s — is a message here rather than an +# arithmetic syntax error four phases in. +for var in IFT_RELAY_TIMEOUT IFT_POLL_INTERVAL IFT_MINT_AMOUNT IFT_SEND_AMOUNT; do + [[ "${!var}" =~ ^[0-9]+$ ]] \ + || die "$var must be a whole number with no unit suffix, got '${!var}'" +done +(( IFT_POLL_INTERVAL > 0 )) || die "IFT_POLL_INTERVAL must be greater than 0" + cmd_init() { check_prerequisites log "--- Phase 1A: Derive keys + render chain configs ---" @@ -163,22 +175,47 @@ cmd_transfer() { run_phase "Phase 5: Relay an IFT transfer A -> B" relay_ift_transfer } +cmd_roundtrip() { + run_phase "Phase 5: Relay an IFT round trip A -> B -> A" relay_ift_roundtrip +} + +# Phases 1-4. Every step is idempotent, so this is the same work whether the +# stack is cold or already up. +bring_up() { + log "╔══════════════════════════════════════════════════╗" + log "║ besu-to-besu: 2 Besu QBFT chains (A, B) + Link ║" + log "╚══════════════════════════════════════════════════╝" + cmd_init + cmd_start + cmd_deploy + cmd_link +} + main() { case "${1:-}" in - clean) clean; exit 0 ;; - "") - log "╔══════════════════════════════════════════════════╗" - log "║ besu-to-besu: 2 Besu QBFT chains (A, B) + Link ║" - log "╚══════════════════════════════════════════════════╝" - cmd_init - cmd_start - cmd_deploy - cmd_link - cmd_transfer + clean) clean; exit 0 ;; + "") bring_up; cmd_transfer; exit 0 ;; + roundtrip) + # Phase 5 on its own against a stack that is already up — this is the + # command to reach for when iterating on the relay itself. Phases 1-4 run + # only when there is nothing usable to relay over. + if stack_ready; then + log "Stack is up and deployed — running phase 5 only." + else + bring_up + fi + cmd_roundtrip exit 0 ;; *) - echo "Usage: $0 [clean]" >&2 + cat >&2 < B + roundtrip relay A -> B -> A. Runs phase 5 alone against a stack that is + already up and deployed, and brings one up first if not. + clean stop containers, remove volumes and chains/local/ +EOF exit 1 ;; esac From d21f0eb1c3008fd4111140685b66a9dc77ca10ed Mon Sep 17 00:00:00 2001 From: Dmitry S <11892559+swift1337@users.noreply.github.com> Date: Thu, 20 Aug 2026 13:34:41 +0200 Subject: [PATCH 3/3] feat(make): improve Makefile, deduplicate commands (#1394) * make: move e2e to its own file; dedup * streamline link/Makefile * streamline e2e/Makefile * cleanup Signed-off-by: Diana Borbe --- .github/workflows/e2e.yml | 10 ++- .github/workflows/license-headers.yml | 6 +- Makefile | 94 +++------------------------ e2e/AGENTS.md | 10 +-- e2e/Makefile | 90 +++++++++++++++++++++++++ e2e/README.md | 23 ++++--- e2e/internal/harness/AGENTS.md | 2 +- link/AGENTS.md | 2 +- link/Makefile | 43 +++++++----- link/README.md | 2 +- 10 files changed, 150 insertions(+), 132 deletions(-) create mode 100644 e2e/Makefile diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.yml index 1785b97f5..c69ce8201 100644 --- a/.github/workflows/e2e.yml +++ b/.github/workflows/e2e.yml @@ -65,10 +65,8 @@ jobs: cache-dependency-path: | link/go.sum e2e/go.sum - - run: make build-link - run: docker info - - run: go test -count=1 ./internal/... ./cmd/... - working-directory: e2e + - run: make -C e2e test-harness env: IBC_BIN: ${{ github.workspace }}/link/bin/ibc @@ -87,7 +85,7 @@ jobs: bun-version: 1.3.14 - uses: foundry-rs/foundry-toolchain@908c540300062bd5a7e473851cdb4282204cee09 # v1 - run: go install github.com/ethereum/go-ethereum/cmd/abigen@v1.17.4 - - run: make check-test-apps + - run: make -C e2e check-stale e2e: needs: changes @@ -104,7 +102,7 @@ jobs: link/go.sum e2e/go.sum - run: docker info - - run: make test-e2e + - run: make -C e2e test matrix: runs-on: ubuntu-latest @@ -115,4 +113,4 @@ jobs: go-version-file: e2e/go.mod cache-dependency-path: e2e/go.sum - run: docker info - - run: make check-e2e-matrix + - run: make -C e2e check-matrix diff --git a/.github/workflows/license-headers.yml b/.github/workflows/license-headers.yml index e5a59b43a..767358c0d 100644 --- a/.github/workflows/license-headers.yml +++ b/.github/workflows/license-headers.yml @@ -18,8 +18,4 @@ jobs: timeout-minutes: 5 steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 - - uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7 - with: - go-version-file: link/go.mod - cache-dependency-path: link/go.sum - - run: make check-license-headers + - uses: apache/skywalking-eyes/header@61275cc80d0798a405cb070f7d3a8aaf7cf2c2c1 # v0.8.0 diff --git a/Makefile b/Makefile index 5952102c4..866551b55 100644 --- a/Makefile +++ b/Makefile @@ -3,93 +3,15 @@ help: ## List repository commands @grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' -E2E_FLAGS ?= -count=1 -E2E_MODE ?= fast LICENSE_EYE_VERSION ?= 0.8.0 -E2E_DIR := e2e -HARNESS_DIR := $(E2E_DIR)/internal/harness -CONTRACT_BINDINGS := $(addprefix $(HARNESS_DIR)/environment/solidityibc/,\ - accessmanager escrow testerc20 counter iftsendcallconstructor iftbatchtransfershim) +lint-license: ## Check SPDX license headers + go run github.com/apache/skywalking-eyes/cmd/license-eye@v$(LICENSE_EYE_VERSION) \ + --config .licenserc.yaml header check -build-link: ## Build the Link binary - $(MAKE) -C link build +run-all-checks: ## Run "all-in-one" code validation step. + $(MAKE) -C link run-all-checks + $(MAKE) -C e2e run-all-checks + $(MAKE) lint-license -install-link: ## Install the Link binary - $(MAKE) -C link install - -doctor-e2e: ## Check the runtime dependencies used by e2e tests - @command -v go >/dev/null || { echo "missing go" >&2; exit 1; } - @command -v docker >/dev/null || { echo "missing docker; Docker is required for e2e modes and matrix generation" >&2; exit 1; } - @docker info >/dev/null || { echo "docker daemon is not reachable" >&2; exit 1; } - -doctor-e2e-tools: ## Check the generation and lint tools used by repository e2e checks - @command -v forge >/dev/null || { echo "missing forge; Forge is required to verify test-app artifacts" >&2; exit 1; } - @command -v bun >/dev/null || { echo "missing bun; bun is required to install Solidity contract dependencies" >&2; exit 1; } - @command -v abigen >/dev/null || { echo "missing abigen; abigen is required to generate typed contract bindings" >&2; exit 1; } - @command -v jq >/dev/null || { echo "missing jq; jq is required to generate typed contract bindings" >&2; exit 1; } - @command -v golangci-lint >/dev/null || { echo "missing golangci-lint; it is required for e2e checks" >&2; exit 1; } - -test-harness: build-link ## Run harness tests, including Docker-backed integrations when available - go -C $(E2E_DIR) test ./internal/... ./cmd/... - -test-e2e: build-link ## Run e2e tests (E2E_MODE=... E2E_FLAGS=...) - # -parallel caps concurrent Docker environments; the GOMAXPROCS default can overload a large machine. - E2E_MODE=$(E2E_MODE) go -C $(E2E_DIR) test . -timeout 60m -parallel 4 $(E2E_FLAGS) - -generate-e2e-matrix: ## Regenerate the E2E provider and topology matrix (requires Docker) - go -C $(E2E_DIR) run ./cmd/e2e-matrix -write test-matrix.md - -check-e2e-matrix: ## Check that the E2E provider and topology matrix is current (requires Docker) - go -C $(E2E_DIR) run ./cmd/e2e-matrix -check test-matrix.md - -lint: lint-link lint-e2e ## Lint all Go modules - -lint-fix: lint-fix-link lint-fix-e2e ## Lint all Go modules and fix errors - -lint-link: ## Lint the Link module - $(MAKE) -C link lint - -lint-fix-link: ## Lint the Link module and fix errors - $(MAKE) -C link lint-fix - -lint-e2e: ## Lint the e2e module, harness included - cd $(E2E_DIR) && golangci-lint run - -lint-fix-e2e: ## Lint the e2e module, harness included, and fix errors - cd $(E2E_DIR) && golangci-lint run --fix - -clean-e2e-dry-run: ## Preview e2e processes and Docker resources - $(E2E_DIR)/scripts/clean.sh --dry-run - -clean-e2e: ## Kill e2e processes and remove Docker resources - $(E2E_DIR)/scripts/clean.sh - -test-apps: ## Rebuild test-app artifacts and typed Go bindings (requires bun, forge, abigen, and jq) - bun install --cwd $(HARNESS_DIR)/environment/solidityibc/contracts --frozen-lockfile - forge build --root $(HARNESS_DIR)/environment/solidityibc/contracts - $(E2E_DIR)/scripts/generate-contract-bindings.sh - -check-test-apps: ## Fail if typed Go contract bindings are stale - bun install --cwd $(HARNESS_DIR)/environment/solidityibc/contracts --frozen-lockfile - forge build --force --root $(HARNESS_DIR)/environment/solidityibc/contracts - $(E2E_DIR)/scripts/generate-contract-bindings.sh - @status="$$(git status --porcelain --untracked-files=all -- $(CONTRACT_BINDINGS))"; \ - test -z "$$status" || { \ - echo "contract bindings are stale — run 'make test-apps' and commit the result" >&2; \ - echo "$$status" >&2; \ - exit 1; \ - } - -check-license-headers: ## Check SPDX license headers - go run github.com/apache/skywalking-eyes/cmd/license-eye@v$(LICENSE_EYE_VERSION) --config .licenserc.yaml header check - -check-link: ## Run Link-local checks - $(MAKE) -C link check - -check-e2e: doctor-e2e doctor-e2e-tools test-harness lint-e2e check-test-apps test-e2e check-e2e-matrix ## Run all repository e2e checks - -check: check-license-headers check-link check-e2e ## Run license, Link, and repository e2e checks - -.PHONY: help build-link doctor-e2e doctor-e2e-tools test-harness test-e2e generate-e2e-matrix check-e2e-matrix lint lint-fix lint-link lint-fix-link lint-e2e lint-fix-e2e \ - clean-e2e-dry-run clean-e2e test-apps check-test-apps check-license-headers check-link check-e2e check +.PHONY: help lint-license run-all-checks diff --git a/e2e/AGENTS.md b/e2e/AGENTS.md index db8e56b13..d2d3fed89 100644 --- a/e2e/AGENTS.md +++ b/e2e/AGENTS.md @@ -7,8 +7,8 @@ This module contains one root repository-level acceptance package: linear Go tes Solidity IBC stack (ICS26Router, ICS20Transfer, ICS27GMP) with attestation light clients and managed attestors. -- Run from the repository root: `make test-e2e` uses fast mode; - `make test-e2e E2E_MODE=complete|production` selects another mode, and +- Run from the repository root: `make -C e2e test` uses fast mode; + `make -C e2e test E2E_MODE=complete|production` selects another mode, and `E2E_FLAGS='-run TestTransfer_AutoRelay -count=1'` focuses a run. `-e2e.mode` in `E2E_FLAGS` overrides `E2E_MODE`. - Tests declare portable EVM, controlled-mining, or node-lifecycle requirements. Fast mode may @@ -25,6 +25,6 @@ managed attestors. paused and resumed it is interval-only, so transaction inclusion may take one second. - `Environment` cleans up managed resources only. Attached chains remain caller-owned and expose no harness mining or node-lifecycle controls. -- Run `make generate-e2e-matrix` after changing requirements or topology, and - `make check-e2e-matrix` to check the committed matrix. Both require Docker. -- After a hard crash: `make clean-e2e-dry-run`, then `make clean-e2e` from the repository root. +- Run `make -C e2e generate-matrix` after changing requirements or topology, and + `make -C e2e check-matrix` to check the committed matrix. Both require Docker. +- After a hard crash: `make -C e2e clean-dry-run`, then `make -C e2e clean` from the repository root. diff --git a/e2e/Makefile b/e2e/Makefile new file mode 100644 index 000000000..f31064f50 --- /dev/null +++ b/e2e/Makefile @@ -0,0 +1,90 @@ +# SPDX-License-Identifier: Apache-2.0 + +E2E_FLAGS ?= -count=1 +E2E_MODE ?= fast +FORGE_BUILD_FLAGS ?= + +HARNESS_DIR := internal/harness +CONTRACT_BINDINGS := $(addprefix $(HARNESS_DIR)/environment/solidityibc/,\ + accessmanager escrow testerc20 counter iftsendcallconstructor iftbatchtransfershim) + +help: ## List e2e commands + @grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' + +doctor: ## Check the runtime dependencies used by e2e tests + @command -v go >/dev/null || { echo "missing go" >&2; exit 1; } + @command -v docker >/dev/null || { echo "missing docker; Docker is required for e2e modes and matrix generation" >&2; exit 1; } + @docker info >/dev/null || { echo "docker daemon is not reachable" >&2; exit 1; } + +doctor-tools: ## Check the generation and lint tools used by e2e checks + @command -v forge >/dev/null || { echo "missing forge; Forge is required to verify test-app artifacts" >&2; exit 1; } + @command -v bun >/dev/null || { echo "missing bun; bun is required to install Solidity contract dependencies" >&2; exit 1; } + @command -v abigen >/dev/null || { echo "missing abigen; abigen is required to generate typed contract bindings" >&2; exit 1; } + @command -v jq >/dev/null || { echo "missing jq; jq is required to generate typed contract bindings" >&2; exit 1; } + @command -v golangci-lint >/dev/null || { echo "missing golangci-lint; it is required for e2e checks" >&2; exit 1; } + +lint: ## Lint code + golangci-lint run + +lint-fix: ## Lint & fix code + golangci-lint run --fix + +test-harness: _build-link ## Run tests for e2e harness itself + go test ./internal/... ./cmd/... -count=1 + +test-apps: ## Rebuild test-app artifacts and typed Go bindings (requires bun, forge, abigen, and jq) + bun install --cwd $(HARNESS_DIR)/environment/solidityibc/contracts --frozen-lockfile + forge build $(FORGE_BUILD_FLAGS) --root $(HARNESS_DIR)/environment/solidityibc/contracts + ./scripts/generate-contract-bindings.sh + +test: _build-link ## Run e2e tests (E2E_MODE=... E2E_FLAGS=...) + # -parallel caps concurrent Docker environments; the GOMAXPROCS default can overload a large machine. + E2E_MODE=$(E2E_MODE) go test . -timeout 60m -parallel 4 $(E2E_FLAGS) + +generate-matrix: ## Regenerate the E2E provider and topology matrix + go run ./cmd/e2e-matrix -write test-matrix.md + +check-matrix: ## Check that the generated E2E matrix is current + go run ./cmd/e2e-matrix -check test-matrix.md + +clean: ## Kill e2e processes and remove Docker resources + ./scripts/clean.sh + +clean-dry-run: ## Preview e2e processes and Docker resources + ./scripts/clean.sh --dry-run + +check-stale: ## Run all checks for stale code + @echo "Checking for stale ABI Go bindings..." + $(MAKE) test-apps FORGE_BUILD_FLAGS=--force + @status="$$(git status --porcelain --untracked-files=all -- $(CONTRACT_BINDINGS))"; \ + test -z "$$status" || { \ + echo "contract bindings are stale — run 'make -C e2e test-apps' and commit the result" >&2; \ + echo "$$status" >&2; \ + exit 1; \ + } + +run-all-checks: doctor doctor-tools ## Run "all-in-one" code validation step. + @echo "==== 1. Linting ====" + $(MAKE) lint + + @echo "==== 2. Checking for stale code ====" + $(MAKE) check-stale + + @echo "==== 3. Testing harness ====" + $(MAKE) test-harness + + @echo "==== 4. Running E2E ====" + $(MAKE) test + + @echo "==== 5. Checking matrix ====" + $(MAKE) check-matrix + +# Hidden commands +_build-link: + $(MAKE) -C ../link build + +.PHONY: help doctor doctor-tools lint lint-fix +.PHONY: test-harness test-apps test +.PHONY: generate-matrix check-matrix +.PHONY: clean clean-dry-run check-stale run-all-checks +.PHONY: _build-link diff --git a/e2e/README.md b/e2e/README.md index 17b184e31..33a7264e9 100644 --- a/e2e/README.md +++ b/e2e/README.md @@ -15,12 +15,11 @@ The root package covers ICS20 transfer, ICS27 GMP, IFT (burn/mint on top of GMP) Run targets from the repository root: ```sh -make doctor-e2e -make build-link -make test-e2e +make -C e2e doctor +make -C e2e test ``` -`make build-link` produces `link/bin/ibc`; `IBC_BIN` overrides that path. The real Link Relayer collects attestor signatures and submits recv, ack, and timeout transactions with attestation proofs, which the attestation light clients verify. +The test target builds `link/bin/ibc`; `IBC_BIN` overrides that path. The real Link Relayer collects attestor signatures and submits recv, ack, and timeout transactions with attestation proofs, which the attestation light clients verify. Execution modes choose providers from each test's declared requirements: @@ -36,15 +35,15 @@ provide those harness controls. `complete` runs each test once with the fastest it does not run every provider permutation. ```sh -make test-e2e -make test-e2e E2E_MODE=complete -make test-e2e E2E_MODE=production -make test-e2e E2E_FLAGS='-run TestIFTTransfer_AutoRelay -count=1' -make test-e2e E2E_MODE=production E2E_FLAGS='-run TestCrossRoute -parallel 1 -count=1' +make -C e2e test +make -C e2e test E2E_MODE=complete +make -C e2e test E2E_MODE=production +make -C e2e test E2E_FLAGS='-run TestIFTTransfer_AutoRelay -count=1' +make -C e2e test E2E_MODE=production E2E_FLAGS='-run TestCrossRoute -parallel 1 -count=1' ``` `-e2e.mode` in `E2E_FLAGS` overrides `E2E_MODE`. After a hard crash, use -`make clean-e2e-dry-run` and then `make clean-e2e`. +`make -C e2e clean-dry-run` and then `make -C e2e clean`. Every environment-backed test calls `t.Parallel()` and boots its own environment; the Makefile caps concurrency at four environments. Pass `E2E_FLAGS='-parallel 1 -count=1'` to serialize when debugging. @@ -113,8 +112,8 @@ specs for all three modes. Generation starts the caller-owned Anvil used by the so Docker is required. ```sh -make generate-e2e-matrix -make check-e2e-matrix +make -C e2e generate-matrix +make -C e2e check-matrix ``` Regenerate the matrix after changing test requirements or topology. The check compares generated diff --git a/e2e/internal/harness/AGENTS.md b/e2e/internal/harness/AGENTS.md index 68f5553aa..1d4c90369 100644 --- a/e2e/internal/harness/AGENTS.md +++ b/e2e/internal/harness/AGENTS.md @@ -17,5 +17,5 @@ corroborates outcomes by reading chain state with its own clients. with `--entrypoint anvil` (PID 1, so `docker stop`'s SIGTERM reaches it and shutdown is prompt instead of waiting out the kill grace). Don't reintroduce a shell-wrapped entrypoint. StopNode/StartNode fault injection is docker pause/unpause; chain state stays in memory. -- Lint with `make lint-e2e` from the repository root; the shared root `.golangci.yml` covers the +- Lint with `make -C e2e lint` from the repository root; the shared root `.golangci.yml` covers the harness and excludes exported-doc mandates for this internal test surface. diff --git a/link/AGENTS.md b/link/AGENTS.md index 2150ff8f6..629e96a0c 100644 --- a/link/AGENTS.md +++ b/link/AGENTS.md @@ -7,4 +7,4 @@ - Repository-wide black-box e2e lives in `../e2e`, with its harness in `../e2e/internal/harness` as a separate Go module. The real Link relayer submits packets through ICS26Router with attestation proofs signed by harness-managed attestors. Keep - `make test-e2e` green when changing that transport contract. + `make -C e2e test` green when changing that transport contract. diff --git a/link/Makefile b/link/Makefile index be85e6fe3..1583105a0 100644 --- a/link/Makefile +++ b/link/Makefile @@ -34,6 +34,9 @@ lint: ## Lint the code lint-fix: ## Lint the code and fix errors golangci-lint run --fix +test: ## Run tests + go test ./... -v + codegen: codegen-sql codegen-proto codegen-abi codegen-mocks ## Generate code codegen-sql: ## Generate Go code based on migrations and queries @@ -42,17 +45,6 @@ codegen-sql: ## Generate Go code based on migrations and queries codegen-proto: ## Generate Go proto bindings cd ../proto && $(BUF) generate -check-proto: ## Fail if Link's generated API is stale or the removed API module returns - @command -v buf >/dev/null || { echo "missing buf; buf is required to verify Link's generated API" >&2; exit 1; } - cd ../proto && buf generate - @status="$$(git -C .. status --porcelain --untracked-files=all -- \ - ':(glob)link/api/v2/**/*.pb.go' \ - ':(glob)link/api/v2/**/*.connect.go')"; \ - test -z "$$status" || { echo "Link API bindings are stale — run 'make codegen-proto' and commit the result" >&2; echo "$$status" >&2; exit 1; } - @test ! -e ../api || { echo "the removed standalone API module still exists" >&2; exit 1; } - @old_api='github.com/cosmos/ibc/'api; \ - ! git -C .. grep -n -F "$$old_api" -- . || { echo "the removed API module is still referenced" >&2; exit 1; } - codegen-mocks: ## Generate mocks for the code go run github.com/vektra/mockery/v3@v3.7.1 --log-level ERROR @@ -72,9 +64,30 @@ codegen-abi: ## Generate Go bindings for EVM contract ABIs --type Contract \ --out internal/chains/evm/contracts/attestation/attestation.go -test: ## Run tests - go test ./... -v +check-stale: ## Run all checks for stale code + @echo "Checking for stale proto bindings..." + $(MAKE) codegen-proto + @status="$$(git -C .. status --porcelain --untracked-files=all -- \ + ':(glob)link/api/v2/**/*.pb.go' \ + ':(glob)link/api/v2/**/*.connect.go')"; \ + test -z "$$status" || { echo "Link API bindings are stale — run 'make codegen-proto' and commit the result" >&2; echo "$$status" >&2; exit 1; } + @test ! -e ../api || { echo "the removed standalone API module still exists" >&2; exit 1; } + @old_api='github.com/cosmos/ibc/'api; \ + ! git --no-pager -C .. grep -n -F "$$old_api" -- . || { echo "the removed API module is still referenced" >&2; exit 1; } + +run-all-checks: ## Run "all-in-one" code validation step. + @echo "==== 1. Linting ====" + $(MAKE) lint + + @echo "==== 2. Checking for stale code ====" + $(MAKE) check-stale + + @echo "==== 3. Building ====" + $(MAKE) build -check: build test lint check-proto ## Run Link build, tests, lint, and generation freshness + @echo "==== 4. Testing ====" + $(MAKE) test -.PHONY: help build install docker-build lint lint-fix test codegen codegen-sql codegen-proto codegen-abi codegen-mocks check-proto check +.PHONY: help build install docker-build lint lint-fix test +.PHONY: codegen codegen-sql codegen-proto codegen-abi codegen-mocks +.PHONY: check-stale run-all-checks diff --git a/link/README.md b/link/README.md index e4b251acc..a1eb3785c 100644 --- a/link/README.md +++ b/link/README.md @@ -42,6 +42,6 @@ worked example. Repository-wide black-box tests live in [`../e2e/`](../e2e/README.md), with the harness in `../e2e/internal/harness` as a separate Go module. From the repository root, -`make doctor-e2e && make test-e2e` runs the Link smoke suite. +`make -C e2e doctor && make -C e2e test` runs the Link smoke suite. The accepted target for the harness is documented in [IBC Environment Architecture](HARNESS-ARCHITECTURE-DESIGN.md).