DRAFT feat(forge): stand up a mainnet-aligned ephemeral devnet via the k8s backend

[apenzk]

• • • • • DRAFT + + + + +

⚠️ WARNING — not yet fully working. Forge stands up the network end-to-end (cluster, genesis with Movement's framework, 4 validators live on chain id 250 producing blocks, GetMetadata green), but the post-genesis migration is not yet validated: every published tools/validator image predates the governed_gas_pool module, so the migration script fails with a LINKER_ERROR. Running the migration needs a tools+validator image built from a framework commit that includes GGP (the branch under test) — an image-versioning prerequisite, not a defect in this change. Also note the fixes here were validated by one live hand-run against a devnet cluster, not yet in CI.

Summary

Adds a forge suite and the supporting backend/chart changes to stand up a mainnet-aligned ephemeral devnet on a bare EKS cluster. The goal is a throwaway network a developer can deploy from their own branch, test against for a couple of hours, then tear down — with the network's chain id and config aligned to mainnet so branch code is exercised under mainnet-like conditions.

The forge k8s backend was written for Aptos's internal infrastructure; the bulk of this change is decoupling it from that infra so it runs on a plain Movement cluster.

What works

Validated on a live cluster:

• Cluster provisions, genesis runs with Movement's framework, 4 validators come up healthy and reach consensus, producing blocks.
• Forge's GetMetadata test passes against the running network.

Changes

• Suite (eph_devnet.rs): 4 validators, mainnet chain params (chain id 250, compressed timings, 100K/1B MOVE staking), on-chain discovery off, validator resources sized for m6a.2xlarge. Registered in main.rs + mod.rs.
• forge backend (cluster_helper.rs): install charts directly via helm instead of the Aptos-internal forge-testnet-deployer image; tolerate a missing Chaos Mesh CRD; env-driven S3 bucket for the genesis blob. Prometheus made optional (swarm.rs).
• Charts: image repos → Movement GHCR (validator, tools); drop hardcoded Aptos blob-upload URL; genesis blob via S3 (job uploads, validators download via node role) instead of the 1 MB-capped k8s Secret.
• Migration: add restore-staking-config.move.

Alternatives

forge-testnet-deployer image (tried, rejected)

The first attempt kept forge's default k8s deploy path, which delegates the install to a forge-testnet-deployer container image (via ForgeDeployerManager in install_testnet_resources). It failed against a real Movement cluster, for reasons that make it a dead end rather than a quick fix.

What happened: the deployer pod sat in ImagePullBackOff. The image is hardcoded in k8s_deployer/constants.rs to Aptos's registry:

• repo us-docker.pkg.dev/aptos-registry/docker/forge-testnet-deployer
• tag f5937ed393eb5214997d1cae0da75d2392cdbc70 (a pinned Aptos main-branch build)

Challenges:

1. Can't pull it — our cluster has no access to Aptos's registry.
2. No Movement build of the deployer image — standing one up from scratch would mean:
   • (image) Write the deployer entrypoint (read FORGE_DEPLOY_VALUES_JSON, helm install genesis → wait for the job → helm install aptos-node) — reverse-engineering an undocumented Aptos-internal contract.
   • (image) Write the Dockerfile: base + helm + kubectl + the two charts + the built framework/genesis modules.
   • (image) Build and publish it to GHCR + wire CI.
   • (cluster) Create a forge service account with cluster-admin — a prerequisite for running the deployer, not part of the image.
3. Branch-agnostic by design — the deployer tag is a fixed constant, not derived from the branch. So even a Movement-built deployer would carry a baked framework, not the branch under test.

Why worse than this PR?

Installing the charts directly (this PR) needs no external image, CI, or RBAC, and the framework comes from the branch under test.