Skip to content

feat(relayer): add Terraform/Ansible lifecycle service - #24

Open
anishnar wants to merge 16 commits into
mainfrom
feat/terraform-relayer-service
Open

feat(relayer): add Terraform/Ansible lifecycle service#24
anishnar wants to merge 16 commits into
mainfrom
feat/terraform-relayer-service

Conversation

@anishnar

Copy link
Copy Markdown
Contributor

No description provided.

anishnar and others added 12 commits July 22, 2026 10:49
…oint

The l1.env age probe called BSD "stat -f %m" first. On GNU coreutils -f is
--file-system and takes no format operand, so stat still wrote a filesystem
report to stdout and exited 1. The || fallback appended the real mtime, and the
arithmetic then aborted the script under set -u. Doctor and install died on any
Linux control host. Probe GNU first and require numeric output.

VM.FUNDING.READY used "grep -q fundedFloat.*true" against a single-line JSON
payload, so .* reached any later "true" and an unfunded P-Chain float reported
PASS. Parse the payload with jq instead.

VM.LISTENERS.LOOPBACK could never fail: the remote command ended in printf after
a "|| true", so an empty result read as proof of loopback-only binding even on a
host without iproute2. The remote side now emits a sentinel and the check WARNs
when it is absent.

run_install is the only lifecycle verb that mutated an existing installation with
no recovery point, while every rollback affordance in the role is gated on
"upgrade". Take a backup first when discovery reports retained state.

Fetched backups defaulted inside the git worktree and contain the plaintext
keystore password, with no ignore rule. Add one, and set the restrictive mode
unconditionally so an early exit cannot skip it.

An empty entry at the reapply password prompt now keeps the installed console
credential instead of deleting it.

Also: strip the sha256sum binary-mode marker correctly (the awk pattern matched
zero backslashes, so it stripped nothing), and create the work directory in the
parent shell so the EXIT trap owns it instead of leaking one per run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…boot

The role installed docker.io unguarded. On Ubuntu 24.04 docker.io depends on
containerd (>= 1.2.6) and containerd.io provides containerd unversioned, so apt
cannot satisfy it on a host that already has docker-ce. The task aborted with
"pkgProblemResolver::Resolve generated breaks" and rc=100, which meant the
relayer could not install at all on any rpc[0] carrying icm_relayer, faucet,
graph_node, blockscout or erpc. Guard the task on an existing Docker instead.

The package stays docker.io deliberately: switching to docker-ce resolves
cleanly on a docker.io host and removes it, and docker.io prerm stops dockerd,
which stops every running Safe container. This mirrors the guard the safe role
already uses.

relayerd tolerated only about 25s of crash-looping (StartLimitBurst=5 with
RestartSec=5s), and relayer-console declared Requires=relayerd.service. After a
reboot relayerd exhausted its starts while avalanchego bootstrapped, systemd
parked it, and the propagated stop took the console down without its own
Restart=on-failure ever firing. Both stayed dead. Drop the start limit, use
Restart=always to match the avalanchego unit, downgrade the console to Wants=,
and reset-failed before the drain so a reapply can recover a parked unit.

The readiness curl had no deadline and systemctl is-failed is false for a
hung-but-running daemon, so make relayer blocked forever. Bound it.

An empty console password on reapply now preserves the installed credential in
both the secrets file and console.env, rather than removing authentication.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ecrets

The pre-restore rollback archive was created by a bare tar under the root umask,
so it landed 0644 while containing the plaintext keystore password, the console
session secret and console.env. The sibling backup path already chmods its
archive to 0600. Do the same here, and create the workspace 0700 rather than
relying on the implicit 0755.

The staged extraction left a plaintext keystore password, the keystore and the
TLS key under the restore workspace forever, one set per restore. Remove the
workspace when the play finishes.

The structural restore problems are deliberately not addressed here. Restore
still aborts on the pre-restore tar when a state directory is missing, and its
rescue still deletes both directories before extracting the rollback. Those need
a live host to verify and are tracked for their own change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ed EOA

ownerType was set to "safe" only when eth_getCode returned bytecode. Every other
outcome fell through to "eoa", including a 200 response carrying a JSON-RPC
error (result undefined, so the default kicked in) and a renounced zero-address
owner, which satisfies the 64-hex shape check. Doctor then printed PASS
"PoAManager owner is a supported EOA" for an L1 whose validator manager has no
authority able to approve a change. Require a non-zero owner and assert the
call succeeded before classifying.

The peer-visibility assert templated fail_msg before evaluating its conditions,
so a validator with avalanchego stopped raised AnsibleUndefinedVariable instead
of the remediation text. Guard on "is defined" first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Twenty of the twenty-one doctor fixture cases fed a level|id|summary|remediation
line into RELAYER_DOCTOR_FIXTURE, which short-circuits the whole doctor and
echoes it straight back. They asserted nothing about real check logic while
their names implied coverage. Keep a couple for the formatter and exit-code
contract and make the rest exercise the extracted doctor functions, including
both STATE.INTEGRITY failure branches.

Add assertions pinning the invariants this review established, so none of them
can regress silently: the Docker guard and its package name, the SSH host-key
default with its override, the reapply backup guard and its ordering, the
non-degenerate WalletConnect placeholder, the checksum marker strip, the jq
funding check, the listener sentinel, and the two systemd settings.

Two are behavioural rather than textual. ssh_target is called for real to prove
its default and override, and the reapply guard is fed a partial and a missing
discovery file to prove it fails closed on a fresh host.

Also guard against a relayer port colliding with a port another role declares.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nt tree

Three defects the register listed as OPEN are implemented in this branch, three
entries were missing from the summary table, and one entry used a status that
the legend never defined. Record the defects this review found, so the register
stays the authoritative backlog it is published as.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… l1.env

NOT RELAYER. This belongs in its own PR; it is committed last so it can be split
out. It gates the relayer, though: the relayer requires POA_MANAGER and
VALIDATOR_MANAGER_PROXY in l1.env and refuses to run without them, and only this
tool creates those contracts.

The Glacier retry budget covered transport errors, 429 and 5xx only. Every other
4xx returned on the first attempt, including the 404 Glacier serves for a
P-Chain transaction it has not indexed yet, so running the tool a minute after
create-l1 -- the normal flow -- aborted immediately. Make 404 and a 2xx without
signedMessage retryable and keep 400/401/403 terminal.

make initialize-validator-manager never forwarded NETWORK, so a mainnet run
queried the fuji Glacier endpoint and got a 404 with no way to override.
NETWORK already defaults to fuji, so this is a no-op for fuji users.

--max-churn-percent was bounded at 100 while the contract rejects 0 and anything
above 20, and churnPeriod was unbounded. Because settings are initialized after
the proxy is upgraded, the revert landed with the proxy already upgraded and
uninitialized. Bound both before anything deploys.

A mid-run failure discarded the partially populated output, so the addresses the
new resume flags need were lost. Return the partial output and add the library
address to it. This is diagnostics: making the resume path real also needs a
playbook change, which is not included here.

The P-Chain GetTx call ran with no timeout on either the client or the context,
so a node that accepts TCP but never answers hung the tool with no output.

cast send discarded its JSON decode error and returned an empty hash with no
error, so a foundry warning on the stream made the playbook report a
transaction as skipped when it had been sent. Parse tolerantly now, the same way
forge create output is already handled. The error stays non-fatal on purpose:
cast has already exited 0 at that point, so the transaction is broadcast, and
failing there would abort the run after the proxy upgrade, initialize or
transferOwnership had already changed on-chain state.

Both l1.env persistence tasks used lineinfile with create:false, which hard-fails
on a missing destination -- after the irreversible on-chain initialization. An
L1 created with avalanche-cli reported a failed play for a fully successful run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
NOT RELAYER. This belongs in its own PR; it is committed last so it can be split
out.

The placeholder project ID was 32 ASCII zeros. Both the post-build verification
and the image-freshness check grep the compiled bundle for that literal, and any
run of 32 or more zeros satisfies it -- a zero address is 40, a bytes32 zero is
64, and Safe bundles contain both. So the build check passed on a UI with no
wallet connector compiled in, and a stale image could be judged fresh. A
non-degenerate value makes both greps meaningful without touching their logic.

Note for whoever lands this: every existing Safe host rebuilds its UI once,
because the freshness check greps for the old literal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Comment thread docs/audits/2026-07-terraform-ansible-live-test-bugs.md Outdated
Comment thread scripts/l1/relayer.sh
Comment thread tools/initialize-validator-manager/glacier.go Outdated
Comment thread tools/initialize-validator-manager/glacier_test.go Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants