This is the result of the analysis of failing o2-exports tests done by Claude Code Fable 5. When running cargo test -all on tests they were randomly failing with the message:
fuel-core process exited before reporting its bound address
The root cause of the issue id fuelup proxy losing a hardlink race while launching fuel-core in parallel (due to parallel test execution).
What I did
I wrapped fuel-core in a script that tees the child's stdout/stderr to files (the fuels-rs error path kills the child without dumping its logs, so the real error was being swallowed), then reproduced the flake. The failed process exited with status 0, wrote nothing to stderr, and printed this to stdout:
Could not create link: /home/kebradalaonda/.fuelup/store/fuel-core-0.48.0/fuel-core-keygen->/home/kebradalaonda/.fuelup/toolchains/testnet-x86_64-unknown-linux-gnu/bin/fuel-core-keygen
Root cause chain
- ~/.fuelup/bin/fuel-core is not fuel-core — it's a hardlink to the fuelup binary acting as a proxy.
- Your repo has a fuel-toolchain.toml pinning fuel-core = "0.48.0" on the testnet channel. Because of that version pin, fuelup's proxy_cli.rs calls Toolchain::add_component() on every single invocation.
- add_component (fuelup src/toolchain.rs), even when the component is already fully installed, unconditionally re-creates the store→toolchain hardlinks. And hardlink() in src/file.rs is remove_file(link) then hard_link(...) — a textbook TOCTOU race.
- Parallel tests each spawn "fuel-core" → multiple proxies relink concurrently → the loser's hard_link hits EEXIST (the symlink fallback fails the same way), the proxy prints the error to stdout and exits 0 without exec'ing fuel-core. fuels-rs sees stderr EOF and reports "fuel-core process exited before reporting its bound address".
Fix
FuelLabs/fuelup add_component should skip relinking when the link already points at the right inode (or take a file lock); the proxy also shouldn't report this failure on stdout with exit code 0. Note the race can even momentarily delete the toolchain's fuel-core binary itself, so other flake modes are possible.
Additional finding
Concurrent proxies sometimes conclude the component is missing entirely and start re-downloading fuel-core from GitHub mid-test-run, then collide in the store directory. That explains the store re-extraction timestamp we saw earlier, and it's worth including in the fuelup issue: the proxy both relinks unconditionally and can trigger redundant re-installs under concurrency.
This is the result of the analysis of failing o2-exports tests done by Claude Code Fable 5. When running
cargo test -allon tests they were randomly failing with the message:The root cause of the issue id fuelup proxy losing a hardlink race while launching fuel-core in parallel (due to parallel test execution).
What I did
I wrapped fuel-core in a script that tees the child's stdout/stderr to files (the fuels-rs error path kills the child without dumping its logs, so the real error was being swallowed), then reproduced the flake. The failed process exited with status 0, wrote nothing to stderr, and printed this to stdout:
Could not create link: /home/kebradalaonda/.fuelup/store/fuel-core-0.48.0/fuel-core-keygen->/home/kebradalaonda/.fuelup/toolchains/testnet-x86_64-unknown-linux-gnu/bin/fuel-core-keygen
Root cause chain
Fix
FuelLabs/fuelup add_component should skip relinking when the link already points at the right inode (or take a file lock); the proxy also shouldn't report this failure on stdout with exit code 0. Note the race can even momentarily delete the toolchain's fuel-core binary itself, so other flake modes are possible.
Additional finding
Concurrent proxies sometimes conclude the component is missing entirely and start re-downloading fuel-core from GitHub mid-test-run, then collide in the store directory. That explains the store re-extraction timestamp we saw earlier, and it's worth including in the fuelup issue: the proxy both relinks unconditionally and can trigger redundant re-installs under concurrency.