Nightly hard fork test (mixed mode): post-fork seed cannot find its genesis ledger tarball

[glyh]

Summary

The nightly hard fork test - mixed mode (./scripts/hardfork/build-and-test.sh --fork-from origin/compatible ...) fails after the fork: the post-fork seed node crashes at boot because it cannot find the genesis ledger tarball it was just given.

[seed] [Error] Could not find or generate a "genesis_ledger" for $root_hash
[seed]   root_hash: "jxQcUzVTMA3FV5tf9D1Zj92RYhmQhVdactbFjVmRkTrWkTtVaC7"
[seed] [Fatal] Unhandled top-level exception:
  "Could not find a ledger tar file for hash 'jxQcUzVTMA3FV5tf9D1Zj92RYhmQhVdactbFjVmRkTrWkTtVaC7'"
  Called from Mina_cli_entrypoint.load_config_files.(fun) in file "src/app/cli/src/cli_entrypoint/mina_cli_entrypoint.ml", line 145

The harness then reports No common genesis for fork network: Failed to query genesis block on node seed.

What happened before the crash

The pre-fork network was healthy (77 blocks, snarked ledger advancing), the fork config validated on every node, and all nodes computed the same fork chain_id. The seed had been assigned fork method advanced (the harness picks methods per node at random), the other nodes auto. Only the seed's boot from the fork config failed.

Likely cause

This matches the root cause of #18979: the fork does not bump the transaction protocol version, so the genesis tarball is written under the next version's name and looked up under the current one.

compatible merged release/mesa on Sep 10 (#19417 / 8d69e1c), which moved protocol_version_transaction from 3 to 4. develop is also 4:

origin/compatible   protocol_version_transaction = 4
8cedbaf17c~5        protocol_version_transaction = 3   (compatible before the merge)
origin/develop      protocol_version_transaction = 4

So --fork-from origin/compatible is now a 4 -> 4 fork, the same shape that crashed in #18979. The failure depends on which fork method the harness samples, which fits it looking intermittent.

Evidence

• Failing: https://buildkite.com/o-1-labs-2/mina-end-to-end-nightlies/builds/4375 (seed method advanced). The branch under test only changes pickles; its circuits and verification keys are identical to develop.
• https://buildkite.com/o-1-labs-2/mina-end-to-end-nightlies/builds/4369 (Sep 9, before the merge) also sampled advanced for the seed and did not hit this error (it failed on a libp2p_helper SIGKILL instead).

Possible fixes

• Bump protocol_version_transaction on develop to 5 so a compatible -> develop fork is a real version step again, or
• change --fork-from to a ref that is still on version 3, or
• fix the tarball naming so a fork without a version bump names and looks up the same file (the general fix Automode mesa->mesa hardfork: post-fork node cannot find its own genesis ledger (missing protocol_version_transaction bump) #18979 asked for).