Repository navigation
Integrate Hyperlane compiler fixes and SDK capabilities - #25
Merged
Mister-Meeseeks merged 44 commits intoOct 19, 2026
Merged
Conversation
gosvm/sdk/system gains TransferSigned (lamports out of a data-less System-owned PDA, the native-vault/ATA-payer shape), Allocate/AllocateSigned, Assign/AssignSigned and CreatePDA, which also creates a PDA whose address already holds lamports (top-up, allocate, assign), matching the create_pda_account helper Solana programs such as Hyperlane's warp routes use. Create/CreateSigned behavior is unchanged. examples/system-pda runs them against the real System program on fogo-v0 and solana-v3 through a generated, staleness-checked testdata/svm.json (TestSDKSystemPDASBF), alongside native encoding and guard tests. The shared SDK probe harness pins the SPL images bundled with LiteSVM 0.8.2 for later token probes. Example SDK snapshots are refreshed.
gosvm/sdk/token adds Program2022ID, IsProgram (classic Token or Token-2022) and builders selected by the token program's index: InitializeMint2, InitializeMetadataPointer (Token-2022), InitializeAccount3, InitializeAccount (owner and Rent sysvar accounts), GetAccountDataSize, and MintToChecked, BurnChecked and TransferChecked with PDA-signed forms. They check program, owners, writability and authority signature or seeds; Token's own errors propagate. The classic checked references are unchanged. examples/token-cpi runs them against the real Token-2022 8.0.0 and Token 3.5.0 images bundled with LiteSVM 0.8.2 on fogo-v0 and solana-v3 (TestSDKTokenCPISBF): a Hyperlane-style 234-byte metadata-pointer PDA mint that is its own mint authority, a self-owned escrow, exact account bytes, GetAccountDataSize return data and propagated Token errors. Native tests cover encodings and guards. Example SDK snapshots are refreshed.
GetReturnData(c, output, program) calls the runtime's sol_get_return_data syscall (by name on fogo-v0, by its static hash 0x5d2245e4 on solana-v3). It returns the full return-data size, copies min(len(output), size) bytes and, when that is nonzero, writes the setter's key into program, exactly as the syscall does. A program buffer other than 32 bytes traps. Native contexts supply a ReadReturnData callback; a nil callback reads as no return data. The C adapter lives in return_data.c.txt and is emitted only for programs that call GetReturnData (usesSDK), so every existing program keeps byte-identical C and ELFs; the pre-Check corpus still matches. sdkOutputs now lists each SDK function's output argument indices, so the evaluation-order analysis sees both buffers GetReturnData writes. SHA256's entry now names its output slice instead of its trailing offset. Tests: native prefix/setter/trap checks, a native-Go/generated-C differential over size and capacity, and a real-SBF suite on both profiles that reads data set by self-CPIs (0 to 1,024 bytes into 0 to 1,024-byte buffers), by the program itself, after a silent callee and in a later top-level instruction (which the runtime clears), plus the trap with rollback. The committed SDK-2 example snapshots are refreshed for the new API file.
Secp256k1Recover(c, hash, recoveryID, signature, output) calls the runtime's sol_secp256k1_recover syscall (by name on fogo-v0, by its static hash 0x17e40350 on solana-v3) and writes the 64-byte uncompressed public key. Wrong hash, signature or output lengths return the new status 2018 without a syscall; runtime statuses (2 invalid recovery ID, 3 invalid signature) propagate unchanged and leave the output untouched. Native contexts supply a RecoverSecp256k1 callback. The C adapter lives in secp256k1.c.txt and is emitted only for programs that call Secp256k1Recover (usesSDK), so every existing program keeps byte-identical C and ELFs; the pre-Check corpus still matches. Tests: native bounds/status/copy checks, a native-Go/generated-C boundary differential, and a real-SBF suite on both profiles that compares recovered keys and statuses with an independent math/big oracle (valid signatures, the other parity, high-s, recovery IDs above 3 and 255, zero and out-of-range scalars, and a wrong-length call). The committed SDK-2 example snapshots are refreshed for the new API file.
Keccak256(c, input, offset, length, output, outOffset) mirrors SHA256's bounded single-region contract and calls the runtime's sol_keccak256 syscall (by name on fogo-v0, by its static hash 0xd7793abb on solana-v3): Ethereum's Keccak-256 with the original 0x01 padding. Out-of-range requests return 2003 without a syscall; native contexts supply a Keccak callback, and a missing callback returns 2009. The C adapter lives in keccak.c.txt and is emitted only for programs that call Keccak256 (usesSDK), so every existing program keeps byte-identical C and ELFs; the pre-Check corpus still matches. Tests: native bounds/copy checks, a test-only reference Keccak-256 checked against published vectors, a native-Go/generated-C boundary differential, and a real-SBF suite on both profiles that compares whole and partial inputs from 0 to 10,240 bytes, across the 136-byte rate boundary, with the reference. The committed SDK-2 example snapshots are refreshed for the new API file.
memmoveHelper prepends only a u8 typedef to libraryMemory, whose overlap test casts pointers to u64, so linking the helper always failed with "use of undeclared identifier 'u64'". Every library-free program for which LLVM emits a memmove call, for example a large struct assigned through two pointers that may alias, could not be built. Tests: the helper compiles on its own under -Werror, and a probe whose pointer-to-pointer copies of a 545-byte value (including a self-assignment) lower to memmove builds and produces the expected account bytes on fogo-v0 and solana-v3. Both fail before this change.
gosvm/sdk/token adds ReadMint and ReadAccount for mints and token accounts owned by classic Token or Token-2022, accepting Token-2022's extended layout (zero padding, account type at byte 165, bounded TLV entries; never the Multisig length) such as a 234-byte metadata-pointer mint or a 170-byte ImmutableOwner ATA. FindExtension locates an extension, ReadTransferFeeConfig decodes TransferFeeConfig and Fee reproduces Token-2022's epoch-selected, capped ceil(amount * bp / 10000) fee without 128-bit integers. The classic checked references are unchanged; Program2022ID/IsProgram match the token instruction branch byte for byte. examples/token-extensions has real Token-2022 write a transfer-fee mint and account whose exact bytes equal the preloaded layouts, decodes Hyperlane's mint/ATA layouts, and compares Fee with what Token-2022 withholds at epochs 4 and 5 (TestSDKTokenExtensionsSBF, fogo-v0 and solana-v3). Native tests cover layout edge cases and 200,090 fee vectors against big.Int.
gosvm/sdk/ata derives the canonical associated token account (Address, Matches) and invokes the real ATA program's Create and CreateIdempotent for classic Token or Token-2022 mints, with a transaction-signer payer or, in the Signed forms, a data-less System-owned PDA payer under the caller, as Hyperlane's warp-route transfer_out does. It checks the ATA, System and token programs, the mint owner, the derived address and the payer; ATA errors propagate. pda.Seeds.Find adds the canonical bump search (255 down, one CreateAddress per candidate, rewriting only the bump byte). examples/ata-create runs against ATA 1.1.1, Token-2022 8.0.0 and Token 3.5.0 on fogo-v0 and solana-v3 (TestSDKATACreateSBF): a 170-byte ImmutableOwner account for a metadata-pointer mint paid by the PDA, idempotent repeat, IllegalOwner for a repeated Create, a classic account, on-chain derivation equal to the host's, and SDK rejections. Program2022ID/IsProgram match the other token branches byte for byte. Example SDK snapshots are refreshed.
FindAddress appends each bump from 255 down to 1 (Solana never tries 0) and returns the first off-curve address with its bump, calling the existing create-program-address boundary once per attempt and skipping on-curve results (status 1). Other statuses propagate; no bump is the new 3004, and a builder with 16 seeds (no room for the bump) is 3001. The builder is left unchanged. It matches 401 vectors from Rust solana-pubkey 3.0.0's try_find_program_address (oracle source in the results directory). The host fixture.FindProgramAddress also tried bump 0; it now stops at 1, with no committed fixture changed. SDK Go only: the full-swap ELFs are unchanged. examples/find-address passes gosvm test --svm on fogo-v0 and solana-v3, including a canonical bump of 252 and a refused noncanonical bump.
Program Go code can now log as Rust programs do with msg!, sol_log_64 and
Pubkey::log: Log writes "Program log: <message>" through sol_log_, Log64
five words in Rust's {:#x} form through sol_log_64_, and LogPubkey a 32-byte
key in base58 through sol_log_pubkey (2019 for another length, without
logging). They return 0. Text that is not valid UTF-8 fails the instruction
in the runtime's syscall, as for Rust; the native SDK panics, and native
contexts receive lines through Context.WriteLog.
sol_log_64_ and sol_log_pubkey join the admitted fogo-v0 imports; solana-v3
uses their static hashes. The logging C runtime is emitted only for programs
that call these, so corpus C and the full-swap ELFs are unchanged. The SDK-2
example snapshots and sdk_sha256 are refreshed. examples/logging passes
gosvm test --svm on fogo-v0 and solana-v3; native Go and generated C agree on
610 vectors.
memmoveHelper prepends only a u8 typedef to libraryMemory, whose overlap test casts pointers to u64, so linking the helper always failed with "use of undeclared identifier 'u64'". Every library-free program for which LLVM emits a memmove call, for example a large struct assigned through two pointers that may alias, could not be built. Tests: the helper compiles on its own under -Werror, and a probe whose pointer-to-pointer copies of a 545-byte value (including a self-assignment) lower to memmove builds and produces the expected account bytes on fogo-v0 and solana-v3. Both fail before this change.
Hyperlane's Sealevel cross-collateral transfer_remote takes 33 accounts, one more than the SDK-2 loader's 32. Low-level builds can now pass -max-accounts 64 (BuildOptions/Program.MaxAccounts = WideAccounts): the entrypoint keeps a 64-entry AccountInfo table (3,584 bytes) and calls Process through a noinline trampoline, because Process inlined beside that table overflows the 4 KiB SBF frame (tutorial 4,224, escrow 4,672 bytes). More than 64 still returns 1001. SDK 1 rejects the option. It is opt-in because the extra call costs +1..+11 CU per transaction on the tutorial, escrow and full swap and would change every SDK-2 ELF; without it every program's C and ELF are unchanged. The SVM fixture harness now accepts 255 account references per instruction (was 32). examples/accounts64 runs on fogo-v0 and solana-v3 with and without the option: 33 unique accounts in one legacy packet, 64 descriptors with aliases beyond index 32, 65 rejected, a System CPI through indices 33..63 and rollback; without it 33 returns 1001. The serialized loader passes 272 UBSan cases at each limit.
… search pda.Seeds.Find duplicated pda.Seeds.FindAddress from the find-address branch (gap4-find-pda) and differed from Solana by also trying bump 0. sdk/pda now carries FindAddress (bumps 255 down to 1, 3004 when none is off-curve) byte for byte as that branch adds it, with its vector test, so the two branches merge without conflict. ata.Address and the ATA probe call FindAddress; derived addresses, bumps and the probe fixtures are unchanged. Example SDK snapshots are refreshed.
solana.Invoke, the SDK-2 C runtime and sdk/cpi allowed 16 account metas and 2 signer groups per CPI; Hyperlane's Sealevel programs need 22 (IGP PayForGas with quoted accounts) and 17 (fee QuoteFee). Seeds keep 16 per group, 32 bytes each and 1,024 encoded bytes. sol_Invoke gets 64-entry meta and 4x16 seed descriptor arrays and is noinline, so its 2,240-byte frame never lands in a caller's frame. cpi.Metas keeps 16 records inline and moves them to one 576-byte heap array at the 17th Add, so small CPIs keep their frame and zeroing cost. examples/cpi-limits and TestCPILimitsSBF run 3 to 64 metas with up to four PDA signers on fogo-v0 and solana-v3 against a C callee that checks its account count, signers and every repeated-account position, plus the 65-meta and 5-group errors through the builders and the raw boundary. The escrow, full-swap and tutorial SDK snapshots are refreshed; the check corpus records the 33 C hashes the runtime change moved.
results/compiler/2026-10-19-cpi-limits holds the before/after CU of the CPI-limits harness and the existing escrow, full-swap and tutorial suites, static frame sizes, and the fresh-rebuild metrics. The full swap's ELFs changed (fogo-v0 bcd9253e, solana-v3 86f0ed15), so verify_rebuild.py now defaults to this reference. HANDOFF records the SDK snapshot change.
goSVM moved bytes with C loops: gosvm_move for copy, append and string
building, and the hidden memcpy/memset/memmove LLVM calls for aggregate
assignment and zeroing. On SBF the byte loop costs about 8 CU per byte and
the word loop 6 CU per 8 bytes; the memory syscalls cost max(10, n/250).
On fogo-v0 and solana-v3, SBF builds now define GOSVM_MEMORY_SYSCALLS and
those routines hand every move of at least one byte to sol_memmove_ or
sol_memset_ (measured: the syscall wins from 1 byte at every alignment).
The entrypoint's duplicate-descriptor copy and ResizeAccount's zero fill do
the same, and two zeroing loops (for i := range s { s[i] = 0 } and
for j := e; j < uint64(len(s)); j++ { s[j] = 0 }) are emitted as a memset.
sol_memcpy_ is never used: the runtime rejects overlapping ranges, including
src == dst. Zero-length moves never reach the VM. Host C and target-less
legacy builds compile the original loops in #else branches, so the legacy
ELFs are unchanged.
Tests: examples/memory-values against native Go as host C (13,660 vectors,
SDK 1 and 2 routines) and on both SBF profiles (5,480 vectors) built with
threshold 1, threshold 64 and loops only; runtime, read-only write and
legacy-identity SBF tests; the corpus check now proves the emitted C is the
recorded C plus only the syscall blocks.
GO_ALLOCATION gains the bulk-memory and zeroing-loop sections, BACKEND_INTERFACE says why a backend must not use sol_memcpy_ for copy, ERROR_CONTRACT records the change as CU/ELF only, and HANDOFF and TARGET_PROFILES point at the new reference ELFs, which verify_rebuild.py now uses by default. results/compiler/2026-10-19-memory-syscalls holds the threshold curves, before/after CU for the Hyperlane Go mailbox (Init 25,182 -> 13,372 CU on fogo-v0, all 64 steps -33.5%), the goSVM examples (-1.4% to -13%, no step worse) and the port patterns (1 KiB tail zeroing 5,232 -> 124 CU), plus the fresh-rebuild check.
SDK-2 programs can now debit an owned writable account and credit any writable one, the `**a.lamports -= x; **b.lamports += x` of Hyperlane's Sealevel fee claims. The runtime's rules are checked before either balance changes: owned, writable, non-executable source (2005/2011/2012), writable non-executable destination (2005/2011), source balance (new 2018) and destination overflow (2014). Moving within one account is a no-op. Rent exemption is left to the application; the runtime rejects a debit below it. The C runtime is emitted only for programs that call it, so corpus C and the full-swap ELFs are unchanged. The SDK-2 example snapshots and sdk_sha256 are refreshed. examples/fee-vault passes gosvm test --svm on fogo-v0 and solana-v3; native Go and generated C agree on 1,580 vectors under UBSan.
Secp256k1Recover already returns 2018 for a wrong hash, signature or output length, and LogPubkey takes 2019, so a MoveLamports source balance below the amount now returns 2020. Nothing else changes. The SDK-2 example snapshots, sdk_sha256 and the fee-vault fixtures are refreshed.
The memset branch of a rewritten zeroing loop took temporary numbers, so the #else branch (host and legacy builds) named its temporaries one higher than earlier compilers did. The branch now reuses the numbers inside its own block. examples/memory-values' C minus the syscall blocks is now the previous compiler's C, and its legacy ELF and the full swap's are unchanged; SBF ELFs are unchanged by this commit.
(*[N]T)(s) passed Check and then failed the built-in backend's C compile (#7): the pointer-conversion rule never looked at the operand, so the emitter cast the C slice struct to a pointer. [N]T(s) was rejected. Both now follow Go. len(s) < N aborts with the bounds trap; the pointer is the slice's data pointer, so writes through either view see each other, and a nil slice gives nil (only N == 0 reaches it); the array is a copy of the first N elements, and [0]T(s) reads nothing. Both are plain operations in gc's evaluation order: calls in the same statement run first, so their panics win and a slice they replace is the one converted. Initializers that convert run at run time instead of at compile time, where the interpreter returned the slice as the pointer. gosvm features reports slice_array_conversions. Host differentials against native Go (output bytes, writes before a failure, panic lines) and SBF runs on fogo-v0 and solana-v3. The corpus's one recorded slice conversion now compiles; every example ELF and the reference ELFs are unchanged.
Pair[uint32, [][32]byte] failed Check with "array type was not declared before use" (#8). Type arguments are validated while instantiations are discovered, before any array type is declared, and typeArgRule only bounded a top-level array (or a pointer to one) without looking it up; arrays inside slices and pointers went to the lookup. Separately, arrays reached only through a field of an instantiated generic struct (V [][3]T, P *[4]T, Rows []Row[T]) were never declared. typeArgRule now descends through pointers and slices, keeping their rejections, and Check and the emitter declare the arrays a struct field holds through pointer, slice, map and array types before using it. Programs whose arrays are all written in source emit the same C: the frozen corpus, every example ELF and the reference ELFs are unchanged. Host differentials against native Go and SBF runs on fogo-v0 and solana-v3.
…te/hyperlane-contributions-20261019
…te/hyperlane-contributions-20261019 # Conflicts: # docs/ERROR_CONTRACT.md
…te/hyperlane-contributions-20261019 # Conflicts: # HANDOFF.md # docs/ERROR_CONTRACT.md
…te/hyperlane-contributions-20261019 # Conflicts: # README.md # docs/SDK2.md # examples/escrow/.gosvm/sdk/solana/api.go # examples/escrow/gosvm.json # examples/full-swap/.gosvm/sdk/solana/api.go # examples/full-swap/gosvm.json # examples/tutorial/.gosvm/sdk/solana/api.go # examples/tutorial/gosvm.json # internal/compiler/builtins.go # internal/compiler/byval.go # internal/compiler/compiler.go # internal/compiler/solana.go # solana/api.go
…te/hyperlane-contributions-20261019 # Conflicts: # README.md # docs/SDK2.md # examples/escrow/.gosvm/sdk/solana/api.go # examples/escrow/gosvm.json # examples/full-swap/.gosvm/sdk/solana/api.go # examples/full-swap/gosvm.json # examples/tutorial/.gosvm/sdk/solana/api.go # examples/tutorial/gosvm.json # internal/compiler/byval.go # internal/compiler/compiler.go # internal/compiler/solana.go # solana/api.go
…te/hyperlane-contributions-20261019 # Conflicts: # HANDOFF.md # docs/ERROR_CONTRACT.md # examples/escrow/gosvm.json # examples/full-swap/gosvm.json # examples/tutorial/gosvm.json # internal/compiler/solana.go
…te/hyperlane-contributions-20261019 # Conflicts: # .github/workflows/ci.yml # HANDOFF.md # docs/ERROR_CONTRACT.md # docs/SDK2.md # examples/escrow/.gosvm/sdk/solana/api.go # examples/escrow/gosvm.json # examples/full-swap/.gosvm/sdk/solana/api.go # examples/full-swap/gosvm.json # examples/tutorial/.gosvm/sdk/solana/api.go # examples/tutorial/gosvm.json # internal/compiler/builtins.go # internal/compiler/compiler.go # internal/compiler/solana.go # solana/api.go
…te/hyperlane-contributions-20261019 # Conflicts: # .github/workflows/ci.yml # HANDOFF.md # docs/ERROR_CONTRACT.md
…te/hyperlane-contributions-20261019
…te/hyperlane-contributions-20261019
…te/hyperlane-contributions-20261019 # Conflicts: # internal/project/project.go # sdk/bundle.go
…te/hyperlane-contributions-20261019 # Conflicts: # HANDOFF.md # docs/ERROR_CONTRACT.md # internal/compiler/solana.go
…te/hyperlane-contributions-20261019 # Conflicts: # HANDOFF.md # docs/ERROR_CONTRACT.md # examples/escrow/gosvm.json # examples/full-swap/gosvm.json # examples/tutorial/gosvm.json # internal/compiler/check_corpus_test.go
…te/hyperlane-contributions-20261019 # Conflicts: # HANDOFF.md # docs/BACKEND_INTERFACE.md # docs/ERROR_CONTRACT.md # internal/compiler/compiler.go # internal/compiler/library.go # internal/compiler/memmove_sbf_test.go # scripts/verify_rebuild.py
…te/hyperlane-contributions-20261019 # Conflicts: # internal/project/project.go # sdk/bundle.go
Mister-Meeseeks
marked this pull request as ready for review
October 19, 2026 15:50
This was referenced Oct 19, 2026
Draft
paulbalaji
added a commit
to hyperlane-xyz/hyperlane-monorepo
that referenced
this pull request
Oct 19, 2026
hyperlane-integration-4 (3ddeff0) is upstream main 939b215, which merged our earlier PRs #9 to #24 in CrocSwap/goSVM#25, plus the open draft PRs: whole-word runtime loads, stores and key comparisons (CrocSwap/goSVM#27), solana.FindAddress on sol_try_find_program_address, also behind pda.Seeds.FindAddress (CrocSwap/goSVM#28), word-at-a-time == on byte arrays (CrocSwap/goSVM#29), the exact MinimumBalanceFor fast path at thresholds 2.0 and 1.0 (CrocSwap/goSVM#30), and word compares in front of hand-written byte compare loops (CrocSwap/goSVM#31). The SDK snapshot is copied from the new CLI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hyperlane's Go port encountered compiler failures and missing runtime/SDK boundaries. This integrates the contributor's PRs #9–#24, preserving their commits, with the SDK snapshots and compiler runtime hooks reconciled together.
The integration fixes native GetReturnData overlap handling and documents silent-CPI clearing. ATA creation delegates canonical-address checking to the validated ATA program, retaining explicit derivation helpers. The 64-account decoder remains opt-in; outgoing CPI supports 64 metas and four signer groups. Token-2022 and ATA error codes are registered in ERROR_CONTRACT.
The combined CPI and memory changes produce new full-swap ELFs. A fresh integration reference records all four reproducible full-swap/clock2 ELFs without modifying earlier results.
Validation: focused compiler/SDK/native-C/SBF tests pass on both profiles, including the frozen corpus, slice-array conversions, generic arrays, CPI limits and real Token/ATA programs. Full root make test and go vet pass. Separate example/service modules and all six SBF application suites pass, with explicit modern-target swap/escrow fixtures and swap lifecycle on all three profiles. Linux and macOS CI job logs pass on final head d698ec9, including all four reference ELFs and release-script checks: https://github.com/CrocSwap/goSVM/actions/runs/39280956340. GitHub's check-summary state lagged the completed job logs; the logs were inspected directly for completion and failures. Results and reproduction: results/compiler/2026-10-19-hyperlane-integration/README.md.
Closes #1
Closes #2
Closes #3
Closes #4
Closes #5
Closes #6
Closes #7
Closes #8