From 3fece1760737f6b3cb7661b629091c1a0c258870 Mon Sep 17 00:00:00 2001 From: Sebastian Scholl Date: Sun, 19 Jul 2026 01:36:33 +0200 Subject: [PATCH 1/2] Add address lookup table (v0) composition support Compose v0 transactions through address lookup tables, and create/extend tables on chain, with a clean single compile path rather than legacy/v0 branching. - TransactionComposer#add_address_lookup_table registers a table and opts the transaction into v0; compose_transaction resolves loadable accounts (non-signer, not the fee payer, not an invoked program id) through the tables in a single pass. merge folds another composer's tables in. - AccountContext#compile takes loaded_accounts:, resolving index_of against the combined v0 account space while keeping only static keys in the message and dropping loaded accounts from the header. Legacy compilation unchanged. - Solace::Accounts::AddressLookupTable: on-chain account model with .deserialize (meta + stored addresses) and #reference for the v0 message reference it contributes. - AddressLookupTableProgram Create/Extend composers + instruction builders. - Connection#get_slot. - Version 0.1.7; docs and CHANGELOG updated. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG | 17 + gem/.rubocop.yml | 6 + gem/Gemfile.lock | 2 +- gem/lib/solace.rb | 4 + .../solace/accounts/address_lookup_table.rb | 134 ++++++ ...ss_lookup_table_program_create_composer.rb | 95 +++++ ...ss_lookup_table_program_extend_composer.rb | 87 ++++ gem/lib/solace/connection.rb | 10 + .../create_lookup_table_instruction.rb | 80 ++++ .../extend_lookup_table_instruction.rb | 69 +++ gem/lib/solace/transaction_composer.rb | 117 +++++- gem/lib/solace/utils/account_context.rb | 54 ++- gem/lib/solace/version.rb | 2 +- .../accounts/address_lookup_table_test.rb | 105 +++++ ...okup_table_program_create_composer_test.rb | 44 ++ ...okup_table_program_extend_composer_test.rb | 56 +++ .../create_lookup_table_instruction_test.rb | 35 ++ .../extend_lookup_table_instruction_test.rb | 42 ++ gem/test/solace/transaction_composer_test.rb | 397 +++++++++++++++++- gem/test/solace/utils/account_context_test.rb | 41 ++ gem/test/support/lookup_table_provisioner.rb | 67 +++ gem/test/test_helper.rb | 1 + site/building/transaction-composer.md | 35 +- site/concepts/account-context.md | 11 + site/concepts/address-lookup-tables.md | 78 +++- site/concepts/connection-and-rpc.md | 1 + 26 files changed, 1561 insertions(+), 29 deletions(-) create mode 100644 gem/lib/solace/accounts/address_lookup_table.rb create mode 100644 gem/lib/solace/composers/address_lookup_table_program_create_composer.rb create mode 100644 gem/lib/solace/composers/address_lookup_table_program_extend_composer.rb create mode 100644 gem/lib/solace/instructions/address_lookup_table_program/create_lookup_table_instruction.rb create mode 100644 gem/lib/solace/instructions/address_lookup_table_program/extend_lookup_table_instruction.rb create mode 100644 gem/test/solace/accounts/address_lookup_table_test.rb create mode 100644 gem/test/solace/composers/address_lookup_table_program_create_composer_test.rb create mode 100644 gem/test/solace/composers/address_lookup_table_program_extend_composer_test.rb create mode 100644 gem/test/solace/instructions/address_lookup_table_program/create_lookup_table_instruction_test.rb create mode 100644 gem/test/solace/instructions/address_lookup_table_program/extend_lookup_table_instruction_test.rb create mode 100644 gem/test/support/lookup_table_provisioner.rb diff --git a/CHANGELOG b/CHANGELOG index f64ea18..c5f19fc 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -20,6 +20,23 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/) and this p --- +## 0.1.7 - 2026-07-19 + +### Added + +1. Added address lookup table (v0) support to `Solace::TransactionComposer`: `#add_address_lookup_table(account:, addresses:)` registers a table and opts the transaction into the v0 format, and `#compose_transaction` loads every eligible account (a non-signer that is not the fee payer and not an invoked program id) through the tables instead of a static account slot. `#merge` folds another composer's tables in, deduped by account. +2. Added `Solace::Accounts` for on-chain account models, with `Solace::Accounts::AddressLookupTable` — a table's address and stored addresses, with `.deserialize` for reading on-chain table state and `#reference` for building the v0 message reference it contributes. +3. Added Address Lookup Table program composers and instruction builders: `Solace::Composers::AddressLookupTableProgram{Create,Extend}Composer` and `Solace::Instructions::AddressLookupTableProgram::{CreateLookupTable,ExtendLookupTable}Instruction`, so tables can be created and extended on chain. +4. Added `Solace::Connection#get_slot`. + +### Changed + +1. `Solace::Utils::AccountContext#compile` accepts `loaded_accounts:`, rebuilding index resolution against the combined v0 account space (static keys followed by loaded addresses) while keeping only the static keys in the message. With no loaded accounts the legacy compilation is unchanged. + +### Fixed + +--- + ## 0.1.6 - 2026-06-22 ### Added diff --git a/gem/.rubocop.yml b/gem/.rubocop.yml index fd35b5f..d9012d1 100644 --- a/gem/.rubocop.yml +++ b/gem/.rubocop.yml @@ -24,6 +24,12 @@ Metrics/ParameterLists: Metrics/MethodLength: Max: 15 +# Composers and utilities collect many small, cohesive methods (e.g. the +# TransactionComposer's account/lookup-table orchestration); 100 is a touch +# tight for them while still discouraging genuinely large classes. +Metrics/ClassLength: + Max: 125 + # Utility modules (e.g. Solace::Utils::Codecs) legitimately collect many small # related methods; the default of 100 is too low for them. Metrics/ModuleLength: diff --git a/gem/Gemfile.lock b/gem/Gemfile.lock index 8290e6f..0b0753b 100644 --- a/gem/Gemfile.lock +++ b/gem/Gemfile.lock @@ -1,7 +1,7 @@ PATH remote: . specs: - solace (0.1.6) + solace (0.1.7) base58 (~> 0.2) ffi (~> 1.15) rbnacl (~> 7.0) diff --git a/gem/lib/solace.rb b/gem/lib/solace.rb index 50fbd46..43b7857 100644 --- a/gem/lib/solace.rb +++ b/gem/lib/solace.rb @@ -55,6 +55,10 @@ module Solace; end require_relative 'solace/message' require_relative 'solace/instruction' require_relative 'solace/address_lookup_table' + +# Accounts (on-chain account models) +require_relative 'solace/accounts/address_lookup_table' + require_relative 'solace/transaction_composer' # Base Classes (Abstract classes) diff --git a/gem/lib/solace/accounts/address_lookup_table.rb b/gem/lib/solace/accounts/address_lookup_table.rb new file mode 100644 index 0000000..77141bf --- /dev/null +++ b/gem/lib/solace/accounts/address_lookup_table.rb @@ -0,0 +1,134 @@ +# frozen_string_literal: true + +module Solace + # On-chain account models — the account state living at an address, as opposed + # to the wire-level structures that reference it. Conventions for these types + # (fetching, deserializing, deriving) are expected to grow here. + module Accounts + # Models an on-chain Address Lookup Table account: its address, the metadata + # the program stores (authority, activation slots), and the full, ordered + # list of addresses it holds. + # + # This is distinct from {Solace::AddressLookupTable}, which is the *reference* + # a v0 message carries (a table account plus the writable/readonly index + # positions into it). A composer decides which of this account's addresses + # load — a v0 concern that needs the account context — and this object owns + # the table-local part: turning the addresses it contributes into that + # message reference. + # + # @example Register a table on a composer + # table = Solace::Accounts::AddressLookupTable.new(account: address, addresses: on_chain_addresses) + # table.reference(loaded_writable, loaded_readonly) # => Solace::AddressLookupTable or nil + # + # @example Read a table's on-chain state + # data = Base64.decode64(connection.get_account_info(address)['data'][0]) + # table = Solace::Accounts::AddressLookupTable.deserialize(StringIO.new(data)) + # table.addresses # => the stored addresses + # + # @see Solace::AddressLookupTable + # @see Solace::TransactionComposer + # @since 0.1.7 + class AddressLookupTable + # Byte offset at which the stored addresses begin — the program reserves a + # fixed-size metadata region ahead of them regardless of its contents. + META_SIZE = 56 + + # @!attribute [r] account + # @return [String, nil] The lookup table's on-chain address + attr_reader :account + + # @!attribute [r] addresses + # @return [Array] The full, ordered list of addresses stored in the table + attr_reader :addresses + + # @!attribute [r] authority + # @return [String, nil] The authority allowed to extend/close the table + attr_reader :authority + + # @!attribute [r] deactivation_slot + # @return [Integer, nil] The slot the table was deactivated in (max while active) + attr_reader :deactivation_slot + + # @!attribute [r] last_extended_slot + # @return [Integer, nil] The slot the table was last extended in + attr_reader :last_extended_slot + + # Deserialize an on-chain lookup table account + # + # The BufferLayout is: + # - [State type (4 bytes, u32 LE)] + # - [Deactivation slot (8 bytes, u64 LE)] + # - [Last extended slot (8 bytes, u64 LE)] + # - [Last extended start index (1 byte)] + # - [Authority (Borsh Option)] + # - [Padding, up to {META_SIZE}] + # - [Addresses (32 bytes each, to end of data)] + # + # @param io [IO, StringIO] The account data to read from + # @return [AddressLookupTable] The parsed table + def self.deserialize(io) + Utils::Codecs.decode_le_u32(io) # state type (1 = lookup table); positional + deactivation_slot = Utils::Codecs.decode_le_u64(io) + last_extended_slot = Utils::Codecs.decode_le_u64(io) + + Utils::Codecs.decode_u8(io) # last extended start index; positional + authority = Utils::Codecs.decode_option_pubkey(io) + + io.seek(META_SIZE) # addresses begin after the fixed-size metadata region + + addresses = [] + addresses << Utils::Codecs.decode_pubkey(io) until io.eof? + + new( + deactivation_slot: deactivation_slot, + last_extended_slot: last_extended_slot, + authority: authority, + addresses: addresses + ) + end + + # Initialize a lookup table account + # + # @param account [#to_s, PublicKey, nil] The lookup table's on-chain address + # @param addresses [Array<#to_s>, nil] The full, ordered address list; may be + # omitted (the composer then loads nothing through this table) + # @param authority [String, nil] The table authority + # @param deactivation_slot [Integer, nil] The deactivation slot + # @param last_extended_slot [Integer, nil] The last extended slot + def initialize(account: nil, addresses: nil, authority: nil, deactivation_slot: nil, last_extended_slot: nil) + @account = account&.to_s + @addresses = Array(addresses).map(&:to_s) + @authority = authority + @deactivation_slot = deactivation_slot + @last_extended_slot = last_extended_slot + end + + # Build the v0 message reference for the addresses this table contributes + # + # @param writable [Array] Loaded writable pubkeys drawn from this table + # @param readonly [Array] Loaded readonly pubkeys drawn from this table + # @return [Solace::AddressLookupTable, nil] The reference, or nil if it loads nothing + def reference(writable, readonly) + writable_indexes = positions_of(writable) + readonly_indexes = positions_of(readonly) + return if writable_indexes.empty? && readonly_indexes.empty? + + Solace::AddressLookupTable.new.tap do |reference| + reference.account = account + reference.writable_indexes = writable_indexes + reference.readonly_indexes = readonly_indexes + end + end + + private + + # Positions of the given pubkeys within this table's address list + # + # @param pubkeys [Array] The pubkeys to locate + # @return [Array] Their positions in {#addresses} + def positions_of(pubkeys) + pubkeys.filter_map { |pubkey| addresses.index(pubkey) } + end + end + end +end diff --git a/gem/lib/solace/composers/address_lookup_table_program_create_composer.rb b/gem/lib/solace/composers/address_lookup_table_program_create_composer.rb new file mode 100644 index 0000000..f01a5a3 --- /dev/null +++ b/gem/lib/solace/composers/address_lookup_table_program_create_composer.rb @@ -0,0 +1,95 @@ +# frozen_string_literal: true + +module Solace + module Composers + # Composer for creating an address lookup table. + # + # Resolves and orders the accounts for a `CreateLookupTable` instruction and + # delegates construction to + # `Instructions::AddressLookupTableProgram::CreateLookupTableInstruction`. + # + # The table address is a program-derived address of `[authority, recent_slot]`; + # derive it (and its bump) with {Solace::Utils::PDA} and pass both in. + # + # Required accounts: + # - **Table**: the uninitialized table account (writable, non-signer) + # - **Authority**: controls the table (readonly, signer) + # - **Payer**: funds the table's rent (writable, signer) + # - **System program** (readonly, non-signer) + # + # @example + # composer = AddressLookupTableProgramCreateComposer.new( + # table: table_address, + # authority: authority, + # payer: payer, + # recent_slot: recent_slot, + # bump: bump + # ) + # + # @see Instructions::AddressLookupTableProgram::CreateLookupTableInstruction + # @since 0.1.7 + class AddressLookupTableProgramCreateComposer < Base + # @return [String] The table's on-chain address + def table + params[:table].to_s + end + + # @return [String] The table authority + def authority + params[:authority].to_s + end + + # @return [String] The rent payer + def payer + params[:payer].to_s + end + + # @return [String] The system program id + def system_program + Solace::Constants::SYSTEM_PROGRAM_ID.to_s + end + + # @return [String] The address lookup table program id + def lookup_table_program + Solace::Constants::ADDRESS_LOOKUP_TABLE_PROGRAM_ID.to_s + end + + # @return [Integer] The slot used to derive the table address + def recent_slot + params[:recent_slot] + end + + # @return [Integer] The bump seed for the table's program-derived address + def bump + params[:bump] + end + + # Setup accounts required for the create lookup table instruction + # + # @return [void] + def setup_accounts + account_context.add_writable_nonsigner(table) + account_context.add_readonly_signer(authority) + account_context.add_writable_signer(payer) + account_context.add_readonly_nonsigner(system_program) + account_context.add_readonly_nonsigner(lookup_table_program) + end + + # Build instruction with resolved account indices + # + # @param account_context [Utils::AccountContext] The account context + # @return [Solace::Instruction] + def build_instruction(account_context) + Solace::Instructions::AddressLookupTableProgram::CreateLookupTableInstruction.build( + recent_slot: recent_slot, + bump: bump, + program_index: account_context.index_of(lookup_table_program), + table_index: account_context.index_of(table), + authority_index: account_context.index_of(authority), + payer_index: account_context.index_of(payer), + system_program_index: account_context.index_of(system_program) + ) + end + end + end +end diff --git a/gem/lib/solace/composers/address_lookup_table_program_extend_composer.rb b/gem/lib/solace/composers/address_lookup_table_program_extend_composer.rb new file mode 100644 index 0000000..e2b6167 --- /dev/null +++ b/gem/lib/solace/composers/address_lookup_table_program_extend_composer.rb @@ -0,0 +1,87 @@ +# frozen_string_literal: true + +module Solace + module Composers + # Composer for extending an address lookup table with new addresses. + # + # Resolves and orders the accounts for an `ExtendLookupTable` instruction and + # delegates construction to + # `Instructions::AddressLookupTableProgram::ExtendLookupTableInstruction`. + # + # Addresses appended here become usable one slot after the extend lands. + # + # Required accounts: + # - **Table**: the table account being extended (writable, non-signer) + # - **Authority**: controls the table (readonly, signer) + # - **Payer**: funds any additional rent (writable, signer) + # - **System program** (readonly, non-signer) + # + # @example + # composer = AddressLookupTableProgramExtendComposer.new( + # table: table_address, + # authority: authority, + # payer: payer, + # addresses: [recipient1, recipient2] + # ) + # + # @see Instructions::AddressLookupTableProgram::ExtendLookupTableInstruction + # @since 0.1.7 + class AddressLookupTableProgramExtendComposer < Base + # @return [String] The table's on-chain address + def table + params[:table].to_s + end + + # @return [String] The table authority + def authority + params[:authority].to_s + end + + # @return [String] The rent payer + def payer + params[:payer].to_s + end + + # @return [String] The system program id + def system_program + Solace::Constants::SYSTEM_PROGRAM_ID.to_s + end + + # @return [String] The address lookup table program id + def lookup_table_program + Solace::Constants::ADDRESS_LOOKUP_TABLE_PROGRAM_ID.to_s + end + + # @return [Array] The addresses to append to the table + def addresses + params[:addresses].map(&:to_s) + end + + # Setup accounts required for the extend lookup table instruction + # + # @return [void] + def setup_accounts + account_context.add_writable_nonsigner(table) + account_context.add_readonly_signer(authority) + account_context.add_writable_signer(payer) + account_context.add_readonly_nonsigner(system_program) + account_context.add_readonly_nonsigner(lookup_table_program) + end + + # Build instruction with resolved account indices + # + # @param account_context [Utils::AccountContext] The account context + # @return [Solace::Instruction] + def build_instruction(account_context) + Solace::Instructions::AddressLookupTableProgram::ExtendLookupTableInstruction.build( + addresses: addresses, + program_index: account_context.index_of(lookup_table_program), + table_index: account_context.index_of(table), + authority_index: account_context.index_of(authority), + payer_index: account_context.index_of(payer), + system_program_index: account_context.index_of(system_program) + ) + end + end + end +end diff --git a/gem/lib/solace/connection.rb b/gem/lib/solace/connection.rb index 06c2ced..fda189a 100644 --- a/gem/lib/solace/connection.rb +++ b/gem/lib/solace/connection.rb @@ -157,6 +157,16 @@ def get_latest_blockhash [@last_fetched_blockhash, @last_fetched_block_height] end + # Get the current slot from the Solana node + # + # @param commitment [String] The commitment level for the request + # @return [Integer] The current slot + # + # @since 0.1.7 + def get_slot(commitment: default_options[:commitment]) + @rpc_client.rpc_request('getSlot', [{ commitment: commitment }])['result'] + end + # Get the minimum required lamports for rent exemption # # @param space [Integer] Number of bytes to allocate for the account diff --git a/gem/lib/solace/instructions/address_lookup_table_program/create_lookup_table_instruction.rb b/gem/lib/solace/instructions/address_lookup_table_program/create_lookup_table_instruction.rb new file mode 100644 index 0000000..b897a9c --- /dev/null +++ b/gem/lib/solace/instructions/address_lookup_table_program/create_lookup_table_instruction.rb @@ -0,0 +1,80 @@ +# frozen_string_literal: true + +module Solace + module Instructions + # The AddressLookupTableProgram module contains instruction builders for the + # Address Lookup Table program. + # + # Address lookup tables let a versioned (v0) transaction reference accounts by + # a compact table index instead of a full 32-byte key, so a single transaction + # can touch far more accounts than the legacy format allows. + # + # @see https://docs.solana.com/developing/lookup-tables + # @since 0.1.7 + module AddressLookupTableProgram + # Instruction for creating a new (uninitialized) address lookup table. + # + # The table's address is a program-derived address of `[authority, recent_slot]`; + # the caller supplies the derived address and its bump seed. + # + # @example Build a CreateLookupTable instruction + # instruction = Solace::Instructions::AddressLookupTableProgram::CreateLookupTableInstruction.build( + # recent_slot: 123, + # bump: 254, + # program_index: 4, + # table_index: 1, + # authority_index: 0, + # payer_index: 0, + # system_program_index: 3 + # ) + # + # @since 0.1.7 + class CreateLookupTableInstruction + # Instruction discriminator for CreateLookupTable (u32 LE) + INSTRUCTION_ID = [0, 0, 0, 0].freeze + + # Builds a Solace::Instruction for creating a lookup table + # + # @param recent_slot [Integer] The slot used to derive the table address + # @param bump [Integer] The bump seed for the table's program-derived address + # @param program_index [Integer] Index of the lookup table program + # @param table_index [Integer] Index of the (uninitialized) table account + # @param authority_index [Integer] Index of the table authority (signer) + # @param payer_index [Integer] Index of the rent payer (writable signer) + # @param system_program_index [Integer] Index of the system program + # @return [Solace::Instruction] + def self.build( + recent_slot:, + bump:, + program_index:, + table_index:, + authority_index:, + payer_index:, + system_program_index: + ) + Instruction.new.tap do |ix| + ix.program_index = program_index + ix.accounts = [table_index, authority_index, payer_index, system_program_index] + ix.data = data(recent_slot, bump) + end + end + + # Instruction data for a create lookup table instruction + # + # The BufferLayout is: + # - [Instruction discriminator (4 bytes, u32 LE)] + # - [Recent slot (8 bytes, u64 LE)] + # - [Bump seed (1 byte)] + # + # @param recent_slot [Integer] The slot used to derive the table address + # @param bump [Integer] The bump seed + # @return [Array] + def self.data(recent_slot, bump) + INSTRUCTION_ID + + Utils::Codecs.encode_le_u64(recent_slot).bytes + + [bump] + end + end + end + end +end diff --git a/gem/lib/solace/instructions/address_lookup_table_program/extend_lookup_table_instruction.rb b/gem/lib/solace/instructions/address_lookup_table_program/extend_lookup_table_instruction.rb new file mode 100644 index 0000000..90812ba --- /dev/null +++ b/gem/lib/solace/instructions/address_lookup_table_program/extend_lookup_table_instruction.rb @@ -0,0 +1,69 @@ +# frozen_string_literal: true + +module Solace + module Instructions + module AddressLookupTableProgram + # Instruction for appending addresses to an existing address lookup table. + # + # Newly added addresses become usable one slot after the extend lands. + # + # @example Build an ExtendLookupTable instruction + # instruction = Solace::Instructions::AddressLookupTableProgram::ExtendLookupTableInstruction.build( + # addresses: [recipient1, recipient2], + # program_index: 4, + # table_index: 1, + # authority_index: 0, + # payer_index: 0, + # system_program_index: 3 + # ) + # + # @since 0.1.7 + class ExtendLookupTableInstruction + # Instruction discriminator for ExtendLookupTable (u32 LE) + INSTRUCTION_ID = [2, 0, 0, 0].freeze + + # Builds a Solace::Instruction for extending a lookup table + # + # @param addresses [Array<#to_s>] The addresses to append to the table + # @param program_index [Integer] Index of the lookup table program + # @param table_index [Integer] Index of the table account (writable) + # @param authority_index [Integer] Index of the table authority (signer) + # @param payer_index [Integer] Index of the rent payer (writable signer) + # @param system_program_index [Integer] Index of the system program + # @return [Solace::Instruction] + def self.build( + addresses:, + program_index:, + table_index:, + authority_index:, + payer_index:, + system_program_index: + ) + Instruction.new.tap do |ix| + ix.program_index = program_index + ix.accounts = [table_index, authority_index, payer_index, system_program_index] + ix.data = data(addresses) + end + end + + # Instruction data for an extend lookup table instruction + # + # The BufferLayout is: + # - [Instruction discriminator (4 bytes, u32 LE)] + # - [Number of addresses (8 bytes, u64 LE)] + # - [Addresses (32 bytes each)] + # + # The address vector uses a u64 length prefix (bincode), not the u32 + # Borsh prefix of {Utils::Codecs.encode_vec_pubkeys}. + # + # @param addresses [Array<#to_s>] The addresses to append + # @return [Array] + def self.data(addresses) + INSTRUCTION_ID + + Utils::Codecs.encode_le_u64(addresses.length).bytes + + addresses.flat_map { |address| Utils::Codecs.encode_pubkey(address) } + end + end + end + end +end diff --git a/gem/lib/solace/transaction_composer.rb b/gem/lib/solace/transaction_composer.rb index 49a2b9e..64fc1b2 100644 --- a/gem/lib/solace/transaction_composer.rb +++ b/gem/lib/solace/transaction_composer.rb @@ -73,6 +73,14 @@ class TransactionComposer # The instruction composers attr_reader :instruction_composers + # @!attribute address_lookup_tables + # The registered address lookup tables + attr_reader :address_lookup_tables + + # @!attribute version + # The transaction version (nil for legacy, 0 for v0) + attr_reader :version + # Initialize the composer # # @param connection [Solace::Connection] The connection to the Solana cluster @@ -80,6 +88,8 @@ def initialize(connection:) @connection = connection @instruction_composers = [] @context = Utils::AccountContext.new + @address_lookup_tables = [] + @version = nil end # Add an instruction composer to the transaction @@ -127,6 +137,7 @@ def insert_instruction(index, composer) # @since 0.1.0 def merge(other, placement: :add, index: nil) merge_accounts(other.context) + merge_address_lookup_tables(other.address_lookup_tables) case placement when :add @@ -154,24 +165,109 @@ def set_fee_payer(pubkey) self end + # Make an address lookup table available to the transaction + # + # Registering a table opts the transaction into the v0 format: every + # compiled account found in a table that is allowed to load (a non-signer + # that is not the fee payer and not an invoked program id) is referenced by + # table index instead of occupying a static account slot. Adding the same + # table (by account) twice is a no-op. + # + # @example + # composer.add_address_lookup_table(account: table_address, addresses: on_chain_addresses) + # + # @param account [#to_s, PublicKey] The lookup table's on-chain address + # @param addresses [Array<#to_s>, nil] The full, ordered list of addresses stored in the table + # @return [TransactionComposer] Self for chaining + # + # @since 0.1.7 + def add_address_lookup_table(account:, addresses: nil) + account = account.to_s + @version = 0 # Lookup tables require a v0 transaction + + unless address_lookup_tables.any? { |table| table.account == account } + address_lookup_tables << Solace::Accounts::AddressLookupTable.new(account: account, addresses: addresses) + end + + self + end + # Compose the final transaction # + # Emits a message at the composer's {#version} — legacy by default, or v0 + # once a lookup table has been added — loading eligible accounts through any + # registered tables. + # # @return [Transaction] The composed transaction (unsigned) def compose_transaction context.compile - message = Solace::Message.new( - header: context.header, - accounts: context.accounts, - instructions: build_instructions, - recent_blockhash: connection.get_latest_blockhash[0] - ) + writable, readonly, references = resolve_address_lookup_tables + context.compile(loaded_accounts: writable + readonly) - Solace::Transaction.new(message: message) + Solace::Transaction.new(message: build_message(references)) end private + # Fold the registered tables into the accounts they load and the references + # the message carries — a single pass over the tables (the order that + # defines the v0 combined account space). + # + # First table wins when tables share an address; within a table the on-chain + # address order is preserved, so the accumulated writable/readonly segments + # match the runtime's loaded-address order once concatenated. + # + # @return [Array(Array, Array, Array)] + # The loaded writable pubkeys, loaded readonly pubkeys, and table references + def resolve_address_lookup_tables + loadable = loadable_accounts + writable = [] + readonly = [] + + references = address_lookup_tables.filter_map do |table| + loaded = table.addresses & loadable + loadable -= loaded + on, off = loaded.partition { |pubkey| context.writable?(pubkey) } + writable.concat(on) + readonly.concat(off) + table.reference(on, off) + end + + [writable, readonly, references] + end + + # Accounts eligible to load through a table: referenced by the transaction, + # not a signer (which covers the fee payer), and not an invoked program id. + # + # @return [Array] The loadable account pubkeys + def loadable_accounts + programs = program_ids + context.accounts.reject { |pubkey| context.signer?(pubkey) || programs.include?(pubkey) } + end + + # Program ids invoked by the built instructions — these must stay static + # + # @return [Array] The invoked program id pubkeys + def program_ids + build_instructions.map { context.accounts[_1.program_index] }.uniq + end + + # Build the composed message at the composer's version (legacy or v0) + # + # @param references [Array] The table references + # @return [Solace::Message] The composed message + def build_message(references) + Solace::Message.new( + version: version, + header: context.header, + accounts: context.accounts, + instructions: build_instructions, + recent_blockhash: connection.get_latest_blockhash[0], + address_lookup_tables: references + ) + end + # Build all instructions with resolved indices # # @return [Array] The built instructions @@ -185,5 +281,12 @@ def build_instructions def merge_accounts(account_context) context.merge_from(account_context) end + + # Merge registered tables from another composer, deduped by account + # + # @param tables [Array] The other composer's tables + def merge_address_lookup_tables(tables) + tables.each { |table| add_address_lookup_table(account: table.account, addresses: table.addresses) } + end end end diff --git a/gem/lib/solace/utils/account_context.rb b/gem/lib/solace/utils/account_context.rb index 950bc84..dc407fb 100644 --- a/gem/lib/solace/utils/account_context.rb +++ b/gem/lib/solace/utils/account_context.rb @@ -62,6 +62,7 @@ class AccountContext def initialize @header = [] @accounts = [] + @loaded_accounts = [] @pubkey_account_map = {} end @@ -173,26 +174,40 @@ def merge_from(other_context) # - Then writable accounts # - Then readonly accounts # - # @return [Hash] The compiled accounts and header - def compile - self.header = calculate_header - self.accounts = order_accounts + # For a v0 (versioned) transaction, accounts resolved through lookup tables + # leave the static account list entirely: the message carries only the + # static keys, and the runtime rebuilds the combined space + # [static..., loaded...] at execution time. Passing them keeps {#index_of} + # resolving against that combined space (so instructions index correctly) + # while dropping them from the static accounts and the header. They must be + # ordered writable-first to match the runtime's combined space. With no + # loaded accounts (the default) this is the legacy compilation, unchanged. + # + # @param loaded_accounts [Array] Pubkeys resolved through lookup tables + # @return [AccountContext] Self + def compile(loaded_accounts: []) + @loaded_accounts = loaded_accounts + + self.header = calculate_header(loaded_accounts) + self.accounts = order_accounts(loaded_accounts) self end - # Index of a pubkey in the accounts array + # Index of a pubkey in the combined account space # # @param pubkey_str [String] The public key of the account - # @return [Integer] The index of the pubkey in the accounts array or -1 if not found + # @return [Integer] The index in the combined space, or -1 if not found def index_of(pubkey_str) indices[pubkey_str] || -1 end - # Get map of indicies for pubkeys in accounts array + # Map of pubkey => index across the combined space (static keys followed by + # any loaded accounts), so instruction indices resolve the same way the + # runtime does once lookup tables are expanded. # - # @return [Hash{String => Integer}] The indices of the pubkeys in the accounts array + # @return [Hash{String => Integer}] The indices of the pubkeys def indices - accounts.each_with_index.to_h + (accounts + @loaded_accounts).each_with_index.to_h end private @@ -214,11 +229,15 @@ def merge_account(pubkey, signer:, writable:, fee_payer: false) self end - # Order accounts by signer, writable, readonly signer, readonly + # Order the static accounts by signer, writable, readonly signer, readonly # - # @return [Array] The ordered accounts - def order_accounts - @pubkey_account_map.keys.sort_by do |pubkey| + # Loaded accounts are excluded — they leave the static account list and are + # resolved through lookup tables at runtime. + # + # @param loaded_accounts [Array] Pubkeys resolved through lookup tables + # @return [Array] The ordered static accounts + def order_accounts(loaded_accounts) + (@pubkey_account_map.keys - loaded_accounts).sort_by do |pubkey| if fee_payer?(pubkey) then 0 elsif writable_signer?(pubkey) then 1 elsif readonly_signer?(pubkey) then 2 @@ -237,9 +256,14 @@ def order_accounts # - The number of readonly signers # - The number of readonly unsigned accounts # + # Loaded accounts no longer occupy the static account list, so they drop + # out of the header. They are never signers, so only the readonly-unsigned + # count changes. + # + # @param loaded_accounts [Array] Pubkeys resolved through lookup tables # @return [Array] The header for the transaction - def calculate_header - @pubkey_account_map.keys.each_with_object([0, 0, 0]) do |pubkey, acc| + def calculate_header(loaded_accounts) + (@pubkey_account_map.keys - loaded_accounts).each_with_object([0, 0, 0]) do |pubkey, acc| acc[0] += 1 if signer?(pubkey) if readonly_signer?(pubkey) then acc[1] += 1 diff --git a/gem/lib/solace/version.rb b/gem/lib/solace/version.rb index 48ceb6f..c4be1f9 100644 --- a/gem/lib/solace/version.rb +++ b/gem/lib/solace/version.rb @@ -2,5 +2,5 @@ module Solace # Latest version of the Solace gem. - VERSION = '0.1.6' + VERSION = '0.1.7' end diff --git a/gem/test/solace/accounts/address_lookup_table_test.rb b/gem/test/solace/accounts/address_lookup_table_test.rb new file mode 100644 index 0000000..36346c4 --- /dev/null +++ b/gem/test/solace/accounts/address_lookup_table_test.rb @@ -0,0 +1,105 @@ +# frozen_string_literal: true + +require 'test_helper' +require 'stringio' + +describe Solace::Accounts::AddressLookupTable do + let(:authority) { Solace::Keypair.generate.address } + let(:address1) { Solace::Keypair.generate.address } + let(:address2) { Solace::Keypair.generate.address } + let(:address3) { Solace::Keypair.generate.address } + + # Build the raw bytes of an on-chain lookup table account + def encode_account_data(authority:, addresses:, deactivation_slot: (2**64) - 1, last_extended_slot: 42) + meta = encode_meta(authority, deactivation_slot, last_extended_slot) + addrs = addresses.flat_map { |address| Solace::Utils::Codecs.base58_to_bytes(address) } + + StringIO.new((meta + addrs).pack('C*')) + end + + # The fixed-size metadata region preceding the stored addresses + def encode_meta(authority, deactivation_slot, last_extended_slot) + codecs = Solace::Utils::Codecs + bytes = codecs.encode_le_u32(1).bytes + # state type = LookupTable + codecs.encode_le_u64(deactivation_slot).bytes + + codecs.encode_le_u64(last_extended_slot).bytes + + [0] + # last extended start index + codecs.encode_option_pubkey(authority) + + bytes + ([0] * (Solace::Accounts::AddressLookupTable::META_SIZE - bytes.length)) + end + + describe '.deserialize' do + it 'reads the metadata and stored addresses' do + io = encode_account_data(authority: authority, addresses: [address1, address2], last_extended_slot: 99) + table = Solace::Accounts::AddressLookupTable.deserialize(io) + + assert_equal authority, table.authority + assert_equal 99, table.last_extended_slot + assert_equal (2**64) - 1, table.deactivation_slot + assert_equal [address1, address2], table.addresses + end + + it 'reads a freshly created table that has no addresses yet' do + table = Solace::Accounts::AddressLookupTable.deserialize( + encode_account_data(authority: authority, addresses: []) + ) + + assert_equal authority, table.authority + assert_empty table.addresses + end + + it 'handles a table with no authority' do + table = Solace::Accounts::AddressLookupTable.deserialize( + encode_account_data(authority: nil, addresses: [address1]) + ) + + assert_nil table.authority + assert_equal [address1], table.addresses + end + end + + describe '#reference' do + let(:table_account) { Solace::Keypair.generate.address } + + let(:table) do + Solace::Accounts::AddressLookupTable.new(account: table_account, addresses: [address1, address2, address3]) + end + + it 'builds a message reference from the loaded addresses' do + reference = table.reference([address2], [address1]) + + assert_instance_of Solace::AddressLookupTable, reference + assert_equal table_account, reference.account + assert_equal [1], reference.writable_indexes # address2 + assert_equal [0], reference.readonly_indexes # address1 + end + + it 'emits the index positions in the order given' do + reference = table.reference([address3, address1], []) + + assert_equal [2, 0], reference.writable_indexes + end + + it 'returns nil when the table contributes no loaded accounts' do + assert_nil table.reference([], []) + end + end + + describe '#initialize' do + it 'normalizes the account and addresses to strings' do + keypair = Solace::Keypair.generate + + table = Solace::Accounts::AddressLookupTable.new(account: keypair, addresses: [keypair]) + + assert_equal keypair.address, table.account + assert_equal [keypair.address], table.addresses + end + + it 'defaults to an empty address list' do + table = Solace::Accounts::AddressLookupTable.new(account: authority) + + assert_empty table.addresses + end + end +end diff --git a/gem/test/solace/composers/address_lookup_table_program_create_composer_test.rb b/gem/test/solace/composers/address_lookup_table_program_create_composer_test.rb new file mode 100644 index 0000000..37cc3d0 --- /dev/null +++ b/gem/test/solace/composers/address_lookup_table_program_create_composer_test.rb @@ -0,0 +1,44 @@ +# frozen_string_literal: true + +require 'test_helper' + +describe Solace::Composers::AddressLookupTableProgramCreateComposer do + before(:all) do + connection = Solace::Connection.new(commitment: 'processed') + @authority = Fixtures.load_keypair('bob') + recent_slot = connection.get_slot - 1 + + @table, bump = Solace::Utils::PDA.find_program_address( + [@authority.address, Solace::Utils::Codecs.encode_le_u64(recent_slot).bytes], + Solace::Constants::ADDRESS_LOOKUP_TABLE_PROGRAM_ID + ) + + tx = Solace::TransactionComposer + .new(connection: connection) + .add_instruction(Solace::Composers::AddressLookupTableProgramCreateComposer.new( + table: @table, + authority: @authority, + payer: @authority, + recent_slot: recent_slot, + bump: bump + )) + .set_fee_payer(@authority) + .compose_transaction + + tx.sign(@authority) + + signature = connection.send_transaction(tx.serialize) + connection.wait_for_confirmed_signature { signature['result'] } + + data = Base64.decode64(connection.get_account_info(@table)['data'][0]) + @lookup_table = Solace::Accounts::AddressLookupTable.deserialize(StringIO.new(data)) + end + + it 'creates a lookup table controlled by the authority' do + assert_equal @authority.address, @lookup_table.authority + end + + it 'starts with no stored addresses' do + assert_empty @lookup_table.addresses + end +end diff --git a/gem/test/solace/composers/address_lookup_table_program_extend_composer_test.rb b/gem/test/solace/composers/address_lookup_table_program_extend_composer_test.rb new file mode 100644 index 0000000..aa500b1 --- /dev/null +++ b/gem/test/solace/composers/address_lookup_table_program_extend_composer_test.rb @@ -0,0 +1,56 @@ +# frozen_string_literal: true + +require 'test_helper' + +describe Solace::Composers::AddressLookupTableProgramExtendComposer do + before(:all) do + connection = Solace::Connection.new(commitment: 'processed') + authority = Fixtures.load_keypair('bob') + recent_slot = connection.get_slot - 1 + + @addresses = [Solace::Keypair.generate.address, Solace::Keypair.generate.address] + + table, bump = Solace::Utils::PDA.find_program_address( + [authority.address, Solace::Utils::Codecs.encode_le_u64(recent_slot).bytes], + Solace::Constants::ADDRESS_LOOKUP_TABLE_PROGRAM_ID + ) + + create_tx = Solace::TransactionComposer + .new(connection: connection) + .add_instruction(Solace::Composers::AddressLookupTableProgramCreateComposer.new( + table: table, + authority: authority, + payer: authority, + recent_slot: recent_slot, + bump: bump + )) + .set_fee_payer(authority) + .compose_transaction + + create_tx.sign(authority) + create_signature = connection.send_transaction(create_tx.serialize) + connection.wait_for_confirmed_signature { create_signature['result'] } + + extend_tx = Solace::TransactionComposer + .new(connection: connection) + .add_instruction(Solace::Composers::AddressLookupTableProgramExtendComposer.new( + table: table, + authority: authority, + payer: authority, + addresses: @addresses + )) + .set_fee_payer(authority) + .compose_transaction + + extend_tx.sign(authority) + extend_signature = connection.send_transaction(extend_tx.serialize) + connection.wait_for_confirmed_signature { extend_signature['result'] } + + data = Base64.decode64(connection.get_account_info(table)['data'][0]) + @lookup_table = Solace::Accounts::AddressLookupTable.deserialize(StringIO.new(data)) + end + + it 'appends the addresses to the table in order' do + assert_equal @addresses, @lookup_table.addresses + end +end diff --git a/gem/test/solace/instructions/address_lookup_table_program/create_lookup_table_instruction_test.rb b/gem/test/solace/instructions/address_lookup_table_program/create_lookup_table_instruction_test.rb new file mode 100644 index 0000000..0c1c180 --- /dev/null +++ b/gem/test/solace/instructions/address_lookup_table_program/create_lookup_table_instruction_test.rb @@ -0,0 +1,35 @@ +# frozen_string_literal: true + +require 'test_helper' + +describe Solace::Instructions::AddressLookupTableProgram::CreateLookupTableInstruction do + describe '.build' do + let(:ix) do + Solace::Instructions::AddressLookupTableProgram::CreateLookupTableInstruction.build( + recent_slot: 123, + bump: 254, + program_index: 4, + table_index: 1, + authority_index: 0, + payer_index: 0, + system_program_index: 3 + ) + end + + it 'returns an instruction' do + assert_kind_of Solace::Instruction, ix + end + + it 'sets the program index' do + assert_equal 4, ix.program_index + end + + it 'orders the accounts [table, authority, payer, system_program]' do + assert_equal [1, 0, 0, 3], ix.accounts + end + + it 'encodes the discriminator, recent slot, and bump' do + assert_equal [0, 0, 0, 0] + [123].pack('Q<').bytes + [254], ix.data + end + end +end diff --git a/gem/test/solace/instructions/address_lookup_table_program/extend_lookup_table_instruction_test.rb b/gem/test/solace/instructions/address_lookup_table_program/extend_lookup_table_instruction_test.rb new file mode 100644 index 0000000..069338d --- /dev/null +++ b/gem/test/solace/instructions/address_lookup_table_program/extend_lookup_table_instruction_test.rb @@ -0,0 +1,42 @@ +# frozen_string_literal: true + +require 'test_helper' + +describe Solace::Instructions::AddressLookupTableProgram::ExtendLookupTableInstruction do + describe '.build' do + let(:recipient1) { Solace::Keypair.generate.address } + let(:recipient2) { Solace::Keypair.generate.address } + + let(:ix) do + Solace::Instructions::AddressLookupTableProgram::ExtendLookupTableInstruction.build( + addresses: [recipient1, recipient2], + program_index: 4, + table_index: 1, + authority_index: 0, + payer_index: 0, + system_program_index: 3 + ) + end + + it 'returns an instruction' do + assert_kind_of Solace::Instruction, ix + end + + it 'sets the program index' do + assert_equal 4, ix.program_index + end + + it 'orders the accounts [table, authority, payer, system_program]' do + assert_equal [1, 0, 0, 3], ix.accounts + end + + it 'encodes the discriminator, a u64 count, and the packed addresses' do + expected = [2, 0, 0, 0] + + [2].pack('Q<').bytes + + Solace::Utils::Codecs.base58_to_bytes(recipient1) + + Solace::Utils::Codecs.base58_to_bytes(recipient2) + + assert_equal expected, ix.data + end + end +end diff --git a/gem/test/solace/transaction_composer_test.rb b/gem/test/solace/transaction_composer_test.rb index 8572c9d..d10c0ce 100644 --- a/gem/test/solace/transaction_composer_test.rb +++ b/gem/test/solace/transaction_composer_test.rb @@ -23,7 +23,7 @@ # Test programs let(:system_program) { Solace::Constants::SYSTEM_PROGRAM_ID } - let(:spl_token_program) { Solace::Constants::SPL_TOKEN_PROGRAM_ID } + let(:spl_token_program) { Solace::Constants::TOKEN_PROGRAM_ID } # Test composers let(:transfer_composer1) do @@ -273,4 +273,399 @@ def connection.get_latest_blockhash assert_equal payer_keypair.address, tx.message.accounts[0] end end + + describe '#add_address_lookup_table' do + let(:table_account) { Solace::Keypair.generate.address } + + it 'registers the table and returns self for chaining' do + result = composer.add_address_lookup_table(account: table_account, addresses: [bob_keypair.address]) + + assert_equal composer, result + assert_equal 1, composer.address_lookup_tables.length + assert_equal table_account, composer.address_lookup_tables.first.account + assert_equal [bob_keypair.address], composer.address_lookup_tables.first.addresses + end + + it 'opts the transaction into the v0 format' do + assert_nil composer.version + + composer.add_address_lookup_table(account: table_account, addresses: []) + + assert_equal 0, composer.version + end + + it 'ignores a table already registered by account' do + composer.add_address_lookup_table(account: table_account, addresses: [bob_keypair.address]) + composer.add_address_lookup_table(account: table_account, addresses: [anna_keypair.address]) + + assert_equal 1, composer.address_lookup_tables.length + assert_equal [bob_keypair.address], composer.address_lookup_tables.first.addresses + end + end + + describe '#compose_transaction with lookup tables' do + let(:table_account) { Solace::Keypair.generate.address } + let(:mint_address) { mint_keypair.address } + let(:from_token_account) { Solace::Keypair.generate.address } + let(:to_token_account) { Solace::Keypair.generate.address } + let(:unrelated_address) { Solace::Keypair.generate.address } + + let(:transfer_checked_composer) do + Solace::Composers::SplTokenProgramTransferCheckedComposer.new( + from: from_token_account, + to: to_token_account, + mint: mint_address, + authority: anna_keypair, + amount: 1_000, + decimals: 6 + ) + end + + before do + # Mock connection to return a blockhash + def connection.get_latest_blockhash + ['EkSnNWid2cvwEVnVx9aBqawnmiCNiDgp3gUdkDPTKN1N', 1000] + end + + composer + .add_instruction(transfer_checked_composer) + .set_fee_payer(payer_keypair) + end + + describe 'when the table covers loadable accounts' do + before do + composer.add_address_lookup_table( + account: table_account, + addresses: [unrelated_address, to_token_account, mint_address, anna_keypair.address, spl_token_program] + ) + + @transaction = composer.compose_transaction + @message = @transaction.message + end + + it 'emits a v0 message' do + assert_predicate @message, :versioned? + assert_equal 0, @message.version + end + + it 'moves loadable accounts out of the static account list' do + refute_includes @message.accounts, to_token_account + refute_includes @message.accounts, mint_address + + # Writable, but not present in the table — stays static + assert_includes @message.accounts, from_token_account + end + + it 'keeps signers, the fee payer, and program ids static even when listed in the table' do + assert_equal payer_keypair.address, @message.accounts[0] + assert_includes @message.accounts, anna_keypair.address + assert_includes @message.accounts, spl_token_program + end + + it 'drops loaded readonly accounts from the readonly unsigned count' do + # payer + authority sign; of the two readonly unsigned accounts + # (mint + token program) only the program remains static + assert_equal [2, 0, 1], @message.header + end + + it 'references loaded accounts through their table positions' do + assert_equal 1, @message.address_lookup_tables.length + + table = @message.address_lookup_tables.first + + assert_equal table_account, table.account + assert_equal [1], table.writable_indexes # to_token_account + assert_equal [2], table.readonly_indexes # mint + end + + it 'resolves instruction indices against the combined v0 account space' do + combined = @message.accounts + [to_token_account, mint_address] + + instruction = @message.instructions.first + + assert_equal spl_token_program, combined[instruction.program_index] + assert_equal( + [from_token_account, mint_address, to_token_account, anna_keypair.address], + instruction.accounts.map { |index| combined[index] } + ) + end + + it 'round-trips through serialization' do + decoded = Solace::Transaction.from(@transaction.serialize).message + + assert_equal 0, decoded.version + assert_equal @message.accounts, decoded.accounts + assert_equal @message.header, decoded.header + + table = decoded.address_lookup_tables.first + + assert_equal table_account, table.account + assert_equal [1], table.writable_indexes + assert_equal [2], table.readonly_indexes + end + end + + describe 'when no table address is loadable' do + before do + composer.add_address_lookup_table( + account: table_account, + addresses: [unrelated_address, anna_keypair.address, spl_token_program] + ) + + @message = composer.compose_transaction.message + end + + it 'composes a v0 message with no table references and every account static' do + assert_equal 0, @message.version + assert_empty @message.address_lookup_tables + assert_includes @message.accounts, to_token_account + assert_includes @message.accounts, mint_address + assert_equal [2, 0, 2], @message.header + end + end + + describe 'when no lookup tables were added' do + before do + @message = composer.compose_transaction.message + end + + it 'composes a legacy message' do + refute_predicate @message, :versioned? + assert_empty @message.address_lookup_tables + end + end + end + + describe '#merge with lookup tables' do + let(:table_a) { Solace::Keypair.generate.address } + let(:table_b) { Solace::Keypair.generate.address } + + it 'folds the tables from the other composer, deduped by account' do + other = Solace::TransactionComposer.new(connection: connection) + other.add_address_lookup_table(account: table_b, addresses: [bob_keypair.address]) + other.add_address_lookup_table(account: table_a, addresses: [anna_keypair.address]) + + composer.add_address_lookup_table(account: table_a, addresses: [anna_keypair.address]) + composer.merge(other) + + assert_equal [table_a, table_b], composer.address_lookup_tables.map(&:account) + assert_equal 0, composer.version + end + end + + describe 'composing v0 transactions against the validator' do + # Land a v0 transfer of `lamports` from `from` to each recipient, loading + # through the given registered tables, and return the composed message. + def land_transfers(connection:, from:, recipients:, tables:) + composer = Solace::TransactionComposer.new(connection: connection) + + recipients.each do |recipient, lamports| + composer.add_instruction(Solace::Composers::SystemProgramTransferComposer.new( + from: from, to: recipient, lamports: lamports + )) + end + + composer.set_fee_payer(from) + tables.each { |account, addresses| composer.add_address_lookup_table(account: account, addresses: addresses) } + + transaction = composer.compose_transaction + transaction.sign(from) + + signature = connection.send_transaction(transaction.serialize) + connection.wait_for_confirmed_signature { signature['result'] } + + transaction.message + end + + describe 'loading writable recipients through a single table' do + before(:all) do + @connection = Solace::Connection.new(commitment: 'processed') + bob = Fixtures.load_keypair('bob') + @recipient1 = Solace::Keypair.generate + @recipient2 = Solace::Keypair.generate + + @table = LookupTableProvisioner.provision( + connection: @connection, authority: bob, addresses: [@recipient1.address, @recipient2.address] + ) + + @message = land_transfers( + connection: @connection, + from: bob, + recipients: { @recipient1 => 5_000_000, @recipient2 => 6_000_000 }, + tables: { @table => [@recipient1.address, @recipient2.address] } + ) + end + + it 'emits a v0 message with the recipients loaded through the table' do + assert_equal 0, @message.version + + refute_includes @message.accounts, @recipient1.address + refute_includes @message.accounts, @recipient2.address + + assert_equal [@table], @message.address_lookup_tables.map(&:account) + assert_equal [0, 1], @message.address_lookup_tables.first.writable_indexes + assert_empty @message.address_lookup_tables.first.readonly_indexes + end + + it 'credits the recipients through the loaded addresses' do + assert_equal 5_000_000, @connection.get_balance(@recipient1.address) + assert_equal 6_000_000, @connection.get_balance(@recipient2.address) + end + end + + describe 'loading accounts across multiple tables' do + before(:all) do + @connection = Solace::Connection.new(commitment: 'processed') + bob = Fixtures.load_keypair('bob') + @recipient1 = Solace::Keypair.generate + @recipient2 = Solace::Keypair.generate + + @table_a = LookupTableProvisioner.provision( + connection: @connection, authority: bob, addresses: [@recipient1.address] + ) + @table_b = LookupTableProvisioner.provision( + connection: @connection, authority: bob, addresses: [@recipient2.address] + ) + + @message = land_transfers( + connection: @connection, + from: bob, + recipients: { @recipient1 => 5_000_000, @recipient2 => 6_000_000 }, + tables: { @table_a => [@recipient1.address], @table_b => [@recipient2.address] } + ) + end + + it 'carries one reference per contributing table' do + assert_equal [@table_a, @table_b], @message.address_lookup_tables.map(&:account) + + refute_includes @message.accounts, @recipient1.address + refute_includes @message.accounts, @recipient2.address + end + + it 'credits recipients loaded from either table' do + assert_equal 5_000_000, @connection.get_balance(@recipient1.address) + assert_equal 6_000_000, @connection.get_balance(@recipient2.address) + end + end + + describe 'loading some accounts while others stay static' do + before(:all) do + @connection = Solace::Connection.new(commitment: 'processed') + bob = Fixtures.load_keypair('bob') + @loaded = Solace::Keypair.generate + @static = Solace::Keypair.generate + + # Only @loaded is stored in the table; @static is not loadable + @table = LookupTableProvisioner.provision( + connection: @connection, authority: bob, addresses: [@loaded.address] + ) + + @message = land_transfers( + connection: @connection, + from: bob, + recipients: { @loaded => 5_000_000, @static => 6_000_000 }, + tables: { @table => [@loaded.address] } + ) + end + + it 'loads the table account and keeps the untabled account static' do + refute_includes @message.accounts, @loaded.address + assert_includes @message.accounts, @static.address + end + + it 'credits both the loaded and the static recipient' do + assert_equal 5_000_000, @connection.get_balance(@loaded.address) + assert_equal 6_000_000, @connection.get_balance(@static.address) + end + end + + describe 'when an address is present in more than one table' do + before(:all) do + @connection = Solace::Connection.new(commitment: 'processed') + bob = Fixtures.load_keypair('bob') + @recipient = Solace::Keypair.generate + + @first_table = LookupTableProvisioner.provision( + connection: @connection, authority: bob, addresses: [@recipient.address] + ) + @second_table = LookupTableProvisioner.provision( + connection: @connection, authority: bob, addresses: [@recipient.address] + ) + + @message = land_transfers( + connection: @connection, + from: bob, + recipients: { @recipient => 5_000_000 }, + tables: { + @first_table => [@recipient.address], + @second_table => [@recipient.address] + } + ) + end + + it 'loads the account from the first table only' do + assert_equal [@first_table], @message.address_lookup_tables.map(&:account) + assert_equal [0], @message.address_lookup_tables.first.writable_indexes + end + + it 'credits the recipient' do + assert_equal 5_000_000, @connection.get_balance(@recipient.address) + end + end + + describe 'loading a readonly account through a table' do + before(:all) do + @connection = Solace::Connection.new(commitment: 'processed') + bob = Fixtures.load_keypair('bob') + payer = Fixtures.load_keypair('payer') + mint = Fixtures.load_keypair('mint') + + bob_ata = Solace::Programs::AssociatedTokenAccount.get_address(owner: bob, mint: mint).first + anna_ata = Solace::Programs::AssociatedTokenAccount.get_address(owner: Fixtures.load_keypair('anna'), mint: mint).first + + @mint_address = mint.address + @amount = 1_000 + + # The mint is the readonly account the transfer references — load it through a table + @table = LookupTableProvisioner.provision( + connection: @connection, authority: payer, addresses: [@mint_address] + ) + + @anna_starting_balance = @connection.get_token_account_balance(anna_ata)['amount'].to_i + + transaction = Solace::TransactionComposer + .new(connection: @connection) + .add_instruction(Solace::Composers::SplTokenProgramTransferCheckedComposer.new( + mint: mint, from: bob_ata, to: anna_ata, + authority: bob, amount: @amount, decimals: 6 + )) + .set_fee_payer(payer) + .add_address_lookup_table(account: @table, addresses: [@mint_address]) + .compose_transaction + + @message = transaction.message + transaction.sign(payer, bob) + + signature = @connection.send_transaction(transaction.serialize) + @connection.wait_for_confirmed_signature { signature['result'] } + + @anna_ending_balance = @connection.get_token_account_balance(anna_ata)['amount'].to_i + end + + it 'loads the mint as a readonly address' do + assert_equal 0, @message.version + refute_includes @message.accounts, @mint_address + + table = @message.address_lookup_tables.first + + assert_equal @table, table.account + assert_empty table.writable_indexes + assert_equal [0], table.readonly_indexes + end + + it 'settles the token transfer through the loaded mint' do + assert_equal @anna_starting_balance + @amount, @anna_ending_balance + end + end + end end diff --git a/gem/test/solace/utils/account_context_test.rb b/gem/test/solace/utils/account_context_test.rb index 320653a..2672f84 100644 --- a/gem/test/solace/utils/account_context_test.rb +++ b/gem/test/solace/utils/account_context_test.rb @@ -137,6 +137,47 @@ end end + describe '#compile with loaded accounts' do + let(:pubkey4) { keypair4.address } + let(:keypair4) { Solace::Keypair.generate } + + before do + context.set_fee_payer(keypair1) + context.add_writable_nonsigner(pubkey2) + context.add_writable_nonsigner(pubkey3) + context.add_readonly_nonsigner(pubkey4) + context.add_readonly_nonsigner(program_id) + + # pubkey3 loads writable, pubkey4 loads readonly + context.compile(loaded_accounts: [pubkey3, pubkey4]) + end + + it 'drops the loaded accounts from the static account list' do + refute_includes context.accounts, pubkey3 + refute_includes context.accounts, pubkey4 + + assert_includes context.accounts, pubkey2 + assert_includes context.accounts, program_id + end + + it 'resolves loaded accounts at the end of the combined space' do + assert_equal context.accounts.length, context.index_of(pubkey3) + assert_equal context.accounts.length + 1, context.index_of(pubkey4) + end + + it 'drops loaded readonly accounts from the readonly-unsigned count' do + # Without loading, the header would be [1, 0, 2] (pubkey4 + program_id). + # Loading pubkey4 (readonly) leaves only program_id readonly-unsigned. + assert_equal [1, 0, 1], context.header + end + + it 'keeps the header intact when only writable accounts are loaded' do + context.compile(loaded_accounts: [pubkey2, pubkey3]) + + assert_equal [1, 0, 2], context.header + end + end + describe '#merge_from' do let(:other_context) { Solace::Utils::AccountContext.new } diff --git a/gem/test/support/lookup_table_provisioner.rb b/gem/test/support/lookup_table_provisioner.rb new file mode 100644 index 0000000..2b669e9 --- /dev/null +++ b/gem/test/support/lookup_table_provisioner.rb @@ -0,0 +1,67 @@ +# frozen_string_literal: true + +# Test helper that provisions a real address lookup table on chain (create + +# extend) using the gem's own composers, then waits for it to become usable. +# Returns the table's address. Back-to-back calls land in different slots, so +# each derives a distinct table address. +module LookupTableProvisioner + extend self + + # Provision a lookup table holding the given addresses + # + # @param connection [Solace::Connection] The connection to use + # @param authority [Solace::Keypair] The table authority and rent payer + # @param addresses [Array<#to_s>] The addresses to store in the table + # @return [String] The table's on-chain address + def provision(connection:, authority:, addresses:) + recent_slot = connection.get_slot - 1 + table, bump = derive_address(authority, recent_slot) + + transaction = Solace::TransactionComposer + .new(connection: connection) + .add_instruction(create_composer(table, authority, recent_slot, bump)) + .add_instruction(extend_composer(table, authority, addresses)) + .set_fee_payer(authority) + .compose_transaction + + transaction.sign(authority) + + signature = connection.send_transaction(transaction.serialize) + connection.wait_for_confirmed_signature { signature['result'] } + + wait_for_next_slot(connection) + + table + end + + # Derive the table's program-derived address from [authority, recent_slot] + def derive_address(authority, recent_slot) + Solace::Utils::PDA.find_program_address( + [authority.address, Solace::Utils::Codecs.encode_le_u64(recent_slot).bytes], + Solace::Constants::ADDRESS_LOOKUP_TABLE_PROGRAM_ID + ) + end + + def create_composer(table, authority, recent_slot, bump) + Solace::Composers::AddressLookupTableProgramCreateComposer.new( + table: table, authority: authority, payer: authority, recent_slot: recent_slot, bump: bump + ) + end + + def extend_composer(table, authority, addresses) + Solace::Composers::AddressLookupTableProgramExtendComposer.new( + table: table, authority: authority, payer: authority, addresses: addresses + ) + end + + # A table extended in slot N becomes usable in slot N + 1 + def wait_for_next_slot(connection) + slot = connection.get_slot + + 50.times do + break if connection.get_slot > slot + + sleep 0.2 + end + end +end diff --git a/gem/test/test_helper.rb b/gem/test/test_helper.rb index fb9d8a2..a829dfd 100644 --- a/gem/test/test_helper.rb +++ b/gem/test/test_helper.rb @@ -19,4 +19,5 @@ require_relative 'support/fixtures' require_relative 'support/factory_bot' +require_relative 'support/lookup_table_provisioner' require_relative 'support/solana_test_validator' diff --git a/site/building/transaction-composer.md b/site/building/transaction-composer.md index 9b8715e..83bbce4 100644 --- a/site/building/transaction-composer.md +++ b/site/building/transaction-composer.md @@ -34,7 +34,8 @@ connection.send_transaction(tx.serialize) | `prepend_instruction(composer)` | `self` | Insert a composer at the front. | | `insert_instruction(index, composer)` | `self` | Insert at a position. | | `set_fee_payer(pubkey)` | `self` | Set the fee payer (`#to_s`); becomes account index 0. | -| `merge(other, placement: :add, index: nil)` | `self` | Merge another `TransactionComposer` (`placement:` `:add`, `:prepend`, or `:insert` with `index:`). | +| `add_address_lookup_table(account:, addresses:)` | `self` | Register an [address lookup table](/concepts/address-lookup-tables); the composed transaction becomes v0. | +| `merge(other, placement: :add, index: nil)` | `self` | Merge another `TransactionComposer` (`placement:` `:add`, `:prepend`, or `:insert` with `index:`); its tables fold in too. | | `compose_transaction` | `Solace::Transaction` | Compile accounts, fetch blockhash, build the message, return an unsigned transaction. | | Accessor | Description | @@ -42,6 +43,8 @@ connection.send_transaction(tx.serialize) | `connection` | The bound `Connection`. | | `context` | The shared `AccountContext`. | | `instruction_composers` | The composers added so far. | +| `address_lookup_tables` | The registered lookup tables (`Solace::Accounts::AddressLookupTable`). | +| `version` | The transaction version — `nil` (legacy) until a table opts it into `0` (v0). | ## Batching several instructions @@ -74,3 +77,33 @@ connection.send_transaction(tx.serialize) This is the layer to reach for when you want several instructions in one atomic transaction, or precise control over the fee payer and signing — without dropping all the way down to hand-built [messages](/concepts/transactions-and-messages). + +## Address lookup tables (v0) + +When a transaction touches more accounts than the legacy format can carry, register the +[address lookup tables](/concepts/address-lookup-tables) it may load through — each as the +table's address plus its full, ordered on-chain address list: + +```ruby +tx = Solace::TransactionComposer.new(connection:) + .add_instruction(swap_composer) + .set_fee_payer(payer.address) + .add_address_lookup_table( + account: table_address, + addresses: table_addresses + ) + .compose_transaction + +tx.message.versioned? # => true — registering a table opts into the v0 format +``` + +`compose_transaction` then emits a **v0 message**: every compiled account found in a table +that is allowed to load (a non-signer that is not the fee payer and not a program id of any +instruction) is referenced by table index instead of occupying a static account slot. +Signers, the fee payer, and program ids always stay static — those are runtime rules, not +options. Register as many tables as you like; when an address appears in several, the first +table wins. `merge` carries a merged composer's tables across too. + +Registering a table sets the composer's `version` to `0`, so the transaction stays v0 even +if nothing ends up loadable. With no tables the composer emits a legacy message, exactly as +before. diff --git a/site/concepts/account-context.md b/site/concepts/account-context.md index 24e6ae2..d53f599 100644 --- a/site/concepts/account-context.md +++ b/site/concepts/account-context.md @@ -58,6 +58,17 @@ ix = Solace::Instructions::SystemProgram::TransferInstruction.build( ) ``` +## Loaded accounts (v0) + +For a versioned (v0) transaction, `compile(loaded_accounts:)` takes the pubkeys resolved +through [address lookup tables](/concepts/address-lookup-tables). Those addresses leave the +static account list entirely — `accounts` (and therefore `Message#accounts`) keeps only the +static keys — while `index_of` still resolves them, against the combined space +`[static..., loaded...]` the runtime rebuilds at execution time. Loaded accounts also drop +out of the header. With no loaded accounts this is exactly the legacy compilation. The +[`TransactionComposer`](/building/transaction-composer#address-lookup-tables-v0) supplies +this for you when tables are registered. + ## You usually don't touch it directly The [`TransactionComposer`](/building/transaction-composer) owns an `AccountContext` diff --git a/site/concepts/address-lookup-tables.md b/site/concepts/address-lookup-tables.md index 7910d83..adc757a 100644 --- a/site/concepts/address-lookup-tables.md +++ b/site/concepts/address-lookup-tables.md @@ -42,9 +42,81 @@ ALTs serialize and deserialize through the [serialization layer](/reference/seri (`AddressLookupTable.deserialize(io)` / `#serialize`), the same path used for the rest of the wire format. +## The on-chain table account + +`Solace::Accounts::AddressLookupTable` models the table *account* itself — its address and +the full, ordered list of addresses it stores — as opposed to the per-transaction reference +above. Register one on a composer (address + addresses), or read one from chain: + +```ruby +data = Base64.decode64(connection.get_account_info(table_address)['data'][0]) +table = Solace::Accounts::AddressLookupTable.deserialize(StringIO.new(data)) + +table.authority # => the table's authority +table.addresses # => the stored addresses +``` + +## Composing v0 transactions + +You rarely build the reference by hand. Register a table on the +[`TransactionComposer`](/building/transaction-composer#address-lookup-tables-v0) and it +selects the loadable accounts, computes the indexes, and emits a v0 message for you: + +```ruby +tx = Solace::TransactionComposer.new(connection:) + .add_instruction(swap_composer) + .set_fee_payer(payer.address) + .add_address_lookup_table( + account: lookup_table_address, + addresses: on_chain_table_addresses + ) + .compose_transaction +``` + +The selection follows the runtime rules: signers, the fee payer, and instruction program +ids always stay static; everything else referenced by the transaction and present in a +table is loaded by index. + +## Creating and extending tables + +Tables are provisioned on chain with the Address Lookup Table program composers. The table +address is a program-derived address of `[authority, recent_slot]`: + +```ruby +recent_slot = connection.get_slot - 1 +table, bump = Solace::Utils::PDA.find_program_address( + [authority.address, Solace::Utils::Codecs.encode_le_u64(recent_slot).bytes], + Solace::Constants::ADDRESS_LOOKUP_TABLE_PROGRAM_ID +) + +create_composer = Solace::Composers::AddressLookupTableProgramCreateComposer.new( + table: table, + authority: authority, + payer: authority, + recent_slot: recent_slot, + bump: bump +) + +extend_composer = Solace::Composers::AddressLookupTableProgramExtendComposer.new( + table: table, + authority: authority, + payer: authority, + addresses: [address1, address2] +) + +tx = Solace::TransactionComposer.new(connection:) + .add_instruction(create_composer) + .add_instruction(extend_composer) + .set_fee_payer(authority) + .compose_transaction +``` + +Addresses added by an extend become usable one slot later. Creating a table is a separate +transaction from composing through it. + ::: tip Scope -Solace models the lookup-table **reference** inside a transaction so it can serialize and -deserialize v0 transactions that use ALTs. Building examples target legacy transactions -unless versioned features are specifically needed; see +Solace models the lookup-table **reference** and the on-chain table **account**, composes +v0 transactions through existing tables, and can create and extend tables. Freezing, +deactivating, and closing tables are not yet covered; see [Transactions & Messages](/concepts/transactions-and-messages#legacy-vs-versioned). ::: diff --git a/site/concepts/connection-and-rpc.md b/site/concepts/connection-and-rpc.md index 1a668f7..9d6617d 100644 --- a/site/concepts/connection-and-rpc.md +++ b/site/concepts/connection-and-rpc.md @@ -35,6 +35,7 @@ test suite and most examples assume. | `get_mint_program_id(mint)` | `String \| nil` | Which token program owns a mint (SPL vs. Token-2022). | | `get_minimum_lamports_for_rent_exemption(space)` | `Integer` | Rent-exempt minimum for `space` bytes. | | `get_program_accounts(program_id, filters)` | `Array` | Accounts owned by a program. | +| `get_slot(commitment:)` | `Integer` | Current slot (defaults to the connection's commitment). | | `get_version` / `get_health` / `get_genesis_hash` | varies | Node metadata. | ## Blockhash and rent From 04eecbdaedf0681fda785b01757ca0839b8f6b80 Mon Sep 17 00:00:00 2001 From: Sebastian Scholl Date: Sun, 19 Jul 2026 01:54:16 +0200 Subject: [PATCH 2/2] Assign composers to variables in ALT tests Extract inline composer instantiations out of the add_instruction chains into named local variables so the setup reads cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...okup_table_program_create_composer_test.rb | 16 +++++----- ...okup_table_program_extend_composer_test.rb | 30 +++++++++++-------- gem/test/solace/transaction_composer_test.rb | 17 ++++++----- 3 files changed, 36 insertions(+), 27 deletions(-) diff --git a/gem/test/solace/composers/address_lookup_table_program_create_composer_test.rb b/gem/test/solace/composers/address_lookup_table_program_create_composer_test.rb index 37cc3d0..0349c72 100644 --- a/gem/test/solace/composers/address_lookup_table_program_create_composer_test.rb +++ b/gem/test/solace/composers/address_lookup_table_program_create_composer_test.rb @@ -13,15 +13,17 @@ Solace::Constants::ADDRESS_LOOKUP_TABLE_PROGRAM_ID ) + create_composer = Solace::Composers::AddressLookupTableProgramCreateComposer.new( + table: @table, + authority: @authority, + payer: @authority, + recent_slot: recent_slot, + bump: bump + ) + tx = Solace::TransactionComposer .new(connection: connection) - .add_instruction(Solace::Composers::AddressLookupTableProgramCreateComposer.new( - table: @table, - authority: @authority, - payer: @authority, - recent_slot: recent_slot, - bump: bump - )) + .add_instruction(create_composer) .set_fee_payer(@authority) .compose_transaction diff --git a/gem/test/solace/composers/address_lookup_table_program_extend_composer_test.rb b/gem/test/solace/composers/address_lookup_table_program_extend_composer_test.rb index aa500b1..e910ec4 100644 --- a/gem/test/solace/composers/address_lookup_table_program_extend_composer_test.rb +++ b/gem/test/solace/composers/address_lookup_table_program_extend_composer_test.rb @@ -15,15 +15,17 @@ Solace::Constants::ADDRESS_LOOKUP_TABLE_PROGRAM_ID ) + create_composer = Solace::Composers::AddressLookupTableProgramCreateComposer.new( + table: table, + authority: authority, + payer: authority, + recent_slot: recent_slot, + bump: bump + ) + create_tx = Solace::TransactionComposer .new(connection: connection) - .add_instruction(Solace::Composers::AddressLookupTableProgramCreateComposer.new( - table: table, - authority: authority, - payer: authority, - recent_slot: recent_slot, - bump: bump - )) + .add_instruction(create_composer) .set_fee_payer(authority) .compose_transaction @@ -31,14 +33,16 @@ create_signature = connection.send_transaction(create_tx.serialize) connection.wait_for_confirmed_signature { create_signature['result'] } + extend_composer = Solace::Composers::AddressLookupTableProgramExtendComposer.new( + table: table, + authority: authority, + payer: authority, + addresses: @addresses + ) + extend_tx = Solace::TransactionComposer .new(connection: connection) - .add_instruction(Solace::Composers::AddressLookupTableProgramExtendComposer.new( - table: table, - authority: authority, - payer: authority, - addresses: @addresses - )) + .add_instruction(extend_composer) .set_fee_payer(authority) .compose_transaction diff --git a/gem/test/solace/transaction_composer_test.rb b/gem/test/solace/transaction_composer_test.rb index d10c0ce..b0d17bb 100644 --- a/gem/test/solace/transaction_composer_test.rb +++ b/gem/test/solace/transaction_composer_test.rb @@ -460,9 +460,11 @@ def land_transfers(connection:, from:, recipients:, tables:) composer = Solace::TransactionComposer.new(connection: connection) recipients.each do |recipient, lamports| - composer.add_instruction(Solace::Composers::SystemProgramTransferComposer.new( - from: from, to: recipient, lamports: lamports - )) + transfer = Solace::Composers::SystemProgramTransferComposer.new( + from: from, to: recipient, lamports: lamports + ) + + composer.add_instruction(transfer) end composer.set_fee_payer(from) @@ -633,12 +635,13 @@ def land_transfers(connection:, from:, recipients:, tables:) @anna_starting_balance = @connection.get_token_account_balance(anna_ata)['amount'].to_i + transfer_composer = Solace::Composers::SplTokenProgramTransferCheckedComposer.new( + mint: mint, from: bob_ata, to: anna_ata, authority: bob, amount: @amount, decimals: 6 + ) + transaction = Solace::TransactionComposer .new(connection: @connection) - .add_instruction(Solace::Composers::SplTokenProgramTransferCheckedComposer.new( - mint: mint, from: bob_ata, to: anna_ata, - authority: bob, amount: @amount, decimals: 6 - )) + .add_instruction(transfer_composer) .set_fee_payer(payer) .add_address_lookup_table(account: @table, addresses: [@mint_address]) .compose_transaction