Skip to content

Commit f8f01ab

Browse files
AIQnetLabclaude
andcommitted
apply: admit BatchTransfers blocks, warm the block's accounts in one multi_get
The batch run proved the whole signed path — 892 transfers/s included on-chain — and then wedged the network: the block-level internal-type guard still listed BatchTransfers among the genesis-only/deprecated variants and hard-rejected any block carrying one. The producer admitted batches through the (opened) mempool gates and included them; every validator rejected the block. Admission, gossip, value-class, validate and apply had all been opened — this fifth site had not. BatchTransfers leaves the forbidden list; CreateAccount, BatchRewardClaims and BatchNodeActivations stay rejected. Every remaining reference to the type across the codebase is now accounted for. Block-apply account warm-up goes through ONE RocksDB multi_get instead of one point read per address. The block already collects and dedupes every affected address (batch recipients included); only the miss path changes — resident accounts keep the exact hit/touch semantics. Measured 240-tx blocks spent 543-851 ms in apply, consistent with hundreds of sequential cold point reads on VPS disks; AccountStore grows load_accounts_batch with a sequential default so non-RocksDB stores are untouched. Ladder on the previous image, for the record: 250 TPS 13433/13433 finalized and 500 TPS 22913/22913 finalized (382 sustained), 100% success both — the loads that previously killed the network. Tests: 420 integration + 176 state, zero warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent bd6a597 commit f8f01ab

4 files changed

Lines changed: 45 additions & 6 deletions

File tree

core/qnet-state/src/state.rs

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1700,6 +1700,14 @@ pub trait AccountStore: Send + Sync {
17001700
/// the canonical "account not found" path).
17011701
fn load_account(&self, address: &str) -> Option<Account>;
17021702

1703+
/// Batched best-effort read; result order matches `addresses`. Default is N
1704+
/// single reads so simple stores stay correct; the RocksDB store overrides
1705+
/// this with one multi_get — the block-apply warm was N sequential point
1706+
/// reads, the dominant cost of applying a loaded block.
1707+
fn load_accounts_batch(&self, addresses: &[String]) -> Vec<Option<Account>> {
1708+
addresses.iter().map(|a| self.load_account(a)).collect()
1709+
}
1710+
17031711
/// Durable batch write of accounts. Called by the eviction sweep BEFORE dropping entries from the
17041712
/// cache; returns true IFF the write durably succeeded. The evictor removes ONLY a successfully-
17051713
/// persisted batch, so a failed persist (I/O error, or no store) keeps the accounts resident —
@@ -1921,11 +1929,37 @@ impl StateManager {
19211929
/// to ensure every sender / receiver / contract address the block
19221930
/// touches is resident before the apply mutates state. Returns the
19231931
/// count of addresses that ended up resident (cache hit + disk hit).
1932+
/// Misses are loaded in ONE multi_get instead of per-address point reads.
19241933
pub fn warm_accounts(&self, addresses: &[String]) -> usize {
19251934
let mut hit = 0usize;
1935+
let mut misses: Vec<String> = Vec::new();
19261936
for addr in addresses {
1927-
if self.warm_account(addr) {
1937+
if self.accounts.contains_key(addr) {
1938+
self.cache_hits.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
1939+
self.touch_access(addr);
19281940
hit = hit.saturating_add(1);
1941+
} else {
1942+
self.cache_misses.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
1943+
misses.push(addr.clone());
1944+
}
1945+
}
1946+
if misses.is_empty() { return hit; }
1947+
let store_guard = self.disk_store.read();
1948+
if let Some(ref store) = *store_guard {
1949+
for (addr, loaded) in misses.iter().zip(store.load_accounts_batch(&misses)) {
1950+
match loaded {
1951+
Some(account) => {
1952+
self.disk_load_hits.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
1953+
// Race-tolerant: keep an entry inserted concurrently — identical
1954+
// bytes, load_accounts_batch is deterministic on the same store.
1955+
self.accounts.entry(addr.clone()).or_insert(account);
1956+
self.touch_access(addr);
1957+
hit = hit.saturating_add(1);
1958+
}
1959+
None => {
1960+
self.disk_load_misses.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
1961+
}
1962+
}
19291963
}
19301964
}
19311965
hit

development/qnet-integration/src/block_pipeline.rs

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2644,15 +2644,15 @@ impl BlockPipeline {
26442644
// Internal-only TX type guard: post-genesis, HARD REJECT the whole
26452645
// block (+ peer reputation penalty) if it carries a genesis-only or
26462646
// deprecated variant (CreateAccount / BatchRewardClaims /
2647-
// BatchNodeActivations / BatchTransfers) — a Byzantine producer
2648-
// could embed one bypassing mempool admission. O(tx_count).
2647+
// BatchNodeActivations) — a Byzantine producer could embed one
2648+
// bypassing mempool admission. BatchTransfers is a live signed value
2649+
// class (bounds + ML-DSA gate run at verify/apply). O(tx_count).
26492650
if mb.height > 0 {
26502651
for tx in &decoded.microblock.transactions {
26512652
let forbidden = matches!(tx.tx_type,
26522653
qnet_state::TransactionType::CreateAccount { .. } |
26532654
qnet_state::TransactionType::BatchRewardClaims { .. } |
2654-
qnet_state::TransactionType::BatchNodeActivations { .. } |
2655-
qnet_state::TransactionType::BatchTransfers { .. }
2655+
qnet_state::TransactionType::BatchNodeActivations { .. }
26562656
);
26572657
if forbidden {
26582658
if is_warn() {

development/qnet-integration/src/node/transactions.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1588,7 +1588,7 @@ impl BlockchainNode {
15881588
// Any post-genesis CreateAccount is an attack.
15891589
// * BatchRewardClaims — DEPRECATED enum variant, never instantiated.
15901590
// * BatchNodeActivations — DEPRECATED enum variant, never instantiated.
1591-
// * BatchTransfers — UNUSED handler-only enum variant.
1591+
// * BatchTransfers — LIVE signed value class; bounds checked below.
15921592
//
15931593
// SCALABILITY: O(1) match per gossip TX. Identical cost at 5 or 5000
15941594
// validators — no cross-node coordination, purely local check.

development/qnet-integration/src/storage/mod.rs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1800,6 +1800,11 @@ impl qnet_state::AccountStore for Storage {
18001800
}
18011801
}
18021802

1803+
fn load_accounts_batch(&self, addresses: &[String]) -> Vec<Option<qnet_state::Account>> {
1804+
// One RocksDB multi_get over the accounts CF (reward_store.rs).
1805+
Storage::load_accounts_batch(self, addresses)
1806+
}
1807+
18031808
fn persist_accounts(&self, accounts: &[(String, qnet_state::Account)]) -> bool {
18041809
match self.persistent.persist_accounts_sync(accounts) {
18051810
Ok(_) => true,

0 commit comments

Comments
 (0)