Fix secondary index scope isolation and harden chain_plugin

docs/kv-ram-billing.md

@@ -28,19 +28,22 @@ shared memory). Key data sizes are billed separately at billing time.
 | Object | Fixed Fields | Indices | Overhead | Billable |
 |--------|-------------|---------|----------|----------|
 | `kv_object` | 8 id + 8 code + 8 payer + 8 key (offset_ptr) + 8 value (offset_ptr) + 1 key_format + 7 padding = 48 | 2 | 64 | **112** + key_size + value_size |
-| `kv_index_object` | 8 id + 8 code + 8 payer + 8 table + 8 sec_key (offset_ptr) + 8 pri_key (offset_ptr) + 1 index_id + 7 padding = 56 | 3 | 96 | **152** + sec_key_size + pri_key_size |
+| `kv_index_object` | 8 id + 8 code + 8 payer + 8 table + 8 sec_key (offset_ptr) + 8 pri_key (offset_ptr) + 1 index_id + 7 padding = 56 | 2 | 64 | **120** + sec_key_size + pri_key_size |
 
 ## KV Key Formats
 
 KV supports two key formats with different RAM trade-offs:
 
-- **Format 1 (standard):** `kv_multi_index` uses 24-byte keys: `[table:8B][scope:8B][pk:8B]`.
-  Embeds legacy table/scope routing into the key. Compatible with SHiP table
-  translation and existing tooling.
+- **Format 1 (standard):** `kv_multi_index` uses 24-byte primary keys:
+  `[table:8B][scope:8B][pk:8B]`. Secondary index entries store
+  `pri_key = [pk:8B]` and `sec_key = [scope:8B][secondary_value]` so that
+  secondary iteration is naturally scoped without chain-side scope parameters.
 
-- **Format 0 (raw):** `kv::raw_table` uses contract-defined key structs,
-  BE-encoded for correct sort order. Key size is controlled by the contract.
-  A uint64 primary key is just 8 bytes. This is the most RAM-efficient option.
+- **Format 0 (raw):** `kv::raw_table` / `kv::indexed_table` use contract-defined
+  key structs, BE-encoded for correct sort order. Key size is controlled by the
+  contract. A uint64 primary key is just 8 bytes. This is the most RAM-efficient
+  option. Secondary indices use `TableName` for table-level isolation; contracts
+  use key discriminators for further partitioning.
 
 ## Per-Row Comparisons (16-byte value, no secondary index)
 
@@ -67,49 +70,65 @@ KV supports two key formats with different RAM trade-offs:
 | Approach | Primary | Secondary | Total |
 |----------|---------|-----------|-------|
 | Legacy | 124 + 108 tid | 128 | **360** |
-| kv_multi_index (fmt=1) | 152 | 152+8+24=184 | **336** (-7%) |
-| kv::raw_table (fmt=0, 8B key) | 136 | 152+8+8=168 | **304** (-16%) |
+| kv_multi_index (fmt=1) | 152 | 120+16+8=144 | **296** (-18%) |
+| kv::raw_table (fmt=0, 8B key) | 136 | 120+8+8=136 | **272** (-24%) |
 
 ### Many rows per table
 
 | Approach | Primary | Secondary | Total |
 |----------|---------|-----------|-------|
 | Legacy | 124 | 128 | **252** |
-| kv_multi_index (fmt=1) | 152 | 184 | **336** (+33%) |
-| kv::raw_table (fmt=0, 8B key) | 136 | 168 | **304** (+21%) |
+| kv_multi_index (fmt=1) | 152 | 144 | **296** (+17%) |
+| kv::raw_table (fmt=0, 8B key) | 136 | 136 | **272** (+8%) |
 
-Note: with raw_table, the `pri_key` stored in `kv_index_object` is also compact
-(8 bytes) vs 24 bytes for standard format. This compounds the savings per
-secondary index row.
+Note: `kv_index_object` has 2 chainbase indices (by_id, by_code_table_idx_seckey).
+For format 1, `sec_key = [scope:8B][secondary_value]` and `pri_key = [pk:8B]`.
+For format 0, both `sec_key` and `pri_key` are contract-defined compact keys.
 
-## EOS Mainnet Estimate (excluding xsat, ~103M primary rows)
+## EOS Mainnet Estimate
 
-Estimated using an EOS v8 snapshot with xsat Bitcoin UTXO data excluded
-(xsat alone has 345M index256 rows that dominate the dataset).
+Estimated using EOS v8 snapshot (`snapshot-2026-03-10-19-eos-v8-0487758850.bin`).
+Secondary index counts: 13.3M idx64, 3.9M idx128, 355.6M idx256 (345M from xsat),
+47K idx_double, 5K idx_long_double.
+
+### Excluding xsat (~103M primary rows, ~28M secondary entries)
 
 | Approach | Total | vs Legacy |
 |----------|-------|-----------|
 | Legacy | 14.4 GB | baseline |
-| kv_multi_index (fmt=1) | 16.0 GB | **+11%** |
-| kv::raw_table (fmt=0, ~10B avg key) | 14.7 GB | **+2%** |
+| kv_multi_index (fmt=1) | 15.1 GB | **+5%** |
+| kv::raw_table (fmt=0, ~10B avg key) | 13.8 GB | **-4%** |
+
+### Including xsat (~262M primary rows, ~373M secondary entries)
+
+Secondary index storage alone (primary rows unchanged by this PR):
+
+| Approach | Secondary Only | vs Legacy |
+|----------|---------------|-----------|
+| Legacy | 56.3 GB | baseline |
+| kv_multi_index (fmt=1) | 62.3 GB | **+11%** |
+| kv::raw_table (fmt=0) | 59.3 GB | **+5%** |
 
-The +11% for `kv_multi_index` comes from 24-byte standard keys replacing
-legacy's 8-byte fixed `uint64_t primary_key`. With `kv::raw_table`, compact
-keys eliminate this gap.
+The `kv_multi_index` premium comes from 24-byte standard keys replacing
+legacy's 8-byte fixed `uint64_t primary_key` and scope-prefixed secondary keys.
+With `kv::raw_table`, compact keys reduce this gap. Dropping the unused
+`by_code_table_idx_prikey` index saves 32 bytes per secondary entry (11.9 GB
+across all 373M entries including xsat).
 
 ## Summary
 
 | Scenario | Legacy | kv_multi_index | kv::raw_table |
 |----------|--------|----------------|---------------|
 | Token balance (1 scope/row, no idx) | 232 | 152 (-34%) | 144 (-38%) |
-| Token balance + 1 idx (1 scope/row) | 360 | 336 (-7%) | 304 (-16%) |
+| Token balance + 1 idx (1 scope/row) | 360 | 296 (-18%) | 272 (-24%) |
 | Dense table (no idx) | 124 | 152 (+23%) | 136 (+10%) |
-| Dense table + 1 idx | 252 | 336 (+33%) | 304 (+21%) |
-| EOS mainnet (no xsat) | 14.4 GB | 16.0 GB (+11%) | 14.7 GB (+2%) |
-
-**Bottom line:** `kv::raw_table` with compact keys achieves near-parity with
-legacy (+2% on EOS-scale data) while providing variable-length key flexibility.
-`kv_multi_index` pays a ~11-33% premium for the convenience of embedding
-table/scope routing in 24-byte standard keys. Contracts without secondary
-indices and with many scopes (like sysio.token) save 34-38% regardless of
-format, thanks to `table_id_object` elimination.
+| Dense table + 1 idx | 252 | 296 (+17%) | 272 (+8%) |
+
+**Bottom line:** Dropping the unused `by_code_table_idx_prikey` chainbase index
+reduced `kv_index_object` overhead from 152 to 120 bytes per secondary entry,
+significantly improving secondary index RAM costs. `kv::raw_table` with compact
+keys now saves 8-24% vs legacy for tables with secondary indices.
+`kv_multi_index` saves 18% for scoped patterns and adds only 17% for dense
+tables — a major improvement from the previous 33% premium. Contracts without
+secondary indices and with many scopes (like sysio.token) save 34-38%
+regardless of format, thanks to `table_id_object` elimination.

libraries/chain/include/sysio/chain/kv_table_objects.hpp

@@ -136,7 +136,6 @@ namespace sysio { namespace chain {
    };
 
    struct by_code_table_idx_seckey;
-   struct by_code_table_idx_prikey;
 
    using kv_index_index = chainbase::shared_multi_index_container<
       kv_index_object,
@@ -154,16 +153,6 @@ namespace sysio { namespace chain {
             >,
             composite_key_compare<std::less<account_name>, std::less<name>, std::less<uint8_t>,
                                   kv_key_less, kv_key_less>
-         >,
-         ordered_unique<tag<by_code_table_idx_prikey>,
-            composite_key<kv_index_object,
-               member<kv_index_object, account_name,  &kv_index_object::code>,
-               member<kv_index_object, name,           &kv_index_object::table>,
-               member<kv_index_object, uint8_t,        &kv_index_object::index_id>,
-               kv_pri_key_extractor
-            >,
-            composite_key_compare<std::less<account_name>, std::less<name>, std::less<uint8_t>,
-                                  kv_key_less>
          >
       >
    >;
@@ -180,7 +169,7 @@ namespace config {
 
    template<>
    struct billable_size<kv_index_object> {
-      static const uint64_t overhead = overhead_per_row_per_index_ram_bytes * 3;  ///< 3 indices: by_id, by_code_table_idx_seckey, by_code_table_idx_prikey
+      static const uint64_t overhead = overhead_per_row_per_index_ram_bytes * 2;  ///< 2 indices: by_id, by_code_table_idx_seckey
       // Fixed fields: 8 id + 8 code + 8 payer + 8 table + 8 sec_key (offset_ptr)
       //             + 8 pri_key (offset_ptr) + 1 index_id + 7 padding = 56
       // sec_key.size() and pri_key.size() are added separately at billing time.

libraries/chain/include/sysio/chain/types.hpp

@@ -96,9 +96,11 @@ namespace sysio::chain {
    }
 
    /// Standard KV key sizes (bytes)
-   inline constexpr size_t kv_key_size        = 24;  ///< [table:8B][scope:8B][pk:8B]
-   inline constexpr size_t kv_prefix_size     = 16;  ///< [table:8B][scope:8B]
-   inline constexpr size_t kv_table_prefix_size = 8; ///< [table:8B]
+   inline constexpr size_t kv_key_size          = 24; ///< [table:8B][scope:8B][pk:8B]
+   inline constexpr size_t kv_prefix_size       = 16; ///< [table:8B][scope:8B]
+   inline constexpr size_t kv_table_prefix_size = 8;  ///< [table:8B]
+   inline constexpr size_t kv_scope_prefix_size = kv_table_prefix_size; ///< [scope:8B] prefix in sec_key
+   inline constexpr size_t kv_pri_key_size      = sizeof(uint64_t); ///< [pk:8B] stored in kv_index_object
 
    /// Build a 24-byte KV key: [table:8B BE][scope:8B BE][pk:8B BE]
    struct kv_key_t {

plugins/chain_plugin/src/chain_plugin.cpp

@@ -1801,15 +1801,31 @@ read_only::get_table_rows_by_seckey( const read_only::get_table_rows_params& p,
    // Encode bounds
    auto lb_bytes = encode_sec_bound(p.key_type, p.lower_bound, p.encode_type);
    auto ub_bytes = encode_sec_bound(p.key_type, p.upper_bound, p.encode_type);
-   auto lb_sv = std::string_view(lb_bytes.data(), lb_bytes.size());
-   auto ub_sv = std::string_view(ub_bytes.data(), ub_bytes.size());
    bool has_lower = !lb_bytes.empty();
    bool has_upper = !ub_bytes.empty();
 
-   // CDT's kv_multi_index encodes secondary pri_key as [scope:8B BE][pk:8B BE]
-   char scope_be[chain::kv_table_prefix_size];
+   // CDT's kv_multi_index encodes sec_key as [scope:8B BE][secondary_value]
+   // and pri_key as [pk:8B BE].  Prepend scope to search bounds so that
+   // B-tree iteration is naturally scoped.
+   char scope_be[chain::kv_scope_prefix_size];
    chain::kv_encode_be64(scope_be, scope);
 
+   auto prepend_scope = [&](const vector<char>& bound) -> vector<char> {
+      vector<char> scoped(chain::kv_scope_prefix_size + bound.size());
+      memcpy(scoped.data(), scope_be, chain::kv_scope_prefix_size);
+      if (!bound.empty())
+         memcpy(scoped.data() + chain::kv_scope_prefix_size, bound.data(), bound.size());
+      return scoped;
+   };
+
+   // If lower bound specified, scope-prefix it; otherwise use bare scope prefix
+   // to position at the first entry in this scope.
+   auto scoped_lb = has_lower ? prepend_scope(lb_bytes)
+                              : vector<char>(scope_be, scope_be + chain::kv_scope_prefix_size);
+   auto scoped_ub = has_upper ? prepend_scope(ub_bytes) : vector<char>();
+   auto lb_sv = std::string_view(scoped_lb.data(), scoped_lb.size());
+   auto ub_sv = has_upper ? std::string_view(scoped_ub.data(), scoped_ub.size()) : std::string_view();
+
    const auto& sec_idx = d.get_index<chain::kv_index_index, chain::by_code_table_idx_seckey>();
 
    uint32_t limit = p.limit;
@@ -1832,31 +1848,38 @@ read_only::get_table_rows_by_seckey( const read_only::get_table_rows_params& p,
    };
 
    auto collect_next = [&](const chain::kv_index_object& obj) {
+      // Validate sec_key has scope prefix and matches our scope
+      if (obj.sec_key.size() < chain::kv_scope_prefix_size ||
+          memcmp(obj.sec_key.data(), scope_be, chain::kv_scope_prefix_size) != 0)
+         return;
       hp.more = true;
-      hp.next_key = decode_sec_key(p.key_type, obj.sec_key.data(), obj.sec_key.size(), p.encode_type);
+      // sec_key is [scope:8B][value] — skip scope prefix for the API response
+      const char* sec_data = obj.sec_key.data() + chain::kv_scope_prefix_size;
+      size_t      sec_size = obj.sec_key.size() - chain::kv_scope_prefix_size;
+      hp.next_key = decode_sec_key(p.key_type, sec_data, sec_size, p.encode_type);
    };
 
    bool reverse = p.reverse && *p.reverse;
 
    if (!reverse) {
-      // Forward iteration
-      auto itr = has_lower
-         ? sec_idx.lower_bound(boost::make_tuple(p.code, p.table, index_id, lb_sv))
-         : sec_idx.lower_bound(boost::make_tuple(p.code, p.table, index_id));
+      // Forward iteration — lb_sv already includes [scope:8B] prefix
+      auto itr = sec_idx.lower_bound(boost::make_tuple(p.code, p.table, index_id, lb_sv));
       uint32_t count = 0;
       for (; itr != sec_idx.end(); ++itr) {
          if (itr->code != p.code || itr->table != p.table || itr->index_id != index_id)
             break;
-         // Upper bound check on secondary key
+         // Scope boundary: sec_key is [scope:8B][value]; entries are sorted,
+         // so once scope prefix differs we're past this scope.
+         if (itr->sec_key.size() < chain::kv_scope_prefix_size ||
+             memcmp(itr->sec_key.data(), scope_be, chain::kv_scope_prefix_size) != 0)
+            break;
+         // Upper bound check on scoped secondary key
          if (has_upper && itr->sec_key_view() > ub_sv)
             break;
-         // Scope filter: first 8 bytes of pri_key must match scope
-         if (itr->pri_key.size() < chain::kv_prefix_size) continue;
-         if (memcmp(itr->pri_key.data(), scope_be, chain::kv_table_prefix_size) != 0)
-            continue;
          if (count >= limit) { collect_next(*itr); break; }
-         // Extract primary key (bytes 8-15 of pri_key)
-         uint64_t pk = chain::kv_decode_be64(itr->pri_key.data() + chain::kv_table_prefix_size);
+         // pri_key is [pk:8B]
+         if (itr->pri_key.size() < chain::kv_pri_key_size) continue;
+         uint64_t pk = chain::kv_decode_be64(itr->pri_key.data());
          auto [data, payer] = fetch_value(pk);
          if (!data.empty()) {
             hp.rows.emplace_back(std::move(data), payer);
@@ -1870,25 +1893,35 @@ read_only::get_table_rows_by_seckey( const read_only::get_table_rows_params& p,
          }
       }
    } else {
-      // Reverse iteration
-      auto end_itr = has_upper
-         ? sec_idx.upper_bound(boost::make_tuple(p.code, p.table, index_id, ub_sv))
-         : sec_idx.upper_bound(boost::make_tuple(p.code, p.table, index_id));
-      auto begin_itr = has_lower
-         ? sec_idx.lower_bound(boost::make_tuple(p.code, p.table, index_id, lb_sv))
-         : sec_idx.lower_bound(boost::make_tuple(p.code, p.table, index_id));
+      // Reverse iteration — bounds already include [scope:8B] prefix.
+      // For no-upper-bound, position past the last entry in this scope
+      // by using scope+1 as the exclusive upper bound prefix.
+      auto end_itr = [&]() {
+         if (has_upper)
+            return sec_idx.upper_bound(boost::make_tuple(p.code, p.table, index_id, ub_sv));
+         if (scope < std::numeric_limits<uint64_t>::max()) {
+            char scope_next_be[chain::kv_scope_prefix_size];
+            chain::kv_encode_be64(scope_next_be, scope + 1);
+            return sec_idx.lower_bound(boost::make_tuple(p.code, p.table, index_id, std::string_view(scope_next_be, chain::kv_scope_prefix_size)));
+         }
+         return sec_idx.upper_bound(boost::make_tuple(p.code, p.table, index_id));
+      }();
+      auto begin_itr = sec_idx.lower_bound(boost::make_tuple(p.code, p.table, index_id, lb_sv));
       auto ritr = boost::make_reverse_iterator(end_itr);
       auto rend = boost::make_reverse_iterator(begin_itr);
       uint32_t count = 0;
       for (; ritr != rend; ++ritr) {
          if (ritr->code != p.code || ritr->table != p.table || ritr->index_id != index_id)
             break;
-         // Scope filter
-         if (ritr->pri_key.size() < chain::kv_prefix_size) continue;
-         if (memcmp(ritr->pri_key.data(), scope_be, chain::kv_table_prefix_size) != 0)
-            continue;
+         // Scope boundary: sec_key is [scope:8B][value]; entries are sorted,
+         // so once scope prefix differs we're past this scope.
+         if (ritr->sec_key.size() < chain::kv_scope_prefix_size ||
+             memcmp(ritr->sec_key.data(), scope_be, chain::kv_scope_prefix_size) != 0)
+            break;
          if (count >= limit) { collect_next(*ritr); break; }
-         uint64_t pk = chain::kv_decode_be64(ritr->pri_key.data() + chain::kv_table_prefix_size);
+         // pri_key is [pk:8B]
+         if (ritr->pri_key.size() < chain::kv_pri_key_size) continue;
+         uint64_t pk = chain::kv_decode_be64(ritr->pri_key.data());
          auto [data, payer] = fetch_value(pk);
          if (!data.empty()) {
             hp.rows.emplace_back(std::move(data), payer);

tests/sysio_util_snapshot_info_test.py

@@ -16,7 +16,7 @@
         "result": {
             "version": 1,
             "chain_id": "144035215e20fd016e2b4b065349c959a1070fcbb0dc3f4784f3130685e774fc",
-            "head_block_id": "0000001db8ed54f83aa97ed6112a0f39568705ead8d2c50ffe4c4c28e193aab3",
+            "head_block_id": "0000001d6e07189b77f366197e2fe3b797323bc51967d3eff34379c8ea24cd2b",
             "head_block_num": 29,
             "head_block_time": "2025-01-01T00:00:14.000"
         }