Reindex optimizations 1 - #240
Draft
techcoderx wants to merge 72 commits into
Draft
Conversation
The reindex ChainActiveAt callback queried electionDb before its Mongo collection was bound (the elections plugin inits after the reindex gate), nil-dereferencing and crash-looping every node on its first restart; it now reads the elections collection directly off the connected DbInstance via a new elections.ChainActiveVersionAt helper. Corrected the consensus-version devnet test's "0.3" vs "0.3.0" target comparison and made candidate visibility race-robust by latching on adoption. Added TestConsensusVersionLateAdoptionReindexDevnet, which takes a node offline through a rollout to cover both the restart/catch-up path and the seeded version-lag full reindex.
…/F21/F22) Fix five deterministic, network-wide chain halts. Each panics before the per-tx recover (executeTxSafely), so it escapes to the block-listener's coarse recover and permanently stops the block pipeline on every node: - F4: an unknown offchain op.Type left a nil VSCTransaction that was dereferenced in Ingest/ExecuteBatch. Unprivileged (any funded user via submitTransactionV1), devnet-confirmed. - F21: a known op with an undecodable, attacker-controlled CBOR payload nil-dereferenced node.MarshalJSON in DecodeTxCbor. - F9: a dropped bls.AggregatePubkeys error fed a nil/invalid pubkey into bls.Verify and panicked. - F13: an empty witness list divided by zero in GenerateSchedule. - F22: a zero-op tx indexed Ops[0] out of range in the block signer path. F4/F21 are handled together: ToTransaction now returns ([]VSCTransaction, error) and errors on the first unknown type or undecodable payload instead of dropping ops or relying on zero-value rejection. An invalid tx fails ATOMICALLY -- no op executes, a fixed 50 RC is charged, and the whole tx is marked FAILED via the same oplog finalization as every other tx (new TxPacket.Invalid branch in ExecuteBatch).
…4-3) An incoming election whose new committee is below MinMembers (e.g. empty) would persist and then drive consensus.GenerateSchedule into witnessList[slot % 0] -- an integer divide-by-zero that panic-halts every validating node. Reject it in TxElectionResult.ExecuteTx before StoreElection so the prior committee stays in charge and the chain keeps producing. Gated on the 0.4.0 consensus line (MinMembersGuardActive), resolved from the version active at the election's submit height (ActiveConsensusVersion(tx.Self.BlockHeight)) -- deterministic on-chain and replay-safe (mainnet history has valid pre-raise 7-member elections). Keep the pure GenerateSchedule empty-list backstop (already on develop as the F13 fix) as defense-in-depth, and log invalid block-proposal rejections symmetrically with the skip/stale cases.
The offchain unstake_hbd op built TxStakeHbd (identical to stake_hbd), so it STAKED funds instead of releasing them -- the wrong direction. Build the dedicated TxUnstakeHbd (ledgerSession.Unstake) instead. The L1 vsc.unstake_hbd path was already correct; only the offchain path was wrong. Correcting the direction changes ledger state on a VALID input, so it is version-gated on the 0.4.0. chain-active version is threaded into OffchainTransaction.ToTransaction via ActiveConsensusVersion at the op's anchored height (same height in Ingest and ExecuteBatch), so a full reindex reproduces historical ledger state and the fix flips network-wide only at the 0.4.0 floor.
Lets any account delegate consensus stake to any node and ALWAYS reclaim it itself, while the node operator can never touch a delegator's stake. Fixes the prior model where the bond pooled onto the node (to) account and only the hive_consensus holder could unstake, so delegators (hive_consensus = 0) were stuck and operators could drain delegated bond. Design: a per-edge delegation balance modelled as a new non-transferable "delegation" asset keyed by composite owner "from::to", which reuses the entire GetBalance snapshot/session/cache machinery. The node's aggregate hive_consensus is left UNTOUCHED, so election weight and pendulum effective bond are unchanged. - consensusversion: bump source 0.4.0 -> 0.5.0; gate via StateEngine.delegatedStakeActive / DelegatedStakeActiveForElection (chain-active election version, deterministic). Pre-0.2.0 behaviour is byte-identical (legacy hive_consensus-holder unstake path retained). - stake: records a from->to edge (LedgerUpdate, asset "delegation") in addition to the unchanged hive debit + node hive_consensus credit. - unstake (delegated): authorize against the SIGNER's edge (operator has no edge to a delegator's stake), debit the NODE's hive_consensus, decrement the edge, and queue the HIVE return to the delegator (action.To = from). - gate threaded via ConsensusParams.Delegated -> stamped into OpLogEvent so the pure record builder (ExecuteOplog) stays deterministic. - migration: BackfillDelegationEdges deterministically reconstructs edges from historical stake #in/#out pairs (the from->to link is otherwise unrecorded), so already-staked users become reclaimable at activation. - slashing: applySlashHaircut is a documented no-op stub returning the full edge (OPEN Q1 — loss attribution undecided; do not enable consensus slashing on a delegated network until resolved). Tests: delegated entitlement invariant (operator cannot unstake a delegator's stake; delegator can; over-unstake rejected; edge drains), legacy path unchanged, and migration edge reconstruction + idempotency. Builds clean; pre-existing Review2/wasm/oracle-price failures are unrelated. Remaining (not in this commit): wire BackfillDelegationEdges to run once at the 0.5.0 activation height (+ all-records scan + marker); GraphQL surface for edges; devnet multi-node verification.
Team decision: every delegator to a slashed node loses the same fraction. Implemented without touching the slash code path: - new slash-immune gross-total asset AssetDelegationTotal (keyed by node), moved by stake/unstake but never by a slash; bond/total = post-slash solvency ratio. - unstake now interprets the amount as GROSS edge removed: the edge + gross total drain by gross (keeping the ratio constant, so the outcome is identical regardless of unstake order = everyone slashed equally), while the node bond + the delegator payout move by released = floor(gross * bond / total), overflow-safe via big.Int. Unslashed -> released == gross. - slashAdjustedRelease replaces the no-op stub; released is computed in ConsensusUnstake and stamped into the oplog (opReleased) so the pure record builder stays deterministic. - migration seeds per-node AssetDelegationTotal (= sum of edges) alongside the edges so the ratio is well-defined for pre-activation stakes too. Tests: pro-rata slash (two delegators each released an equal 5000 from a 50%-slashed node, order-independent, bond fully distributed) + existing entitlement/legacy/migration all green. build/gofmt/vet clean.
Exposes the per-edge delegation balance to the frontend (e.g. the unstake
form), reusing the ledger session's GetBalance:
- new type ConsensusDelegation { from, to, delegated, claimable }
- getConsensusDelegation(from, to, height): delegated = gross edge stake;
claimable = slash-adjusted (mirrors slashAdjustedRelease — full unless the
node bond is slashed below its gross delegated total, then delegated *
bond / total, overflow-safe via big.Int).
Generated gqlgen code (generated.go/models.go) is gitignored and regenerated
at build (Dockerfile.devnet / `go run gqlgen generate`); only the schema and
the resolver impl are committed.
…ivation Backfills per-delegator stake edges from history exactly once when consensus 0.2.0 activates, so accounts that delegated BEFORE the upgrade can reclaim their stake (their edge is otherwise invisible — the from->to link lives only in the paired #in/#out ledger rows). - db/vsc/ledger: GetLedgerRecordsByType(types, toBlock) — all-account scan of consensus_stake/unstake history, deterministically ordered (+ mock). - ledger-system: MigrateDelegationEdgesOnce(blockHeight) — marker-guarded, idempotent (persisted system:delegation_migration row; also short-circuits a node restored from a post-activation snapshot). Scans history -> BackfillDelegationEdges -> StoreLedger edges + per-node totals + marker. Deterministic: same canonical ledger => identical edges on every node. - state-processing: ExecuteBatch runs it (gated on delegatedStakeActive) BEFORE the slot's session/txs, stamped at lastBlockBh, so a delegated unstake in the activation slot already sees its edge. - test mocks updated for the new interface methods. Test: TestMigrateDelegationEdgesOnce — a pre-0.2.0 delegation (legacy #in/#out only, edge invisible) becomes reclaimable after the backfill; second run is a marker no-op (edge not doubled). build/gofmt clean. NOTE: the on-chain wiring (run-once placement, height-visibility of seeded edges vs slot tx heights, checkpoint-sync) is consensus-critical and must be verified on a devnet with 0.2.0 ACTIVE before mainnet activation. Pre-existing rc-system mock breakage (missing CancelPendingSafetySlashBurn etc.) is unrelated.
End-to-end multi-node proof of the per-delegator feature: pins consensus 0.2.0 via the version floor (FloorEpoch must be non-zero — 0 = "no floor"), funds userA, has userA delegate 5.000 consensus stake to operatorB (from != to), and asserts: - the per-edge delegation reads 5000 on EVERY node (deterministic), and - userA's undelegate drains the edge to 0 on every node. Uses the bare-auth ledgerOp path (the legacy hive_ops.go Unstake helper sends a "hive:"-prefixed required_auth that Hive L1 rejects) and waits for the deposit to credit before staking. Verified PASS on a 5-node docker devnet (409s).
…take
Found in self-review of the staking/unstaking paths. Neither was caught by the
devnet test (which ran at 0.2.0, exercising only the delegated path).
1) Oplog Params fork (CRITICAL). The oplog — including OpLogEvent.Params — is
CBOR-encoded into the L2 oplog block CID (block-producer MakeOplog), which all
nodes must agree on. ConsensusStake/ConsensusUnstake stamped delegated/released
into Params UNCONDITIONALLY, so a node on the new binary produced a different
oplog CID than an old node for ANY block with a stake/unstake — an old/new
fork BEFORE 0.2.0 even activates, defeating the gate. Fix: only stamp the new
keys on the delegated path; legacy consensus_stake carries NO Params and
consensus_unstake carries ONLY {epoch}, byte-identical to before. Guarded by
TestLegacyConsensusOplogParamsUnchanged.
2) Migration gate not fail-stop. ExecuteBatch gated the one-time backfill on
delegatedStakeActive -> ActiveConsensusVersion -> GetElectionByHeight, which
returns a ZERO version on a transient election-DB error. One node could skip
the migration for a slot while peers ran it, then reject a delegated unstake
the peers accepted -> fork. Fix: gate on the FAIL-STOP GetElectionInfoOrBlock
(same read the consensus_stake/unstake handlers use), so every node decides
identically (blocks until the DB recovers).
Delegated-path behaviour is unchanged (devnet result still holds). Tests +
node build green.
…ion modes (v0.3.0)
Follow-up to per-delegator consensus stake/unstake (PR 219): delegators now earn
their fair share of pendulum rewards, and node operators opt in to receiving
delegations via a published mode.
Gated on a NEW consensus version line, **v0.3.0** — kept separate from the
already-shipped 0.2.0 batch (try/catch ICC + pendulum LP floor), whose mainnet
activation heights were fixed without this behavior. Bundling new consensus
rules into 0.2.0 retroactively would diverge upgraded vs not-yet-upgraded nodes
at that fixed height; a fresh 0.3.0 floor rise activates delegation on its own
coordinated schedule. Pre-activation behaviour is byte-identical to 0.2.0.
Operator delegation mode (deactivated | share | custom):
- New leaf package modules/common/delegationmode (constants + Normalize /
AllowsDelegation / SharesRewards helpers).
- Configured in identityConfig.DelegationMode (+ SetDelegationMode setter),
published in the node announcement (vsc_node.delegation_mode), stored on the
witness record, and readable via GraphQL (Witness.delegation_mode +
getNodeDelegationMode query).
- Default "deactivated": delegation is strict opt-in. Self-stake (from==to) is
always allowed; unstaking an existing delegation is never gated.
Stake-time enforcement:
- StateEngine.NodeDelegationMode reads the operator's announced mode from the
witness DB (deterministic; defaults to deactivated).
- TxConsensusStake rejects third-party delegation (from!=to) to a node that has
not opted in ("node does not accept delegations"). Pre-0.3.0 the gate is inert.
Fair-share rewards (share mode):
- The per-delegator split is baked into the attested, BLS-signed, byte-compared
settlement record, so payouts come from the signed record. Inter-node pro-rata
math is unchanged; each share-mode node's distribution is expanded into
per-delegator entries summing to the same amount (rounding remainder ->
operator), so TotalDistributed/Residual and the conservation invariant are
untouched. Operator earns purely via its own self-stake edge (no on-chain
commission); custom keeps rewards at the operator for off-chain settlement.
- New ledgerSystem.AllDelegationEdges + StateEngine.PendulumShareDelegations feed
settlement.ComposeInputs.ShareDelegations; ExpandShareDistributions performs
the split; producer, apply-time re-derivation, and structural validator all
read the same source at the pinned SnapshotRangeTo. validatePendulumSettlement
relaxed to allow share-delegator distributions (strict reject + replay
self-heal on transient read failure; never diverges).
Version plumbing:
- consensusversion: currentConsensus 2 -> 3, add V0_3_0 line + Version0_3_0Active;
delegatedStakeMinVersion keys off V0_3_0. 0.2.0 history entry restored to its
ICC + LP-floor meaning; delegation documented under a new 0.3.0 entry.
Tests: unit (delegationmode, edge grouping, reward split + ComposeRecord
conservation), handler gate (deactivated/share/custom/self-stake; inert at 0.2.0,
active at 0.3.0), integration (PendulumShareDelegations + NodeDelegationMode over
a real LedgerSystem + witness DB), and a 5-node devnet pinned at 0.3.0 (delegate
-> edge=5000 on every node -> undelegate -> 0). State-processing failures
byte-identical to the develop baseline (zero new regressions).
Also fixes a pre-existing build break in modules/rc-system/rc_system_test.go
(duplicate mock methods + missing LedgerSystem interface methods).
…not a wrong default Harden the non-fail-stop reads the delegation feature added on consensus paths: the AssetDelegation/Total balance read now uses ledgerRangeOrBlock (was a swallowed error plus a nil-slice deref that panicked a node mid-unstake), the timelock ingest and NodeDelegationMode witness reads block via a new getWitnessAtHeightOrBlock (a transient blip could otherwise persist a divergent maturity or silently gate to deactivated), and MigrateDelegationEdgesOnce blocks on its marker/history/store reads (also closing the edge-misclassification a failed-then-retried migration could cause). The GraphQL getNodeDelegationMode now uses a non-blocking best-effort read that returns null when the mode cannot be determined rather than blocking the request or reporting a wrong default, with the effective-mode resolution shared as one pure core between the consensus and API paths. Also re-key two stale version comments (devnet-setup, rewards test) to 0.5.0.
end slash allow window at the same height
…recovery snapshot) Recovery snapshot of the v6 TSS vault-rotation build after the originating session bricked mid-M1.3-council. Bundles three dark-launched milestones, all inert until VaultRotationV2ActivationHeight / MaxNewMembersPerElection are set (0 on every network today): - M1.1a: deterministic governance FLAG halt for BTC keysign (vsc.tss_halt custom_json + solvency_gate.go, gate at tss.go SignAction choke). Council- hardened to FLAG-only; SIGNAL scaffolded/fail-open. Mainnet-safe, inert. - M1.2: churn RATE cap MaxNewMembersPerElection 0->1 mainnet only. - M1.3: reshare->keygen rotation, PER-KEYID gate-off (U-1) + fresh keyId per gen + keygen reward-sourcing G15 (ScoreTssKeygenExclusion). Dark-launched behind VaultRotationV2Enabled(bh). 5 mandatory TSS checks authored. Builds green (magid + host pkgs); unit tests pass. Full state in /mnt/o/MAGI-TSS-BLOCKTRADES-AUDIT-2026-07-07/BUILD-LEDGER.md. Deferred/outstanding: M1.3 council (lost in brick, re-run), M1.4, S2 devnet rotation proof, M1.3b NR-9 check-sig ceremony. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…f tests, deploy guardrail Re-ran the M1.3 council (4 Opus lenses) after the originating session bricked mid-review. M1.3 PASSES (shipped diff is correct, deterministic, byte-identically inert at flag=0, honest dark-launch). These are the hardening fixes from the adjudication (none says M1.3 is broken); all inert until VaultRotationV2Enabled. - corr-F1 split-brain guard (state_engine.go): reject a "keygen" commitment whose keyId is already active — contract keyId-reuse would keep the stale on-chain PublicKey while the DKG wrote new shares, freezing the vault. Reshare (same pubkey) unaffected. Gated on VaultRotationV2Enabled -> byte-identical when inert. - compl-F1 tests: extracted shouldSkipReshareForVaultRotation (behaviour-preserving) + TestShouldSkipReshareForVaultRotation (7 cases) + TestVaultRotationV2Enabled. - adv-F1/corr-F3/compl-F3 deploy guardrail (convergent, 3 lenses): enabling the flag with only M1.3 shipped freezes the vault. Hard go-live precondition doc at the flag def (params.go) + explicit VaultRotationV2ActivationHeight: 0 in all 4 configs. - corr-F2 documented global keygen scoring (intended); corr-F4 record accuracy. Builds green (magid + host pkgs); new + existing tss/params/rewards tests pass. 4 pre-existing state-processing failures (pendulum-oracle/review2, DB-harness) fail identically at base 69d4493 — flagged for team, not a regression. Synthesis: /mnt/o/MAGI-TSS-BLOCKTRADES-AUDIT-2026-07-07/methodology-run/M1.3-council-synthesis.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Last Month-1 containment item. Adds zeroizeKeystoreEntry — best-effort secure erase (overwrite stored value with same-length zeros + clear in-memory copy) of a retired TSS key share, wired into the keystore-cleanup delete path. DORMANT TWICE OVER: (1) gated on VaultRotationV2Enabled(bh) -> byte-identical to the bare delete on every network today (flag 0); (2) the cleanup block is under KeyRetirementEnabled (false on mainnet, never flipped) -> does not run until S5's fund-gated retire path calls the same helper. Wiring secure-erase to the current TIME-gated retirement would turn a stalled-migration freeze into unrecoverable loss, so dormancy is safety-critical, not cosmetic. Honest limitation documented: shares are already AES-256-GCM at rest (the primary control); on flatfs a Put is temp-file+rename, so this is a logical overwrite + unlink, not a guaranteed physical erase. Defence-in-depth, not a substitute. Builds green; TestZeroizeKeystoreEntry + full tss package pass. 5 TSS checks authored (local keystore op, no party-list/CID/consensus output). Light-gated (self-verified, like M1.2) — not a full 4-lens council. Record: /mnt/o/MAGI-TSS-BLOCKTRADES-AUDIT-2026-07-07/methodology-run/M1.4-implementation.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Applied the fix batch from the pruned audit methodology over Month-1 (7 decorrelated lenses). Month-1 shipped diff was SOUND as-shipped; these are hardening + one framing correction. All build green + unit tests pass. - Churn-cap activation height [4 lenses -> HIGH]: add MaxNewMembersActivationHeight + EffectiveMaxNewMembers gate; election-proposer reads the effective cap. The 0->1 cutover is now an atomic single-block flip (dark-launchable), so a rolling binary upgrade can't split old(cap=0)/new(cap=1) nodes into divergent election member sets. Mainnet value stays 1 but INERT until governance pins a future height. +TestEffectiveMaxNewMembers. - Un-wire M1.4 zeroize [key-lifecycle L1 -> MED]: removed the time-gated zeroize caller; kept zeroizeKeystoreEntry+test as the primitive S5's fund-gated retire wires. The time-gate had zero fund check -> could destroy a still-funded key's share (recoverable freeze -> permanent loss) if KeyRetirementEnabled flipped pre-S5. - Solvency-math overflow [money-math F1]: observeBtcSolvencyInsolvent computed in uint64 with underflow guard + reject implausibly-large Supply (was int64 cast, inverted for uint64 >= 2^63). Dead M1.1b code; fix-before-wiring. - Halt fail-open, read path [3 lenses]: refreshChainConsensusCache retains prior cache on a consensus-state Get() error (fail closed for BtcKeysignHalted/ ProcessingSuspended) instead of resetting to zero-value. - Halt fail-open, write path [error-handling]: ERROR (was Warn) on SetBtcKeysignHalt write failure - the node did NOT apply the emergency halt; re-broadcast the op. - Manual reshare fail-open [2 lenses]: KeyReshare BTC refusal fails CLOSED before the first BlockTick (lastBlockHeight==0) when v2 is configured. Deferred (tracked): V-8 evacuation whitelist (S3), zeroize-all-epochs (S5), btcclient URL-escape/non-negative (M1.1b), pre-existing retire-status race (team). The one election-proposer test failure (TestH6GatewayPoPGate) is pre-existing (gateway-PoP, MocknetConfig cap=0 -> gate is a no-op; fails identically at base). Synthesis: methodology-run/MONTH1-PRUNED-METHODOLOGY.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… V-8 S3.0-S3.3: the fund-safety heart of the spine's node layer. A retiring/ draining vault generation's TSS key may sign ONLY a migration sweep whose every output pays the committed successor vault's P2WSH — anything else is refused (ends "sign blind"; closes the theft-oracle the reconstructable old key would otherwise be). - lib/btcvault: dependency-light lib-lift (like lib/btcclient) — vault registry + TxSpends decode, msgp SigningData decode (tolerant of the optional amount field), successor P2WSH derivation and BIP143 (segwit-v0) sighash recompute, all byte-for-byte mirrors of the contract. - modules/tss/output_scoping.go: deterministic pre-issuance gate at the SignAction choke (tss.go:1036). Reads the vault list + pending spends from consensus contract state (readContractStateKey, pinned to bh) and the successor PRIMARY from COMMITTED tss_keys (never a contract field — the C-C guard). Binds the untrusted template to the signature by recomputing the sighash (a SigHashAll digest commits to the outputs). Fails CLOSED on any uncertainty. Inert until VaultRotationV2Enabled (activation height 0). - V-8: a proven successor-scoped sweep is exempt from the M1.1a solvency halt so the honest evacuation to the new vault can still complete. Determinism: pure local skip-or-issue before any session/party-list/CID work (keysign results are never CID'd/BLS-collected) — Constraints 1-3 safe by construction; every input is consensus-state/committed, so all nodes reach the identical verdict. 5 mandatory TSS checks authored (S3-DESIGN.md). Tests: lib/btcvault (4) + output_scoping (9) green — successor sweep allow, active-gen unrestricted, inert pre-fold, non-sweep digest refuse, non- successor output refuse, lying-decoy recompute-catch, missing-amount fail- closed, unknown/non-fund-holding refuse, corrupt-registry fail-safe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ry (HIGH) Council converged (3 lenses: adversarial, fail-safe, correctness) on a fail-OPEN in the output-scoping gate: evaluateScope mapped resolveVaultView->not-ok to scopeAllow, but not-ok conflated a genuinely ABSENT registry (pre-fold, allow correct) with a PRESENT-but-corrupt one (malformed / 0-active / 2+-active). The latter fell open, letting a lying/compromised contract disable NN#1 by writing a bad "v" and get the reconstructable retiring key to sign an attacker output. Inert until the flag, but must-fix before pinning VaultRotationV2ActivationHeight. - resolveVaultView is now TRI-STATE: vaultAbsent (allow ONLY gen-0 "main" — a higher-gen keyId cannot exist without a registry -> refuse), vaultUnresolvable (registry present but corrupt/ambiguous -> refuse ALL BTC-vault keysigns, fail closed + recoverable), vaultResolved (the status switch). Once "v" is present, any failure is unresolvable, never absent. - Extracted btcSignGateDecision(verdict, halted) (pure) so the V-8 halt exemption is unit-testable without a live datalayer. Tests: TestEvaluateScope_UnresolvableRegistryFailsClosed (two/zero-active + malformed -> refuse, incl. active-gen keyId frozen on a corrupt registry), absent-non-main -> refuse, TestBtcSignGateDecision_V8 (proven sweep issues during a halt; all else freezes). Full tss + btcvault suites green, vet clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…try (HIGH) + resolve-once (F-1)
Two node fixes from the pruned methodology (both inert until the flag; must land
before pinning VaultRotationV2ActivationHeight).
- money-math HIGH: resolveVaultView keyed byKeyId by VaultKeyName(Generation) and
counted activeCount over the raw slice, so two entries sharing a Generation
(one Retiring, one Active) aliased onto one map key — last-write-wins could hide
the reconstructable retiring key behind an Active entry, returning scopeAllow
(a fail-open worse than the council's, since it never reaches the sighash/output
check). Now reject any duplicate Generation -> vaultUnresolvable (fail closed).
Honest state never reuses a generation (monotonic).
- assumptions F-1: the gate re-ran GetLastOutput once per key ("v"/"va"/"p" + one
"d-<txid>" per pending spend) under the global TSS lock. Added
contractStateReaderAt: resolve the committed output + state databin ONCE, then
read every key from that one databin. Cuts N+3 GetLastOutput to 1, bounding work
under the lock. (A context-bounded GetRaw is a tracked hardening — needs a
datalayer API that takes a context; contract-state leaves at a processed bh are
local, so a network stall is an edge.) btcScopeDeps.readKey is now a bound
1-arg closure; evaluateScope/resolveVaultView/retiringSignPaysSuccessor drop bh.
Tests: TestEvaluateScope_DuplicateGenerationFailsClosed (dup-gen -> refuse) added;
all output_scoping + lib/btcvault tests green (reader closure + evaluateScope
signatures updated); magid builds; vet clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ions at the flag def
WARDEN flagged that the deploy-ordering constraints lived only in O-drive
markdown, not in the code where the other go-live preconditions are. Add to the
VaultRotationV2ActivationHeight doc: (d) pin the churn cap too (else the V-A
liveness mitigation is vacuous), (e) verify ContractId("BTC") is populated (empty
= all three BTC gates go inert together), (f) reorg-harden the migration confirm
flow + pause-exempt the confirm path (tracked S2). Comment-only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s into the deploy gate The 5-lens failure-state/brick council found NO permanent brick reachable today (shares never deleted + CSV backstop + inert), but named the constraints that keep it that way. Record them at the flag def as hard pin preconditions: (g) build the check-signature-before-activate ceremony (FS3-1: else funds route to an unsignable vault); (h) ★ S5 retirement must gate on SPV-proven ZERO L1 balance, NEVER contract registry-emptiness (FS5-1: the one forward-coupling to PERMANENT LOSS — a stranded delete-at-build gen reads registry-empty while funds are on L1); (i) raise contract MaxBlockRetention >= CSV timelock + reorg; (j) don't let ProcessingSuspended deprecate a fund-holding gen while renewKey is blocked. Comment-only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…uspend + reshare extends expiry Two node key-lifecycle fixes from the failure-state/brick council (both prevent a recoverable freeze from becoming reachable, both deterministic on-chain state). - BRK-5 (FS-1/FS-2): while the chain is processing-suspended, FREEZE the deprecation/retirement clock. A suspend blocks renewKey (a non-recovery contract op), so the unconditional epoch clock would deprecate a fund-holding gen's key with no in-suspend cure (freeze until unsuspend). The chain is halted anyway, so no lifecycle should advance; it resumes normally when the suspend lifts. Gated on the on-chain ProcessingSuspended flag (refreshed just above). - BRK-8 (FS3-2 / L-1): a RESHARE now extends the key's ExpiryEpoch (like activation does), so an actively-reshared in-use key does NOT deprecate mid-life on the fixed ~3-month epoch clock (the manual-renew time-bomb). A key that stops resharing still deprecates at its last-reshare expiry; reshare only; legacy no-expiry keys (Epochs==0) unchanged. Deterministic (commitment.Epoch + Epochs are on-chain). Builds green + vet clean; consensus_version/suspend tests pass; full state-processing package shows only the 4 pre-existing Mongo-harness failures (unchanged). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…k (i)/(j) done The 4-lens methodology on the brick fixes verified them sound (deterministic, fail- safe, abuse-resistant, accounting-neutral). Fold in the determinism lens's deploy- ordering caveat: BRK-5/BRK-8 change consensus key-lifecycle timing once Epochs>0 keys exist, so the fleet must run the binary uniformly before v2 keys are minted (inert on today's legacy Epochs==0 keys). Mark (i) MaxBlockRetention + (j) suspend- deprecation-freeze as implemented (BRK-4a / BRK-5). Comment-only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… + fee-funding BRK-1 (delete-at-confirm migration) is built + council-proven on the contract (feat/vault-rotation-s1 @ a5cf0a0), so precondition (f) — reorg-hardened confirm flow + pause-exempt migration confirm — is now DONE. Add precondition (l): BRK-1's registry-based NN#3 makes rotation-liveness depend on every superseded gen fully draining. A sub-sweep-fee dust deposit to a superseded gen's still-matchable address is credited but un-sweepable (V-1/V5-4 fee abort) → NN#3 freezes rotation (funds-safe, liveness DoS). Ship the V-1 dust escape (or a min-deposit floor / S5 SPV-zero prune) before pinning. Also: FeeSupply must be funded (calcVscFee=0 today rejects every sweep at the build-time reserve check — fail-safe can't-start, not stuck-on-L1). Comment-only; VaultRotationV2ActivationHeight stays 0 (inert). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The contract is pull-driven: it only builds a migration sweep when someone calls
migrateVault, and getMigrationInputs returns at most MaxMigrationInputs UTXOs per
call ("the caller drains it in successive tranches"). Nothing in the bot drove
that loop, so a rotated-out generation kept most of its BTC forever — the sweep
was never built, the generation never drained, retireVault could not advance it
(fund-gated), and the committee's bond stayed locked. Broadcasting and settling
are already generic over pending spends (HandleUnmap -> HandleConfirmations ->
confirmSpend), so a sweep rides that pipeline once it exists; only the driving
side was missing.
- vault_rotation.go: per block, build the next tranche (migrateVault), fall back
to writeOffDust when a residual is provably un-sweepable, and retireVault once
a generation is drained. Rate-limited (every op costs RC), never stacks a
second sweep on an in-flight one (that would draw the fee reserve down twice
and race the same UTXO set), fails CLOSED on an unreadable vault registry, and
is a no-op on any contract without a vault registry (dash/ltc/... and a
pre-rotation BTC deploy). The whole policy is a pure decideVaultAction() so it
is unit-testable without a chain.
- vault_state.go: batched vault- and UTXO-registry reads (one query for every
UTXO object, not one per UTXO); reuses the node's UnmarshalVaultRegistry so the
91-byte stride cannot drift from the contract.
- contract-interface/const.go: the "v" / "ms-" / "us-" state keys, mirrored from
the contract.
- call_contract_l2.go: a per-action RC limit (rcLimitFor). The bot attaches one
global RcLimit (default 10000) to every L2 tx, but the vault ops are
SPV/secp256k1/P2WSH-derivation heavy (~8M); at the default every migrateVault
would abort with a cost-limit-exceeded. Raised for the four vault ops ONLY —
not globally, because a high rc_limit reserves HBD against RC and surfaces as a
spurious insufficient-balance on ops that move HBD (the vault ops move none).
The vault ops are owner-or-operator gated; until governance appoints this bot via
setVaultOperator the driver latches on the not-owner rejection and reports it
once, rather than burning RC on guaranteed-failing retries.
… mixed-version Devnet coverage for the paths the PR left unproven or soft-asserted. Every assertion reads committed contract state (the vault "v" and UTXO "r" registries), not transaction status — several existing cases passed while transitioning nothing. - vault_drain_complete_test.go: settles EVERY migration tranche and proves a multi-UTXO generation drains fully to zero, then genuinely Inactive, then Purged. Replaces the vacuous Stage-5 retire asserts (which only checked that the retireVault tx confirmed, while retireVault succeeds transitioning nothing). - vault_operator_drain_test.go: a non-owner appointed via setVaultOperator drives the full drain + retire; a stranger is refused; a non-owner cannot appoint; a revoked operator is refused again. The delegation end-to-end on a live chain. - vault_writeoffdust_test.go: the deadlock escape hatch — a sub-dust residual that migrateVault refuses as uneconomic is cleared by writeOffDust so the generation drains and retires. - vault_redrive_test.go: the L7 re-drive at the contract level — a premature re-drive is refused (staleness gate), a stranger is refused (operator-scoped), and the operator re-drives a stale sweep into a higher-fee replacement. - vault_mixed_version_test.go: rolling-upgrade proof — PR nodes + merge-base nodes in one fleet, v2 flag off (the post-merge mainnet state), asserting block-for-block cross-node convergence so the ungated per-block theft-halt read and the new consensus_unstake Params["from"] neither fork nor stall a half-upgraded network. - vault_batched_test.go: register/activate now retry the BRK-2 check-sig instead of a single racing read — without which the VL-PEN-15 set-once and VL-PEN-10 two-keygens negative cases passed VACUOUSLY (a second op "rejected" only because the first never landed). - vault_stage4_test.go: factor migrateAndSettleAs(opNode) out of migrateAndSettle so a sweep can be driven from an appointed operator's identity.
A migration sweep that is broadcast but never confirms (built at too low a fee for the live BTC market) leaves the driver's in-flight check true forever, so the drain wedges: the generation never empties, retireVault cannot advance it, and the committee's bond stays locked. The existing bot only REPORTS staleness via /health (manual operator action); nothing drove redriveSpend. Track the contract height at which each in-flight sweep was first seen and, once a sweep has been in flight past the stale window, call redriveSpend to replace it with a higher-fee tx. The contract's own staleness gate (RedriveStaleBlocks) is the final arbiter, so an early call is simply refused; the driver only avoids issuing calls it knows will be rejected. Settled sweeps are forgotten so a recycled txid starts a fresh clock rather than inheriting a stale one. redriveSpend also gets the raised vault-op RC limit (it builds a fresh signed sweep — as heavy as migrateVault). HasMigrationSweepInFlight becomes InFlightSweepTxIds (returns the sweep txids the re-drive works off); FetchContractHeight reads the contract "h" for the staleness clock. Unit-tested (TestNoteSweepsInFlightStaleness): too-early does not redrive, stale does, a settled sweep is forgotten, a recycled txid is not instantly stale.
…nnot steal Proves the flip side of TestVaultOperatorDrivenDrain: the scoped operator's blast radius is exactly the four self-validating ops and nothing more. Appoints an operator, then confirms that operator CANNOT: - writeOffDust a generation holding a large, still-sweepable UTXO (no fund destruction — the op runs but the un-sweepability gate deletes nothing) - retireVault a still-funded generation (no premature purge / orphaning) - setVaultOperator (no privilege escalation / self re-appointment) - pause the contract (no governance seizure) and that gen-0's funds are intact after every attack. 5/5 on devnet.
Devnet proof of the bond-lock: a committee member whose generation holds a retiring, not-yet-drained BTC vault key cannot unstake its consensus bond, and can once that generation is drained. The SAME consensus_unstake op is issued twice — refused while gen-0 is retiring+funded, accepted after gen-0 is fully drained — so the difference isolates the bond-lock and its release (an insufficient-stake refusal would fail BOTH; a bond-lock fails only the first). Observed via the ledger_actions pending consensus_unstake amount (the same predicate GetAccountPendingConsensusUnstake uses): 0 while locked, > 0 once released. 3/3 on devnet.
…ection Proves the ungated, live-on-merge governance halt end-to-end: a gateway-multisig vsc.tss_halt(true) sets consensus_state.btc_keysign_halted on EVERY node, and tss_halt(false) clears it — the op-plumbing integration the mixed-version test could not reach (it broadcast once, before vsc.gateway's authority was populated). This version retries the op until the flag actually propagates, which also confirms VSC accepted it (a Hive-level broadcast success is not acceptance), using the known-working deterministic-BLS gateway-multisig setup. The flag->freeze logic itself is unit-tested (TestBtcKeysignFrozen_FlagAndScope). Also fixes countHaltFlag: it queried the "consensus_state" collection, but the halt flag lives in "chain_consensus_state" — so it always read 0 regardless of the real state. Verified against a live devnet (node logs "vsc.tss_halt applied" on all nodes + chain_consensus_state.singleton.btc_keysign_halted=true). This latent bug also affected the mixed-version helper.
The reshare-skip set (retiring/draining/inactive) excluded PURGED — the terminal retired state. And nothing deactivates a gen's tss_key at purge: FindEpochKeys selects on status:"active" (blind to vault status), no purge handler touches tss_keys, and KeyRetirementEnabled is false. So a purged generation's key stayed active, kept being returned by FindEpochKeys, and — not being in the skip set — was RESHARED every cycle, resurrecting a retired key's shares in the current committee. That defeats the PR's headline property (a reconstructed key becomes worthless within one rotation): the purged key stayed live and reshared forever. Add ReshareSkipKeyIds — every NON-active generation (retiring/draining/inactive AND purged) — and point the reshare loop at it. Kept SEPARATE from KeyIds on purpose: the #11 bond-lock and V-A signing consumers must RELEASE a purged gen's members (it is drained + past grace, nothing to sign, no reason to stay bond-locked), so Purged must NOT enter KeyIds. Only the single ACTIVE gen ever reshares. Inert until VaultRotationV2Enabled, like the rest of the skip. TestComputeRetiringSignerSet_PurgedSkipsReshareButReleasesBond: a purged gen is in ReshareSkipKeyIds but NOT KeyIds; the active gen is in neither.
POA admission batch, part 1 of the build map at MAGI-POA-BUILD-2026-07-20/00-BUILD-MAP-v2-IMPLEMENTABLE.md. S0 - activation surface. New consensus line V0_7_0 with five resolvers (PoaAdmissionOps/SeatGate/FlatWeight/ChurnCap/ExitHalt), all flipping as one batch. 0.7.0 rather than 0.4.0 because 0.4/0.5 are develop's delegated-stake batch and 0.6 is vault-protection's: a shared line means one floor rise silently activates two unrelated batches after a merge. Version-gated, not height-gated, so a laggard is excluded from the committee rather than forking across a height gap. New ConsensusParams: PoaAdmitVoteWindowBlocks, PoaExitHaltBlocks, PoaMaxNewMembersPerElection, each with a non-zero fallback (a zero window expires proposals instantly, a zero halt is no halt, a zero cap wedges admission permanently). S1 - poa_seats collection. Append-only by construction: the interface has no delete, because voting is entry-only (signers must not be able to vote each other out - a smaller set is a cheaper capture, and the admit threshold is self-protecting only while the set cannot shrink). Height-addressed reads so a reindex reproduces the same elections as a live node. Accounts normalised through one helper: a "hive:" prefix mismatch does not error, it matches nothing, and in the election path matching nothing means an empty committee. S2 - seat maintenance at the ratified-election consensus point. Two jobs: bootstrap seeding, and the seating/exit bookkeeping that creates the "when did this account leave the set" fact - which exists nowhere in the codebase today (elections carry no reason, no status, no departure height), and which the collateral exit-halt is counted from. Bootstrap seeding is the load-bearing safety property. The seat gate is an allowlist over candidacy; activated against an empty registry it deletes every candidate. That is not hypothetical - the structurally identical H-6 key-admission gate starved the mainnet committee below the floor at epoch 1699, halted elections, and is still disabled today. So the first ratified election after activation seeds the registry from its own member set: no operator action, no flag day. Empty registry keeps the gate inert; a failed registry read skips maintenance entirely rather than reading as "nobody is seated" (which would arm the halt against every operator at once). Exit recording is idempotent: once an exit height is set, later elections that also exclude the account cannot push it forward. Without that guard an operator who exits and stays out has their 3-day clock reset every election interval, and a temporary lock becomes a permanent seizure. Inert below 0.7.0 - no registry writes at all, so this binary produces byte-identical state to the current one until the floor rises. Tests: 11 seat-maintenance (whitebox, local doubles - test_utils imports this package so an in-package test cannot import it back), 10 registry, 5 gate/param. Note: cmd/vsc-node needs `make generate` first on this branch; gqlgen's generated.go/models.go are gitignored and absent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…6,S7) S3 - seat gate. Election candidacy is restricted to ratified seats, so stake alone no longer buys a place in the committee. Placement is load-bearing: the gate runs BEFORE the stake loop, the churn cap and the bond floor guard, so the floor guard remains the LAST membership-shrinking step and can still backfill incumbents. Three independent anti-starvation layers, because this is the same gate shape that halted mainnet at epoch 1699: version-gated on the PRIOR election's version (one epoch of slack after the floor rises), inert while the registry is empty (bootstrap seeding fills it), and fail-stop on a read error (abort the attempt, retry next slot) rather than building a committee from an unknown seat set. S5 - flat seat-weight. Every seat carries params.PoaSeatWeight (1). Stake keeps every other role - MinStake floor, maturity window, established grace, slashable bond - but stops being consensus weight, so 2/3 means 14 seats of 20 rather than two-thirds of stake. Not a novel path: pType "initial" already flattens weights today, and every downstream consumer is a ratio rule over whatever weights the election carries. TSS thresholds are member-count based and untouched. Two secondary sites flatten too, and both matter. The bond floor guard's backfill would otherwise re-seat an incumbent at its stake-derived weight, putting one stake-proportional member into an otherwise flat committee - restoring the exact capture vector through the liveness patch. distWeight is inert today (REQUIRED_ELECTION_MEMBERS is empty) but is computed as a share of the total, so flattening it stops one-seat-one-vote being quietly undone later by adding a required member. S6 - churn cap. The existing cap is dead code everywhere: EffectiveMaxNewMembers needs MaxNewMembersActivationHeight pinned and it is 0 on every network, mainnet included. (Four devnet tests set the cap value and assert on it while the height stays 0.) POA activates it off the version gate rather than pinning a height, which removes the mis-pin footgun documented on the field itself - a bare value lets old-binary cap=0 and new-binary cap=N nodes compute different member sets for one election and stall the epoch on failed BLS aggregation. S7 - collateral exit-halt. A seat's bond is unwithdrawable while it is in the set, and for PoaExitHaltBlocks after it leaves. This is what makes the bond a deterrent: theft cannot be prevented (a threshold signature confirms on Bitcoin in ~10 minutes regardless of Magi), so it is deterred by collateral the thief cannot extract before detection. Enforced at BOTH the unstake submission and the matured-payout release, and the second is what makes it real - the submission check alone is bypassable by ordering (submit while comfortably seated, then leave and let the 5-epoch maturity elapse, and the bond pays out with no halt ever applied). Fail-closed on an unreadable registry: a delayed withdrawal is bounded, a thief's collateral leaving during the detection window is not. Termination is proven, not assumed: unstaking drops weight, the next election records an exit, and the bond releases a bounded time later. An operator who stays seated stays held - by their own choice, and the refusal message names the release height. Tests: 7 election-path (gate excludes unseated / inert when empty / fail-stop / inert below activation, flat weight ignores a 10,000x whale, weight still tracks stake below activation, churn cap defers newcomers but never incumbents), 18 seat + exit-halt. Pre-existing failures unchanged by this work, verified against c21610f: TestH6GatewayPoPGate (the H-6 gate it asserts is disabled for the epoch- 1699 liveness fix), and 4 in state-processing (pendulum-oracle x2, review2 x2). The election-proposer suite cannot run from /mnt/o at all - NewContractTest chmods data/config and DrvFs refuses; run it from an ext4 checkout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The only way a new operator ever enters the set. A clone of the existing witness-vote governance mechanism (vsc.reserve_payout / vsc.reserve_vote): propose-is-first-vote, ceil(2/3) beneficiary-excluded threshold, expiry window, one-shot terminality. It reuses that engine's pure core and its proposal store rather than re-implementing the tally, so the determinism argument is the already-reviewed one. Two deliberate differences from the governance trio: The electorate is SEATS, not the elected committee, one vote each. A seat temporarily out of the committee (dropped for a liveness fault) still votes, because it is still an admitted operator; and no amount of stake buys a second vote. There is no create op. Every vote for the same (candidate, ubo) converges on one proposal id derived from that pair, so the first vote opens the proposal. Folding a tx id into the id - as reserve_payout does, correctly, for its chosen amounts - would scatter votes across per-tx proposals and make the threshold unreachable by construction. Why the threshold IS the theft threshold: admission at ceil(2/3) of seats is self-protecting. A coalition below 2/3 cannot vote accomplices in to REACH 2/3, and one already at 2/3 gains nothing - it already controls the vault. That holds only while the set cannot SHRINK, which is why there is no removal op anywhere in this build (A6, asserted structurally) and why the seat gate is placed where it cannot starve the committee. The UBO is part of the proposal id, not loose payload, so a coalition cannot gather approvals for one beneficial owner and then seat a different one. A blank ubo_id is refused rather than defaulted: every blank collides under the sparse unique index, so the per-owner cap would silently stop binding. The electorate is snapshotted at proposal creation. Re-reading it per vote would move the denominator under an open proposal - seats admitted mid-window raising the bar for a vote already cast, and a shrinking set LOWERING it. A failed registry read refuses to tally at all, for the same reason: a partial electorate is a lower 2/3 bar. WHAT THIS DOES NOT DO: it does not vet anybody. The chain enforces that a UBO string is present and unique; it cannot verify the string is TRUE. KYC/UBO vetting is an off-chain precondition to casting a vote, and reading the uniqueness check as vetting would be a serious misreading. Tests: 14. Threshold (2 of 4 refused, 3 of 4 admits), no vote stacking, non-seats ignored, second seat per owner refused, blank ubo refused, expiry closes and stays terminal, applied is terminal, already-seated candidate is a no-op, electorate snapshot pinned, read failure refuses to tally, wrong net_id ignored, and two structural checks that no removal path exists in the interface or the payload. Suite state on this branch, all verified identical at base c21610f: 4 state-processing + 1 election-proposer (H6, whose gate is disabled) fail pre-existing; 6 modules/db packages fail on a 30s timeout because no MongoDB is running. The Mongo-backed poaseats implementation is therefore NOT covered by a live-DB test - only the semantics-mirroring mock is. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Found by the seven Phase-0 component models run against S0-S4. Ordered by consequence. CRITICAL - bootstrap seeded ONE seat, then halted the chain. UboId was the only optional Seat field without bson omitempty, so every bootstrap seat wrote an explicit empty ubo_id. A Mongo SPARSE unique index only exempts ABSENT fields - an explicitly stored empty string is indexed - so seat #1 inserted and #2..N failed with duplicate-key errors that the loop logged and continued past. The registry then held one seat, which is not empty, so the seat gate's inert-while-empty guard did not fire; the gate deleted every other candidate, the committee fell below MinMembers, and HoldElection aborted. That is the epoch-1699 committee-starvation halt, reintroduced by the mechanism written to prevent it - and the comment above the index confidently asserted the opposite. No mock could catch it: it is a property of the struct tag and the storage engine, so the regression test asserts on bson marshalling instead. CRITICAL - the seat gate had no starvation guard, only an ordering argument. It now computes the gated list speculatively and DECLINES to apply it if that would drop the committee below MinMembers, logging at ERROR. One epoch of permissionless candidacy is recoverable; a stalled chain is not. MinMembers rather than bondCommitteeFloor: the latter folds in the hardcoded 8-key gateway floor, which exceeds a whole 3-node devnet and would make POA untestable, and mainnet already pins MinMembers to 8 because of that same gateway floor. HIGH - what got seated was not what was voted on. The seat write used the crossing vote's freshly-parsed payload rather than the proposal's stored fields, so the last voter, not the electorate, determined the admission. Now seats from prop.Candidate/prop.UboId - and, because that made the store load-bearing, SaveProposal was extended to actually persist those two fields. It hand-rolls its set map and had silently dropped both, which would have made every admission write an empty account and fail. The two fixes are only correct together; either alone is worse than neither. HIGH - flat weight applied without the seat gate. Only the candidacy block checked poaSeats != nil; flat-weight, churn, backfill and distWeight gated on version alone. A node with no registry would admit any staked witness AND flatten everyone to weight 1 - committee weight bought at MinStake per seat, cheaper than the stake-weighting POA replaces and free of the vetting it adds. All POA election rules now share one predicate, so they apply as a set or not at all. HIGH - proposal-id delimiter was attackable. The id is sha256 over candidate, a NUL byte, and the owner id - but both fields were raw JSON strings, and a JSON u0000 escape decodes to a real NUL that passes through both normalisers, so a crafted pair shifts the field boundary and two different (candidate, owner) pairs collide into one proposal. Candidates are now bounded to Hive's account charset and owner ids to printable non-space ASCII. Case and surrounding whitespace are still NORMALISED rather than rejected, and before the id is derived, so differently-spelled votes converge on one proposal instead of splitting the electorate. HIGH - registry writes were best-effort on a consensus path. ExitHeight decides whether an unstake is refused or paid; a write failing on one node while succeeding on peers diverges the ledger result for an identical tx, and SetExit's idempotency means no later election can repair it. Now fail-stop via blockingRetry, matching bond_lock and the safety-slash paths. MEDIUM - bootstrap could enshrine a degraded committee, permanently. Seats are append-only and widening the set needs 2/3 OF THE SEATS, so whatever the first post-activation election happened to contain became a permanent floor with the survivors holding a veto. Bootstrap now refuses to seed below MinMembers and retries next election. It also only fires at the actual transition (previous election below the POA line) rather than whenever the registry looks empty - otherwise a node that merely LOST its poa_seats collection would re-seed from the current committee and diverge silently, since poa_seats is not merklized and the reindex trigger does not cover it. MEDIUM - non-deterministic bootstrap order and a releasable hold. Bootstrap iterated a Go map, so under any partial failure WHICH seats survived was node-dependent. Now sorted. SetSeating gained a monotonic guard: clearing exit_height is the one write that RELEASES a collateral hold, so an older election reprocessed by a replay or reorg must not be able to drive it. Also: poaseats.NormalizeAccount now case-folds, so the write path (governance.NormalizeAccount, which folds) and the read paths agree by construction rather than by relying on Hive's account charset, which nothing here stated or tested. 72 POA tests, all passing. Unchanged pre-existing failures, verified identical at c21610f: 4 in state-processing, TestH6GatewayPoPGate, and the modules/db packages that need a live MongoDB. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Same class as the omitempty bug fixed in 02c3e17, found by checking the rest of the struct for it. Both silent, both invisible to every in-memory test, and one of them was INTRODUCED by that commit is own monotonic guard. last_seated_height and exit_height are QUERIED BY VALUE: SetSeating filters last_seated_height $lte, SetExit filters exit_height == 0 together with last_seated_height $gt 0. A MongoDB value query does not match a document where the field is ABSENT - only an explicit null or $exists:false does. Both fields carried bson omitempty, so on a freshly admitted seat, where both values are 0 and therefore omitted entirely: - SetSeating matched nothing, so a voted-in seat was never recorded as seated - it never acquired a LastSeatedHeight, so it could never acquire an exit either; - SetExit matched nothing, so NO seat ever had an exit recorded, which silently disables the entire collateral exit-halt. The halt would have looked implemented, passed all 72 tests, and protected nothing. A Go map has no notion of an absent field, so no in-memory double can express this failure; the regression test asserts on bson marshalling, which needs no database. Note the two fields now differ deliberately from UboId directly above them: UboId MUST omit (its unique index is sparse, and a sparse index does not exempt an explicit empty string), these two MUST NOT (they are filtered by value). Both directions are now stated at the field. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Found by the collateral-bypass hunt. poaseats.NormalizeAccount stripped the prefix and THEN lowercased; governance.NormalizeAccount (the write path) lowercases and then strips. TrimPrefix is case-sensitive, so "HIVE:alice" kept its prefix on the read path and normalised to "hive:alice" while the write path produced "alice" - the two helpers disagreeing on exactly the mixed-case input that case-handling exists to cover. Same silent-membership-mismatch class as the rest: an excluded operator in the election path, a wrongly-armed collateral halt in the maintenance path. Guarded out on the live security path today (tx.From requires a case-sensitive lowercase hive: prefix at submission) and it failed closed where reachable, so this is defence in depth rather than a live break - but the whole point of routing every comparison through one helper is that its correctness must not depend on which caller reaches it. The same hunt found no exit-halt bypass: all seven attempted escapes traced to BLOCKED, including the strongest lead (consensus_stake credits To rather than From) - that path mints new bond from liquid HIVE and never moves an existing hive_consensus balance, and hive_consensus has no transfer path anywhere in the ledger. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…epoch Two findings from the Phase-1 adversarial hunt, both in fixes made earlier in this same audit. CRITICAL - the entire POA batch could never activate. The re-seed guard added in 02c3e17 resolved its activation check via ActiveConsensusVersion(blockHeight), while its transition check read prevElection. GetElectionByHeight filters block_height with a strict less than, and the election being processed is stored AT that height, so the first call returns the PREVIOUS election - the exact row the second call tests. The gate demanded that row be at/above the POA line while the transition check demanded it be below: a contradiction no chain state can satisfy. bootstrapPoaSeats could therefore never fire, on any node, at any epoch. The consequence was not a fork - every node computes the same wrong answer - but a silent, permanent defeat of the batch: the registry stays empty forever, so the seat gate stays inert and vsc.admit_vote is a no-op, while flat seat-weight and the churn cap DO still fire. That is precisely the "worse than either regime" hybrid this build guards against elsewhere: candidacy open to anyone with MinStake, but every such candidate carrying the same weight as a vetted operator. A full reindex reproduces it identically, so nothing would have surfaced it. Now resolved from the election being processed, which is both correct and the more honest reading of the question being asked. The hunt agent compiled a proof of the dead path before I fixed it; the regression test models the real shape - a previous election below the line and a current one at it - which the old test doubles could not express, because they returned one fixed version for every query. That is the third time in this audit a double hid a defect by being more forgiving than reality. HIGH - the churn cap could stall the epoch. The seat gate's starvation guard simulates only the seat-gate step, but the churn cap runs afterwards and defers new entrants, so it could re-shrink a committee the guard had just approved back under MinMembers - turning the guard's promised "one epoch of open candidacy" into the halt it exists to prevent. Deferring an entrant is always safe to skip, so the cap now yields when it would breach the floor. It is a rate limit, not a safety property. Also: AdmitSeat was left best-effort at both call sites while every sibling registry write was made fail-stop in 02c3e17. Now fail-stop too, with deterministic duplicate-key refusals surfaced rather than retried - blockingRetry on a deterministic error would wedge block processing forever. 77 POA tests. Unchanged pre-existing failures verified at c21610f. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ness
RETRACTION. I reported that the ceil(2/3) admission threshold is only
"self-protecting for one round" and that a 13-of-20 coalition reaches
permanent control for one external vote, rated HIGH and folded into a
CRITICAL chain. The operator challenged it and the challenge is right:
holding 13 of 20 seats means 13 vetted, KYC'd, UBO-capped operators are
already colluding, which IS the catastrophe the vetting exists to
prevent. Framing "one more vote" as a ladder was close to restating that
the threshold is 14.
I also under-weighted that the "one external vote" is a vote to admit a
candidate who must first pass off-chain vetting with a distinct
beneficial owner - the actual load-bearing control, which I glossed.
Re-rated LOW/INFO. What survives is narrow and worth keeping only as a
note: required(W) = ceil(2W/3) increases by 0 for W congruent to 2 mod 3,
so at W=20->21 admitting a seat does not raise the bar. A property of
specific set sizes, not a capture path.
The code comment is corrected in both directions. It previously asserted
a claim that was too strong, then I replaced it with a warning that was
also too strong. It now states plainly what the threshold buys (a
minority acting alone can never grow itself; the set can never shrink)
and where the trust actually sits (vetting, plus no-shrink).
The chain that finding was part of does NOT collapse - it relocates onto
the half that never needed it: bootstrap founds the permanent,
append-only registry from whichever committee the OLD permissionless
stake-only regime elected, at a moment chosen by a hand-pinned version
floor. Those seats never pass vetting and there is no un-seed. That needs
no ratchet and no admission vote. It is the finding I should have led
with.
Also adds minerharness_test.go (POA_MINE=1): drives the real seat state
machine through 40 randomised runs to emit a {fn,state_before,state_after}
history for the lever3 trace-miner. This retires the earlier
"not-forkable" waiver, which was simply wrong - the miner needs a
transition history, not a fork. 1,115 transitions mined 30 laws (29
novel), including an empirically re-derived append-only property; the
break-loop then ran 2,944 campaigns with mutation-control proven on
30/30 targets and broke none of them.
Caveat recorded in phase1/miner-README.md: the break-loop ran against a
JS model mirroring the Go guards, not against production Go, and the
mined laws are aggregate counters - they say nothing about the
cross-component properties that matter most.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Quality review of what PRUNED (security) does not grade. Two findings, both
fixed; behavior-preserving.
1. isDuplicateSeatErr classified duplicate-seat failures by matching error
MESSAGE SUBSTRINGS ("already holds a seat", "E11000"). That is the fragile
pattern the repo already avoids elsewhere (mongo.IsDuplicateKeyError). A
mis-classification here is not cosmetic: a duplicate is a DETERMINISTIC
refusal, and if a future wrapper changed the message text the classifier
would silently treat it as transient and blockingRetry would wedge block
processing forever — the exact failure the classifier exists to prevent.
poaseats now exports typed sentinels (ErrSeatExists, ErrUboExists);
AdmitSeat wraps them and maps the storage-layer E11000 (insert race) back to
the same sentinel; isDuplicateSeatErr uses errors.Is. Both mocks now return
the same typed errors so a future test cannot pass against classification
the real store would fail.
2. poaSeatElectorate returned a []string seatAccounts that BOTH callers
discarded (one with an explicit _ = seatAccounts). Dropped it to a 2-value
return.
POA tests green after the change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Seven failure-state tests covering "what happens when something is already wrong" — the conditions three of the four criticals in this build hid in. - BootstrapDoesNotHangOnADeterministicDuplicate: proves blockingRetry SURFACES a deterministic ErrSeatExists instead of looping on it (would wedge block processing). Timeout-guarded so a regression fails, not hangs. - ExitHaltFailsClosedWithoutConfig: nil sconf -> HOLD, no panic. - AdmitVoteIgnoresAForeignProposalType + IdsNeverCollide: the admit handler refuses a foreign-typed row in the SHARED governance_proposals store, and admit/reserve-payout ids are type-prefixed so they cannot collide. - RG1_NeverSeatedMemberIsNotHalted_KNOWN_GAP: a CHARACTERIZATION test that pins the current (buggy) ratification-gap behaviour so a future fix is detectable; labelled to be INVERTED when RG-1 is closed. - AdmitVoteNoPanicOnNilDependencies: every partial-wiring combination is a safe no-op. - SeatMaintenanceToleratesDegenerateElectionMembers: duplicate/empty/prefix- only members are sanitised; writing the test surfaced that the MinMembers floor guard correctly refuses a sub-floor distinct set (my first expectation was wrong, the code was right). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The exit-halt armed only from first SEATING (SetSeating at ratification), but a seat is elected at GENERATION height and marked seated only at RATIFICATION. In the window between, LastSeatedHeight is still 0, and the old never-seated branch returned false — so a member could unstake in that window, drain hive_consensus (the slashable pool) to ~0, and still become a committee member one block later with an unslashable bond. The council verified the defeated slash is already wired and testnet-live (double-sign + invalid-block), so this was a HIGH, not a latent finding. Under POA a seat is ELECTABLE the moment it is admitted (the seat gate admits any seat holder), so "holds a seat" is the correct trigger for the halt, not "has been seated once". IsPoaExitHalted now holds any admitted seat, including the pre-seating window an RG-1 attacker occupies. TRADEOFF, flagged for team review: a seat admitted but never elected has its bond held with no timed release (ExitHeight is only set on a seated->absent transition). This errs toward holding collateral (safe) over convenience; a governance release for a genuinely-never-serving operator is a follow-up. It cannot re-open RG-1 — any release keyed on admission height would let the attacker (an admitted seat at unstake time) through again, so the never-seated hold stays unconditional until a real exit or explicit governance action. Two characterization tests that PINNED the vulnerable behaviour are inverted into regression guards (TestExitHaltHoldsAnAdmittedNeverSeatedSeat, TestRG1_AdmittedSeatIsHaltedBeforeFirstSeating). 54 state-processing POA tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lectable The prior fix (6638da1) armed only the never-seated branch, leaving an equivalent RE-ELECTION gap the adversarial round found: a seat seated once, left, and whose halt window has ELAPSED (released) returns not-halted — which is exactly its state during the generation->ratification window of a RE-election. Same drain, different seat state, untouched. Root cause: the seat lifecycle fields (LastSeatedHeight/ExitHeight) only update at ratification, so they LAG the window in which an election is decided but not yet on-chain. Gating on them alone can never cover that window. The complete trigger is ELECTABILITY, not the seat clock: in both the first-election and re-election gaps the attacker must be an electable witness to win the pending election. IsPoaExitHalted now holds the bond whenever the account is an electable witness (isElectableWitness reads the exact set the election proposer draws from — GetWitnessesAtBlockHeight + EnabledOnly, deterministic and freshness-filtered). Only once the operator can no longer be elected (disabled/stale witness) does the clock govern release. This also fixes F2 (the freeze-forever the prior fix introduced): a never-served admitted seat now has a release path — disable the witness, wait a window from admission. And the refusal message is corrected: for an electable hold it tells the operator to disable its witness (the real action), rather than falsely claiming a fixed release height. Fail-closed throughout: nil witness store or a read error holds. Determinism preserved: isElectableWitness is a pure function of on-chain witness state at height, the same call elections already depend on. New regression guards: HoldsAnElectableWitnessAfterWindowElapsed (F1), NeverSeatedSeatReleasesAfterDisablingWitness (F2), TerminatesOnlyWhenNoLongerElectable. Test env gains a fakeWitnesses whose electability is derived from the seat registry with an explicit disable() for wind-down. All state-processing POA tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nt-in-time The electability fix (44db718) sampled isElectableWitness at the UNSTAKE height U, but committee membership is decided at generation G < U. An attacker enabled at G (-> elected) could disable its witness at D in (G,U] and unstake at U, where it reads non-electable and (if the seat clock had elapsed) released -- then be seated at ratification from the frozen-at-G membership with a drained, unslashable bond. The fix does not need the off-chain generation membership. The generation->ratification window is a handful of blocks; the halt window is ~3 days. So the bond is now held while the account is electable now OR has had ANY witness announcement (enabled or just-disabled) within the last `window` blocks -- because within a window of witness activity an election it was in may still be in flight. Release requires the account to have been witness-SILENT for a full window, by which point any in-flight election has provably ratified (and would have re-seated it -> held on the seated branch). No timing seam remains, and this stays entirely in the halt predicate -- no surgery on the audited slash/ledger code. hadRecentWitnessActivity reads GetWitnessAtHeight (the account's latest announcement height); fail-closed on a transient read, a genuine never-announced returns not-held. The refusal message and PoaExitHaltReleaseHeight mirror the same condition. Test env gains dated witness announcements (disable now takes a height); new guard TestRG1c_ExitedElapsedButRecentlyDisabledIsHeld plus the F1/F2 guards updated to the max(seat-clock, disable+window) release. All state-processing POA tests green. This is the third and final iteration on the halt predicate; the earlier two each left a timing seam this closes at the root (witness activity, the only on-chain signal that tracks electability across the generation gap). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- drop debug_data from vsc_blocks collection - remove redundant ledgers from transaction_pool and removed ledger filters in find transactions query - se: early return when executing empty batches - se: memoize schedule - db: add missing indexes on contract state, ledger claims, tss keys and transaction pool - unordered bulk write on hive_blocks - precompiled regex for hive and eth address checks - transaction ingestion improvements - ledger: balance cache retention across txns - tss key lifecycle only runs on new election
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
seprof=debuglog level. Example:debug_datafrom vsc_blocks collectionCGO_LDFLAGSenv var for go tests if on macOS