Skip to content

feat(validator_store): use decided votes for aggregation - #1128

Merged
mergify[bot] merged 7 commits into
sigp:epbsfrom
shane-moore:feat/1113-decided-vote-aggregation
Jul 15, 2026
Merged

feat(validator_store): use decided votes for aggregation#1128
mergify[bot] merged 7 commits into
sigp:epbsfrom
shane-moore:feat/1113-decided-vote-aggregation

Conversation

@shane-moore

@shane-moore shane-moore commented Jul 10, 2026

Copy link
Copy Markdown
Member

Problem, Evidence, and Context (Required)

Committee signing from #1103 uses the QBFT-decided vote, but aggregation still derived aggregate-attestation and sync-contribution requests from the local slot seed. Under Gloas, this can omit the decided AttestationData.index from the requested root or use a block root the committee did not sign, causing missed or mismatched aggregation.

Closes #1113.

Related context:

Change Overview (Required)

  • Cache each committee's successful decision in the slot-scoped voting context. The first decision wins, equal repeats are idempotent, and conflicting repeats fail closed before signature collection.
  • Select one vote per SSV committee during aggregation, falling back to the local slot seed when no decision is available.
  • Deduplicate Beacon API requests using complete identities: aggregate root plus beacon committee index, and block root plus sync subnet.
  • Assemble consensus data deterministically using the existing Go-compatible ordering.
  • Keep wire types, SSZ layout, P2P role and message formats, fetch deadlines, and partial-result behavior unchanged.

The commits are ordered for review: the first establishes the decision handoff and conflict safety, and the second consumes those decisions in aggregation.

Design rationale and data flow

Before this change, aggregation could use the slot's locally selected seed vote for every SSV committee. The fetched aggregate-attestation results were effectively associated by beacon committee index, and sync contributions by subnet. That was sufficient while every SSV committee used the same block and attestation-data inputs.

After #1103, each SSV committee can complete QBFT with its own SlotVote. In Gloas, that vote also contains the decided AttestationData.index. Two committees can therefore request the same beacon committee index or sync subnet against different decided roots. Reusing a result based only on committee index or subnet can attach data fetched for vote A to the consensus data for vote B.

The implementation separates decision retention from fetch-result deduplication:

  1. VotingContext retains the first successful decision for each SSV committee. It is already slot-scoped, so decisions expire with the existing voting context and require no independent eviction policy. If a committee has not decided before aggregation runs, lookup falls back to the slot seed to preserve the previous partial-progress behavior.
  2. MetadataService resolves one vote for each SSV committee, then derives the exact Beacon API requests required by that committee.
  3. Requests are deduplicated globally by their complete API identity:
    • aggregate attestation: (attestation_data_root, beacon_committee_index)
    • sync contribution: (block_root, sync_subnet_id)
  4. Fetched results are held only for the duration of that aggregation build. They are not a long-lived service cache. Each committee reconstructs the same composite keys from its selected vote, so it can consume only matching results.
  5. Matching candidates are passed through the existing deterministic ordering rules before the wire value is built.

The composite keys are intentional. Keying results only by SSV committee would prevent useful sharing when committees agree and would still need another level for multiple beacon committees or sync subnets. Keying only by beacon committee index or subnet is unsafe once committee votes diverge. The complete request identity provides isolation while retaining cross-committee deduplication.

Risks, Trade-offs, and Mitigations (Required)

  • Divergent committee decisions can create more requests than the previous global maps. Exact composite-key deduplication still shares identical requests.
  • Missing decisions deliberately fall back to the local slot seed instead of blocking the duty.
  • Conflicting completed decisions reject that signing attempt to preserve consistency. The cached first value remains unchanged and the conflict is logged with context.
  • Production assembly tests cover foreign-result isolation and exact list association and ordering.

Validation (Required)

  • cargo test -p anchor_validator_store (57 passed)
  • cargo clippy -p anchor_validator_store --tests -- -D warnings -D clippy::allow_attributes
  • cargo fmt --all -- --check
  • git diff --check upstream/epbs...HEAD
  • The first commit independently passed the validator-store test suite and Clippy before the aggregation changes were applied.

New tests cover attestation-first and sync-first decisions, fallback, idempotence, conflicting decisions, Gloas index binding, composite-key deduplication and isolation, partial results, and deterministic ordering.

Rollback (Required for behavior or runtime changes; optional otherwise)

Revert the two commits. The cache is in-memory and slot-scoped, with no database, configuration, or network migration.

Blockers / Dependencies (Optional)

#1103 is merged into epbs. This branch is based on its merge commit, 3b174388.

@shane-moore
shane-moore marked this pull request as ready for review July 10, 2026 18:12
@codecov-commenter

codecov-commenter commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.85542% with 74 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (epbs@3b17438). Learn more about missing BASE report.

Files with missing lines Patch % Lines
anchor/validator_store/src/metadata_service.rs 84.50% 73 Missing ⚠️
anchor/validator_store/src/lib.rs 99.18% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             epbs    #1128   +/-   ##
=======================================
  Coverage        ?   66.21%           
=======================================
  Files           ?      163           
  Lines           ?    29345           
  Branches        ?        0           
=======================================
  Hits            ?    19430           
  Misses          ?     9915           
  Partials        ?        0           
Flag Coverage Δ
rust 66.21% <88.85%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@shane-moore

Copy link
Copy Markdown
Member Author

@claude, please take a look

@claude-code-actions-sigp

claude-code-actions-sigp Bot commented Jul 10, 2026

Copy link
Copy Markdown

Claude encountered an error after 11m 49s —— View job


Claude PR Review

Working on the review...

  • Read PR diff against origin/epbs
  • Review validator_store/src/lib.rs changes (decision cache in VotingContext)
  • Review validator_store/src/metadata_service.rs changes (aggregation vote selection)
  • Review test additions
  • Post inline comments for confirmed issues
  • Publish review summary
    • Branch: feat/1113-decided-vote-aggregation

@shane-moore

Copy link
Copy Markdown
Member Author

@claude, please take a look

@claude-code-actions-sigp

claude-code-actions-sigp Bot commented Jul 11, 2026

Copy link
Copy Markdown

Claude finished @shane-moore's task in 10m 5s —— View job


Claude PR Review

  • Read PR diff against origin/epbs
  • Review validator_store/src/lib.rs changes (decision cache in VotingContext)
  • Review validator_store/src/metadata_service.rs changes (aggregation vote selection)
  • Review test additions
  • Publish review summary

Reviewed the two commits against origin/epbs. This is a clean, well-reasoned change and the design writeup matches the code. The composite-key approach ((attestation_data_root, committee_index) and (block_root, subnet_id)) correctly isolates divergent committee decisions while still deduplicating identical requests, and build_consensus_data_from_candidates recomputes keys per-committee from that committee's own vote, so a result fetched for vote A can never leak into vote B's consensus data. The test coverage for isolation, ordering, fallback, idempotence, and conflict is thorough.

I did not find any correctness or safety issues I'm confident are bugs. A few non-blocking observations below.

Observations (non-blocking)

1. New failure mode injected into the signing hot pathlib.rs:1485-1486

let decided_vote = result?;
voting_context.remember_decided_vote(committee_id, decided_vote.clone())?;

After QBFT successfully decides, a CommitteeDecisionConflict from remember_decided_vote now aborts the entire signing attempt, even though the decision itself is valid. The doc comment argues this can't happen because attestation and sync signing share one QBFT instance and therefore observe the same completion — that reasoning is sound and the fail-closed default is the safe choice. Worth confirming this is the intended trade-off: a hypothetical bug in the shared-instance invariant would now turn a would-be-successful sign into an error rather than degrading silently. No change requested; flagging for visibility.

2. Repeated tree_hash_root in the aggregation buildmetadata_service.rs:591-620

aggregate_fetch_key runs aggregate_fetch_attestation_data + tree_hash_root inside both the retain closure and again in the filter_map over attestation_committee_indexes, so each surviving aggregator's key is hashed roughly twice per committee. n is small here so the cost is negligible, but if you wanted to tidy it you could compute the key once per aggregator up front. Purely optional.

3. Confirm SpecificError::CommitteeDecisionConflict classification — the new variant falls through to the _ => "other_specific_error" arm in instrumentation.rs:87 and NonCollection in classify_ptc_collection_failure. That's correct behavior (no compile break, sensible default), just noting it's intentional given the wildcard arms.

Nothing here blocks merge from my perspective — the core isolation and dedup logic is correct and well-tested.
• Branch: feat/1113-decided-vote-aggregation

@shane-moore

shane-moore commented Jul 11, 2026

Copy link
Copy Markdown
Member Author

@claude, for the three observations:

  1. Fail-closed on remember_decided_vote conflict is intentional. The shared-instance invariant is structural: both signing paths hold the same slot-scoped Arc<VotingContext> (same seed vote, same instance key), and the QBFT instance sends the identical decided value to every on_completed waiter and replays the stored value to post-decision initializers. TimedOut returns before the cache write, so a conflict requires two divergent Success values, which would need the instance map entry cleaned up mid-context plus a different re-decision at the same height. If that ever happened, aborting is preferable to signing a vote that diverges from what aggregation will use.

  2. Keeping the repeated aggregate_fetch_key as-is: a few extra hashes per aggregator once per slot, and recomputing keys from the committee's own vote is the isolation property the design leans on.

  3. CommitteeDecisionConflict cannot reach either wildcard in practice: failure_reason is only called from the proposer spans and classify_ptc_collection_failure only from PTC collection failures, neither of which is on the committee path where this error originates. The arms are compile-time fallthrough only.

@claude-code-actions-sigp

claude-code-actions-sigp Bot commented Jul 11, 2026

Copy link
Copy Markdown

Claude finished @shane-moore's task in 1m 11s —— View job


Responses confirmed

I re-checked each point against the current branch (4603c44). All three hold up.

1. Fail-closed on remember_decided_vote conflict — confirmed intentional and correctly ordered.
lib.rs:1478-1486 returns Err(SpecificError::Timeout) for Completed::TimedOut before result? unwraps, so the cache write at 1486 is only reached on Completed::Success. remember_decided_vote (lib.rs:2207-2231) is first-write-wins, treats an equal repeat as idempotent (entry.get() == &vote → Ok), and only returns CommitteeDecisionConflict on a genuinely divergent second Success at the same height. Your reasoning about the shared-instance invariant matches the structure, and aborting rather than signing a vote that diverges from what aggregation consumes is the right default. Agreed.

2. Repeated aggregate_fetch_key — fine to leave as-is.
Recomputing the key from each committee's own vote is exactly the per-committee isolation the design leans on, and the cost is a handful of hashes once per slot. Not worth trading the clarity of that property for a micro-optimization.

3. CommitteeDecisionConflict never reaches either wildcard in practice — confirmed.

  • failure_reason (instrumentation.rs:64) is only invoked from the proposer_randao_reveal (lib.rs:2696) and proposer_sign_block (lib.rs:2874) spans.
  • classify_ptc_collection_failure (instrumentation.rs:44) is only invoked from the PTC path (lib.rs:874).

The conflict originates in remember_decided_vote on the committee attestation/sync signing path, which is disjoint from both callsites. So _ => "other_specific_error" and _ => NonCollection are compile-time fallthroughs only, as you said — sensible defaults if that ever changes.

Nothing outstanding from my side. All three observations were non-blocking and your responses resolve them.
• Branch: feat/1113-decided-vote-aggregation

@jnhsigmap jnhsigmap left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved b/c none of these are blocking but just some suggestions from claude and analysis 🙂
another one that came up was an integration test for build_consensus_data_for_all_committees to ensure that it feeds each committee's decided vote and not the seed into the fetch keys when the two committees differ.

let mut aggregate_requests = HashSet::new();
let mut sync_requests = HashSet::new();

for ssv_committee_id in &ssv_committees {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit nit-picky but here build_consensus_data_for_all_committees creates and populates committee_votes in the first loop by iterating ssv_committees, then the second loop iterates over ssv_committees again and returns Err if no vote is found for a ssv_committee_id?

  let vote = committee_votes
      .get(&ssv_committee_id)
      .ok_or_else(|| format!("Missing vote for committee {ssv_committee_id:?}"))?;

committee_votes contains all of the ssv_committee_id values that we want, so ok_or_else technically should never be reached. Could we loop over committee_votes directly here?:

  let mut result = HashMap::with_capacity(committee_votes.len());
  for (ssv_committee_id, vote) in &committee_votes {
      let consensus_data = self.build_consensus_data_for_committee(
          slot,
          ssv_committee_id,
          attesters_by_ssv_committee.get(ssv_committee_id),
          sync_by_ssv_committee.get(ssv_committee_id),
          vote,
          &fetch_results,
      )?;
      if let Some(data) = consensus_data {
          result.insert(*ssv_committee_id, Arc::new(data));
      }
  }

@shane-moore shane-moore Jul 15, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved in e62322f7: iterate committee_votes directly and remove the unreachable missing-vote branch.

}

#[test]
fn voting_context_returns_committee_decision_or_seed() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could be worth adding a doc comment like "Tests vote_for_committee lookup rule. Does a committee get its own decided vote back once one is recorded, and does an undecided committee fall back to the slot seed." to make it clear what is being tested?

@shane-moore shane-moore Jul 15, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved in adc3858c: document the committee decision lookup and seed-fallback behavior covered by the test.

}

#[test]
fn build_consensus_data_uses_composite_results_and_preserves_wire_order() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair bit going on here, test itself is sound, just could do with an explanation of objectives.

Objectives:

  1. Foreign-result isolation: Results fetched under a different vote must never be consumed by a committee with foreign_vote.
  2. Wire ordering: De-duplicated/surviving aggregators/contributors and their index lists come out in the exact go-ssv-compatible order as per "The sorting order MUST match SSV-Go exactly for consensus compatibility."

@shane-moore shane-moore Jul 15, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved in 73159658: document the composite-key isolation and go-ssv-compatible wire-ordering objectives.

@mergify

mergify Bot commented Jul 15, 2026

Copy link
Copy Markdown

Queued — the merge queue status continues in this comment ↓.

@mergify

mergify Bot commented Jul 15, 2026

Copy link
Copy Markdown

Merge Queue Status

  • Entered queue2026-07-15 17:00 UTC · Rule: default · triggered by rule Add outdated, ready-to-merge PRs to merge queue
  • Checks passed · on draft merge queue: checking epbs (309f57a) and #1128 together #1141
  • Merged2026-07-15 17:12 UTC · at 1ce66f62e49357f1aec52d4b6a33a5aa34f3d4b9 · squash

This pull request spent 12 minutes 11 seconds in the queue, including 10 minutes 44 seconds running CI.

Required conditions to merge
  • check-success=test-suite-success

mergify Bot added a commit that referenced this pull request Jul 15, 2026
@mergify
mergify Bot merged commit 8f2aeb3 into sigp:epbs Jul 15, 2026
23 checks passed
@mergify mergify Bot removed the queued label Jul 15, 2026
petarjuki7 added a commit to petarjuki7/anchor that referenced this pull request Jul 16, 2026
…ions

epbs (sigp#1082/sigp#1103/sigp#1128) added `spec` and `forced_gloas_index` to the
shared test HarnessOptions. Append `..Default::default()` to the two
ProposerPreferences failure-test constructions, matching the sibling
payload_attestation tests, so the suite compiles on the rebased base.

Part of sigp#1063.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants