chore(main): release 0.3.0 - #23
Closed
github-actions[bot] wants to merge 756 commits into
Closed
Conversation
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the
Comment |
github-actions
Bot
force-pushed
the
release-please--branches--main--components--platform
branch
16 times, most recently
from
January 16, 2026 14:11
9220655 to
d0ed82e
Compare
github-actions
Bot
force-pushed
the
release-please--branches--main--components--platform
branch
10 times, most recently
from
January 22, 2026 10:27
4963fe0 to
1b9d0d9
Compare
github-actions
Bot
force-pushed
the
release-please--branches--main--components--platform
branch
2 times, most recently
from
February 3, 2026 15:13
502898c to
fec7d93
Compare
- Add P2PSender trait with try_send() method to network.rs - Implement RealP2PSender wrapping mpsc::Sender<P2PCommand> - Implement MockP2PSender for testing (cfg(test)) - Export P2PSender, RealP2PSender from lib.rs - Update main.rs to use Arc<dyn P2PSender> - Pure refactor - no behavior change - Foundation for Task 5: send_timeout() implementation
- Add p2p_channel_capacity (default 4096) to P2PConfig - Add heartbeat_channel_capacity (default 256) for future use - Add rpc_p2p_channel_capacity (default 64) to P2PConfig - Update main.rs to use config values for channel creation - Default behavior unchanged - foundation for Tasks 9, 10
- Add ChannelMetrics struct with atomic counters - Track capacity, dropped_count, send_count - Update RealP2PSender to increment counters - Add with_metrics() constructor for tracking - Add 6 unit tests for metrics functionality - Log warning when messages dropped
CORE FIX for channel overflow bug: - Add MessagePriority enum (CRITICAL/HIGH/MEDIUM/LOW) - Add send_timeout() to P2PSender trait - Map P2PMessage variants to priorities - CRITICAL: 500ms (Proposal, PrePrepare, Prepare, Commit) - HIGH: 200ms (StateMutationProposal/Vote) - MEDIUM: 100ms (Heartbeat, WeightVote) - LOW: 10ms (StorageProposal, StorageVote) - Update tests to use send_timeout with consumers - TDD green: all 5 channel_overflow tests pass
- Change default p2p_channel_capacity from 4096 to 16384 - Add comment: "4x increase for burst absorption" - Update in Default impl, development(), production() - Provides larger buffer for burst handling
4x increase to reduce RPC bottleneck Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- Add RetryQueue with exponential backoff (500ms→1s→2s→4s→5s) - Max 5 retries before giving up with error log - Add retry_count to ChannelMetrics - Queue only CRITICAL priority messages - process_retries() for external retry handling - Add 11 unit tests for retry logic
- Add WARN-level logs for heartbeat message rejections (was DEBUG) - Add heartbeat_rejections_total counter metric for monitoring - Make stale threshold configurable via VALIDATOR_STALE_THRESHOLD_MS env var - Change default stale threshold from 90s to 300s (5 minutes) - Add unit test for configurable threshold Root cause: Validators were marked inactive after 90s without visible logs showing why heartbeats were rejected. Now operators can diagnose P2P issues and configure appropriate thresholds for their network conditions. Fixes issue where all validators became inactive simultaneously due to network hiccups or message validation failures going unnoticed.
- Add process_retries() periodic task (500ms interval) - Wire ChannelMetrics with RealP2PSender::with_metrics() - Replace all 15 try_send() calls with send_timeout() by priority: - HIGH (200ms): StateMutationProposal/Vote (2 calls) - MEDIUM (100ms): Heartbeat (1 call) - LOW (10ms): Storage/Challenge broadcasts (12 calls) - Fix bug: increment dropped_count when max retries exceeded - Add retry queue integration tests Fixes P2P broadcast channel overflow that was dropping messages without backpressure, causing 'no available capacity' errors.
- Add PendingWeight struct with epoch, weights, attempt tracking - Implement WeightSubmissionQueue with enqueue/dequeue/mark_submitted - Duplicate epoch rejection to prevent double submission - Immediate durability via flush() after every write - 7 unit tests covering queue operations and crash recovery
- Add WeightCommand enum with Submit, HotkeyChanged, Shutdown variants - Implement WeightTaskHandle for controlling background task - spawn_weight_task() creates tokio task for async weight submission - Exponential backoff: 10s -> 20s -> ... -> 60s capped - Transport errors trigger retry, permanent errors stop immediately - mark_submitted() only called after ALL mechanisms succeed
- Add WeightSubmissionQueue with sled persistence for crash recovery - Spawn weight_task before main select! loop - Replace 447 lines of inline submission with enqueue() call - Add graceful shutdown handling - Remove old is_transport_error function (moved to weight_task) Weights are now submitted in a background task, decoupled from P2P event processing. The queue persists to disk, surviving crashes.
- test_duplicate_epoch_rejected: verify dedup contract - test_crash_recovery_persistence: verify queue survives process kill - test_hotkey_rotation_clears_queue: verify hotkey change clears queue
- Create background_weight_handler.rs with hourly timer loop - Check every 60s, trigger at minute==0 (heure pile) - Fetch weights from chain.platform.zip RPC via subnet_getWeights - Automatic retry on RPC failure (no WASM fallback) - Dedup protection via last_submission_hour tracking - Graceful shutdown handling Co-authored-by: Platform Bot <bot@platform.network>
- Add minute zero detection tests (heure pile) - Add hour epoch calculation tests - Add deduplication prevention tests - Add RPC response parsing tests for error handling All 9 tests passing. Co-authored-by: Platform Bot <bot@platform.network>
The previous limit of 100 messages per second was too low for active validators running consensus, storage sync, and heartbeats. Bootstrap validator was getting rate limited, blocking all P2P traffic. Changes: - DEFAULT_RATE_LIMIT: 100 -> 10000 (100x increase) - RATE_LIMIT_WINDOW_MS: 1000 -> 10000 (1s -> 10s window) This allows ~1000 messages/sec sustained throughput.
Add independent weight submission thread completely decoupled from P2P rate limiting and block sync: - Create standalone_weight_submitter.rs module - Timer checks every 60s for minute() == 0 (heure pile) - Fetches weights from chain.platform.zip/rpc - Submits directly via Subtensor::set_weights - Retry logic with backoff [5s, 10s, 30s] - Graceful shutdown handling Integration in main.rs: - Spawn after subtensor/signer creation - Shutdown in Ctrl+C handler This ensures weights are submitted hourly even when P2P is rate limited or experiencing issues.
Change from direct set_weights to WeightSubmitter which handles: - CRv4 timelock encryption automatically - Chain auto-reveals when DRAND pulse arrives - No manual reveal needed The WeightSubmitter detects chain version and encrypts weights with DRAND-based timelock encryption.
- Change health check interval from 300s to 60s (5 blocks at 12s/block) - Change threshold from 300s to 60s for faster dead connection detection - Add unit tests for health check timeout behavior Fixes connection death after ~547s where 300s health check plus websocket idle timeout caused validator event loop to stop receiving block events. Now detects and reconnects within 60s of block silence.
…sion - Store config (endpoint, netuid, signer_seed) instead of persistent connection - Create fresh SubtensorClient in fetch_and_submit() for each hourly submission - Connection automatically dropped after each submission - Prevents stale connection issues - each submission starts fresh - CRv4 commit-reveal still works via WeightSubmitter
- BlockSync now reconnects on health check failure (not just subtensor) - Weights submitted immediately at startup (not just hourly) - Prevents complete block event starvation after WebSocket death - Fresh BlockSync ensures blocks resume flowing after reconnect Both issues could leave validator in zombie state: 1. Old: BlockSync never reconnected, only metagraph client 2. Old: No weights at startup, waited up to 60min for first submit
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This release is too large to preview in the pull request body. View the full release notes here: https://github.com/PlatformNetwork/platform/blob/release-please--branches--main--components--platform--release-notes/release-notes.md