Skip to content

Show aggregate P2P bandwidth on dashboard #103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tdenisenko merged 1 commit into from
Jul 2, 2026
Merged

Show aggregate P2P bandwidth on dashboard #103

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 45 additions & 48 deletions ROADMAP.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,78 +4,75 @@

LogEx starts from a recent consensus checkpoint, tracks the live head, reverse-syncs execution history toward genesis, stores compressed verified logs, and serves the dashboard, SQL query API, JSON-RPC, gRPC, and live ERC20 transfer subscriptions.

Current branch: `fix/auto-refresh-stale-checkpoint`.
Current branch: `feature/dashboard-p2p-bandwidth`.

The Mac mini is running this branch against the saved full-sync data directory. Public dashboard forwarding through `157.245.195.72:18683` is active, and the client can bridge a stale restart gap from the saved execution head to a refreshed consensus checkpoint.
This branch updates the dashboard bandwidth tile and `/status` payloads so P2P bandwidth includes download and upload rates across execution sync, historical execution sync, and consensus sync.

## Completed Since Last Run

- Added automatic startup recovery for stale persisted consensus state and stale local execution progress.
- Added a checkpoint-gap bridge that validates the EL parent chain up to a fresh CL checkpoint anchor before ingesting the missing logs.
- Pipelined checkpoint-gap body/receipt fetching with bounded lookahead so long-offline forward catch-up no longer waits for one chunk to fully ingest before requesting the next.
- Fixed checkpoint-gap progress accounting so dashboard logs/sec is sampled once per ingested batch instead of once per block after a batch write.
- Kept the existing EL/log storage and known peer cache intact during checkpoint refresh.
- Verified the remote dashboard path through the VPS and restarted the native Mac mini client with the fix.
- Renamed the dashboard metric from `P2P download` to `P2P bandwidth`.
- Added aggregate dashboard display for download and upload throughput in Mbps.
- Added consensus-layer P2P payload download/upload rates and cumulative totals to `/status`.
- Added execution-layer upload accounting for outbound requests and data served from the local serve cache.
- Included execution header responses in the existing execution download metric.
- Added server/status and serve-cache test coverage for the new fields.

## Remaining TODOs

No remaining TODOs for the stale-checkpoint restart recovery task.
- Merge the bandwidth dashboard task after review/CI.
- Reason: The code is implemented and locally validated, but the branch still needs the normal PR merge workflow.
- Completion criteria: PR is created, checks pass, and the branch is merged into `master`.

## Design Decisions

- Archive stale `cl/consensus_state.json` instead of deleting it.
- Why: The client can recover automatically while preserving a diagnostic copy of the old trusted state.
- Alternatives considered: fail startup and require manual data-dir surgery. That was operationally fragile and caused the dashboard outage.
- Tradeoff: The data directory may retain a small archived consensus snapshot after recovery.
- Track payload-level P2P bandwidth instead of OS network-interface throughput.
- Why: The dashboard should reflect data LogEx processes, not unrelated host traffic or encrypted transport overhead.
- Alternatives considered: polling system network counters. That would include non-LogEx traffic and vary by OS.
- Tradeoff: Mbps is application payload throughput, not exact TCP wire bytes.

- Bridge long-offline gaps with EL parent-chain validation ending at a fresh CL checkpoint anchor.
- Why: CL historical sync is intentionally not required; the fresh checkpoint execution hash is enough to validate the canonical EL ancestor chain back to the saved head.
- Alternatives considered: require a fresh data directory, or reintroduce CL historical sync. Both add unnecessary operational cost for this restart case.
- Tradeoff: The bridge is correctness-first and less optimized than the normal historical pipeline.
- Aggregate EL and CL bandwidth in the dashboard instead of replacing individual network-layer fields.
- Why: Existing API consumers can still inspect layer-specific data, while the main UI shows the user-facing total.
- Alternatives considered: a single top-level bandwidth field. That would hide useful debugging detail.
- Tradeoff: UI aggregation must handle missing per-layer fields as zero.

- Account checkpoint-gap progress per contiguous batch.
- Why: The gap bridge ingests rows in batches; per-block progress updates after a batch write distort live logs/sec because each block update can be separated by only microseconds.
- Alternatives considered: keep per-block progress updates. That made the dashboard report impossible rates during catch-up.
- Tradeoff: The live forward metric is chunk-granular during restart recovery, which matches the actual batch-oriented work.
- Use short rolling windows for CL and EL upload rates.
- Why: Upload events are bursty, especially when serving peers or sending small RPC requests.
- Alternatives considered: cumulative average since startup. That would be too stale for a live dashboard.
- Tradeoff: The displayed upload rate drops to zero when no recent upload payloads were observed.

## Challenges and Resolutions

- Challenge: A full synced data directory failed to restart after being offline because the persisted consensus state and local execution head were older than the recent checkpoint window.
- Resolution: Startup now resolves a fresh checkpoint, archives stale CL state, and resumes using the existing EL/log storage.
- Remaining: None known for correctness.

- Challenge: The first recovery attempt hit the consensus reorg guard because the fresh CL anchor was ahead of the persisted recent-header window.
- Resolution: Future-only anchors are classified as restart gaps, then bridged by fetching and validating the EL header chain to the CL anchor.
- Challenge: The prior dashboard metric only showed execution-layer download payloads.
- Resolution: Added upload fields, CL bandwidth fields, and aggregate UI formatting.
- Remaining: None known.

- Challenge: The dashboard briefly reported impossible multi-billion logs/sec during checkpoint-gap catch-up.
- Resolution: Progress tracking now supports batched forward updates, and the gap bridge records one live rate sample per ingested chunk.
- Remaining: None known.
- Challenge: Execution upload is served partly through Reth provider callbacks.
- Resolution: Instrumented `ServeCacheProvider` return paths and outbound request helpers to account for local EL P2P upload payloads.
- Remaining: Payload sizes are estimates of decoded protocol payloads, not encrypted TCP bytes.

## Dead Code and Obsolescence Cleanup

- Inspected the stale startup guards, consensus reorg classifier, checkpoint-gap bridge, and progress tracker.
- No obsolete code was safely removable in this pass; the new bridge reuses existing validation, peer, and storage primitives.
- Removed no files.
- Inspected the old `P2P download` UI labels and status fields.
- Replaced obsolete dashboard copy with `P2P bandwidth`.
- Kept existing download fields for compatibility and added upload fields rather than renaming API keys.
- No files were removed.

## Git Workflow

- Current branch: `fix/auto-refresh-stale-checkpoint`.
- New branch created from `master`.
- Commits made during this run:
- `7dc7c6fd fix: recover stale checkpoint restarts`
- `af3a59a3 fix: pipeline stale checkpoint catchup`
- Pull request status: draft PR #102 opened for `fix/auto-refresh-stale-checkpoint` into `master`.
- Merge status: not merged yet.
- Current branch: `feature/dashboard-p2p-bandwidth`.
- New branch created from latest `master`.
- Commits made during this run: pending.
- Pull request status: pending.
- Merge status: pending.
- Validation run:
- `cargo test -p logex-node archived_consensus_state`
- `cargo test -p logex-node restart_guard`
- `cargo test -p logex-sync locate_consensus_reorg`
- `cargo test -p logex-sync checkpoint_gap_pipeline_depth`
- `cargo test -p logex-sync forward_batch_progress_records_one_live_rate_sample`
- `cargo clippy -p logex-node -p logex-sync --all-targets -- -D warnings`
- `cargo fmt --check`
- `cargo check -p logex-types -p logex-cl -p logex-sync -p logex-server`
- `cargo test -p logex-server`
- `cargo test -p logex-sync p2p::serve_cache`
- `cargo test -p logex-cl network::tests`
- `cargo clippy -p logex-types -p logex-cl -p logex-sync -p logex-server -- -D warnings`

## Known Issues or Risks

- The checkpoint-gap bridge is only for long-offline restart recovery. Normal historical reverse sync and live tip-following paths are unchanged.
- The local Codex sandbox cannot directly curl the public VPS dashboard, but the VPS can reach `10.66.0.2:18683` and packet capture showed public TCP/18683 traffic being forwarded and answered.
- Bandwidth metrics report decoded application payload bytes, not full encrypted TCP wire bytes or host network-interface counters.
- The execution upload metric estimates served receipt payloads from encoded receipts and computed blooms; it is intended for dashboard throughput visibility, not byte-perfect packet accounting.
Loading
Loading