Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions docs/snapshot-serving-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@ The peers endpoint is a separate feature and was not implemented in this phase.

### Configuration

A single CLI-only option (not config file — single-use bootstrap):
The endpoint is CLI-only because bootstrap is a single-use operation:

```
--snapshot-endpoint URL Fetch snapshot from URL and bootstrap.
Expand All @@ -261,6 +261,12 @@ A single CLI-only option (not config file — single-use bootstrap):
https://snap.example.com/50000 → fetches block 50000
```

Snapshot bootstrap is an attended operation against an operator-selected endpoint. It reports connection/request phases
and, every five seconds during transfer, downloaded bytes, percentage, rate, and ETA when `Content-Length` is available.
SIGINT cancels pending resolver or socket work and retains atomic temporary-file cleanup. The existing
`chain-state-db-size-mb` setting bounds both fixed-length and chunked response bodies. No snapshot-specific resource
options are added.

The block number is encoded as a trailing path component of the URL. If the last path segment is a decimal number, it's treated as a specific block request (POST to `/v1/snapshot/by_block`); otherwise POST to `/v1/snapshot/latest`.

### Files
Expand All @@ -281,7 +287,9 @@ The block number is encoded as a trailing path component of the URL. If the last
```
This works naturally with `--delete-all-blocks` which clears state before snapshot handling.
3. **Fetch metadata:** POST to `/v1/snapshot/latest` or `/v1/snapshot/by_block` depending on URL format.
4. **Download snapshot:** Uses `fc::http_client::post_to_file()` to POST to `/v1/snapshot/download` and save binary response to local snapshots directory.
4. **Download snapshot:** Uses bounded `fc::http_client::post_to_file()` streaming to POST to
`/v1/snapshot/download`, enforce size and disk-headroom bounds, and atomically save the response to the local
snapshots directory.
5. **Root hash verification:** Uses `threaded_snapshot_reader::load_index()` to read the footer and compare the stored root hash against the advertised `root_hash`. This is a fast metadata-only check that catches download corruption. Full integrity verification (re-hashing all sections) happens during snapshot loading, and on-chain attestation verification happens after syncing.
6. **Continue normal loading:** Sets `snapshot_path` to downloaded file and `snapshot_auto_fetched = true`. No `--genesis-json` needed — snapshot contains genesis.

Expand Down
61 changes: 56 additions & 5 deletions libraries/libfc/include/fc/network/http/http_client.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,44 @@
#include <fc/network/url.hpp>
#include <fc/crypto/blake3.hpp>

#include <cstdint>
#include <filesystem>
#include <functional>
#include <optional>

namespace fc {

/** Current phase of a streamed HTTP file download. */
enum class http_file_download_phase {
connecting,
sending_request,
waiting_for_response,
downloading,
complete,
};

/** Periodic status for a streamed HTTP file download. */
struct http_file_download_status {
/// Current request or transfer phase.
http_file_download_phase phase;
/// Decoded response-body bytes persisted so far.
uint64_t downloaded_bytes;
/// Expected response-body bytes when the peer supplied Content-Length.
std::optional<uint64_t> total_bytes;
/// Time elapsed since the download request started.
microseconds elapsed;
};

/** Resource limits and status reporting for a streamed HTTP file download. */
struct http_file_download_options {
/// Maximum accepted response-body size in bytes.
uint64_t max_response_body_bytes;
/// Retry a failed request once when it used a cached connection. Only enable for idempotent requests.
bool retry_failed_reused_connection = false;
/// Receive phase transitions and periodic transfer status. The callback must not throw.
std::function<void(const http_file_download_status&)> status_callback;
};

class http_client {
public:
http_client();
Expand All @@ -29,19 +63,36 @@ class http_client {
return post_sync(dest, payload_v, deadline);
}

/// Download binary response from a POST request to a file.
/// Uses a temp file during download and renames on completion.
/// Clean up on failure.
/**
* Download a binary POST response using explicit resource limits.
*
* The response is written to a temporary sibling of @p output and renamed only after
* the complete bounded body has been persisted successfully.
*/
void post_to_file(const url& dest,
const variant& payload,
const std::filesystem::path& output,
const time_point& deadline = time_point::maximum());
const http_file_download_options& options);

/**
* Set a predicate used to cancel synchronous HTTP operations.
*
* The predicate is polled while resolver and socket operations are pending. Returning true
* cancels the active operation so callers can interrupt otherwise unbounded requests.
*/
void set_cancel_check(std::function<bool()> cancel_check);

void add_cert(const std::string& cert_pem_string);
void set_verify_peers(bool enabled);

private:
friend struct http_client_test_access;

/// Override the filesystem free-space query for deterministic transport tests.
void set_space_available_provider_for_testing(
std::function<uint64_t(const std::filesystem::path&)> provider);

std::unique_ptr<class http_client_impl> _my;
};

}
}
Loading
Loading