feat: smart HTTP file cache with parallel downloads and 20% pre-cache trigger - #203
Open
tsirysndr wants to merge 12 commits into
Open
feat: smart HTTP file cache with parallel downloads and 20% pre-cache trigger#203tsirysndr wants to merge 12 commits into
tsirysndr wants to merge 12 commits into
Conversation
tsirysndr
commented
May 27, 2026
tsirysndr
commented
Owner
- Add rockbox-cache crate: SHA-256 URL keying, parallel Range-request downloads, LRU eviction, disk-space guard, atomic rename, auto-skip for live streams (no Content-Length) and configurable no_cache_patterns
- netstream: check cache on open, serve from BufReader on hit, start background fetch on miss
- settings: wire all 6 cache fields (enabled, dir, max_size_mb, min_free_space_mb, parallel_parts, no_cache_patterns) from settings.toml
- cli: spawn precache-monitor thread — at 20% through current track, pre-fetch the next HTTP track in the background
- docs: add HTTP file cache section to mintlify/configuration.mdx and README
… trigger - Add rockbox-cache crate: SHA-256 URL keying, parallel Range-request downloads, LRU eviction, disk-space guard, atomic rename, auto-skip for live streams (no Content-Length) and configurable no_cache_patterns - netstream: check cache on open, serve from BufReader<File> on hit, start background fetch on miss - settings: wire all 6 cache fields (enabled, dir, max_size_mb, min_free_space_mb, parallel_parts, no_cache_patterns) from settings.toml - cli: spawn precache-monitor thread — at 20% through current track, pre-fetch the next HTTP track in the background - docs: add HTTP file cache section to mintlify/configuration.mdx and README
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
…pc crate Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…t dir On Android, HOME is set to the app sandbox root (not a Unix home dir), so $HOME/.config/rockbox.org is wrong for any crate whose Lazy static fires before configure_environment runs. - configure_environment now sets ROCKBOX_CONFIG_DIR to $config_dir/.config/rockbox.org before any threads start - CacheConfig::with_defaults() prefers ROCKBOX_CONFIG_DIR, falling back to $HOME/.config/rockbox.org on desktop/WASM where the var is unset - load_settings default cache dir uses the same logic
- with_defaults() now calls create_dir_all so the dir exists from the first CACHE static access, regardless of whether configure() is called - start_background_fetch: all skip paths now log at info level so the reason is always visible in the default log filter - perform_download: no-Content-Length skip promoted from debug to info (catches transcoded Navidrome streams that never have a content-length) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Create ~/.config/rockbox.org/cache in parse_args() immediately after tracing is initialized — this is the very first Rust code that runs, so the dir exists before load_settings, configure(), or any HTTP open. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…sibility librockbox_server.a and librockbox_cli.a each compile their own copy of tracing_core, giving them separate GLOBAL_INIT statics. parse_args() only sets the CLI copy's dispatcher, so all logs from server-compiled code (rockbox_cache::start_background_fetch, configure, etc.) were silently dropped. Initialize the server copy's dispatcher in start_server() so cache, netstream, and server logs become visible. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…sibility librockbox_server.a and librockbox_cli.a each compile their own copy of tracing_core, giving them separate GLOBAL_INIT statics. parse_args() only sets the CLI copy's dispatcher, so all logs from server-compiled code (rockbox_cache::start_background_fetch, configure, etc.) were silently dropped. Initialize the server copy's dispatcher in start_server() so cache, netstream, and server logs become visible.
…tent-Length Some servers omit Content-Length on HEAD but respond to a range probe with 206 + Content-Range: bytes 0-0/<total>. Parse the total from the Content-Range header so these files are still cached instead of being skipped as live streams. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…Length Transcoding servers like Navidrome advertise one Content-Length on HEAD but stream a slightly larger payload (e.g. 11 527 315 vs 11 617 210). The strict downloaded != content_length check was discarding every completed download as "incomplete". Changed to downloaded < content_length so over-delivery is accepted and the .tmp is promoted to .cache.
tsirysndr
force-pushed
the
master
branch
2 times, most recently
from
July 12, 2026 17:58
9e2abea to
b23d182
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.