Add /staging/* to Caddy route allowlist#74
Open
magent-cryptograss wants to merge 14 commits into
Open
Conversation
BT clients construct webseed URLs as base_url + torrent_name + file_path,
but IPFS gateway serves files without the torrent name directory level.
New /webseed/{cid}/ Caddy route rewrites:
/webseed/{cid}/{torrent_name}/{file} → /ipfs/{cid}/{file}
Also updates enrich endpoint to use the new webseed URL.
Use $1/$2 (uri replacement syntax) instead of {re.1}/{re.2}
(matcher syntax).
BEP 19 webseed behavior differs by torrent type: - Single-file: client fetches the URL directly - Multi-file: client appends name/path to the URL Most releases (23/26) are single video files. Using single-file format means the webseed URL can point directly to the IPFS gateway without any path rewriting needed. Multi-file torrents (albums) still use the /webseed/ rewrite route.
The endpoint now returns the actual webseed URLs used in the torrent, so Blue Railroad can store them in Release YAML and the PHP renderer can use them directly in magnet links. This is needed because single-file and multi-file torrents use different webseed URL formats.
Instead of relying solely on webseeds (which require torrent metadata
that magnet-link-only clients can't bootstrap without peers), this adds
a proper libtorrent seeder that participates in the BitTorrent swarm.
- New seeder service: manages libtorrent session, seeds all enriched
content from /staging/seeding/{cid}/ directories
- Torrent file serving: GET /torrent/{infohash}.torrent endpoint
- Seeder status: GET /torrent/status for monitoring
- Enrich endpoint now adds torrents to seeder and returns torrent_url
- Dockerfile exposes port 6881 for BitTorrent traffic
- Playbook forwards BT ports (6881 TCP+UDP) and adds /torrent* route
- Added libtorrent dependency to requirements.txt
Restores the AV1+Opus HLS transcoding integration that was in the
original Node.js pinning service but was lost in the Python rewrite.
- POST /transcode-coconut: pins source to IPFS, submits to Coconut API
- POST /webhook/coconut: receives completion, downloads HLS, pins to IPFS
- GET /job/{id}: poll for job status (camelCase to match arthel frontend)
- GET /jobs: list recent jobs
- COCONUT_API_KEY added to config and docker-compose env
- Local ffmpeg transcoding remains as fallback
- Video finalization now tries Coconut.co first (AV1+Opus HLS), falls back to local ffmpeg if Coconut unavailable or fails - New transcoding_strategy field on ContentFinalizeRequest: auto (default), coconut, local, none - New transcoding-submitted SSE event for async Coconut path - All draft responses include commit hash (GIT_COMMIT env var) for staleness detection - config.get_commit() helper exposes build identity Depends on Dockerfile baking GIT_COMMIT at build time: docker build --build-arg GIT_COMMIT=$(git rev-parse --short HEAD) ...
- Add transcoding_qualities field to ContentFinalizeRequest — lets callers specify output heights (e.g. [1080, 720, 480]). Defaults to [720, 480] when not provided. Threaded through to Coconut API. - Document available quality values and bitrate tiers in coconut.py - Add test suite: 13 tests covering job config building (default and custom qualities, bitrate tiers, AV1+Opus codec enforcement), job persistence (save/load/list), model defaults
The regex \w+\d*\.hunter only matched single-level subdomains like magent1.hunter.cryptograss.live but not pickipedia.justin.hunter. Changed to [\w.-]+ to match any depth of subdomain.
Add action parameter to HMAC token creation/verification to support separate "upload:" and "finalize:" token prefixes. Upload tokens allow any logged-in wiki user to push bytes to staging. Finalize tokens (issued only to users with finalize-release permission) are required to pin to IPFS. New require_finalize_auth dependency used on /finalize endpoints. Backwards-compatible: old upload tokens still accepted for finalize with a deprecation warning.
Deploy pickipedia and delivery-kid together — no need to accept upload-prefixed tokens at finalize endpoints.
New GET /staging/drafts/{draft_id}/{filename} endpoint serves uploaded
files from staging for preview (video embed, audio playback) on wiki
ReleaseDraft pages.
Supports both header auth (HMAC/API key/wallet) and query param auth
(?token=&user=×tamp=) so <video src="..."> tags work without
JS fetch workarounds. Path traversal protection via component
sanitization and resolved-path validation. FileResponse handles
HTTP range requests natively for video seeking.
…llback - Use a dict for MIME type registration instead of repeated add_type calls - Move require_auth import to module top level (no circular import issue) - Add comment clarifying why HTTPException is caught and passed
The staging endpoint was deployed but Caddy's explicit path allowlist didn't include it, so requests fell through to the default fallback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The staging file endpoint (#73) was deployed but Caddy's explicit path allowlist didn't include
/staging/*, so requests fell through to the default "delivery-kid pinning service" fallback response instead of reaching the FastAPI app.One-line fix: add
reverse_proxy /staging/* localhost:3001to the Caddyfile template.