Skip to content

Coconut-first transcoding, commit provenance in drafts#69

Open
magent-cryptograss wants to merge 8 commits into
cryptograss:mainfrom
magent-cryptograss:unified-release-drafts
Open

Coconut-first transcoding, commit provenance in drafts#69
magent-cryptograss wants to merge 8 commits into
cryptograss:mainfrom
magent-cryptograss:unified-release-drafts

Conversation

@magent-cryptograss

Copy link
Copy Markdown
Contributor

Summary

  • Video finalization now tries Coconut.co first (AV1+Opus HLS), falls back to local ffmpeg if unavailable
  • New transcoding_strategy field on ContentFinalizeRequest: auto (default), coconut, local, none
  • New transcoding-submitted SSE event for async Coconut path
  • All draft responses include commit hash (GIT_COMMIT env var) for staleness detection

Note: This PR also includes the libtorrent seeder and webseed commits from multi-step-album-upload that haven't been merged to main yet.

Deployment note

Dockerfile needs GIT_COMMIT baked at build time:

docker build --build-arg GIT_COMMIT=$(git rev-parse --short HEAD) ...

Requires COCONUT_API_KEY in container env for Coconut (graceful fallback to local ffmpeg without it).

Companion PRs

  • pickipedia#61: Unified ReleaseDraft lifecycle
  • blue-railroad-import: Bot promotes drafts to Release pages

Test plan

  • Upload video with COCONUT_API_KEY set → confirm Coconut submission in logs
  • Upload video without key → confirm local ffmpeg fallback
  • Verify commit field in draft API responses
  • Verify transcoding-submitted SSE event on Coconut path

jMyles added 7 commits March 15, 2026 02:21
BT clients construct webseed URLs as base_url + torrent_name + file_path,
but IPFS gateway serves files without the torrent name directory level.

New /webseed/{cid}/ Caddy route rewrites:
  /webseed/{cid}/{torrent_name}/{file} → /ipfs/{cid}/{file}

Also updates enrich endpoint to use the new webseed URL.
Use $1/$2 (uri replacement syntax) instead of {re.1}/{re.2}
(matcher syntax).
BEP 19 webseed behavior differs by torrent type:
- Single-file: client fetches the URL directly
- Multi-file: client appends name/path to the URL

Most releases (23/26) are single video files. Using single-file
format means the webseed URL can point directly to the IPFS gateway
without any path rewriting needed.

Multi-file torrents (albums) still use the /webseed/ rewrite route.
The endpoint now returns the actual webseed URLs used in the torrent,
so Blue Railroad can store them in Release YAML and the PHP renderer
can use them directly in magnet links. This is needed because
single-file and multi-file torrents use different webseed URL formats.
Instead of relying solely on webseeds (which require torrent metadata
that magnet-link-only clients can't bootstrap without peers), this adds
a proper libtorrent seeder that participates in the BitTorrent swarm.

- New seeder service: manages libtorrent session, seeds all enriched
  content from /staging/seeding/{cid}/ directories
- Torrent file serving: GET /torrent/{infohash}.torrent endpoint
- Seeder status: GET /torrent/status for monitoring
- Enrich endpoint now adds torrents to seeder and returns torrent_url
- Dockerfile exposes port 6881 for BitTorrent traffic
- Playbook forwards BT ports (6881 TCP+UDP) and adds /torrent* route
- Added libtorrent dependency to requirements.txt
Restores the AV1+Opus HLS transcoding integration that was in the
original Node.js pinning service but was lost in the Python rewrite.

- POST /transcode-coconut: pins source to IPFS, submits to Coconut API
- POST /webhook/coconut: receives completion, downloads HLS, pins to IPFS
- GET /job/{id}: poll for job status (camelCase to match arthel frontend)
- GET /jobs: list recent jobs
- COCONUT_API_KEY added to config and docker-compose env
- Local ffmpeg transcoding remains as fallback
- Video finalization now tries Coconut.co first (AV1+Opus HLS),
  falls back to local ffmpeg if Coconut unavailable or fails
- New transcoding_strategy field on ContentFinalizeRequest:
  auto (default), coconut, local, none
- New transcoding-submitted SSE event for async Coconut path
- All draft responses include commit hash (GIT_COMMIT env var)
  for staleness detection
- config.get_commit() helper exposes build identity

Depends on Dockerfile baking GIT_COMMIT at build time:
  docker build --build-arg GIT_COMMIT=$(git rev-parse --short HEAD) ...
Returns:
Coconut API response dict
"""
if qualities is None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where can this be passed? What qualities are available? Maybe it makes sense to make this an option in the (I guess, video pinning?) UI?

@jMyles jMyles left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lotta reasonably idiomatic pyhont; not a lot of tests.

- Add transcoding_qualities field to ContentFinalizeRequest — lets
  callers specify output heights (e.g. [1080, 720, 480]). Defaults
  to [720, 480] when not provided. Threaded through to Coconut API.
- Document available quality values and bitrate tiers in coconut.py
- Add test suite: 13 tests covering job config building (default and
  custom qualities, bitrate tiers, AV1+Opus codec enforcement),
  job persistence (save/load/list), model defaults
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants