delivery-kid: port Coconut submit to V2 schema (preview path only)#90
Merged
jMyles merged 1 commit intoMay 31, 2026
Merged
Conversation
Coconut V2 rejects the V1 job shape with a chain of 400s — verified
iteratively against api.coconut.co/v2/jobs:
- "The job 'notification' is not a valid object."
→ V2 renamed webhook → notification: {type, url}
- "[mp4_preview] Format Container 'mp4_preview' is not valid."
→ outputs dict key is the format-spec (e.g. mp4, mp4:480p,
httpstream), not a free-form label
- "You must define the 'storage' URL if you specify an output with a 'path'."
→ V2 requires explicit storage; defaults are gone
- "The storage service is not valid." (the helpful one — lists every
valid service: s3, gcs, wasabi, dospaces, coconut, ipfs_pinata, …)
- "Output param key 'video' is not valid."
→ V2 outputs don't accept nested video/audio blocks; codec/bitrate
customization goes into the format-spec key instead
The minimal V2 shape that gets 201 from Coconut:
{
"input": {"url": "..."},
"storage": {"service": "coconut"},
"outputs": {"mp4": {"path": "/preview.mp4"}},
"notification": {"type": "http", "url": "..."}
}
This PR ports only the preview path (include_preview=True). The HLS /
finalize path uses an httpstream output block whose exact shape needs
more discovery work — left raising NotImplementedError so failure is
loud rather than silent.
Also: rewrap the 4xx error so Coconut's response body is included in
the exception message. The previous resp.raise_for_status() lost the
body and the diagnostics panel only saw a generic httpx message —
which is exactly what made today's iterative debugging painful.
Tested with a real POST to V2 (HTTP 201, job id returned). Webhook
callback shape on the V2 side may or may not match what
_update_draft_preview expects today — to be discovered on the next
real upload + observed.
6 tasks
jMyles
added a commit
that referenced
this pull request
Jun 1, 2026
…fast path Three related fixes for the regression that produced empty Release pages (the Nine Pound Hammer case: Release CID points at an IPFS dir containing only metadata.json, no playable content). ## 1. submit_to_coconut: V2 httpstream for finalize Replace #90's NotImplementedError. Verified against api.coconut.co/v2/jobs that the minimal V2 shape returns HTTP 201: { "input": {"url": "..."}, "storage": {"service": "coconut"}, "outputs": {"httpstream": {"hls": {"path": "/hls"}}}, "notification": {"type": "http", "url": "..."} } Quality variants / trim / codec customization aren't yet exposed because the V2 schema for those inside httpstream isn't documented and we get 201 on the minimal shape. Falls back to Coconut V2 defaults for now; warning logged when callers pass qualities/trim that won't land. Follow-up to expose those. ## 2. Disable the finalize fast path The fast path at finalize_sse_generator was reusing state.preview_cid as the Release CID when wants_transcode + preview_cid + no trim. Safe pre-V2 (preview produced full HLS), broken post-V2 (preview produces a single mp4 — different artifact than finalize). Worse: when the V2 webhook download wasn't ported yet, preview_cid pointed at an IPFS dir containing only metadata.json, and finalize blindly reused that. That's the chain that produced the empty Release page. Always go through the finalize transcode path until preview_cid can be trusted to point at finalize-grade HLS. ## 3. download_hls_outputs: handle V2 callback shape Coconut V2's webhook callback returns ``outputs`` as a list of dicts (``[{"key": "httpstream", "url": ..., ...}]``), not the V1 dict shape (``{"hls_av1_720p": {"url": ...}}``). Normalise both into a uniform list, recognise the httpstream key, and chase the master playlist → variant playlists → segments. Drops V1 keys silently. Added _download_hls_variants_from_master that parses EXT-X-STREAM-INF entries from the master playlist and fetches each variant + segments. ## Why this is the right scope right now This is the minimum to get finalize producing real Release pages with real HLS content again, in time for tour shows. The variant/codec/trim customization is a clear follow-up but doesn't block uploads working.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replace
submit_to_coconutbody with the minimal V2 schema we just verified end-to-end againstapi.coconut.co/v2/jobs(HTTP 201 on a real POST).Also: capture Coconut's 4xx response body in the raised exception so the diagnostics panel surfaces the real reason instead of httpx's generic
Client error '400 Bad Request' for url ....Discovery summary
V2 rejected V1's shape with a chain of 400s, each pointing at one rename/restructure:
The job 'notification' is not a valid object.webhook→notification: {type, url}[mp4_preview] Format Container 'mp4_preview' is not valid.outputsdict key is the format-spec (mp4,mp4:480p,httpstream), not a free-form labelYou must define the 'storage' URL if you specify an output with a 'path'.storage— no default hostingThe storage service is not valid. Valid services are s3, gcs, wasabi, dospaces, …, coconut, ipfs_pinata, ipfs_infura.coconutis a valid serviceOutput param key 'video' is not valid.video/audioblocks anymoreMinimal V2 shape that gets 201:
```json
{
"input": {"url": "..."},
"storage": {"service": "coconut"},
"outputs": {"mp4": {"path": "/preview.mp4"}},
"notification": {"type": "http", "url": "..."}
}
```
Scope
Preview path (
include_preview=True): fully V2, single 480p MP4 output, no nested codec blocks. Verified against the real API.HLS / finalize path (
qualities,trim_start,trim_end): raisesNotImplementedError. V2 uses anhttpstreamoutput block with HLS+DASH variants whose exact codec/variant shape needs more discovery. Punted to a follow-up PR rather than guessing and re-introducing the silent-400 hell we just escaped.Today's failure mode:
NotImplementedError→ upload_log gets a clear "V2 HLS not yet ported" entry → diagnostics panel surfaces it cleanly. No worse than the silent 400 it produced before, much louder to debug.What may still need to change after deploy
The V2 job creation shape is nailed down. The webhook callback shape from V2 is not yet observed —
_update_draft_previewmay need updates to handle a different payload structure (e.g.outputsarrived as an array in the creation response, not a dict; the callback might do the same).Recommendation: deploy this, retry the upload, and observe the next failure (if any). The improved error capture should make the next iteration ~30 seconds rather than another curl-discovery session.
Why now
Pre-existing breakage of the entire preview transcoding path — any video upload reaches
preview_status: failedimmediately. Surfaced today by Sky's diagnostics-panel PR #82 finally rendering the underlying httpx 401/400 messages, which let us see exactly what Coconut was rejecting.Test plan
--rebuildneeded — Python change only)preview_statusadvances pastfailedand a Coconut job ID appears in the preview_logpreview_status: ready, side-effect: bot snapshot fires,ReleaseDraft:{id}/diagnosticssub-page should materialize (validates the delivery-kid: snapshot draft diagnostics to wiki sub-page #87 bot path too)