Skip to content

delivery-kid: webhook writes finalized state to ReleaseDraft wiki YAML#96

Merged
jMyles merged 2 commits into
cryptograss:productionfrom
magent-cryptograss:finalize-write-wiki-yaml
Jun 1, 2026
Merged

delivery-kid: webhook writes finalized state to ReleaseDraft wiki YAML#96
jMyles merged 2 commits into
cryptograss:productionfrom
magent-cryptograss:finalize-write-wiki-yaml

Conversation

@magent-cryptograss

Copy link
Copy Markdown
Contributor

Why

#95 closed half the gap — `_update_draft_finalize` now writes `status=finalized` + `final_cid` into delivery-kid's `draft.json` when the Coconut webhook arrives. But the wiki `ReleaseDraft:{id}` page YAML stays on the pre-finalize state because the historical "write the YAML edit" step was done by browser-side JS responding to an SSE `complete` event that the slow Coconut path never sends.

The Blue Railroad Imports bot's `find_cid_from_history` greps wiki edit summaries for "Finalized: pinned to IPFS as " to know which drafts to promote. With no such edit ever being made for V2-Coconut finalizes, no Release page ever appears, even though the HLS has been pinned end-to-end.

What

Three connected changes:

1. New `pickipedia_client.write_finalized_to_releasedraft` — loads the current ReleaseDraft YAML, sets the three finalized-state fields (`status`, `final_cid`, `finalized_at`), saves with the bot-trigger summary. Preserves all user-supplied YAML fields by parse+update+dump rather than rewriting.

2. `_update_draft_finalize` fires the wiki write as an asyncio task right after the existing draft.json update and diagnostics snapshot. Same fire-and-forget pattern as the snapshot — wiki blips don't mask the underlying finalize outcome.

3. Stage-by-stage `finalize_log` entries so the wiki diagnostics panel narrates the webhook-side work instead of jumping straight from "transcoding-submitted" to "complete". All entries land in a single `draft.json` save (we don't have a stream back to the browser; the panel picks them up on its next poll).

Also: PyYAML added to requirements.txt — not used elsewhere in pinning-service, so wasn't guaranteed installed.

Test plan

  • Deploy delivery-kid with `--rebuild` (new PyYAML dep needs to be baked into the image)
  • Re-finalize the terrapin draft
  • Watch `docker logs pinning-service`: expect `pickipedia_client: marked ReleaseDraft:{id} finalized` after the Coconut webhook
  • Wiki page revision history should show a "Finalized: pinned to IPFS as Qm..." edit by Magent@magent
  • On the next Blue Railroad Imports cron run, `Release:{hls_cid}` appears
  • New Release page resolves to a directory with `master.m3u8` + variant playlists + segments — real playable video, not a 421-byte `metadata.json` again

Known followups (not in this PR)

  • Coconut V2 quality variants (`mp4:480p`, `mp4:720p` style) — current finalize uses Coconut V2 defaults
  • Trim (V2 syntax to be discovered)
  • The modal UI on the wiki page still flashes through stages because it's SSE-driven and the SSE closes early. Real fix would be longer-lived SSE that monitors the webhook. The diagnostics panel (poll-based) does narrate progress correctly after this PR; the modal is the same UX gap as before.

jMyles added 2 commits June 1, 2026 17:33
PR cryptograss#95 closed half the gap — _update_draft_finalize now updates
delivery-kid's draft.json with status=finalized + final_cid when
Coconut V2 completes. But the wiki ReleaseDraft:{id} page YAML stays
on the pre-finalize state because the historical "write the YAML
edit" step was done by browser-side JS responding to an SSE 'complete'
event that the slow Coconut path never sends (the SSE closes after
'transcoding-submitted' minutes before Coconut actually completes).

Net effect through PR cryptograss#95: HLS was pinned, draft.json reflected it,
but no Blue Railroad Imports bot trigger fired because the bot reads
the wiki page edit history for "Finalized: pinned to IPFS as <cid>"
summaries — and that edit was never made.

This PR closes the second half:

## 1. pickipedia_client: write_finalized_to_releasedraft

New method that:
  - Loads the current ReleaseDraft:{id} YAML
  - Sets status=finalized, final_cid=<cid>, finalized_at=<iso>
  - Saves with summary "Finalized: pinned to IPFS as <cid>"
  - Preserves all user-supplied YAML fields (title, description,
    venue, performers, etc.) by parse+update+dump rather than
    rewriting from scratch

Async wrapper runs the sync mwclient calls in a thread so the
FastAPI event loop isn't blocked during the wiki round-trip.

## 2. coconut.py _update_draft_finalize: call it on success

After the existing draft.json update + diagnostics-snapshot, fire
the wiki YAML update as another asyncio task (fire-and-forget,
same pattern as the snapshot). On the next Blue Railroad bot run,
the bot's find_cid_from_history picks up the summary and creates
Release:{cid}.

## 3. Also: narrate the webhook stages in finalize_log

The wiki diagnostics panel had a 2-minute gap between
"transcoding-submitted" and "complete" — single jump. Add separate
log entries for webhook-received / HLS-pinned / complete so the
panel renders the progression. They all land in one draft.json
save (we don't have a stream back to the browser; the panel picks
them up on its next poll).

## Test plan after deploy

- Re-finalize the terrapin draft (or any in-flight)
- Watch docker logs: should see "pickipedia_client: marked
  ReleaseDraft:<id> finalized" after the Coconut callback
- Wiki page revision history should show a "Finalized: pinned to
  IPFS as Qm..." edit by Magent@magent
- Blue Railroad Imports bot creates Release:<cid> on next cron
- New Release page resolves to a directory with master.m3u8 +
  variant playlists + segments — real playable video, not 421-byte
  metadata.json
@jMyles
jMyles merged commit e55ec8b into cryptograss:production Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants