fix: prevent HF token leakage into Nostr events and pipeline artifacts#64
Merged
Conversation
Livestream recording URLs can carry a Hugging Face access token in a query param (e.g. ?token=hf_...). Previously this URL flowed unchanged through extractRecordingUrl -> episode 'audio' tag -> .transcript-mapping.json, which GitHub push protection blocks (failing the Process Episodes workflow), and the same token was republished to Nostr relays on every episode update. Changes: - scripts/lib/conversion-utils.ts: add redactUrlSecrets() (strips token / hf_token / access_token / jwt query params + inline hf_... literals) and redactEventSecrets() (scrubs an entire NostrEvent). extractRecordingUrl() now redacts before returning and warns when it changed something. - scripts/transcribe-audio.ts: sanitize every result event via redactEventSecrets before writing .transcript-mapping.json. - .gitignore: ignore .episodes-to-transcribe.json, .transcript-mapping.json, .show-notes-mapping.json so they cannot sneak back in. - Untrack the two currently-tracked mapping files. Note: previously published episode events may still carry a token-bearing audio URL on Nostr relays. Source livestream events should be republished without the token, and any leaked HF token rotated. Workflow yaml change (.github/workflows/process-episodes.yml: drop the 'git add .transcript-mapping.json || true' line from the Commit transcripts step) is held back until the push token's OAuth workflow scope is granted; will be applied in a follow-up commit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Process Episodes workflow failing daily since push protection blocks transcript commits:
Root cause: livestream recording URL carries
?token=hf_...→ flows unchanged throughextractRecordingUrl→ episodeaudiotag → saved into.transcript-mapping.json→ push protection blocks commit. Worse, same token gets republished to Nostr relays on every episode update (real public leak).Changes (this PR)
scripts/lib/conversion-utils.ts: addredactUrlSecrets()(stripstoken/hf_token/access_token/jwtquery params + inlinehf_...literals) andredactEventSecrets()(scrubs entire event).extractRecordingUrl()now redacts before returning.scripts/transcribe-audio.ts: sanitize every result event viaredactEventSecretsbefore writing mapping JSON..gitignore: ignore.episodes-to-transcribe.json,.transcript-mapping.json,.show-notes-mapping.json.Smoke-tested: HF token query stripped, inline literals
[REDACTED], clean URLs untouched.tsc+ ESLint clean on touched files.Follow-up commit (needs
workflowOAuth scope)Held back from this push — push token lacks
workflowscope. Apply aftergh auth refresh -s workflow.Required operator actions (code cannot)
HF_TOKENGitHub secret on kurt-croix/podstr belongs to kurt-croix, not auggie-lahey.download/recording/streamingURL contains?token=hf_...— repoint to kurt-croix HF space or token-free source. Otherwise the next run republishes the same leak.Vibed with MKStack