Skip to content

Latest commit

 

History

History
319 lines (232 loc) · 12.9 KB

File metadata and controls

319 lines (232 loc) · 12.9 KB

How ABSP works

Notes from building a downloader that keeps working. Everything below was measured against live sites while building this, not taken from documentation.


1. YouTube needs a JavaScript runtime

This is the one that breaks naive wrappers hardest, because the failure looks like something else entirely.

YouTube protects its media URLs with an n parameter challenge — a piece of JavaScript that must be executed to derive the real URL. yt-dlp ships the solver, but it needs a JS runtime to run it in. Without one you get this:

WARNING: n challenge solving failed: Some formats may be missing.
WARNING: Only images are available for download.

ID  EXT   RESOLUTION  PROTO  VCODEC
sb3 mhtml 48x27       mhtml  images  storyboard
sb2 mhtml 80x45       mhtml  images  storyboard

Storyboard thumbnails. Nothing playable. The error never says "install a JavaScript runtime" — it says formats may be missing, and then the format selector fails with Requested format is not available, which sends everyone chasing the wrong problem.

yt-dlp supports Deno (default), Node, QuickJS and Bun. Bundling Deno would mean another ~40 MB binary to ship and update.

What ABSP does instead: use the runtime it already has. In web mode that is the Node process running the server; in the desktop app it is Electron, which becomes Node when ELECTRON_RUN_AS_NODE=1 is present in the environment.

args.push('--js-runtimes', `node:${process.execPath}`);
// …and for the desktop build, spawn yt-dlp with:
{ ...process.env, ELECTRON_RUN_AS_NODE: '1' }

Verified by stripping node from PATH entirely and pointing yt-dlp at the Electron binary — it still resolved 2160p. Zero extra download.


2. Client choice decides 4K vs 360p

YouTube's player API is not one thing. yt-dlp can impersonate several clients, and in 2025–26 two separate mechanisms decide what each one gets back:

  • PO tokens (proof-of-origin) — some clients now refuse to hand out media URLs without one. Generating them needs a separate token-provider service.
  • SABR — YouTube's own streaming protocol. Clients enrolled in it get formats with no direct URL at all, so yt-dlp skips them.

The practical consequence is that which client you pick sets your maximum quality, and most wrappers never think about it. Measured against the same video (Big Buck Bunny 4K), with no PO token provider configured:

Client Result
android_vr Full DASH ladder to 2160p60, no PO token needed
web_embedded Full DASH ladder to 2160p60, no PO token needed
tv SABR-gated → HLS only, caps at 1080p
tv_simply PO token required → one 360p format
ios PO token required → nothing usable

Same video, same moment, same network. The difference between a 4K download and a 360p one is a single argument.

ABSP defaults to android_vr,web_embedded,tv — the first two for quality, tv last as an HLS fallback, which is what live streams need. It is exposed in Settings, because YouTube will move again.

--extractor-args youtube:player_client=android_vr,web_embedded,tv

The argument is namespaced to youtube:, so it is inert for every other site.


3. The engine cannot be pinned

YouTube breaks extractors on a timescale of days. A downloader that bundles a fixed yt-dlp is broken from the moment it ships; the only question is how long until the user notices.

ABSP does not bundle it. On first run it fetches yt-dlp from the latest release and ffmpeg/ffprobe from ffmpeg-static, for the right platform and architecture, into the user's data folder. There is an Update engine button in Settings that re-fetches yt-dlp.

This solves three problems at once:

  1. One installer builds for every platform without shipping platform binaries.
  2. The engine updates independently of the app — which matters, because the app cannot auto-update on macOS without a paid signing certificate.
  3. Installer size stays around 100 MB instead of 250 MB.

"Click Update engine" fixes the large majority of it stopped working reports.


4. The bug that deletes your video

This one is worth the whole document.

Extracting audio is a two-step operation: yt-dlp downloads the source media, then ffmpeg extracts the audio track, then yt-dlp deletes the source. That is correct and desirable — you asked for an MP3, not a 200 MB intermediate.

Now consider the obvious implementation, writing straight into the user's downloads folder:

  1. You download a video → Great Video [abc123].mp4
  2. Later you want the audio, so you download the MP3 of the same video
  3. yt-dlp computes the same output name, sees Great Video [abc123].mp4 already there, and with --no-overwrites adopts that existing file as its download source instead of downloading again
  4. It extracts the audio
  5. It deletes the source — which is your video

You get the MP3. The video is gone. The job reports success. Nothing in the log says a file was destroyed.

It is worse than it sounds, because it is intermittent: it only fires when the chosen format lands on the same extension as the file you already have. In testing it reproduced on X every time, but Pinterest and Instagram happened to dodge it because their formats route through ffmpeg differently. A first fix that only relocated intermediate files looked correct on those two sites and still destroyed files on X.

The fix: every job runs in a private scratch directory and results are moved into the user's folder only after the process exits successfully.

const workDir = fs.mkdtempSync(path.join(os.tmpdir(), 'absp-work-'));
args.push('--paths', `home:${workDir}`);
// …on success only:
files.map((f) => moveIntoPlace(f, workDir, outputDir));

yt-dlp can no longer see, reuse or delete anything you already have. A repeat download becomes name (2).mp4 rather than overwriting.

There is one consequence worth handling: cancelling now discards the scratch directory, which for a 40-video playlist would throw away everything already finished. So cancellation moves completed items into place before cleaning up — they are whole files, not partial ones.


5. Why macOS calls unsigned apps "damaged"

If you build an Electron app and hand the .dmg to someone, they may see:

"ABSP" is damaged and can't be opened. You should move it to the Trash.

Nothing is damaged, and it is not a corrupted download. Electron ships its binaries ad-hoc signed. electron-builder then rewrites the bundle — injects app.asar, renames the executable — which invalidates that signature without replacing it:

$ codesign --verify --deep --strict "ABSP.app"
ABSP.app: code has no resources but signature indicates they must be present

Apple Silicon refuses to execute arm64 code with a broken signature, and Gatekeeper reports that to users as "damaged" — a dialog whose only button is Move to Trash.

The fix is to re-sign ad-hoc after packaging, inside-out. But the two architectures fail differently:

  • arm64 ships linker-signed (the kernel demands it), so only the bundle seal is stale.
  • x64 ships completely unsigned — Intel never required signatures — so sealing the outer bundle fails with code object is not signed at all on nested helpers like chrome_crashpad_handler.

Signing only .app and .framework directories is therefore not enough. ABSP's build/afterPack.js walks the bundle, detects Mach-O files by magic number, signs every nested binary first, then nested bundles, then the outer app, and fails the build if verification does not pass:

• ad-hoc signed and verified  mac-arm64/ABSP.app  (15 binaries, 8 bundles)

The result is still not notarised, so users get the ordinary "unidentified developer" prompt — but that has an Open Anyway button, which "damaged" does not.


6. Portrait video and the "1920p" problem

A vertical clip reports its long edge as height. A 1080×1920 Reel is "height 1920", so a quality menu built naively from height offers 1920p — which reads as better than 4K for what is a 1080-wide video.

ABSP keeps width alongside height and labels portrait video by its real dimensions (1080×1920) while landscape keeps the familiar 1080p. Small thing; it removes a real "why is my 4K download so small" question.

Estimated file sizes come from the same probe, with the best audio track's size added to video-only formats so the number reflects the merged file rather than the video stream alone.


7. What each site actually needs

Site Notes
YouTube JS runtime + client choice, as above. Playlists supported
Pinterest Only combined streams — no separate video/audio, so the H.264 merge path never applies and the selector falls through to best-single-stream. Boards work as playlists. Image pins have nothing to download
Instagram Public posts and Reels work with no cookies. yt-dlp marks its profile extractor as broken, so ABSP treats every IG link as a single item rather than starting a doomed bulk job
Facebook Public videos work with no cookies. Often serves VP9, so "prefer H.264" has to fall back rather than fail
X (Twitter) Both x.com and twitter.com. Public posts work with no cookies. Posts marked sensitive need a session

Cookies are not equivalent across browsers

Tested on macOS:

Browser Result
Chrome Works
Safari FailsOperation not permitted reading Cookies.binarycookies. macOS protects Safari's container; the app needs Full Disk Access
Firefox / Edge / Brave Not installed on the test machine — unverified

Both failure modes are translated into instructions rather than surfaced raw.


8. Error messages are a feature

yt-dlp's errors are accurate and almost useless to a non-technical user. A dead Facebook link produces:

ERROR: [facebook] 111111111: Cannot parse data; please report this issue on
https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate
issue template. Confirm you are on the latest version using yt-dlp -U

That is not a bug and the user should not file anything. ABSP maps the failures people actually hit onto one actionable sentence each — rate limits, login walls, sensitive posts, suspended accounts, image-only posts, dead links, cookie permission failures — and unrecognised errors pass through unchanged rather than being swallowed.

There is a unit test asserting the string report this issue can never reach a user.


9. Architecture

One codebase, two shells:

Electron  ──┐
            ├──►  Node HTTP server  ──►  yt-dlp + ffmpeg
Browser   ──┘         (Express)

The desktop app starts the same server on a random localhost port and loads it in a BrowserWindow. The only difference is a preload bridge exposing native "reveal in Finder" and a folder picker. There is no second UI implementation, so a fix lands in both at once.

Progress comes from a --progress-template rather than scraping yt-dlp's pretty output:

ABSP_PROGRESS<TAB>%(progress.status)s<TAB>%(progress.downloaded_bytes)s<TAB>…

One machine-readable line per tick, pushed to the UI over server-sent events.

Jobs persist to jobs.json, so the queue survives a restart; anything that was mid-flight comes back as a retryable failure. A job producing no output at all for the stall timeout is killed and marked retryable — liveness counts any line from yt-dlp, so a long quiet merge is not mistaken for a hang.


10. Testing something that depends on five external sites

Two suites, split by what they can promise:

  • npm test — 53 assertions, deterministic and offline. Source detection, playlist rules, format selection, every error translation, path validation, settings I/O. This gates CI.
  • npm run test:e2e — 142 assertions against a live server. Real probes and downloads across all five sites, every audio format, quality ladders, overwrite protection, job control, pause/resume/retry, playlist ranges, trimming, HTTP range requests, SSE, concurrency, history persistence. Every output file is verified with ffprobe, not just checked for existence.

The end-to-end suite is not a merge gate, because five third-party sites make it non-deterministic by construction. It is what you run before cutting a release.

It has earned its keep. It caught WAV downloads failing outright (cover art cannot be embedded in a WAV container, and yt-dlp fails the whole job), and it caught a download-folder validator that rejected /Volumes/… — which would have quietly made external drives unusable.

It also produced two false failures worth mentioning, because the lesson generalises: a cancellation assertion that assumed the old discard-everything behaviour, and two timeouts caused by the suite itself hammering one video fifteen times in a row. Both looked like product bugs. Neither was.