diff --git a/README.md b/README.md index e572bf3..af9b735 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,10 @@ # plx -Spotify ⇄ Deezer playlist converter CLI. +Playlist converter CLI for Spotify, Deezer, and YouTube Music. -Converts playlists in both directions — reads Spotify **without an account, Premium, or OAuth**; writes via your session cookies. +Converts a playlist from any of the three services into any other — six directions — using your own logged-in browser session. No developer account, no OAuth, no API keys. -> ⚠️ **Personal, non-commercial use.** Uses unofficial web endpoints that can change or break without notice. Not affiliated with Spotify or Deezer. +> ⚠️ **Personal, non-commercial use.** Uses unofficial web endpoints that can change or break without notice. Not affiliated with Spotify, Deezer, or YouTube. ## Install @@ -20,6 +20,7 @@ Requires Node.js ≥ 22. plx # interactive menu plx --url "https://open.spotify.com/playlist/ID" --to deezer plx --url "https://www.deezer.com/playlist/ID" --to spotify +plx --url "https://music.youtube.com/playlist?list=ID" --to deezer plx --url "" --to deezer --dry-run # preview matches without writing ``` diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 3d7c368..3bd2ead 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -4,15 +4,16 @@ Source of truth for the system's design and its external contracts. Read this be ## Goal -`plx` is a personal Spotify ⇄ Deezer playlist converter distributed as a CLI (`npm install -g plx`). It reads a Spotify playlist **without a Spotify account, Premium, or OAuth**: +`plx` is a personal playlist converter for Spotify, Deezer, and YouTube Music, distributed as a CLI (`npm install -g plx`). It uses the user's own logged-in browser session, never a developer account or an official API (ADR 0001): ```bash plx --url "https://open.spotify.com/playlist/PLAYLIST_ID" --dry-run ``` -Both directions are supported: -- **Spotify → Deezer** — reads Spotify anonymously, writes via a Deezer `arl` cookie. -- **Deezer → Spotify** — reads via `arl`, writes via a Spotify `sp_dc` web-session cookie. +All six directions are supported — every provider is a peer, so any can be the source and any the +target. Each is authenticated by a browser session credential: Spotify `sp_dc`, Deezer `arl`, and a +whole cookie header for YouTube Music. Reading a public Spotify playlist needs no credential at all, +which is what makes a credential-free `--dry-run` possible. ## Flow @@ -52,10 +53,10 @@ direction-specific code. The pieces: Providers meet only in `Conversion`, which is why the conversion tests supply providers directly and mock each one's `fetch`. -| Capability | Spotify | Deezer | -|---|---|---| -| `reorder` | ✗ — no move primitive in the endpoints plx uses; a Spotify target is dedupe-and-append | ✓ by track id | -| `resolveTrack` | ✓ — every match into a Spotify target is verified | ✗ — matches into Deezer are unverified | +| Capability | Spotify | Deezer | YouTube Music | +|---|---|---|---| +| `reorder` | ✗ — no move primitive in the endpoints plx uses; a Spotify target is dedupe-and-append | ✓ by track id | ✓ by `setVideoId`, mapped from track id inside the provider | +| `resolveTrack` | ✓ — every match into a Spotify target is verified | ✗ — matches into Deezer are unverified | ✓ via `player` | ## Module map @@ -63,7 +64,9 @@ mock each one's `fetch`. |---|---| | `src/cli.ts` | Entry. Arg dispatch, interactive menu (`@clack/prompts`), credential prompt + semi-auto fill + persist. Names no provider — it drives off the registry. | | `src/registry.ts` | The `ProviderSpec` list: label, link hosts, credential prompt, ref parsing, validation, and how to build each provider. The one place a provider is registered. | -| `src/ytmusic.ts` | YouTube Music session credential: the cookie bundle, the shared cookie-header joiner, SAPISIDHASH request signing, InnerTube transport, and live session validation. | +| `src/ytmusic.ts` | YouTube Music credential + transport: the cookie bundle, the shared cookie-header joiner, SAPISIDHASH signing, InnerTube calls, session validation, playlist-ref parsing, mix detection. | +| `src/ytmusic-parse.ts` | Pure parsing of InnerTube responses. Pinned by fixtures recorded from the real service (`tests/fixtures/ytmusic/`). No I/O. | +| `src/ytmusic-provider.ts` | `YtMusicProvider` (a `Provider`): read, search songs then videos, create/add, verify, and reorder by `setVideoId`. | | `src/args.ts` | Pure typed CLI parsing → `CliOptions`. No I/O. | | `src/config.ts` | Config dir + `credentials.json` read/write. Holds **one opaque string per provider, keyed by provider name** — it never parses a credential's contents. Per-provider env overrides, `tryAutoFillCredentials`. | | `src/browser.ts` | Reads `arl`/`sp_dc` from a logged-in browser (Chromium family decrypt + Safari binarycookies parse). macOS only. | @@ -101,6 +104,11 @@ The project intentionally uses **unofficial web endpoints**. They can change wit | Session credential | The browser cookie bundle for `.youtube.com`, sent as one `Cookie` header | **`__Secure-1PSIDTS` / `__Secure-3PSIDTS` are required.** Without them the `*PSID` cookies are unbound and the request authenticates as **logged out — with a 200 and no error**. Cookies must also come from one profile and prefer `.youtube.com` over `.google.com`, which set the same names to *different* values. | | Request signing | `Authorization: SAPISIDHASH _ https://music.youtube.com")>`, plus `x-origin` | Mirrors what the web client computes. `SAPISID` may instead be `__Secure-3PAPISID`/`__Secure-1PAPISID`. | | Session validation | InnerTube `POST /youtubei/v1/account/account_menu`, client `WEB_REMIX` | **A rejected session returns 200 with a signed-out menu**, so validity is judged by whether the response names an account, never by the status code. | +| Playlist read | `POST browse` with `browseId: VL`, following `continuationItemRenderer` tokens | Entries are `musicResponsiveListItemRenderer`, ~6 levels deep; the path differs between browse, search, and continuation, so the parser walks rather than indexes. Title comes from `microformat`. | +| Song vs music video | `watchEndpointMusicConfig.musicVideoType` | Only `MUSIC_VIDEO_TYPE_ATV` is a catalog song. `OMV`, `UGC`, `OFFICIAL_SOURCE_MUSIC` and a missing type are all treated as videos — their stated artist is the uploading channel. | +| Search | `POST search` with the web client's filter `params` — songs and music videos are separate filters | A playlist read puts duration in a fixed column; a search trails it on the subtitle line (`Artist • Album • 3:49`). | +| Create / add / reorder | `POST playlist/create` (`privacyStatus: PRIVATE`), `POST browse/edit_playlist` with `ACTION_ADD_VIDEO` / `ACTION_MOVE_VIDEO_BEFORE` | Reorder moves by `setVideoId`, an opaque per-item handle that is **not** the video id — the same song added twice has two. Resolved inside the provider (ADR 0002). | +| Track metadata (verify) | `POST player` → `videoDetails` | Flat, unlike everything else here. A catalog track's `author` is the artist's auto-generated `" - Topic"` channel. | ### Deezer @@ -124,6 +132,9 @@ The project intentionally uses **unofficial web endpoints**. They can change wit 1. `exact` — normalized title + first artist equal, duration within ±3s. 2. `fuzzy-duration` — title + artist equal, duration ignored (live/remaster variants). 3. `fuzzy-title` — one normalized title contains the other + artist equal. +4. `video` — **YouTube Music only**, and only after the catalog returns nothing. The artist rule relaxes to containment (the source artist must appear in the video's title *or* channel name) because a video's stated artist is usually the uploader; in exchange the duration tolerance becomes **mandatory**, and a candidate with no duration is rejected. Duration is the only guard left, and it is what separates the same recording uploaded as a video from someone else's cover. + +The tier set is open, not a closed enum every provider shares (ADR 0003) — `video` is the first tier only one provider can produce. `normalize` strips accents, parentheticals, `(feat./ft.)` labels, punctuation, and collapses whitespace. diff --git a/docs/CONFIG.md b/docs/CONFIG.md index e68a8ea..e2acd6f 100644 --- a/docs/CONFIG.md +++ b/docs/CONFIG.md @@ -5,8 +5,9 @@ Setup guide for credentials, config directory, and storage. ## Prerequisites - **Node.js ≥ 22** -- **Deezer account** (free is fine) — needed to *write* Deezer playlists. -- **No Spotify Developer account or app registration** — reading is anonymous; writing needs only a logged-in Spotify web session. +- An account on whichever services your conversion touches — free tiers are fine. +- **No developer account, app registration, or API key for any provider.** plx acts as you, using + the browser session you already have (ADR 0001). ## Quick start @@ -19,7 +20,8 @@ Spotify → Deezer reads Spotify anonymously — no Premium, no OAuth. ## Credentials -`plx` needs at most **two** cookies, both optional depending on direction: +One browser session credential per provider. **All are optional** — you only need the ones your +chosen direction actually touches, and a `--dry-run` from a public Spotify playlist needs none: | Cookie | Env var | Used for | Needed when | |---|---|---|---| @@ -41,16 +43,19 @@ Reading a Spotify playlist for a dry run needs **no credentials**. 1. Log in to [deezer.com](https://www.deezer.com) / [open.spotify.com](https://open.spotify.com) in a browser. 2. Open DevTools (F12) → **Application** (Chrome/Edge) or **Storage** (Firefox) → **Cookies** → the site domain. -3. Find `arl` (Deezer) or `sp_dc` (Spotify) and copy its value. +3. Find `arl` (Deezer) or `sp_dc` (Spotify) and copy its value. For YouTube Music, copy the whole `cookie:` header instead — see above. ### Semi-auto fill (macOS) -Instead of hand-copying, the `arl` / `sp_dc` inputs can **read the cookie straight -from a logged-in browser** — you only accept the macOS Keychain "Allow" dialog. It -works for the **Chromium family** (Brave, Chrome, Edge, Chromium, Vivaldi, Opera) -and **Safari** (Safari needs **Full Disk Access** in System Settings → Privacy & -Security). Pick **"Ambil kredensial otomatis"** from the menu, or it runs -automatically when a credential is missing. Manual paste stays as fallback. +Instead of hand-copying, plx can **read the credentials straight from a logged-in +browser** — you only accept the macOS Keychain "Allow" dialog. It works for the +**Chromium family** (Brave, Chrome, Edge, Chromium, Vivaldi, Opera) and **Safari** +(Safari needs **Full Disk Access** in System Settings → Privacy & Security). Pick +**"Auto-fetch credentials (from browser)"** from the menu, or it runs automatically +when a credential is missing. Manual paste stays the fallback everywhere else. + +Each credential it finds is then checked against the live service, so you are told +whether it actually works rather than just that it was saved. ## Env vars @@ -101,7 +106,9 @@ else in the file, including `recentUrls`, survives untouched. ## Report output -Default CSV report path is `conversion_report.csv`. Override with `--output ` or the interactive menu's **Laporan** option. +Default CSV report path is `conversion_report.csv`. Override with `--output ` or the interactive menu's **Report** option. + +Columns: `playlist,source,target,title,artist,isrc,matched,target_id,method,note`. `source` and `target` name the providers, and `target_id` is the matched identifier in the target's own terms — the same meaning in all six directions. ## Security diff --git a/docs/CONTINUATION.md b/docs/CONTINUATION.md index 8cb1f3c..4867fc9 100644 --- a/docs/CONTINUATION.md +++ b/docs/CONTINUATION.md @@ -4,6 +4,42 @@ Updated: 2026-08-17 +## 2026-08-17 (e) — the provider is complete + +**#3 provider-neutral report, #7 read, #8 write, #9 music video fallback, #10 reorder.** All six +directions now run. `ProviderSpec.convertible`, the temporary flag from #6, is deleted. + +### Verified against the live service, not just stubs + +- **YouTube Music → Deezer**: a 180-entry playlist read with pagination — 38 songs converted, + **141 music videos skipped** and recorded as their own report row. +- **Spotify → YouTube Music** and **Deezer → YouTube Music**: 4/4 matched and verified, playlist + created private, re-running into the same playlist added 0 (dedupe), then **deleted** — the test + account is left as it was. +- **Reorder**: applied to a real playlist, order matched exactly; a second call on the now-ordered + playlist returned false without editing. +- A **mix** is refused with its reason; `music.youtube.com` and `youtube.com` links resolve to the + same playlist. + +### Things worth knowing + +- **Only `MUSIC_VIDEO_TYPE_ATV` is a catalog song.** `OMV`, `UGC`, `OFFICIAL_SOURCE_MUSIC`, and + entries with no type at all are videos or unavailable. All four appear in a real playlist and all + four are in the fixture. +- **`parseDuration` must return null, not 0, for non-durations.** A playlist read has a duration + column and a search result does not; callers tell them apart by null. Returning 0 for an empty + column silently swallowed the subtitle fallback — caught by the recorded fixtures, which is + exactly why the ticket demanded real responses over hand-written literals. +- **Verification uses `player` → `videoDetails`**, which is flat, rather than the ~6-level renderer + nesting everything else needs. A catalog track's `author` is the `" - Topic"` channel. +- **`error.name` → `error.message` in Conversion.** Every throw here is a plain `Error`, so the + report's one diagnostic field always read the literal "Error" — which silently swallowed the mix + refusal's explanation. This was flagged in the #2 review and only became an AC failure here. +- **Reorder moves by `setVideoId`**, an opaque per-item handle — the same song added twice has two. + Mapped from track id inside the provider; it never reaches the conversion interface (ADR 0002). +- `YtMusicProvider.deletePlaylist` is deliberately **not** on the `Provider` interface. plx never + deletes during a conversion; it exists so a live write test can clean up after itself. + ## 2026-08-17 (d) **YouTube Music browser session credential** (issue #6). Unblocks #7 and #8. Nothing converts yet — diff --git a/package.json b/package.json index 97818c0..2c6721d 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "plx-converter", "version": "0.1.1", - "description": "Spotify ⇄ Deezer playlist converter CLI", + "description": "Convert playlists between Spotify, Deezer, and YouTube Music using your own browser session", "license": "MIT", "type": "module", "bin": { diff --git a/src/args.ts b/src/args.ts index 5164acd..5c171f3 100644 --- a/src/args.ts +++ b/src/args.ts @@ -16,14 +16,15 @@ The source provider is inferred from the link, so you only name the target. Options: -u, --url playlist URL, URI, or ID (repeatable) - -t, --to target provider (required unless --dry-run) + -t, --to target provider: spotify | deezer | ytmusic (required) -o, --output CSV report path (default: conversion_report.csv) -d, --dry-run match only; do not create or modify any playlist -h, --help show this help Examples: plx --url https://open.spotify.com/playlist/37i9dQZF1DXcBWIGoYBM5M --to deezer - plx --url https://www.deezer.com/playlist/1234567890 --to spotify + plx --url https://www.deezer.com/playlist/1234567890 --to ytmusic + plx --url https://music.youtube.com/playlist?list=PL... --to spotify plx --url https://open.spotify.com/playlist/37i9dQZF1DXcBWIGoYBM5M --to deezer --dry-run plx `; diff --git a/src/cli.ts b/src/cli.ts index e11a39b..66b09cc 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -2,7 +2,7 @@ import { confirm, intro, isCancel, outro, password, select, text } from '@clack/prompts'; import { credential, loadConfig, saveCredential, saveRecentUrl, tryAutoFillCredentials, type Config } from './config.js'; import { Conversion } from './conversion.js'; -import { PROVIDERS, convertibleProviders, inferSource, providerFor, targetKeys, type ProviderSpec } from './registry.js'; +import { PROVIDERS, inferSource, providerFor, targetKeys, type ProviderSpec } from './registry.js'; import { HELP_TEXT, parseArgs, type CliOptions } from './args.js'; import type { Provider } from './types.js'; @@ -99,7 +99,9 @@ async function pickSourcePlaylist(cfg: Config, spec: ProviderSpec, provider: Pro : String(pick); if (isCancel(ref) || !ref || String(ref).toLowerCase() === 'q') return null; const id = await spec.parseRef(String(ref)); - return { id, name: await spec.playlistName(id), ref: String(ref) }; + // A provider that can resolve its own titles does it better than the registry's fallback. + const name = (await provider.playlistName?.(id).catch(() => '')) || await spec.playlistName(id); + return { id, name, ref: String(ref) }; } async function pickFromAccount(provider: Provider, spec: ProviderSpec, role: string): Promise<{ id: string; title: string } | null> { @@ -143,9 +145,9 @@ async function pickTargetPlaylist(spec: ProviderSpec, provider: Provider): Promi // One flow for every direction: pick source provider, pick target provider, then convert. async function runConversion(cfg: Config, options: CliOptions): Promise { - const sourceSpec = await pickProvider('Source provider', convertibleProviders()); + const sourceSpec = await pickProvider('Source provider', PROVIDERS); if (!sourceSpec) return; - const targetSpec = await pickProvider('Target provider', convertibleProviders(), sourceSpec); + const targetSpec = await pickProvider('Target provider', PROVIDERS, sourceSpec); if (!targetSpec) return; const source = await build(cfg, sourceSpec, false); @@ -249,14 +251,12 @@ async function main() { ? `Unknown target provider '${options.target}'. Valid targets: ${targetKeys().join(', ')}.` : `No target provider. Name one with --to <${targetKeys().join('|')}>.`); } - if (!targetSpec.convertible) throw new Error(`Writing into ${targetSpec.label} is not built yet. Valid targets: ${targetKeys().join(', ')}.`); const cfg = await loadConfig(); if (cfg.legacyCredentials) console.log('⚠️ Saved credentials are in an older format and were not carried over. Run `plx` and re-enter them (one browser dialog).'); for (const raw of options.urls) { const sourceSpec = inferSource(raw); - if (sourceSpec && !sourceSpec.convertible) throw new Error(`Reading from ${sourceSpec.label} is not built yet.`); if (!sourceSpec) throw new Error(`Cannot tell which provider '${raw}' belongs to. Use a link from one of: ${PROVIDERS.map((p) => p.label).join(', ')}.`); if (sourceSpec === targetSpec) throw new Error(`Source and target are both ${sourceSpec.label}.`); @@ -267,7 +267,8 @@ async function main() { const conversion = new Conversion(source, target, options.output); try { const id = await sourceSpec.parseRef(raw); - const name = await sourceSpec.playlistName(id); + // A provider that can resolve its own titles does it better than the registry's fallback. + const name = (await source.playlistName?.(id).catch(() => '')) || await sourceSpec.playlistName(id); const result = await conversion.matchPlaylist({ name, uri: id }, options.dryRun); if (!options.dryRun && result.matchedIds.length) await conversion.writePlaylist(name, result.matchedIds); } finally { diff --git a/src/conversion.ts b/src/conversion.ts index adf1a87..e0b4c66 100644 --- a/src/conversion.ts +++ b/src/conversion.ts @@ -15,17 +15,34 @@ const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms)); export class Conversion { readonly rows: ReportRow[] = []; constructor(private readonly source: Provider, private readonly target: Provider, private readonly output: string) {} - private record(row: ReportRow) { this.rows.push(row); } + // Every row carries the direction that produced it, so the file still makes sense a month later + // and a report from one of six directions is not mistaken for another. + private record(row: ReportRow) { this.rows.push({ ...row, source: this.source.name, target: this.target.name }); } async matchPlaylist(playlist: PlaylistRef, dryRun = false): Promise { let result; try { result = await this.source.readPlaylist(playlist.uri); } - catch (error) { this.record({ playlist: playlist.name, title: '', artist: '', isrc: null, matched: false, note: `failed to read playlist: ${error instanceof Error ? error.name : 'Error'}` }); return { matchedIds: [], total: 0, truncated: false }; } + catch (error) { + // The message, not the name: every throw in this codebase is a plain Error, so `error.name` is + // always the literal "Error" and the one diagnostic the report carries would say nothing. It + // is also where a refusal explains itself — a mix, say — so it has to reach the user. + const reason = error instanceof Error ? error.message : String(error); + this.record({ playlist: playlist.name, title: '', artist: '', isrc: null, matched: false, note: `failed to read playlist: ${reason}` }); + console.log(`[${playlist.name}] ✗ ${reason}`); + return { matchedIds: [], total: 0, truncated: false }; + } if (result.truncated) { // The provider hit its own read cap; how many it managed is the only limit the path knows. this.record({ playlist: playlist.name, title: '', artist: '', isrc: null, matched: false, note: `WARNING: truncated at ${result.tracks.length} tracks; remainder not converted` }); console.log(`[${playlist.name}] ⚠️ Truncated at ${result.tracks.length} tracks.`); } + // Entries the source refused to convert are recorded before the matched ones, as their own kind + // of row: they were never searched for, so calling them "no match" would misreport them. + for (const entry of result.skipped ?? []) { + this.record({ playlist: playlist.name, title: entry.name, artist: entry.artist, isrc: null, matched: false, note: 'skipped: music video (channel name is not a reliable artist)' }); + } + if (result.skipped?.length) console.log(`[${playlist.name}] ${result.skipped.length} music video${result.skipped.length === 1 ? '' : 's'} skipped.`); + const matchedIds: string[] = []; let verified = 0; for (let i = 0; i < result.tracks.length; i += 1) { @@ -33,13 +50,13 @@ export class Conversion { let match: Match | null = null; for (let attempt = 0; attempt < 3; attempt += 1) { try { match = await this.target.search(track); break; } - catch (error) { if (attempt === 2) console.log(` ⚠️ search failed: ${error instanceof Error ? error.name : 'Error'}`); else await sleep(2 ** attempt * 1000); } + catch (error) { if (attempt === 2) console.log(` ⚠️ search failed: ${error instanceof Error ? error.message : 'error'}`); else await sleep(2 ** attempt * 1000); } } if (match) { const flag = await this.verify(track, match); if (this.target.resolveTrack && flag === null) verified += 1; matchedIds.push(match.id); - this.record({ playlist: playlist.name, title: track.name, artist: track.artist, isrc: null, matched: true, deezer_id: match.id, method: match.method, note: [flag, dryRun ? 'dry-run' : null].filter(Boolean).join('; ') || null }); + this.record({ playlist: playlist.name, title: track.name, artist: track.artist, isrc: null, matched: true, target_id: match.id, method: match.method, note: [flag, dryRun ? 'dry-run' : null].filter(Boolean).join('; ') || null }); console.log(` [${i + 1}/${result.tracks.length}] ✓ ${match.method} — ${track.name} — ${track.artist}${flag ? ` — ${flag}` : ''}`); } else { this.record({ playlist: playlist.name, title: track.name, artist: track.artist, isrc: null, matched: false, note: dryRun ? 'dry-run: no match' : 'no match' }); diff --git a/src/csv.ts b/src/csv.ts index f0f5819..ff27d42 100644 --- a/src/csv.ts +++ b/src/csv.ts @@ -1,7 +1,7 @@ import { writeFile } from 'node:fs/promises'; import type { ReportRow } from './types.js'; -const fields = ['playlist', 'title', 'artist', 'isrc', 'matched', 'deezer_id', 'method', 'note'] as const; +const fields = ['playlist', 'source', 'target', 'title', 'artist', 'isrc', 'matched', 'target_id', 'method', 'note'] as const; const cell = (value: unknown) => { const text = value == null ? '' : String(value); return /[",\n]/.test(text) ? `"${text.replaceAll('"', '""')}"` : text; diff --git a/src/matcher.ts b/src/matcher.ts index 136d3c4..1578187 100644 --- a/src/matcher.ts +++ b/src/matcher.ts @@ -1,4 +1,4 @@ -import type { Candidate, Match, Track } from './types.js'; +import type { Candidate, Match } from './types.js'; export function normalize(value: string): string { return value.normalize('NFKD').replace(/\p{M}/gu, '').toLowerCase().replace(/\(.*?\)/g, '').replace(/[\[(]\s*(feat\.?|ft\.?)\s*/gi, '').replace(/\]/g, '').replace(/[^\p{L}\p{N}_\s]/gu, '').replace(/\s+/g, ' ').trim(); @@ -15,4 +15,26 @@ export function matchCandidates(title: string, artist: string, durationMs: numbe for (const c of candidates) { const ct = normalize(c.title); if (na === artistKey(c.artist) && nt && ct && (nt.includes(ct) || ct.includes(nt))) return { id: c.id, title: c.title, artist: c.artist, method: 'fuzzy-title' }; } return null; } -export function matchTrack(track: Track, candidates: Candidate[]): Match | null { return matchCandidates(track.name, track.artist, track.durationMs, candidates); } + +// Match rules for a music video, used only after the catalog has produced nothing. +// +// A music video's stated artist is usually the uploading channel, so the artist rule relaxes from +// equality to containment: the source artist must appear somewhere in the video's title or channel +// name. That is a much weaker guard, so in exchange the duration tolerance becomes **mandatory** +// rather than optional — a candidate with no duration is rejected outright. Duration is what +// separates the same recording uploaded as a video from someone else's cover of it. +export function matchVideoCandidates(title: string, artist: string, durationMs: number | null | undefined, candidates: Candidate[]): Match | null { + if (durationMs == null) return null; // nothing left to guard with + const nt = normalize(stripFeat(title)); + const na = artistKey(firstArtist(artist)); + if (!nt || !na) return null; + for (const c of candidates) { + if (c.duration == null || !matchDurationMs(durationMs, c.duration)) continue; + const ct = normalize(c.title); + // The source artist has to show up somewhere attributable — the video's title or its channel. + if (!artistKey(c.title).includes(na) && !artistKey(c.artist).includes(na)) continue; + if (!ct.includes(nt) && !nt.includes(ct)) continue; + return { id: c.id, title: c.title, artist: c.artist, method: 'video' }; + } + return null; +} diff --git a/src/registry.ts b/src/registry.ts index 0ff5004..7da0d5e 100644 --- a/src/registry.ts +++ b/src/registry.ts @@ -1,6 +1,7 @@ import { DeezerClient, resolveDeezerPlaylistId } from './deezer.js'; import { SpotifyProvider, anonymousToken, authenticatedToken, parsePlaylistId, playlistName } from './spotify.js'; -import { validateSession } from './ytmusic.js'; +import { parsePlaylistRef, validateSession } from './ytmusic.js'; +import { YtMusicProvider } from './ytmusic-provider.js'; import type { Provider } from './types.js'; // Everything the CLI needs to know about a provider that is not part of converting: what to call it, @@ -18,10 +19,6 @@ export type ProviderSpec = { loginSite: string; // Whether this provider can read a playlist with no credential at all (so a dry run needs none). anonymousRead: boolean; - // Whether this provider can take part in a conversion yet. A provider whose credential works but - // whose read/write side is not built is offered in Credentials and hidden from Convert, so it is - // never picked as a direction that cannot run. Drop the flag once every provider is convertible. - convertible: boolean; // Prove the stored credential against the live service, resolving to something to show the user // (an account name, say). Throws with a specific message when the session is expired or rejected. validate(credential: string): Promise; @@ -42,7 +39,6 @@ const spotify: ProviderSpec = { credentialHint: 'login open.spotify.com → F12 → Application → Cookies → sp_dc → copy value', loginSite: 'Spotify (open.spotify.com)', anonymousRead: true, - convertible: true, parseRef: async (ref) => parsePlaylistId(ref), playlistName, // An sp_dc mints a full search+write token; without one the anonymous token still reads public @@ -60,7 +56,6 @@ const deezer: ProviderSpec = { loginSite: 'Deezer (deezer.com)', // Deezer's search is public, but reading a *playlist* goes through the authenticated GraphQL API. anonymousRead: false, - convertible: true, parseRef: resolveDeezerPlaylistId, playlistName: async (id) => id, validate: async (cred) => `account ${(await new DeezerClient(cred).getMe()).id}`, @@ -82,25 +77,21 @@ const ytmusic: ProviderSpec = { credentialHint: 'login music.youtube.com → F12 → Network → any request → Request Headers → copy the whole "cookie:" value', loginSite: 'YouTube Music (music.youtube.com)', anonymousRead: false, - // Reading and writing land in #7 and #8; this release only proves the credential works. - convertible: false, - parseRef: async (ref) => new URL(ref, 'https://music.youtube.com').searchParams.get('list') ?? ref.trim(), + parseRef: async (ref) => parsePlaylistRef(ref), + // The title needs an authenticated browse, so YtMusicProvider resolves it; this is the fallback. playlistName: async (id) => id, - create: () => { throw new Error('Converting with YouTube Music is not built yet.'); }, + create: async (cred) => new YtMusicProvider(cred), validate: validateSession, }; export const PROVIDERS: ProviderSpec[] = [spotify, deezer, ytmusic]; -// The providers that can actually take part in a conversion right now. -export const convertibleProviders = (): ProviderSpec[] => PROVIDERS.filter((p) => p.convertible); - export function providerFor(key: string): ProviderSpec | undefined { return PROVIDERS.find((p) => p.key === key.toLowerCase()); } -// The keys valid after --to. Only providers that can actually be written into are offered. -export const targetKeys = (): string[] => convertibleProviders().map((p) => p.key); +// The keys valid after --to. +export const targetKeys = (): string[] => PROVIDERS.map((p) => p.key); // Which provider a playlist link belongs to. The link already names its service, so a command line // that gives one never has to restate the source. diff --git a/src/types.ts b/src/types.ts index 86b4b38..b00ac27 100644 --- a/src/types.ts +++ b/src/types.ts @@ -1,9 +1,12 @@ export type Track = { name: string; artist: string; durationMs?: number | null }; export type Candidate = { id: string; title: string; artist: string; duration?: number | null }; export type Match = { id: string; title: string; artist: string; method: MatchMethod }; -export type MatchMethod = 'exact' | 'fuzzy-duration' | 'fuzzy-title'; +// The tier that accepted a match. Open by design: a provider may produce a tier no other provider +// can, and 'video' is the first — YouTube Music's music-video fallback (ADR 0003). Widening this is +// how a new provider adds a tier; it is not a closed enum every provider must share. +export type MatchMethod = 'exact' | 'fuzzy-duration' | 'fuzzy-title' | 'video'; export type PlaylistRef = { name: string; uri: string }; -export type ReportRow = { playlist: string; title: string; artist: string; isrc: string | null; matched: boolean; deezer_id?: string | null; method?: string | null; note?: string | null }; +export type ReportRow = { playlist: string; source?: string; target?: string; title: string; artist: string; isrc: string | null; matched: boolean; target_id?: string | null; method?: string | null; note?: string | null }; export type MatchResult = { matchedIds: string[]; total: number; truncated: boolean }; // A music service plx can both read playlists from and write playlists to. Every provider is a @@ -11,12 +14,17 @@ export type MatchResult = { matchedIds: string[]; total: number; truncated: bool // capabilities: a conversion whose target lacks one degrades quietly rather than failing (ADR 0002). export type Provider = { readonly name: string; - readPlaylist(playlistId: string): Promise<{ tracks: Track[]; truncated: boolean }>; + // `skipped` is for entries the provider read but deliberately refused to convert — a YouTube Music + // music video, whose stated artist is the uploading channel. They are reported as their own kind + // of row, distinct from a track that was searched for and not found. + readPlaylist(playlistId: string): Promise<{ tracks: Track[]; truncated: boolean; skipped?: Array<{ name: string; artist: string }> }>; search(track: Track): Promise; createPlaylist(title: string): Promise; addTracks(playlistId: string, trackIds: string[]): Promise; getPlaylistTrackIds(playlistId: string): Promise>; listPlaylists(): Promise<{ id: string; title: string }[]>; + // Optional. Resolve a playlist's display name from its id, where the provider can do so. + playlistName?(playlistId: string): Promise; // Optional. Reorder the playlist to `trackIds`, keyed on track id — a provider whose native // reorder uses an opaque per-item handle looks that up internally. Resolves true if it moved anything. reorder?(playlistId: string, trackIds: string[]): Promise; diff --git a/src/ytmusic-parse.ts b/src/ytmusic-parse.ts new file mode 100644 index 0000000..5759bc4 --- /dev/null +++ b/src/ytmusic-parse.ts @@ -0,0 +1,105 @@ +import type { Candidate } from './types.js'; + +// Parsing for YouTube Music's InnerTube responses, split out from transport so it can be pinned by +// fixtures recorded from the real service. The renderer nesting is ~6 levels deep and is not a +// contract: everything here reads defensively and returns nothing rather than throwing. + +// A playlist entry or search result, in this provider's own terms. +export type YtItem = { + videoId: string; + // The opaque per-item handle a reorder moves. The same song added twice has two of them, which is + // why it can never key the conversion interface (ADR 0002). Present only on playlist reads. + setVideoId?: string; + title: string; + // For a song this is the artist; for a music video it is the uploading channel, which is exactly + // why a music video's metadata is not trusted. + artist: string; + durationSec: number | null; + isSong: boolean; +}; + +// Only a catalog release (ATV) counts as a song. Everything else — an official music video, an +// official-source upload, a user upload, a podcast — is a video whose stated artist is the channel. +// An entry with no type at all is an unavailable/removed track and is treated the same way. +const SONG_TYPE = 'MUSIC_VIDEO_TYPE_ATV'; + +type Runs = { runs?: Array<{ text?: string }> }; +const runsText = (t: Runs | undefined) => t?.runs?.map((r) => r.text ?? '').join('') ?? ''; + +// "3:49" → 229, "1:02:03" → 3723. Anything that is not a clock reading is null, *not* zero — a +// playlist read has a duration column and a search result does not, and callers tell the two apart +// by null. Returning 0 for an empty column silently swallows the fallback. +export function parseDuration(text: string): number | null { + const trimmed = text.trim(); + if (!/^\d+(:[0-5]?\d)+$/.test(trimmed)) return null; + return trimmed.split(':').map(Number).reduce((total, part) => total * 60 + part, 0); +} + +type Renderer = { + flexColumns?: Array<{ musicResponsiveListItemFlexColumnRenderer?: { text?: Runs } }>; + fixedColumns?: Array<{ musicResponsiveListItemFixedColumnRenderer?: { text?: Runs } }>; + playlistItemData?: { videoId?: string; playlistSetVideoId?: string }; + overlay?: { musicItemThumbnailOverlayRenderer?: { content?: { musicPlayButtonRenderer?: { playNavigationEndpoint?: WatchEndpoint } } } }; +}; +type WatchEndpoint = { watchEndpoint?: { videoId?: string; watchEndpointMusicSupportedConfigs?: { watchEndpointMusicConfig?: { musicVideoType?: string } } } }; + +// Every musicResponsiveListItemRenderer anywhere in the response. The path to them differs between +// a playlist browse, a search, and a continuation, and none of those paths is stable, so this walks. +export function renderers(node: unknown, found: Renderer[] = []): Renderer[] { + if (!node || typeof node !== 'object') return found; + const record = node as Record; + if (record.musicResponsiveListItemRenderer) found.push(record.musicResponsiveListItemRenderer as Renderer); + for (const child of Object.values(record)) renderers(child, found); + return found; +} + +// The continuation token for the next page, if the response has one. +export function continuationToken(node: unknown): string | null { + if (!node || typeof node !== 'object') return null; + const record = node as Record; + const token = (record.continuationItemRenderer as { continuationEndpoint?: { continuationCommand?: { token?: string } } } | undefined) + ?.continuationEndpoint?.continuationCommand?.token; + if (token) return token; + for (const child of Object.values(record)) { + const found = continuationToken(child); + if (found) return found; + } + return null; +} + +export function playlistTitle(response: unknown): string { + const micro = (response as { microformat?: { microformatDataRenderer?: { title?: string } } })?.microformat; + return micro?.microformatDataRenderer?.title ?? ''; +} + +function parseItem(renderer: Renderer): YtItem | null { + const watch = renderer.overlay?.musicItemThumbnailOverlayRenderer?.content?.musicPlayButtonRenderer?.playNavigationEndpoint?.watchEndpoint; + const videoId = renderer.playlistItemData?.videoId ?? watch?.videoId; + if (!videoId) return null; + const columns = renderer.flexColumns?.map((c) => runsText(c.musicResponsiveListItemFlexColumnRenderer?.text)) ?? []; + const title = columns[0] ?? ''; + if (!title) return null; + + // A playlist read puts the duration in its own fixed column. A search result has no fixed column + // and instead trails it on the subtitle line: "Artist • Album • 3:49" (song), or + // "Channel • 178M views • 4:10" (video). Either way the duration is the last •-separated part. + const fixed = runsText(renderer.fixedColumns?.[0]?.musicResponsiveListItemFixedColumnRenderer?.text); + const subtitleParts = (columns[1] ?? '').split('•').map((s) => s.trim()).filter(Boolean); + const trailing = subtitleParts.length > 1 ? subtitleParts[subtitleParts.length - 1] : ''; + const durationSec = parseDuration(fixed) ?? parseDuration(trailing); + + return { + videoId, + setVideoId: renderer.playlistItemData?.playlistSetVideoId, + title, + artist: subtitleParts[0] ?? '', + durationSec, + isSong: watch?.watchEndpointMusicSupportedConfigs?.watchEndpointMusicConfig?.musicVideoType === SONG_TYPE, + }; +} + +export function parseItems(response: unknown): YtItem[] { + return renderers(response).flatMap((r) => parseItem(r) ?? []); +} + +export const toCandidate = (item: YtItem): Candidate => ({ id: item.videoId, title: item.title, artist: item.artist, duration: item.durationSec }); diff --git a/src/ytmusic-provider.ts b/src/ytmusic-provider.ts new file mode 100644 index 0000000..00df91e --- /dev/null +++ b/src/ytmusic-provider.ts @@ -0,0 +1,160 @@ +import { matchCandidates, matchVideoCandidates, searchQuery } from './matcher.js'; +import { innertube, isMix, parsePlaylistRef } from './ytmusic.js'; +import { continuationToken, parseItems, playlistTitle, toCandidate, type YtItem } from './ytmusic-parse.js'; +import type { Match, Provider, Track } from './types.js'; + +// Search filter params, as the web client sends them: catalog songs, and music videos. +const SONGS = 'EgWKAQIIAWoKEAkQBRAKEAMQBA%3D%3D'; +const VIDEOS = 'EgWKAQIQAWoKEAkQBRAKEAMQBA%3D%3D'; + +const secToMs = (s: number | null | undefined) => (s == null ? null : s * 1000); + +export class YtMusicProvider implements Provider { + readonly name = 'YouTube Music'; + constructor(private readonly cookie: string) {} + + private call(path: string, body: Record): Promise { return innertube(path, this.cookie, body); } + + // Every page of a playlist, following continuations. + private async browseAll(playlistId: string): Promise<{ pages: unknown[]; items: YtItem[] }> { + const pages: unknown[] = []; + const items: YtItem[] = []; + let token: string | null = null; + for (;;) { + const page: unknown = token + ? await this.call('browse', { continuation: token }) + : await this.call('browse', { browseId: `VL${parsePlaylistRef(playlistId)}` }); + pages.push(page); + items.push(...parseItems(page)); + token = continuationToken(page); + if (!token) return { pages, items }; + } + } + + // Music videos are read but not converted: their stated artist is the uploading channel, so + // matching on it produces confident wrong answers. They come back as `skipped` so the report can + // show them as their own kind of row rather than silently dropping them. + async readPlaylist(playlistId: string): Promise<{ tracks: Track[]; truncated: boolean; skipped: Array<{ name: string; artist: string }> }> { + if (isMix(playlistId)) { + throw new Error('That is a mix, not a playlist. A mix is generated fresh every time it is opened, so it has no fixed contents to convert — open the playlist you want and use its link.'); + } + const { items } = await this.browseAll(playlistId); + return { + tracks: items.filter((i) => i.isSong).map((i) => ({ name: i.title, artist: i.artist, durationMs: secToMs(i.durationSec) })), + truncated: false, + skipped: items.filter((i) => !i.isSong).map((i) => ({ name: i.title, artist: i.artist })), + }; + } + + async playlistName(playlistId: string): Promise { + if (isMix(playlistId)) return parsePlaylistRef(playlistId); + const page = await this.call('browse', { browseId: `VL${parsePlaylistRef(playlistId)}` }); + return playlistTitle(page) || parsePlaylistRef(playlistId); + } + + private async searchItems(track: Track, params: string): Promise { + const response = await this.call('search', { query: searchQuery(track.name, track.artist), params }); + return parseItems(response); + } + + // Catalog songs first. Only when nothing in the catalog matches does a music video get a look, and + // then under stricter rules — see matchVideoCandidates (ADR 0003: the provider owns the search + // strategy, the matcher owns the rules). + async search(track: Track): Promise { + const songs = await this.searchItems(track, SONGS); + const song = matchCandidates(track.name, track.artist, track.durationMs, songs.filter((i) => i.isSong).map(toCandidate)); + if (song) return song; + + const videos = await this.searchItems(track, VIDEOS); + return matchVideoCandidates(track.name, track.artist, track.durationMs, videos.map(toCandidate)); + } + + async createPlaylist(title: string): Promise { + const response = await this.call<{ playlistId?: string }>('playlist/create', { title, description: '', privacyStatus: 'PRIVATE' }); + if (!response.playlistId) throw new Error('YouTube Music did not return a playlist id'); + return response.playlistId; + } + + // Not part of the Provider interface — plx never deletes a user's playlist during a conversion. + // It exists so a live write can be undone, which is the only way to exercise the write path + // against the real service without leaving litter behind. + async deletePlaylist(playlistId: string): Promise { + await this.call('playlist/delete', { playlistId: parsePlaylistRef(playlistId) }); + } + + async addTracks(playlistId: string, videoIds: string[]): Promise { + const response = await this.call<{ status?: string }>('browse/edit_playlist', { + playlistId: parsePlaylistRef(playlistId), + actions: videoIds.map((videoId) => ({ action: 'ACTION_ADD_VIDEO', addedVideoId: videoId })), + }); + if (response.status && response.status !== 'STATUS_SUCCEEDED') throw new Error(`YouTube Music refused the add (${response.status})`); + return videoIds.length; + } + + async getPlaylistTrackIds(playlistId: string): Promise> { + const { items } = await this.browseAll(playlistId); + return new Set(items.map((i) => i.videoId)); + } + + async listPlaylists(): Promise<{ id: string; title: string }[]> { + const response = await this.call('browse', { browseId: 'FEmusic_liked_playlists' }); + const found: { id: string; title: string }[] = []; + (function walk(node: unknown) { + if (!node || typeof node !== 'object') return; + const record = node as Record; + const row = record.musicTwoRowItemRenderer as { navigationEndpoint?: { browseEndpoint?: { browseId?: string } }; title?: { runs?: Array<{ text?: string }> } } | undefined; + const browseId = row?.navigationEndpoint?.browseEndpoint?.browseId; + const title = row?.title?.runs?.[0]?.text; + // Anything not a playlist (an artist row, say) has no VL-prefixed browse id. + if (browseId?.startsWith('VL') && title) found.push({ id: browseId.slice(2), title }); + for (const child of Object.values(record)) walk(child); + })(response); + return found; + } + + // Verification: re-resolve a match by its own id and hand back what the service says it is. + // `player` reports flat `videoDetails` rather than the deep renderer nesting everything else uses, + // which makes it both cheaper and far less likely to break. + async resolveTrack(videoId: string): Promise { + const response = await this.call<{ videoDetails?: { title?: string; author?: string; lengthSeconds?: string } }>('player', { videoId }); + const details = response.videoDetails; + if (!details?.title) throw new Error('YouTube Music returned nothing for that track'); + const seconds = Number(details.lengthSeconds); + return { + name: details.title, + // A catalog track's uploader is the artist's auto-generated " - Topic" channel. + artist: (details.author ?? '').replace(/\s*-\s*Topic$/, ''), + durationMs: Number.isFinite(seconds) ? seconds * 1000 : null, + }; + } + + // Reorder capability (ADR 0002), keyed on track id. YouTube Music moves by `setVideoId`, an opaque + // per-item handle that is *not* the track id — the same song added twice has two of them. That + // lookup happens here, inside the provider; the handle never reaches the conversion interface. + async reorder(playlistId: string, trackIds: string[]): Promise { + const desired = [...new Set(trackIds)].filter(Boolean); + if (desired.length < 2) return false; + const { items } = await this.browseAll(playlistId); + + const handles = new Map(); + for (const item of items) if (item.setVideoId && !handles.has(item.videoId)) handles.set(item.videoId, item.setVideoId); + const movable = desired.filter((id) => handles.has(id)); + const current = items.map((i) => i.videoId).filter((id) => movable.includes(id)); + if (current.length === movable.length && movable.every((id, i) => current[i] === id)) return false; + + // Walk backwards placing each track before the one that should follow it, so a single pass + // leaves the run in source order without needing to know where anything moved to. + for (let i = movable.length - 2; i >= 0; i -= 1) { + await this.call('browse/edit_playlist', { + playlistId: parsePlaylistRef(playlistId), + actions: [{ + action: 'ACTION_MOVE_VIDEO_BEFORE', + setVideoId: handles.get(movable[i]), + movedSetVideoIdSuccessor: handles.get(movable[i + 1]), + }], + }); + await new Promise((resolve) => setTimeout(resolve, 100)); + } + return true; + } +} diff --git a/src/ytmusic.ts b/src/ytmusic.ts index 16b8d55..102481d 100644 --- a/src/ytmusic.ts +++ b/src/ytmusic.ts @@ -112,3 +112,24 @@ export async function validateSession(cookie: string): Promise { if (!account) throw new Error(EXPIRED); return account; } + +// --- Playlist ids ---------------------------------------------------------- + +// A YouTube Music playlist id *is* a YouTube playlist id, so both hosts name the same thing. +export function parsePlaylistRef(ref: string): string { + const input = ref.trim(); + if (/^https?:\/\//i.test(input)) { + try { + const list = new URL(input).searchParams.get('list'); + if (list) return list; + const last = new URL(input).pathname.replace(/\/+$/, '').split('/').pop() ?? ''; + return last.replace(/^VL/, ''); + } catch { /* fall through to the raw value */ } + } + return input.replace(/^VL/, ''); +} + +// A mix (radio) regenerates on every visit: its contents are not reproducible, so it is not a +// playlist and plx refuses it as a source. Every mix id starts with RD — RDCLAK, RDAMVM, RDEM… +// Checked on the id so the reason is explicit, rather than surfacing as an empty read. +export const isMix = (playlistId: string) => /^RD/i.test(parsePlaylistRef(playlistId)); diff --git a/tests/args.test.ts b/tests/args.test.ts index dd9adf7..a9c491c 100644 --- a/tests/args.test.ts +++ b/tests/args.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest'; import { parseArgs } from '../src/args.js'; -import { PROVIDERS, convertibleProviders, inferSource, providerFor, targetKeys } from '../src/registry.js'; +import { PROVIDERS, inferSource, providerFor, targetKeys } from '../src/registry.js'; describe('parseArgs', () => { it('takes a link plus an explicit target provider', () => { @@ -58,22 +58,19 @@ describe('the provider registry', () => { expect(providerFor('napster')).toBeUndefined(); }); - it('offers only providers that can be written into as --to targets', () => { - expect(targetKeys()).toEqual(['spotify', 'deezer']); - expect(targetKeys()).not.toContain('ytmusic'); // registered for credentials, not yet convertible + it('offers every provider as a --to target', () => { + expect(targetKeys()).toEqual(['spotify', 'deezer', 'ytmusic']); }); // Every direction is source × target minus the identity pairs, so three providers give six. - it('reaches N×(N-1) directions once every provider is convertible', () => { + it('reaches all six directions across three providers', () => { const directions = PROVIDERS.flatMap((s) => PROVIDERS.filter((t) => t !== s).map((t) => `${s.key}->${t.key}`)); expect(directions).toHaveLength(PROVIDERS.length * (PROVIDERS.length - 1)); - expect(directions).toHaveLength(6); - }); - - it('currently runs the two directions whose providers are both built', () => { - const ready = convertibleProviders(); - expect(ready.flatMap((s) => ready.filter((t) => t !== s).map((t) => `${s.key}->${t.key}`))) - .toEqual(['spotify->deezer', 'deezer->spotify']); + expect(directions).toEqual([ + 'spotify->deezer', 'spotify->ytmusic', + 'deezer->spotify', 'deezer->ytmusic', + 'ytmusic->spotify', 'ytmusic->deezer', + ]); }); it('gives every provider the fields the menu and command line read', () => { diff --git a/tests/conversion-into-spotify.test.ts b/tests/conversion-into-spotify.test.ts index b2d4268..3167216 100644 --- a/tests/conversion-into-spotify.test.ts +++ b/tests/conversion-into-spotify.test.ts @@ -54,8 +54,8 @@ describe('conversion into a Spotify target', () => { expect(created).toContain('spclient.wg.spotify.com'); expect(added).toEqual([['spotify:track:777']]); const csv = await readFile(output, 'utf8'); - expect(csv).toContain('My Playlist,Song One,Artist One,,true,spotify:track:777,exact,'); - expect(csv).toContain('My Playlist,Song Two,Artist Two,,false,,,no match'); + expect(csv).toContain('My Playlist,Deezer,Spotify,Song One,Artist One,,true,spotify:track:777,exact,'); + expect(csv).toContain('My Playlist,Deezer,Spotify,Song Two,Artist Two,,false,,,no match'); }); // The target can resolve a track by identifier, so every match is re-resolved and re-checked diff --git a/tests/conversion.test.ts b/tests/conversion.test.ts index 4dc54eb..3b3f25e 100644 --- a/tests/conversion.test.ts +++ b/tests/conversion.test.ts @@ -69,7 +69,9 @@ describe('Conversion', () => { expect(result.truncated).toBe(false); expect(converter.rows).toHaveLength(1); expect(converter.rows[0].matched).toBe(true); - expect(converter.rows[0].deezer_id).toBe('42'); + expect(converter.rows[0].target_id).toBe('42'); + expect(converter.rows[0].source).toBe('Spotify'); + expect(converter.rows[0].target).toBe('Deezer'); expect(converter.rows[0].method).toBe('exact'); expect(converter.rows[0].note).toBe('dry-run'); }); @@ -188,10 +190,11 @@ describe('CSV quoting', () => { it('quotes fields containing commas and quotes', async () => { const dir = await mkdtemp(join(tmpdir(), 'plx-')); const path = join(dir, 'out.csv'); - await writeCsv(path, [{ playlist: 'A, B', title: 'Song "X"', artist: 'Artist', isrc: null, matched: true, deezer_id: '123', method: 'exact', note: 'ok' }]); + await writeCsv(path, [{ playlist: 'A, B', title: 'Song "X"', artist: 'Artist', isrc: null, matched: true, target_id: '123', method: 'exact', note: 'ok', source: 'Spotify', target: 'Deezer' }]); const content = await readFile(path, 'utf8'); expect(content).toContain('"A, B"'); expect(content).toContain('"Song ""X"""'); expect(content).toContain(',123,exact,ok'); + expect(content.split('\n')[0]).toBe('playlist,source,target,title,artist,isrc,matched,target_id,method,note'); }); }); diff --git a/tests/fixtures/ytmusic/playlist-mixed-types.json b/tests/fixtures/ytmusic/playlist-mixed-types.json new file mode 100644 index 0000000..2826bbf --- /dev/null +++ b/tests/fixtures/ytmusic/playlist-mixed-types.json @@ -0,0 +1,400 @@ +{ + "microformat": { + "microformatDataRenderer": { + "title": "Retrogade" + } + }, + "contents": { + "items": [ + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Mothman" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Haarper" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Mothman" + } + ] + } + } + } + ], + "fixedColumns": [ + { + "musicResponsiveListItemFixedColumnRenderer": { + "text": { + "runs": [ + { + "text": "1:53" + } + ] + } + } + } + ], + "playlistItemData": { + "playlistSetVideoId": "21D2A4324C732A32", + "videoId": "2Fdi3l5Xu88", + "voteSortValue": 1616204680 + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CMkKEMjeAiITCMT1v8WAqJYDFQPyVQkdmhQ3LsoBBB6bSC8=", + "watchEndpoint": { + "videoId": "2Fdi3l5Xu88", + "playlistId": "PLk5rTSgAZ9W0Z5y1L1h6gKOiqR13A9LUC", + "params": "ogMLX3pOR1F3eWl2aTQ%3D", + "playerParams": "iAQB8AUBygYQMjFEMkE0MzI0QzczMkEzMrAIAg%3D%3D", + "playlistSetVideoId": "21D2A4324C732A32", + "loggingContext": { + "vssLoggingContext": { + "serializedContextData": "GiJQTGs1clRTZ0FaOVcwWjV5MUwxaDZnS09pcVIxM0E5TFVD" + } + }, + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_ATV" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "ciscaux - hey! hi! hello! featuring 1nonly and lilbubblegum (prod. okayjml)" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "splash.mp3" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": {} + } + } + ], + "fixedColumns": [ + { + "musicResponsiveListItemFixedColumnRenderer": { + "text": { + "runs": [ + { + "text": "2:30" + } + ] + } + } + } + ], + "playlistItemData": { + "playlistSetVideoId": "52152B4946C2F73F", + "videoId": "UAjUiiNFWi0", + "voteSortValue": 1615980306 + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CPULEMjeAiITCMT1v8WAqJYDFQPyVQkdmhQ3LsoBBB6bSC8=", + "watchEndpoint": { + "videoId": "UAjUiiNFWi0", + "playlistId": "PLk5rTSgAZ9W0Z5y1L1h6gKOiqR13A9LUC", + "playerParams": "iAQB8AUBygYQNTIxNTJCNDk0NkMyRjczRrAIAg%3D%3D", + "playlistSetVideoId": "52152B4946C2F73F", + "loggingContext": { + "vssLoggingContext": { + "serializedContextData": "GiJQTGs1clRTZ0FaOVcwWjV5MUwxaDZnS09pcVIxM0E5TFVD" + } + }, + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_UGC" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Nos Monster" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "TrippyThaKid" + }, + { + "text": " & " + }, + { + "text": "ZCR" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": {} + } + } + ], + "fixedColumns": [ + { + "musicResponsiveListItemFixedColumnRenderer": { + "text": { + "runs": [ + { + "text": "1:39" + } + ] + } + } + } + ], + "playlistItemData": { + "playlistSetVideoId": "090796A75D153932", + "videoId": "vCCSeIMv_x8", + "voteSortValue": 1615980687 + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "COYLEMjeAiITCMT1v8WAqJYDFQPyVQkdmhQ3LsoBBB6bSC8=", + "watchEndpoint": { + "videoId": "vCCSeIMv_x8", + "playlistId": "PLk5rTSgAZ9W0Z5y1L1h6gKOiqR13A9LUC", + "playerParams": "iAQB8AUBygYQMDkwNzk2QTc1RDE1MzkzMrAIAg%3D%3D", + "playlistSetVideoId": "090796A75D153932", + "loggingContext": { + "vssLoggingContext": { + "serializedContextData": "GiJQTGs1clRTZ0FaOVcwWjV5MUwxaDZnS09pcVIxM0E5TFVD" + } + }, + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_OMV" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "lilbubblegum x lames - drift (prod shenrxn)" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "lilbubblegum" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": {} + } + } + ], + "fixedColumns": [ + { + "musicResponsiveListItemFixedColumnRenderer": { + "text": { + "runs": [ + { + "text": "1:45" + } + ] + } + } + } + ], + "playlistItemData": { + "playlistSetVideoId": "4949B9D0837AA5B0", + "videoId": "D2V0Nt-9OkQ", + "voteSortValue": 1627658362 + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CIADEMjeAiITCMT1v8WAqJYDFQPyVQkdmhQ3LsoBBB6bSC8=", + "watchEndpoint": { + "videoId": "D2V0Nt-9OkQ", + "playlistId": "PLk5rTSgAZ9W0Z5y1L1h6gKOiqR13A9LUC", + "playerParams": "iAQB8AUBygYQNDk0OUI5RDA4MzdBQTVCMLAIAg%3D%3D", + "playlistSetVideoId": "4949B9D0837AA5B0", + "loggingContext": { + "vssLoggingContext": { + "serializedContextData": "GiJQTGs1clRTZ0FaOVcwWjV5MUwxaDZnS09pcVIxM0E5TFVD" + } + }, + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_OFFICIAL_SOURCE_MUSIC" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Pablo" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Markie-D" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Unchained" + } + ] + } + } + } + ], + "fixedColumns": [ + { + "musicResponsiveListItemFixedColumnRenderer": { + "text": { + "runs": [ + { + "text": "3:04" + } + ] + } + } + } + ], + "playlistItemData": { + "playlistSetVideoId": "B0EAEBDFE250D593", + "videoId": "o_QWxcPLGJs", + "voteSortValue": 1623215386 + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": {} + } + } + } + } + } + ] + } +} \ No newline at end of file diff --git a/tests/fixtures/ytmusic/playlist.json b/tests/fixtures/ytmusic/playlist.json new file mode 100644 index 0000000..b31f26e --- /dev/null +++ b/tests/fixtures/ytmusic/playlist.json @@ -0,0 +1,347 @@ +{ + "microformat": { + "microformatDataRenderer": { + "title": "My Hanni" + } + }, + "contents": { + "items": [ + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Rising" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "tripleS" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "ASSEMBLE" + } + ] + } + } + } + ], + "fixedColumns": [ + { + "musicResponsiveListItemFixedColumnRenderer": { + "text": { + "runs": [ + { + "text": "2:40" + } + ] + } + } + } + ], + "playlistItemData": { + "playlistSetVideoId": "56B44F6D10557CC6", + "videoId": "ZftSRpQFTew", + "voteSortValue": 1753192307 + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CPEOEMjeAiITCNnC_LCAqJYDFa1PnQkdVnk6FsoBBB6bSC8=", + "watchEndpoint": { + "videoId": "ZftSRpQFTew", + "playlistId": "PLk5rTSgAZ9W0GBaPy00PTTb-06YUlyHKr", + "playerParams": "iAQB8AUBygYQNTZCNDRGNkQxMDU1N0NDNrAIAg%3D%3D", + "playlistSetVideoId": "56B44F6D10557CC6", + "loggingContext": { + "vssLoggingContext": { + "serializedContextData": "GiJQTGs1clRTZ0FaOVcwR0JhUHkwMFBUVGItMDZZVWx5SEty" + } + }, + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_ATV" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Girls' Capitalism" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "tripleS" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "LOVElution <ↀ>" + } + ] + } + } + } + ], + "fixedColumns": [ + { + "musicResponsiveListItemFixedColumnRenderer": { + "text": { + "runs": [ + { + "text": "3:39" + } + ] + } + } + } + ], + "playlistItemData": { + "playlistSetVideoId": "289F4A46DF0A30D2", + "videoId": "G0hLAX-bg3w", + "voteSortValue": 1753192309 + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CN4OEMjeAiITCNnC_LCAqJYDFa1PnQkdVnk6FsoBBB6bSC8=", + "watchEndpoint": { + "videoId": "G0hLAX-bg3w", + "playlistId": "PLk5rTSgAZ9W0GBaPy00PTTb-06YUlyHKr", + "playerParams": "iAQB8AUBygYQMjg5RjRBNDZERjBBMzBEMrAIAg%3D%3D", + "playlistSetVideoId": "289F4A46DF0A30D2", + "loggingContext": { + "vssLoggingContext": { + "serializedContextData": "GiJQTGs1clRTZ0FaOVcwR0JhUHkwMFBUVGItMDZZVWx5SEty" + } + }, + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_ATV" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Girls Never Die" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "tripleS" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "" + } + ] + } + } + } + ], + "fixedColumns": [ + { + "musicResponsiveListItemFixedColumnRenderer": { + "text": { + "runs": [ + { + "text": "3:08" + } + ] + } + } + } + ], + "playlistItemData": { + "playlistSetVideoId": "017208FAA85233F9", + "videoId": "gVl5RmTRBXY", + "voteSortValue": 1753192312 + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CMsOEMjeAiITCNnC_LCAqJYDFa1PnQkdVnk6FsoBBB6bSC8=", + "watchEndpoint": { + "videoId": "gVl5RmTRBXY", + "playlistId": "PLk5rTSgAZ9W0GBaPy00PTTb-06YUlyHKr", + "playerParams": "iAQB8AUBygYQMDE3MjA4RkFBODUyMzNGObAIAg%3D%3D", + "playlistSetVideoId": "017208FAA85233F9", + "loggingContext": { + "vssLoggingContext": { + "serializedContextData": "GiJQTGs1clRTZ0FaOVcwR0JhUHkwMFBUVGItMDZZVWx5SEty" + } + }, + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_ATV" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Hit the Floor" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "tripleS" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Visionary Vision " + } + ] + } + } + } + ], + "fixedColumns": [ + { + "musicResponsiveListItemFixedColumnRenderer": { + "text": { + "runs": [ + { + "text": "2:58" + } + ] + } + } + } + ], + "playlistItemData": { + "playlistSetVideoId": "52152B4946C2F73F", + "videoId": "mnvh9kaI1Lc", + "voteSortValue": 1753192314 + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CLgOEMjeAiITCNnC_LCAqJYDFa1PnQkdVnk6FsoBBB6bSC8=", + "watchEndpoint": { + "videoId": "mnvh9kaI1Lc", + "playlistId": "PLk5rTSgAZ9W0GBaPy00PTTb-06YUlyHKr", + "playerParams": "iAQB8AUBygYQNTIxNTJCNDk0NkMyRjczRrAIAg%3D%3D", + "playlistSetVideoId": "52152B4946C2F73F", + "loggingContext": { + "vssLoggingContext": { + "serializedContextData": "GiJQTGs1clRTZ0FaOVcwR0JhUHkwMFBUVGItMDZZVWx5SEty" + } + }, + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_ATV" + } + } + } + } + } + } + } + } + } + } + ] + } +} \ No newline at end of file diff --git a/tests/fixtures/ytmusic/search-songs.json b/tests/fixtures/ytmusic/search-songs.json new file mode 100644 index 0000000..e567f3f --- /dev/null +++ b/tests/fixtures/ytmusic/search-songs.json @@ -0,0 +1,304 @@ +{ + "contents": { + "items": [ + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Fortnight (feat. Post Malone)" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Taylor Swift" + }, + { + "text": " • " + }, + { + "text": "THE TORTURED POETS DEPARTMENT" + }, + { + "text": " • " + }, + { + "text": "3:49" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "313M plays" + } + ] + } + } + } + ], + "playlistItemData": { + "videoId": "eXrmLd5mer4" + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CPMCEMjeAiITCOjjnLGAqJYDFSlGnQkd8m8W-8oBBB6bSC8=", + "watchEndpoint": { + "videoId": "eXrmLd5mer4", + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_ATV" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Fortnight (Acoustic Version) (feat. Post Malone)" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Taylor Swift" + }, + { + "text": " • " + }, + { + "text": "Fortnight (Acoustic Version)" + }, + { + "text": " • " + }, + { + "text": "3:49" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "5.6M plays" + } + ] + } + } + } + ], + "playlistItemData": { + "videoId": "ikzjwDcxaNc" + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "COECEMjeAiITCOjjnLGAqJYDFSlGnQkd8m8W-8oBBB6bSC8=", + "watchEndpoint": { + "videoId": "ikzjwDcxaNc", + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_ATV" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Guilty as Sin?" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Taylor Swift" + }, + { + "text": " • " + }, + { + "text": "THE TORTURED POETS DEPARTMENT" + }, + { + "text": " • " + }, + { + "text": "4:15" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "120M plays" + } + ] + } + } + } + ], + "playlistItemData": { + "videoId": "rpLCx-fkEX0" + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CM8CEMjeAiITCOjjnLGAqJYDFSlGnQkd8m8W-8oBBB6bSC8=", + "watchEndpoint": { + "videoId": "rpLCx-fkEX0", + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_ATV" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Fortnight (BLOND:ISH Remix) (feat. Post Malone)" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Taylor Swift" + }, + { + "text": " & " + }, + { + "text": "BLOND:ISH" + }, + { + "text": " • " + }, + { + "text": "Fortnight (BLOND:ISH Remix)" + }, + { + "text": " • " + }, + { + "text": "3:37" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "6.1M plays" + } + ] + } + } + } + ], + "playlistItemData": { + "videoId": "MpOvl2zZscQ" + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CL0CEMjeAiITCOjjnLGAqJYDFSlGnQkd8m8W-8oBBB6bSC8=", + "watchEndpoint": { + "videoId": "MpOvl2zZscQ", + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_ATV" + } + } + } + } + } + } + } + } + } + } + ] + } +} \ No newline at end of file diff --git a/tests/fixtures/ytmusic/search-videos.json b/tests/fixtures/ytmusic/search-videos.json new file mode 100644 index 0000000..0442a05 --- /dev/null +++ b/tests/fixtures/ytmusic/search-videos.json @@ -0,0 +1,255 @@ +{ + "contents": { + "items": [ + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Fortnight (feat. Post Malone)" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Taylor Swift" + }, + { + "text": " • " + }, + { + "text": "178M views" + }, + { + "text": " • " + }, + { + "text": "4:10" + } + ] + } + } + } + ], + "playlistItemData": { + "videoId": "q3zqJs7JUCQ" + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CKsCEMjeAiITCJPnurGAqJYDFeicSwUd3EorOMoBBB6bSC8=", + "watchEndpoint": { + "videoId": "q3zqJs7JUCQ", + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_OMV" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Fortnight (feat. Post Malone)" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Taylor Swift" + }, + { + "text": " • " + }, + { + "text": "32M views" + }, + { + "text": " • " + }, + { + "text": "3:49" + } + ] + } + } + } + ], + "playlistItemData": { + "videoId": "b7kmP1fsGg8" + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CJwCEMjeAiITCJPnurGAqJYDFeicSwUd3EorOMoBBB6bSC8=", + "watchEndpoint": { + "videoId": "b7kmP1fsGg8", + "playerParams": "0gcJCfQA-iE-y48-", + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_OMV" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Taylor Swift feat. Post Malone - Fortnight (Lyrics)" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "TrendingTracks" + }, + { + "text": " • " + }, + { + "text": "5.4M views" + }, + { + "text": " • " + }, + { + "text": "3:49" + } + ] + } + } + } + ], + "playlistItemData": { + "videoId": "V-iZOohyftg" + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CI0CEMjeAiITCJPnurGAqJYDFeicSwUd3EorOMoBBB6bSC8=", + "watchEndpoint": { + "videoId": "V-iZOohyftg", + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_UGC" + } + } + } + } + } + } + } + } + } + }, + { + "musicResponsiveListItemRenderer": { + "flexColumns": [ + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Fortnight x About You - Taylor Swift & The 1975 ft. Post Malone (Mashup)" + } + ] + } + } + }, + { + "musicResponsiveListItemFlexColumnRenderer": { + "text": { + "runs": [ + { + "text": "Joshuel Mashups" + }, + { + "text": " • " + }, + { + "text": "448K views" + }, + { + "text": " • " + }, + { + "text": "4:46" + } + ] + } + } + } + ], + "playlistItemData": { + "videoId": "h1T4ZKkSu3g" + }, + "overlay": { + "musicItemThumbnailOverlayRenderer": { + "content": { + "musicPlayButtonRenderer": { + "playNavigationEndpoint": { + "clickTrackingParams": "CP8BEMjeAiITCJPnurGAqJYDFeicSwUd3EorOMoBBB6bSC8=", + "watchEndpoint": { + "videoId": "h1T4ZKkSu3g", + "watchEndpointMusicSupportedConfigs": { + "watchEndpointMusicConfig": { + "musicVideoType": "MUSIC_VIDEO_TYPE_UGC" + } + } + } + } + } + } + } + } + } + } + ] + } +} \ No newline at end of file diff --git a/tests/ytmusic-parse.test.ts b/tests/ytmusic-parse.test.ts new file mode 100644 index 0000000..f36a210 --- /dev/null +++ b/tests/ytmusic-parse.test.ts @@ -0,0 +1,124 @@ +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { describe, expect, it } from 'vitest'; +import { continuationToken, parseDuration, parseItems, playlistTitle } from '../src/ytmusic-parse.js'; +import { isMix, parsePlaylistRef } from '../src/ytmusic.js'; + +// Every fixture here was recorded from the live service and trimmed by hand to the paths the parser +// reads. Hand-written literals are rejected for this provider: the renderer nesting is ~6 levels +// deep, so a literal written from assumption pins the assumption and stays green while production +// goes red. A failure here should mean YouTube Music changed. +const fixture = (name: string) => JSON.parse(readFileSync(join(import.meta.dirname, 'fixtures/ytmusic', `${name}.json`), 'utf8')) as unknown; + +describe('parseDuration', () => { + it('reads the durations the service actually prints', () => { + expect(parseDuration('2:40')).toBe(160); + expect(parseDuration('3:49')).toBe(229); + expect(parseDuration('1:02:03')).toBe(3723); + }); + it('returns nothing for text that is not a duration', () => { + expect(parseDuration('313M plays')).toBeNull(); + expect(parseDuration('')).toBeNull(); + }); +}); + +describe('a recorded playlist read', () => { + const response = fixture('playlist'); + + it('takes the playlist title from the microformat block', () => { + expect(playlistTitle(response)).toBe('My Hanni'); + }); + + it('reads title, artist, duration and ids off each entry', () => { + const items = parseItems(response); + expect(items).toHaveLength(4); + expect(items[0]).toMatchObject({ title: 'Rising', artist: 'tripleS', durationSec: 160, videoId: 'ZftSRpQFTew', isSong: true }); + // The per-item reorder handle, which is not the track id. + expect(items[0].setVideoId).toBeTruthy(); + expect(items[0].setVideoId).not.toBe(items[0].videoId); + }); + + it('gives every entry a duration', () => { + expect(parseItems(response).every((i) => i.durationSec != null)).toBe(true); + }); +}); + +// One real entry of each type the service returned for a mixed playlist. +describe('a recorded playlist of mixed entry types', () => { + const items = parseItems(fixture('playlist-mixed-types')); + + it('counts only a catalog release as a song', () => { + // ATV is the catalog track. UGC, OMV, official-source, and an entry with no type at all are + // videos or unavailable — their stated artist is the uploading channel, not the performer. + expect(items.filter((i) => i.isSong)).toHaveLength(1); + expect(items.filter((i) => !i.isSong).length).toBeGreaterThanOrEqual(3); + }); + + it('still parses the videos it will not convert, so they can be reported', () => { + for (const item of items) { + expect(item.videoId).toBeTruthy(); + expect(item.title).toBeTruthy(); + } + }); +}); + +describe('a recorded song search', () => { + const items = parseItems(fixture('search-songs')); + + it('parses the trailing duration off the subtitle line', () => { + // "Taylor Swift • THE TORTURED POETS DEPARTMENT • 3:49" — artist first, duration last. + expect(items[0]).toMatchObject({ title: 'Fortnight (feat. Post Malone)', artist: 'Taylor Swift', durationSec: 229 }); + }); + + it('marks catalog results as songs', () => { + expect(items.every((i) => i.isSong)).toBe(true); + }); +}); + +describe('a recorded video search', () => { + const items = parseItems(fixture('search-videos')); + + it('never marks a video result as a song', () => { + expect(items.some((i) => i.isSong)).toBe(false); + }); + + it('parses the channel as the artist and the trailing duration', () => { + // "Taylor Swift • 178M views • 4:10" — the middle part is a view count, not an album. + expect(items[0]).toMatchObject({ artist: 'Taylor Swift', durationSec: 250 }); + }); + + it('reads a user upload whose channel is not the artist', () => { + // The real UGC case the containment rule exists for: artist is in the title, channel is not it. + const ugc = items.find((i) => i.artist === 'TrendingTracks'); + expect(ugc?.title).toContain('Taylor Swift'); + expect(ugc?.durationSec).toBe(229); + }); +}); + +describe('continuationToken', () => { + it('returns nothing when a response has no further pages', () => { + expect(continuationToken(fixture('search-songs'))).toBeNull(); + }); +}); + +describe('playlist references', () => { + it('resolves music.youtube.com and youtube.com links to the same id', () => { + const id = 'PLk5rTSgAZ9W0GBaPy00PTTb-06YUlyHKr'; + expect(parsePlaylistRef(`https://music.youtube.com/playlist?list=${id}`)).toBe(id); + expect(parsePlaylistRef(`https://www.youtube.com/playlist?list=${id}`)).toBe(id); + expect(parsePlaylistRef(id)).toBe(id); + expect(parsePlaylistRef(`VL${id}`)).toBe(id); // the browse-id form + }); + + it('refuses a mix, whatever form its id arrives in', () => { + expect(isMix('RDCLAK5uy_kLWIr9gv1XLlPbaDS965-Db4ODFbLD0Q')).toBe(true); + expect(isMix('https://music.youtube.com/playlist?list=RDAMVMabc123')).toBe(true); + expect(isMix('RDEMabc')).toBe(true); + }); + + it('accepts user playlists, Liked Music, and auto-generated album playlists', () => { + expect(isMix('PLk5rTSgAZ9W0GBaPy00PTTb-06YUlyHKr')).toBe(false); // user playlist + expect(isMix('LM')).toBe(false); // Liked Music + expect(isMix('OLAK5uy_kZm0dJmZOJ7yQxKmL5Vw2gU1TVzFvXqYs')).toBe(false); // album playlist + }); +}); diff --git a/tests/ytmusic-provider.test.ts b/tests/ytmusic-provider.test.ts new file mode 100644 index 0000000..19197ba --- /dev/null +++ b/tests/ytmusic-provider.test.ts @@ -0,0 +1,246 @@ +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { matchCandidates, matchVideoCandidates } from '../src/matcher.js'; +import { YtMusicProvider } from '../src/ytmusic-provider.js'; +import { Conversion } from '../src/conversion.js'; +import { DeezerClient } from '../src/deezer.js'; + +const fixture = (name: string) => JSON.parse(readFileSync(join(import.meta.dirname, 'fixtures/ytmusic', `${name}.json`), 'utf8')) as unknown; + +// Stub the one fetch the provider makes, answering by InnerTube path. +function stubInnertube(handler: (path: string, body: Record) => unknown): void { + vi.stubGlobal('fetch', vi.fn().mockImplementation(async (url: string, init?: RequestInit) => { + const path = new URL(String(url)).pathname.replace('/youtubei/v1/', ''); + const body = JSON.parse(String((init as RequestInit).body)) as Record; + const result = handler(path, body); + return { ok: true, status: 200, text: async () => JSON.stringify(result ?? {}) }; + })); +} + +const provider = () => new YtMusicProvider('SAPISID=test'); + +// ── The music video rule (#9), at the matcher level where match rules live (ADR 0003) ──────────── +describe('the music video match rule', () => { + const song = { title: 'Fortnight', artist: 'Taylor Swift', durationMs: 229_000 }; + + it('accepts a video whose channel is the artist', () => { + const match = matchVideoCandidates(song.title, song.artist, song.durationMs, [ + { id: 'v1', title: 'Fortnight (feat. Post Malone)', artist: 'Taylor Swift', duration: 229 }, + ]); + expect(match).toMatchObject({ id: 'v1', method: 'video' }); + }); + + // The real UGC shape: the artist is in the video's title, the channel is some uploader. + it('accepts a video whose title contains the artist even when the channel does not', () => { + const match = matchVideoCandidates(song.title, song.artist, song.durationMs, [ + { id: 'v2', title: 'Taylor Swift feat. Post Malone - Fortnight (Lyrics)', artist: 'TrendingTracks', duration: 229 }, + ]); + expect(match?.id).toBe('v2'); + }); + + // Duration is the only guard left once the artist rule relaxes, so it becomes mandatory. + it('rejects a video outside the duration tolerance even when title and artist look right', () => { + expect(matchVideoCandidates(song.title, song.artist, song.durationMs, [ + { id: 'live', title: 'Fortnight', artist: 'Taylor Swift', duration: 300 }, + ])).toBeNull(); + }); + + it('rejects a video with no duration at all, rather than letting it through', () => { + expect(matchVideoCandidates(song.title, song.artist, song.durationMs, [ + { id: 'v3', title: 'Fortnight', artist: 'Taylor Swift', duration: null }, + ])).toBeNull(); + }); + + it('rejects a video by someone else entirely', () => { + expect(matchVideoCandidates(song.title, song.artist, song.durationMs, [ + { id: 'cover', title: 'Fortnight', artist: 'Some Cover Band', duration: 229 }, + ])).toBeNull(); + }); + + it('is stricter than the catalog rule, which tolerates a missing duration', () => { + const candidates = [{ id: 'x', title: 'Fortnight', artist: 'Taylor Swift', duration: null }]; + expect(matchCandidates(song.title, song.artist, song.durationMs, candidates)).not.toBeNull(); + expect(matchVideoCandidates(song.title, song.artist, song.durationMs, candidates)).toBeNull(); + }); +}); + +// ── Searching: catalog first, video only as a fallback (#8, #9) ────────────────────────────────── +describe('YtMusicProvider.search', () => { + afterEach(() => vi.unstubAllGlobals()); + + it('never chooses a music video when a catalog song matches', async () => { + let videoSearched = false; + stubInnertube((path, body) => { + if (path !== 'search') return {}; + if (String(body.params).includes('EgWKAQIQ')) videoSearched = true; + return fixture(String(body.params).includes('EgWKAQIQ') ? 'search-videos' : 'search-songs'); + }); + const match = await provider().search({ name: 'Fortnight', artist: 'Taylor Swift', durationMs: 229_000 }); + expect(match).toMatchObject({ id: 'eXrmLd5mer4', method: 'exact' }); // the ATV catalog track + expect(videoSearched).toBe(false); // the catalog answered, so videos were never asked for + }); + + it('falls back to a music video when the catalog has nothing', async () => { + stubInnertube((path, body) => { + if (path !== 'search') return {}; + return String(body.params).includes('EgWKAQIQ') ? fixture('search-videos') : {}; + }); + const match = await provider().search({ name: 'Fortnight', artist: 'Taylor Swift', durationMs: 250_000 }); + expect(match).toMatchObject({ method: 'video' }); + }); + + it('reports no match rather than substituting a video of the wrong length', async () => { + stubInnertube((path, body) => { + if (path !== 'search') return {}; + return String(body.params).includes('EgWKAQIQ') ? fixture('search-videos') : {}; + }); + expect(await provider().search({ name: 'Fortnight', artist: 'Taylor Swift', durationMs: 30_000 })).toBeNull(); + }); +}); + +// ── Reading a playlist (#7) ────────────────────────────────────────────────────────────────────── +describe('YtMusicProvider.readPlaylist', () => { + afterEach(() => vi.unstubAllGlobals()); + + it('refuses a mix with a reason, without calling the service', async () => { + const spy = vi.fn(); + vi.stubGlobal('fetch', spy); + await expect(provider().readPlaylist('RDCLAK5uy_abc')).rejects.toThrow(/mix.*no fixed contents/i); + expect(spy).not.toHaveBeenCalled(); + }); + + it('converts the catalog songs and holds the music videos back as skipped', async () => { + stubInnertube(() => fixture('playlist-mixed-types')); + const { tracks, skipped } = await provider().readPlaylist('PLtest'); + expect(tracks).toHaveLength(1); // only the ATV entry + expect(skipped.length).toBeGreaterThanOrEqual(3); + expect(skipped.every((s) => s.name)).toBe(true); + }); + + it('accepts Liked Music and album playlists as sources', async () => { + stubInnertube(() => fixture('playlist')); + for (const id of ['LM', 'OLAK5uy_kZm0dJmZOJ7yQxKmL5Vw2gU1TVzFvXqYs']) { + await expect(provider().readPlaylist(id)).resolves.toMatchObject({ truncated: false }); + } + }); +}); + +// A skipped music video is its own kind of report row, not an unmatched track. +describe('skipped videos in the report', () => { + afterEach(() => vi.unstubAllGlobals()); + + it('records them distinctly from tracks that were searched for and not found', async () => { + stubInnertube(() => fixture('playlist-mixed-types')); + const deezer = new DeezerClient('arl'); + vi.spyOn(deezer, 'search').mockResolvedValue(null); // the one real song finds no match + const conversion = new Conversion(provider(), deezer, 'report.csv'); + await conversion.matchPlaylist({ name: 'Mixed', uri: 'PLtest' }); + + const skipped = conversion.rows.filter((r) => r.note?.startsWith('skipped:')); + const unmatched = conversion.rows.filter((r) => r.note === 'no match'); + expect(skipped.length).toBeGreaterThanOrEqual(3); + expect(unmatched).toHaveLength(1); + expect(skipped[0].note).toContain('music video'); + expect(conversion.rows.every((r) => r.source === 'YouTube Music' && r.target === 'Deezer')).toBe(true); + }); +}); + +// ── Reorder (#10) ──────────────────────────────────────────────────────────────────────────────── +describe('YtMusicProvider.reorder', () => { + afterEach(() => vi.unstubAllGlobals()); + + // A playlist read shaped like the real one, with the per-item handles reorder depends on. + const playlistOf = (entries: Array<[videoId: string, setVideoId: string]>) => ({ + contents: { + items: entries.map(([videoId, setVideoId]) => ({ + musicResponsiveListItemRenderer: { + flexColumns: [{ musicResponsiveListItemFlexColumnRenderer: { text: { runs: [{ text: videoId }] } } }], + playlistItemData: { videoId, playlistSetVideoId: setVideoId }, + overlay: { musicItemThumbnailOverlayRenderer: { content: { musicPlayButtonRenderer: { playNavigationEndpoint: { watchEndpoint: { videoId, watchEndpointMusicSupportedConfigs: { watchEndpointMusicConfig: { musicVideoType: 'MUSIC_VIDEO_TYPE_ATV' } } } } } } } }, + }, + })), + }, + }); + + it('moves tracks into source order using the per-item handle, not the track id', async () => { + const edits: Array> = []; + stubInnertube((path, body) => { + if (path === 'browse') return playlistOf([['C', 'setC'], ['A', 'setA'], ['B', 'setB']]); + if (path === 'browse/edit_playlist') { edits.push((body.actions as Record[])[0]); return { status: 'STATUS_SUCCEEDED' }; } + return {}; + }); + expect(await provider().reorder('PLtest', ['A', 'B', 'C'])).toBe(true); + // Handles, never track ids, cross the wire. + expect(edits.every((e) => String(e.setVideoId).startsWith('set'))).toBe(true); + expect(edits.map((e) => [e.setVideoId, e.movedSetVideoIdSuccessor])).toEqual([['setB', 'setC'], ['setA', 'setB']]); + }); + + it('leaves a target that is already in order alone', async () => { + let edited = false; + stubInnertube((path) => { + if (path === 'browse') return playlistOf([['A', 'setA'], ['B', 'setB'], ['C', 'setC']]); + if (path === 'browse/edit_playlist') { edited = true; return {}; } + return {}; + }); + expect(await provider().reorder('PLtest', ['A', 'B', 'C'])).toBe(false); + expect(edited).toBe(false); + }); + + it('keeps the handle out of the conversion interface', () => { + // The capability is declared as (playlistId, trackIds) — the same shape Deezer implements. + expect(YtMusicProvider.prototype.reorder).toHaveLength(2); + }); +}); + +// ── Writing (#8) ───────────────────────────────────────────────────────────────────────────────── +describe('YtMusicProvider writes', () => { + afterEach(() => vi.unstubAllGlobals()); + + it('creates the playlist private', async () => { + let sent: Record | null = null; + stubInnertube((path, body) => { + if (path !== 'playlist/create') return {}; + sent = body; + return { playlistId: 'PLnew' }; + }); + expect(await provider().createPlaylist('[plx] Mine')).toBe('PLnew'); + expect(sent).toMatchObject({ title: '[plx] Mine', privacyStatus: 'PRIVATE' }); + }); + + it('fails loudly rather than returning a playlist id it did not get', async () => { + stubInnertube(() => ({})); + await expect(provider().createPlaylist('x')).rejects.toThrow(/did not return a playlist id/); + }); + + it('adds tracks as one edit action per video', async () => { + let actions: Array> = []; + stubInnertube((path, body) => { + if (path === 'browse/edit_playlist') { actions = body.actions as Array>; return { status: 'STATUS_SUCCEEDED' }; } + return {}; + }); + expect(await provider().addTracks('PLx', ['a', 'b'])).toBe(2); + expect(actions).toEqual([ + { action: 'ACTION_ADD_VIDEO', addedVideoId: 'a' }, + { action: 'ACTION_ADD_VIDEO', addedVideoId: 'b' }, + ]); + }); + + it('throws when the service refuses the add instead of reporting success', async () => { + stubInnertube((path) => (path === 'browse/edit_playlist' ? { status: 'STATUS_FAILED' } : {})); + await expect(provider().addTracks('PLx', ['a'])).rejects.toThrow(/refused the add/); + }); + + // Verification reads flat videoDetails rather than the deep renderer nesting everything else uses. + it('resolves a track for verification, dropping the "- Topic" channel suffix', async () => { + stubInnertube((path) => (path === 'player' + ? { videoDetails: { title: 'Fortnight', author: 'Taylor Swift - Topic', lengthSeconds: '229' } } + : {})); + expect(await provider().resolveTrack('abc')).toEqual({ name: 'Fortnight', artist: 'Taylor Swift', durationMs: 229_000 }); + }); + + it('reports an unresolvable track rather than inventing metadata', async () => { + stubInnertube(() => ({})); + await expect(provider().resolveTrack('gone')).rejects.toThrow(/returned nothing/); + }); +});