feat(markers): design spec for multi-server intro/credits detection#241
Draft
stevezau wants to merge 17 commits into
Draft
feat(markers): design spec for multi-server intro/credits detection#241stevezau wants to merge 17 commits into
stevezau wants to merge 17 commits into
Conversation
Supersedes the stalled PR #191 (Plex-only, blackdetect+silencedetect heuristic that failed on Superman II and NCIS:Sydney). Designs a multi-server pipeline that: - Reads existing markers from Plex/Emby/Jellyfin via their public APIs - Detects via a 4-tier cascade: TheIntroDB lookup, chromaprint cross-episode (TV), adaptive binary-search blackdetect (movies), optional PaddleOCR (Tier 4, GPU) - Emits canonical .markers.json sidecars next to media - Publishes per-server: Plex SQLite (provenance/restore for non-Pass); Emby via sydlexius/Segment_Reporting; Jellyfin via a new C# companion plugin (MediaPreviewSegments) shipped in this repo - Adds a Marker Inspector UI page (sibling of BIF Inspector) Phased rollout over 4 PRs (A scaffolding + TheIntroDB, B algorithms, C publishers + Jellyfin plugin, D polish + OCR). Architecture Review run pre-commit; 3 LOW findings (operator precedence in code block, class name nits, file-path drift) addressed inline. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- TheIntroDB submit-back is opt-in only (default off) - Jellyfin plugin: extend existing Jellyfin.Plugin.MediaPreviewBridge rather than ship a second plugin. Adds MarkerBridgeController + MarkerSegmentProvider to the same project; bumps version to 10.11.1.0; reuses existing CI workflow and manifest URL. - Inspector unified at /inspector with tabs (Frames / Markers / Audio); /bif-viewer 301-redirects to /inspector?tab=frames - OCR Tier 4 still open — clarified the three concrete options (drop entirely, bundle, separate :with-ocr tag) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plex's native credits detection uses OCR-style scrolling-text recognition combined with black-frame detection (Plex 2023 blog). EmbyCredits uses Tesseract OCR. Only Jellyfin's intro-skipper omits OCR, and movies are its known gap. Adopting OCR brings non-Pass users to feature parity with Plex's gold standard. Tag separation keeps the default image lean (~2GB smaller). Settings flag is runtime-detected so the UI only offers it when paddleocr is importable — users switch tags by changing one line in compose. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8 tasks
Live-tested the API: real path is /v1/media (not /media), real param is tmdb_id (not tmdbId). Earlier research agent had both wrong. Real coverage (from /v1/stats today): - 1,744 shows + 272 movies, 45,274 episodes, 323,404 submissions - 295 contributors, 81,435 accepted timestamps - Empirical hit rate: ~40% popular TV S1E1, ~18% popular movies Rate limits are binding: 100/day anonymous per IP, 500/day per authenticated user. No bulk-export endpoint. Implication: Tier 1 is opportunistic (new items via webhook, Inspector lookups, slow background backfill) NOT a library-bootstrap fast path. Mirrored OpenAPI spec to docs/design/theintrodb-openapi-v2.yaml so implementation can work offline. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…s, Plex DB write details
CRITICAL: TheIntroDB v1 sunsets 2026-08-16. Every /v1/* response carries
sunset header. Spec now uses /v2/media throughout.
Added Tier 1b (IntroDB.app, ~73% hit rate on TV but mostly single-
submission entries — gated by confidence floor) and Tier 1c
(anime-skip.com for anime libraries — AniList-keyed, separate ecosystem,
only fires on kind="anime" libraries).
Sustainability note: TheIntroDB is one-maintainer (Pasithea0) but
community-active (295 contributors, real consensus voting, 3 plugin
releases in 10 weeks). MED risk over 12 months — mitigated by caching
positive hits forever.
Plex DB write section rewritten with verified 2026 details:
- REST marker API still 400s (Plex admin confirmed in Feb-Apr 2026
dev forum thread #931778)
- Use narrow endpoints PUT /library/metadata/{id}/intro and /credits,
not the broad /analyze (markers-only, async fire-and-forget)
- ONE tags row (tag_type=12) for ALL markers; intro/credits/commercial
differentiated by taggings.text — earlier spec had this wrong
- MUST write BOTH taggings AND media_parts.extra_data (the load-bearing
detail Casvt's script misses but MarkerEditorForPlex gets right —
without the part-level denormalized JSON, markers vanish on re-analyze)
- Editions vs versions handled separately
- extra_data JSON shape version-specific (PMS ≥1.40 vs older)
- Provenance lives in side-car DB, not in Plex; thumb_url sentinel
is a legacy hack now abandoned
- Plex Pass requirement is on PLAYER account, not just server admin
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…h-review findings Empirical finding from 160-file probe of user's /data library: - 20% TV / 16% movies have explicit credit-marker chapter names - Netflix/Amazon WEBDLs heavily over-indexed (frame-accurate boundaries authored by the streamer) - Bluray rips rarely have semantically-labelled chapters New Tier 0 piggybacks on the EXISTING pymediainfo MediaInfo.parse() call in processing/generator.py:291 — Menu track exposes chapter names with HH_MM_SSmmm key format and 'en:'-prefixed values. Live-verified. Other findings incorporated: - AcoustID side-channel DEAD (empirically 0/5 popular shows match; in-show audio mixes differ from MusicBrainz recordings) - Subtitle dialogue gaps too noisy (top gap was a scene break, not intro) - HLS/SCTE-35 doesn't transfer to MKV containers (RFC 9559 verified) - Sonarr/Radarr release group becomes a routing hint, not marker source - ChapterDB.org → chapterdb.plex.tv read-only mirror as Phase D opportunistic Tier 3b for movies Cascade now has 5 tiers. Phase A starts with Tier 0 (free) + Tier 1 TheIntroDB v2 + Tier 1c anime-skip read-only. Architecture Review pre-commit found 2 MED + 1 LOW; all addressed: - MED: original spec claimed Tier 0 piggybacks on ffprobe; codebase actually uses pymediainfo. Verified live that Menu-track parse exposes chapter names equivalently — spec rewritten around pymediainfo. - MED: global settings nested under `settings.markers` but actual settings.json layout is flat at root. Now top-level `markers` key sibling of gpu_config/media_servers. - LOW: Phase A claimed Inspector tab, but unified-Inspector shell doesn't exist yet. Phase A now ships standalone /markers page; Phase D handles the BIF→tabbed-Inspector port. Also added empirical hit-rate data, expanded risk section to 9 items with concrete mitigations, and committed to TheIntroDB v2 URL. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five parallel research streams (Plex/Jellyfin/Emby community sentiment, cross-server patterns, UX feature-request mine) surfaced concrete gaps no existing tool fills. Major additions: NEW §7.5 — Community-validated UX features (the differentiator): - 7.5.1 Locked markers + per-show overrides (5.75-year community pain) - 7.5.2 Apply-pattern-to-season (nurbles 2020, never built) - 7.5.3 Per-client compatibility matrix in Inspector (truth-table showing which clients render which markers — Swiftfin/Roku/Tizen blind spots make this critical UX) - 7.5.4 Multi-OP anime mode (intro-skipper #658 dismissed) - 7.5.5 Cross-server marker bridge (Plex → Jellyfin/Emby) — THE killer feature solving shared-user entitlement gap + migration - 7.5.6 EDL sidecar export (free Kodi support) - 7.5.7 Submit-back to TheIntroDB + SkipMe.db (both, not just one) - 7.5.8 Cache sharding from day one (50TB lib = 500k files) - 7.5.9 Bulk-edit grid view - 7.5.10 Webhook on segment events NEW Tier 2.5 — Subtitle-keyword recap detection (THE holy-grail feature no tool ships — intro-skipper #763 open May 2026, #136 open since 2024, Emby #120144, Plex #787447 all punted). bw_plex demonstrated subtitle keyword "Previously on" approach works. ~10ms per episode, no GPU, near-100% subtitle coverage on streamer WEBDLs. Closes a 5-year community gap. MarkerSegment additions: - locked: bool — re-detection never overrides - seed: bool — user-blessed fingerprint anchor for apply-to-season - schema_version: bumped to 1 - source enum expanded with introdb/anime_skip/skipmedb/manual/ chapter/subtitle_recap Chromaprint algorithm constants corrected against intro-skipper PluginConfiguration.cs canonical values: - MaximumFingerprintPointDifferences = 6 (was 8 in spec) - BlackFrameThreshold = 28 of 255 (was 0.10) - ChromaprintConstants.SampleDuration = 4096/11025/3 ≈ 0.124s - All other defaults verified Known-failure-mode documentation added: - Bob's Burgers S01 chromaprint failure (skipme.db #29) - Plex 10-min ceiling matches Emby ceiling (NCIS:Sydney pattern) - Plex multi-version file replacement bug (#782058, since March 2022) - Plex shared-user entitlement gating (#937108) Settings schema expanded: - per_show_overrides (the killer setting) - subtitle_recap_keywords (multilingual) - client_compatibility_path - anime_max_op_clusters - skipmedb_submit, edl_export, webhook_events - cache_shard_depth (operational correctness) Spec is now 1239 lines (was 1003). PR #241 updated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Architecture Review pre-commit run against the 1239-line spec found: - 5 HIGH severity issues - 5 MED severity issues - 3 LOW severity issues All HIGHs and most MEDs fixed inline: HIGH #1 — MediaInfo.parse line number: corrected from :291 (startup probe) to :769 (per-file pipeline call site). Tier 0's "free piggyback" claim is preserved. HIGH #2 + #3 — Phase A scope expansion: tightened from claimed-1-week- but-actually-2-3-weeks to a real 5-7-day MVP. Tier 1c, IntroDB fallback, Emby/Jellyfin reads, sidecar writes, cache sharding moved to Phase B. Settings migration writes ONLY Phase A keys (9 keys) instead of all 18 to avoid stranding feature flags. HIGH #4 — Tier 2.5 subtitle recap detector now wired into §9 pipeline as sub-phase 2c (subtitle extract) + 2d (subtitle scan). Full 7-step detection-phase table added showing tier ordering, skip conditions, output shape. Tier 2 (chromaprint) clarified as "deposits fingerprint, pass-2 runs post-dispatch via new marker_post_detection job type". HIGH #5 — Tier numbering inconsistency: §0 TL;DR now says "seven-step cascade — five primary tiers (0/1/2/3/4) with sub-tier fallbacks (1b, 1c, 2.5)". Consistent across §0/§5/§12. MED #6 — Library.kind clarified: existing servers/base.py:118 field is opaque and set to movie/show by enumeration. We add a separate per- library setting markers.anime_mode_libraries: list[str] instead of repurposing kind. Phase D may add auto-detection from genre. MED #7 — MarkerSegment.locked/seed propagated to: - Canonical sidecar JSON example (now shows all 9 fields) - Provenance DB schema (added locked, seed, source, detector columns) - Plex restore loop (checks locked before re-INSERT; locked=1 wins over re-detection candidate) - Consumer rules section documenting forward-compat behaviour MED #8 — Cross-server bridge now has its own subsection in §9 describing job type (source=bridge), failure scope (per-server try/except), and confidence-gate bypass for native_plex segments. MED #9 — Plex /intro and /credits endpoints: added explicit BoolInt param shape (force=1 not True), full server.query() snippet, and mandatory test asserting isinstance(params['force'], int) and not isinstance(params['force'], bool) to catch the BoolInt 500 trap. MED #10 — media_parts.extra_data merge spec: full input-state matrix (NULL/empty/legacy/≥1.40), key invariants (array always, url URL-encoded, final true only on credits, keys alphabetized, attributeName matches parent key), worked example with before/after byte-for-byte, Phase C test count 18 rows. LOW #11 — Worker._process_item vs process_item conflated: §9 clarifies hook point is inside Worker._process_item at jobs/worker.py:340 AFTER the existing BIF call. LOW #12 — Phase A plugin version check fixed: JellyfinMarkerPublisher returns READ_ONLY unconditionally in Phase A regardless of plugin version, so existing trickplay users don't see spurious Setup Health warnings. Version gate enabled in Phase C with the writes. Spec is now 1374 lines (was 1239). All cross-references verified. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…x Tier 1/2
Background agent ran the proposed algorithms against the user's actual
/data library (4856 TV dirs, 9753 movie dirs). Concrete findings either
confirmed, refined, or REFUTED spec claims.
REFUTED:
- Tier 2.5 subtitle-keyword recap detection — 0/30 hits on real files.
"Previously on" is a graphical title card on streaming releases, NOT
in subtitle streams. Mr. Robot S01E06's 90s recap montage has
subtitled dialogue but no keyword chyron in SRT. Reason no existing
tool ships this approach: it doesn't work.
→ DELETED Tier 2.5 from cascade. Recap handling routes through
Tier 0 chapter names (verified working — Dead Boy Detectives
S01E05 has "Recap" chapter), Tier 1 TheIntroDB's recap segments,
Tier 4 OCR (optional, chyron-region keyword scan), or manual
Inspector edits.
REFINED:
- Tier 0 hit rate: spec said 20% TV / 16% movies; measured 40% TV /
23% movies. Spec was conservative by 2× on TV. Updated.
- Tier 1 hit rate: spec said ~57% TV; measured 5% on user's
heterogeneous library (1/20 random IMDb-keyed lookups). Popular-show
cross-check confirmed 70% on mainstream — the 57% figure is a
popularity-biased number. Added tmdb_popularity_min setting + popularity
gate before query to avoid wasted rate-limit quota.
- Tier 2 chromaprint: validated working (Hart to Hart S01 = 102/231
pair matches; theme region 0-78s identified consistently across
pairs). One Piece anime = 10/10 pairs, OP 0-110s — anime myth
busted. BUT discovered release-source variance: cross-encoder
pairs (WEBDL-GRAVE vs Bluray-COLL3CTiF) match poorly.
→ ADDED release-source grouping to Pass 2: extract release group
from filename, run pairwise within-source first (near-100%),
then cross-source with relaxed thresholds.
CONFIRMED:
- Tier 3 adaptive blackdetect: 5 random movies, all converged in
7 probes (1.5-6.3s elapsed, median 4s). PR #191's naive linear
scan was 131s on equivalent 4K HDR. ~30x speedup verified.
- TheIntroDB v2 API + response shape matches spec exactly. null
semantics ("from beginning" / "to end") confirmed.
UNCONFIRMED:
- Tier 5 Emby: local Emby (4.9.3.0) doesn't have MediaPreviewBridge
or Segment_Reporting installed; can't live-verify Phase C writes.
Surfaced as Setup Health requirement.
New §11.5 section captures all the empirical evidence with citations
to test artifacts at /tmp/intro_validation/. Future implementers can
re-run the tests against new content to verify accuracy stays high.
Spec is now 1488 lines (was 1374). PR #241 commit count: 11.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
External SOTA research stream (Korolkov 2025, Hao 2021, AWS, TwelveLabs, Cognitive Mill, Chapter-Llama, intro-skipper survey) provided concrete F1 ceilings: - Academic SOTA: 0.91 F1 (Korolkov CLIP+attention; no public weights) - Commercial baseline: 0.73 F1 (Hao/AWS Rekognition) - Our cascade: ~0.75-0.85 F1 (FOSS Pareto frontier) - Cognitive Mill 99.99%: unverified marketing, dismissed Two HIGH-leverage additions surfaced: 1. Tier 2c — SoundFingerprinting escalation path. When chromaprint Pass 2 match rate within season < 30% (cross-source Hart to Hart problem), automatically escalate to peak-pair fingerprinting (MIT, used by plex-credits-detect). 3× slower but handles re-encodes. 2. Tier 5 — Chapter-Llama (MIT, weights public on HF). First open-source SOTA-class option. Llama-3.1-8B + LoRA, 16+GB VRAM, F1 45.3 on chapter generation. Deferred to future Phase E; documented as the realistic open-source path past 0.85 F1. Documented: - AWS Rekognition pricing math ($11,250 to bootstrap 5000-ep library) as paid-tier escape hatch, not v1 dependency - TwelveLabs Marengo Bedrock as alternative paid embedding source - Signal hierarchy by marker type (audio fp wins for themed TV intros, OCR wins for credits, blackdetect wins for boundaries, etc.) - Explicit acknowledgement: chromaprint is 2017-era, ~18 F1 below academic SOTA. Survives as free fast-path, not best signal. Spec is now 1547 lines (was 1488). PR #241 commit 12. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Background agent extracted Plex Media Server binary from linuxserver/plex:latest Docker image and ran strings/readelf/xxd analysis. The COMPLETE Plex algorithm is now known: §11.5.1 Pipeline decoded: 1. Decode (FFmpeg + EAE for Dolby only — myth: EAE is NOT a fingerprinter) 2. Essentia mel-bands + MFCC features (2048 frame @ 22050 Hz, 1024 hop) 3. Music.tflite classifier (3.1MB CNN: 8 Conv2D + 2 Dense + Sigmoid) 4. Chromaprint fingerprinting (Plex Media Fingerprinter = fpcalc statically linked, SAME library as intro-skipper) 5. IntroDetectorManager: Hamming-LCS + ≥50% quorum rule 6. Persist to metadata_item_setting_markers (NEW: differs from MarkerEditor's taggings table — flagged for Phase C verification) 7. Cloud submit to metadata.provider.plex.tv §11.5.2 BogoHash = SHA-256 of first 4096 bytes. Confirmed by binary class hierarchy + Plex staffer ChuckPa forum post. We can compute this EXACTLY — possible future cloud-query capability, documented but not committed (ToS grey area). §11.5.3 Credits = OpenCV EAST text-bbox + Music.tflite + blackdetect. Our PaddleOCR PP-OCRv5 beats EAST on stylized text (Plex only finds bboxes, doesn't read text content). §11.5.4 Plex Commercial Skipper IS literally comskip (GPLv2 OSS, Erik Kaashoek's). Same CSV columns, same comskip.ini config. §11.5.5 Anime NCOP regex extracted verbatim from binary — we adopt it for filename-based intro detection (NCOP files ARE the opening). §11.5.6 Accuracy moat = ZERO. Plex's components are: - chromaprint (same as us) - Essentia (open AGPLv3) - 3MB CNN (replaceable with YAMNet) - comskip (we'd ship same) - OpenCV EAST (we use better — PaddleOCR) A third-party impl matches Plex intro within ~2%, credits within ~5-10%. The only moat is the metadata.provider.plex.tv cached DB which becomes irrelevant once we run local detection once. §5 Tier 2 chromaprint algorithm now includes Plex's binary-verified 50% quorum rule: "if fewer than half of episodes in season match, drop ALL intros for the season". Spec is now 1632 lines (was 1547). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Background research stream did IL decompilation of Emby.Providers.dll (closed-source Emby Premiere) and line-by-line analysis of intro-skipper source. The findings collapse the implementation uncertainty for Tier 2 chromaprint. Algorithm lineage confirmed: VictorBitca/matcher (Go 2019) → puzzledsam (Python 2019) → chefbennyj1 (C# 2020) → Emby Premiere (C# 2022, 4.7+ licensed in) Both Emby and Jellyfin's algorithms decoded with code snippets included: - Jellyfin: inverted-index, O(N) per pair, known LHS/RHS mismatch bug - Emby: sliding-window cross-correlation, O(N²) per pair, single consistent offset, 3-sample lookahead filters spurious matches Recommendation: start with Jellyfin's approach, FIX the LHS/RHS- mismatch bug via triple tracking (shift, lhsRange, rhsRange), ADD pairwise consensus (≥2 pairs agreeing) — strictly stronger than intro-skipper's longest-wins-per-episode rule which explains the ~80% accuracy ceiling. Canonical constants captured for Python re-impl: - SAMPLE_DURATION = 4096/11025/3 (exact) - MAX_BIT_DIFF = 6 (Jellyfin tightened from VictorBitca's 8) - MAX_TIME_SKIP = 3.5s, MIN_INTRO = 15s, MAX_INTRO = 120s - INVERTED_INDEX_SHIFT = 2 - ANALYSIS_PERCENT = 0.25, ANALYSIS_LENGTH_LIMIT = 600s - MIN_PAIRS_FOR_CONSENSUS = 2 (OUR addition, fixes 80% ceiling) SWAR popcount formula included — bit-identical to BOTH Emby's GetFastHammingDistance AND intro-skipper's BitOperations.PopCount, verified by manual IL trace. Boundary refinement spec (intro-skipper's three-stage; Emby skips): chapter snap → silence snap → keyframe snap, asymmetric windows favor shrinking over extending. Per-mode analyzer chain ordering documented (Intro=Chapter+Chromaprint, Credits non-anime=Chapter+BlackFrame+Chromaprint, Credits anime flipped, Recap/Preview/Commercial=Chapter only). TEST FIXTURE BREAKTHROUGH: VictorBitca/matcher repo has Jojo-01.wav through Jojo-10.wav (10 episodes, 17MB each) as canonical pre- validated test corpus. Intros start near 0, end near 60s. Phase B's first test. Spec is now ~1700 lines. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ECT verdict Larger empirical sweep across 20 shows in 10 content categories corrected and calibrated earlier numbers: CORRECTED: - Tier 0 chapters: ~11% library-wide (was 40%) — small sample bias in round 1. Almost exclusively anime fansubs. Western Bluray rips ship generic "Chapter NN". - Tier 1 TheIntroDB: ~50% on mixed library (was 5%) — round 1's 5% came from random-IMDb sampling that hit recent/niche shows. VALIDATED: - Multi-OP cluster mode: One Piece S01 vs S10 vs S20 = 0% match (chromaprint correctly distinguishes different OPs). Simple k-means on fingerprint medians will cluster cours. - Release-source grouping is MANDATORY: Hart to Hart within-source 100%, cross-source 33%. 67-point drop validates spec. - Bob's Burgers "skipme.db #29 failure" debunked: 47% within -CtrlHD source; the reported failure was cross-source pairs. NEW per-content-type strategy (verdict table in §11.5.9): - Talk shows (LWT): 100% match, 0.6s stddev — Tier 2 wins - Western cartoons: 47-100% — Tier 2 wins with source grouping - Long-running anime within-cour: 100% — Tier 2 wins, cluster by cour - Sitcoms: 40-73% — Tier 2 + Tier 1 fallback - Modern drama: 20-33% weak chromaprint, Tier 1 carries (BrBa, Severance both 5/5 TheIntroDB) - Anime rotating OP: 27% (multi-OP confuses) — cluster mandatory - Reality TV (Survivor): 0% — REJECT, no recurring theme - Mini-series (Chernobyl): 0% — REJECT, unique cold opens - Single-season prestige (Watchmen): 0% — REJECT, same problem NEW: Tier 2 confidence threshold function based on empirical data: - HIGH ≥80% match + ≤5s stddev - MEDIUM 40-80% + ≤20s - LOW <40% or >30s - REJECT 0 matches (don't synthesize low-confidence guess) NEW: REJECT verdict for content with structurally no marker signal. Reality/mini-series/single-season → Inspector shows "no markers detected" rather than wrong markers. Better UX than guessing. Spec is now ~1900 lines. PR #241 commit 15. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eline
Self-contained HTML mockup at docs/design/inspector-mockup.html
demonstrating the Inspector UI across every state we have to handle:
01. Single-item Inspector — Severance S01E03 with audio waveform,
server markers vs detected markers vs recap, confidence,
per-client compatibility matrix, full action set
02. Per-server matrix — 8 distinct server states (Plex Pass,
Plex no-Pass DB write, Emby+Premiere+plugin, Emby plugin
missing, Jellyfin upgraded, Jellyfin needs update, Jellyfin
no plugin, server unreachable)
03. Library coverage dashboard — bootstrap progress for 5000-ep
library with per-library breakdown, ETA, REJECT bucket
04. Cross-server bridge — Plex → Jellyfin marker push UI
05. Bulk editor — Phase D spreadsheet view with shift-N-seconds
06. Pipeline diagram — shows what reuses existing infra (BIF,
FFmpeg, worker pool, SocketIO) vs what's new
07. Edge cases — REJECT verdict, rate limit, plugin missing,
locked markers, mini-series partial
Design language: dark theme, IBM Plex Sans + Mono + Bricolage
Grotesque, teal/amber/purple marker palette (intro/credits/recap).
No external dependencies beyond Google Fonts.
Open in a browser: file:///path/to/docs/design/inspector-mockup.html
Or via PR #241 raw URL.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous mockup used a custom dark aesthetic (Bricolage Grotesque + IBM Plex Sans, teal/amber accent colors, custom CSS framework) that didn't match the existing app. Steve correctly called this out. Rewritten using the live app's design tokens verbatim: - Bootstrap 5.3 (same CDN + version as base.html) - Bootstrap Icons (bi-*) - Plex orange (--plex-orange #e5a00d) as scalpel for primary CTA - Cool-slate light theme + dark Plex theme via data-bs-theme - *-bg-subtle + *-text-emphasis pattern for status badges (matches style.css guidance) - .page-header / .page-title / .page-subtitle pattern from base.html - .card with 1px border, no shadow (matches bif_viewer.html) - .nav nav-pills nav-fill for tabs - Same theme-toggle JS pattern The mockup now reads as a NEW PAGE INSIDE the existing app rather than a separate design exploration. Drops into web/templates/ shape. Marker color palette retained (intro=#5EEAD4 / credits=#F59E0B / recap=#A78BFA) as data-accent colors only — these are content colors for the timeline visualization, not brand colors. Same 7 sections preserved: single-item Inspector, per-server matrix (8 states), library coverage, cross-server bridge, bulk editor, pipeline integration, edge cases. Plus an alert callout for the integration gaps that still need design work (frame cache reuse, GPU pool sharing, SocketIO events, setup wizard, scheduler). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Draft PR carrying the design spec for adding multi-server intro/credits detection (Plex + Emby + Jellyfin). No code yet — this PR is the design document only, opened in draft so it can be reviewed in GitHub-context before any implementation work lands.
Spec lives at
docs/design/2026-05-17-intro-credits-detection-design.md.Supersedes #191 (Plex-only,
blackdetect+silencedetectheuristic that failed on Superman II and NCIS:Sydney — see the spec's §1 retrospective).Headline design choices
<canonical_path>.markers.jsonnext to media. One detection run, multiple publishers fan out.OutputAdapterpattern:episode.analyze()for Plex Pass; direct SQLitetaggingswrite (off by default, provenance/restore) for non-Pass.GET /Items/{Id}?Fields=Chapters; write via the existing communitysydlexius/Segment_Reportingplugin.MediaPreviewSegments) implementingIMediaSegmentProvider. Built in our CI, distributed via a plugin-manifest URL hosted by our web app./inspector/markerspage (sibling of BIF Inspector). Search → audio waveform + timeline + side-by-side server vs detected markers + manual nudge + one-click apply.Why the write-path matrix is the core constraint
All three native HTTP write paths are unusable. Plugin/DB is the only sustainable route, and Jellyfin requires us to ship a plugin (which is why this is multi-week work and not multi-day).
Phased rollout
The spec breaks into four PRs against
feat/markers-detection:Open questions (also tracked in spec §13)
:with-ocrtag?Test plan (for the spec itself, not implementation)
🤖 Generated with Claude Code