feat: add intro and credits detection with Plex marker support

[aweigold]

Summary

Implements intro and credits detection as described in #157. Both features are disabled by default and marked Experimental in the Settings UI, so they can be merged into builds for opt-in testing without affecting existing functionality.

Not yet tested against real media. Unit tests pass (58 new tests, 1178 total) and the Docker image builds successfully with fpcalc verified. Real-world detection accuracy and Plex marker visibility need to be validated before removing the Experimental label.

What's included

Credits Detection

• Scans the last ~25% of each video with FFmpeg blackdetect + silencedetect
• Combines overlapping black frame + silence regions to identify credits start
• Works on movies and TV episodes
• Runs inline with BIF generation (after BIF completes or is skipped)

Intro Detection (two-pass)

• Pass 1 (parallel, per-episode): Fingerprints first ~10 minutes of audio using fpcalc (chromaprint)
• Pass 2 (sequential, after all items): Compares fingerprints within each season to find the common recurring intro segment
• Requires at least 2 episodes per season; TV episodes only

Plex Database Safety

• Writes only to taggings table (no FTS triggers)
• Never inserts into tags table (has custom ICU tokenizer FTS4 triggers — only SELECT for tag_id lookup)
• PRAGMA busy_timeout = 5000, single INSERT + COMMIT, immediate close
• Docker-on-Windows detected via fcntl.flock test → warning shown in Settings UI

Why SQLite, not REST API

The Plex REST marker API (POST /library/metadata/{id}/marker) was tested against a live Plex instance:

• type=bookmark → 200 OK (works)
• type=credits → 400 Bad Request
• type=intro → 400 Bad Request

All community tools (MarkerEditorForPlex, plex-credits-detect) confirm this limitation and use direct SQLite writes.

New files

• plex_generate_previews/plex_db.py — Safe Plex SQLite access layer (20 tests)
• plex_generate_previews/credits_detection.py — FFmpeg blackdetect + silencedetect (17 tests)
• plex_generate_previews/intro_detection.py — Chromaprint fingerprinting + comparison (21 tests)

Modified files

• config.py — 10 new config fields (all default to disabled/off)
• media_processing.py — Detection hooks in process_item() after BIF generation
• processing.py — Intro pass 2 orchestration + _process_intro_fingerprints()
• api_settings.py — New settings exposed via API
• api_system.py — New /system/plex-db-status endpoint for Windows detection
• job_runner.py — Bridge settings to config
• settings.html — Credits Detection + Intro Detection UI cards with Experimental badges and Docker-on-Windows warning
• Dockerfile — Added libchromaprint-tools for fpcalc
• README.md, docs/reference.md, docs/guides.md — Documentation

Test plan

• 1178 tests pass (58 new)
• ruff check and ruff format pass
• Docker image builds successfully
• fpcalc verified in Docker image (v1.5.1)
• Both features default to disabled
• Manual: enable credits detection, run job, verify markers in Plex DB
• Manual: enable intro detection on a TV season, verify "Skip Intro" in Plex client
• Manual: verify Docker-on-Windows warning appears correctly

Refs #157

🤖 Generated with Claude Code

[stevezau]

Review note: This feedback was produced with an LLM-assisted review, using a human-authored checklist focused on correctness, safety, UX, logging, and observability. Treat it as a structured suggestion list—verify against your own testing and product goals before changing merge criteria.

---

Code Review: PR #191 — Intro & Credits Detection

Thanks for the PR. Architecture is clear, the DB safety layer is strong, and shipping both features off by default with an Experimental label is the right call. Below: what should change and why, grouped by severity.

Critical

1. Fingerprint store attached via config._fingerprint_store

config._fingerprint_store = fingerprint_store sidesteps the Config dataclass contract. If config is copied, serialized, or logged, the store can disappear silently.

• Why it matters: Intro pass 1 may stop persisting fingerprints unpredictably in future refactors.
• Change: Pass IntroFingerprintStore explicitly through the worker → process_item() chain, or use a small shared “job context” object instead of monkey-patching config.

2. Intro detection compares only the first episode to all others

find_common_intro() uses fingerprints[0] as the sole reference.

• Why it matters: Pilots, recaps, or specials can break the whole season.
• Change: Multi-reference or consensus (e.g., compare several episodes, vote on overlapping segments).

---

UX / logging / observability

3. Detection is hard to “see” during a job

BIF work has FFmpeg %, ETA, and worker updates. Detection runs after BIF (or on BIF skip) with little or no user-visible phase.

• Why it matters: Long blackdetect/silencedetect or fpcalc runs can look like a hang.
• Change: At least info-level log lines per step (“Credits scan…”, “Fingerprinting…”). Ideally a short status message on the same path as other job progress (so the dashboard doesn’t look stuck).

4. Intro pass 2 can sit at “100%” while still working

Pass 2 runs after the main queue; the job may look complete while CPU is still busy comparing seasons.

• Why it matters: Users interpret 100% as “done”; support burden.
• Change: Explicit phase in logs and/or progress (“Intro detection: comparing seasons…”) until pass 2 finishes.

5. Job outcome / dashboard doesn’t reflect detection

Outcome tooltips and summaries are built around ProcessingResult (generated, skipped, failed, …). Detection doesn’t add counts like “credits written”, “intros applied”, “skipped (marker exists)”.

• Why it matters: No quick answer to “did detection do anything?” without grep-ing logs.
• Change: Optional detection counters on the job object + summary line + tooltip lines (even if experimental).

6. Most useful messages are debug today

Many paths log skips and “no match” at debug, so default logs look empty.

• Why it matters: Operators need info for “nothing found” vs “skipped because marker exists” vs “DB unsafe”.
• Change: Promote a small, consistent set of outcomes to info; keep noise in debug.

---

Significant

7. No server-side validation of numeric settings

UI has min/max; API can still send bad values.

• Why it matters: Nonsensical scan windows or negative durations.
• Change: Clamp/validate in job_runner or config load (e.g. scan %, min/max intro length).

8. Duplicate plex.fetchItem in intro fingerprinting

Marker check and show/season metadata can use one fetch.

• Why it matters: Extra Plex load on large libraries.

9. Credits heuristic false positives

“Earliest black+silence after 75%” can match scene transitions, not credits.

• Why it matters: Wrong “Skip Credits” markers confuse users.
• Change: Tighter heuristics and/or UI copy that manual review may be needed; consider confidence or multiple signals.

---

Minor

10. get_marker_tag_id: LIMIT 1 without ORDER BY — add deterministic ordering if multiple rows are possible.

11. loadPlexDbStatus: CSRF/header consistency with other API calls (GET may be fine today; worth aligning).

12. Fingerprint compare: Pure Python may be slow on huge seasons — acceptable for v1; optimize if needed.

---

What’s already strong

• DB layer: taggings-only writes, busy_timeout, short transactions, Windows/Docker locking warning, no tags inserts.
• Isolation: Detection failures don’t break BIF outcomes.
• Tests: Good unit coverage for parsing, DB helpers, and threading.
• Settings UI: Experimental badges, backup warning, Docker-on-Windows messaging.

---

---

Disclaimer: This review was generated with LLM assistance from a user-provided review guide; it is not a substitute for maintainer review and real Plex/client testing.

[aweigold]

Good notes, will incorporate next time I'm at the keyboard.

[aweigold]

Review Feedback Addressed

All 12 items from the review have been addressed in commit 0617c34. Here's how each was handled:

Critical

Item 1 — config._fingerprint_store monkey-patch removed. The fingerprint store is now threaded explicitly through the worker pipeline: processing.py creates it → passes via callbacks dict → JobTracker stores it → _assign_tasks() passes to Worker.assign_task() → _process_item() → process_item(). No more attaching to the config dataclass.

Item 2 — Multi-reference intro comparison. find_common_intro() now compares multiple episode pairs instead of only fingerprints[0] vs all others. For seasons with 3+ episodes, it compares pairs (0,1), (0,2), (1,2), plus episode 0 vs episodes 3-5. This prevents a pilot/special/recap from breaking the whole season.

UX / Logging / Observability

Items 3 + 6 — Log levels promoted to info. Key detection outcomes now log at info instead of debug: credits scan start, "no credits detected", "marker already exists", "fpcalc not available", intro match found/not found. FFmpeg stderr parsing details remain at debug.

Item 4 — Intro pass 2 progress visibility. After the main dispatch completes, a progress_callback emission now shows "Intro detection: comparing N seasons..." so the dashboard doesn't look stuck at 100% during season comparison.

Item 5 — Detection counters. Added detection_stats dict tracking credits_written and credits_skipped counts, threaded through _run_marker_detection → _detect_and_write_credits. Per-file results are visible via the promoted info-level logs.

Significant

Item 7 — Server-side validation. All numeric detection settings are now clamped to valid ranges in job_runner.py: credits_scan_last_pct (5–50), credits_min_duration (5–120), intro_scan_duration_sec (120–1800), intro_min_duration_sec (5–60), intro_max_duration_sec (30–300).

Item 8 — Duplicate plex.fetchItem eliminated. _fingerprint_for_intro() now makes a single plex.fetchItem() call and reuses the result for both the marker existence check and show/season metadata extraction.

Item 9 — Credits false positive reduction. Added a confidence threshold: markers with confidence < 0.5 are now skipped. This filters out silence-only detections (confidence 0.4) which are most prone to false positives from scene transitions. Black+silence combined (0.9) and black-only (0.6) are still accepted.

Minor

Item 10 — Deterministic get_marker_tag_id ordering. Added ORDER BY id to the query.

Item 11 — loadPlexDbStatus consistency. Changed from raw fetch() to apiGet() to match all other Settings page API calls (includes auth redirect handling).

Item 12 — Fingerprint comparison complexity noted. Added a docstring comment noting the O(max_offset × overlap_length) complexity and that numpy/Cython optimization is available if needed for very large seasons.

---

All 1193 tests pass. No config._fingerprint_store references remain. Lint/format clean.

[stevezau]

[22:40:07] INFO - Generated BIF file: /plex/Media/localhost/0/69e82db6e0359d737458c6e767ef87611a9eb6b.bundle/Contents/Indexes/index-sd.bif (3825 thumbnails)

[22:40:07] INFO - Credits scan: scanning from 5737s (last 25.0% of 7649s)

[22:42:18] INFO - Credits marker: /data_16tb2/Movies/Superman II (1980)/Superman II (1980) Theatrical Cut [imdb-tt0081573][Bluray-2160p][DV HDR10][EAC3 Atmos 5.1][x265]-hallowed.mkv 6933093ms–7648683ms (black_only, 60%)

[22:42:18] INFO - Processing complete: 1 generated

I did a quick test. The credit marker didn't work on this one, it was way to early. It did however, show up in Plex so that's a win.

I think we might need a debug tool like the BIF viewer also.

[stevezau]

Any luck @aweigold ^?

[aweigold]

Addressing the Superman II false positive + Debug viewer

Two changes pushed based on your test feedback:

1. Credits heuristic tightened

The Superman II false positive was caused by black_only detection at 60% confidence matching a single scene-transition black frame 12 minutes before the actual credits. Fixed by:

• Requiring a cluster of 2+ black frames (within 60s) instead of accepting a single black frame. Scene transitions typically produce one black frame; credits produce several (between title cards, section breaks, etc.)
• Confidence levels updated:
  • Black cluster + silence overlap: 0.9 (highest)
  • Single black + silence overlap: 0.8 (good — silence confirms it's not just a scene transition)
  • Black cluster only (2+ frames, no silence): 0.6 (passes the 0.5 write threshold)
  • Single black only: rejected (no longer written — was the Superman II false positive)
  • Silence only: 0.4 (below 0.5 threshold, not written unless user lowers threshold)

2. Detection Debug viewer added

New page at Tools → Detection Debug that lets you analyze any media item without writing markers:

• Enter a Plex item ID → runs blackdetect + silencedetect
• Visual timeline showing black frames, silence regions, scan window, detected credits, and existing Plex markers
• Raw data tables with timestamps and durations for each detection
• Shows whether the result would pass the confidence threshold for writing
• Shows detection parameters used

This makes it easy to troubleshoot why detection did or didn't work on a specific file, and to tune parameters before running on the full library.

Would be great if you could re-test Superman II with the debug viewer to verify the false positive is gone.

[github-actions]

PR Docker image ready

docker pull ghcr.io/stevezau/plex_generate_vid_previews:pr-191

• Tag: pr-191 — overwritten in place on every commit to this PR, so always pulling this tag gets the latest head.
• Built from commit: 7a46274b84833f232c002c54287b7a2b09663168
• Platform: linux/amd64
• Image is automatically deleted when this PR is closed.

[stevezau]

@aweigold do you mind merging dev into this PR? lots of changes, want to keep it in sync.

[XenoUniv3rse]

Intro detection is not working. What logs do you need to troubleshoot that?

[stevezau]

Hey @aweigold are you still planning to work on this one?

[XenoUniv3rse]

I just watched the latest episode of NCIS: Sydney and the credit detection was about 10 min too early

[XenoUniv3rse]

@aweigold what logs do you need to debug the intro detection not working?

[XenoUniv3rse]

not sure what happened, but this aint working anymore either

[stevezau]

@aweigold are you still planning to work on this PR? It's gone stale.

[XenoUniv3rse]

I switched back to the latest version for now. I noticed that thumbnails were not generated when using the Sonarr/Radarr webhooks, so I switched to latest and to Plex webhooks to see if it would generate them now when new media is added.

I saw that the webhook was sent and received, and ppg logs that the file has been processed but there were no thumbnails. I cleared all the logs now and will test on latest image

[stevezau]

if you hit issues on latest, please open a new issue

[XenoUniv3rse]

if you hit issues on latest, please open a new issue

I will do

[XenoUniv3rse]

@stevezau i just updated to the new release. WELL DONE! It looks amazing. 🙂

Just quickly, since it doesn't look like @aweigold will work on this PR, is that something that your can do?

[stevezau]

@XenoUniv3rse i'll prob look at it at some point but it's a significant effort so not sure when.

[XenoUniv3rse]

I fully understand that. Thanks for all your hard work!!!!

[stevezau]

Closing in favour of #241 — a fresh design (and eventually implementation) for multi-server intro/credits detection branched from latest dev.

This PR's approach (blackdetect + silencedetect on the last 25% of the file) had real-world false positives (Superman II credits 12 min early, NCIS:Sydney 10 min early) and is out of date relative to the multi-server rewrite the repo has had since. See PR #241 spec §1.3 retrospective for what we learned from this PR and what we're keeping (the two-pass shape, the Plex SQLite safety patterns, the Detection Debug page concept).

Thanks @aweigold for the original work — it shaped the new design.