Skip to content

Harden sync and subtitle-upgrade metadata parsing - #3388

Open
mjc wants to merge 16 commits into
morpheus65535:developmentfrom
mjc:mjc/wanted-search-sync-upgrade
Open

Harden sync and subtitle-upgrade metadata parsing#3388
mjc wants to merge 16 commits into
morpheus65535:developmentfrom
mjc:mjc/wanted-search-sync-upgrade

Conversation

@mjc

@mjc mjc commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR hardens sync, profile parsing, subtitle cache, subtitle sync, and subtitle-upgrade paths that feed later wanted-search normalization work.

The goal is to preserve valid Radarr/Sonarr/profile/history/cache data while skipping malformed entries instead of letting one bad payload crash or corrupt the broader sync/upgrade operation.

What changed

  • Skips malformed Radarr/Sonarr language profile entries instead of assuming every item is a dictionary.
  • Skips profile entries with missing IDs or non-string language/name fields.
  • Handles malformed cutoff profile IDs while resolving profile cutoffs.
  • Handles non-list movie/series API responses without touching the database or progress state.
  • Skips malformed movie/series entries in bulk sync loops while preserving valid entries from the same payload.
  • Validates Radarr movie-file IDs, paths, and sizes before file-size checks, .strm detection, or movie parsing.
  • Skips subtitle-upgrade attempts when a history language value cannot be parsed into a usable language.
  • Handles missing or malformed profile payloads when checking whether a subtitle language is still wanted.
  • Handles missing audio-language data, missing history scores, and notification results without messages during upgrade searches.
  • Preserves missing scene names as None rather than converting them to a literal string.
  • Purges malformed subtitle-cache entries during cache lookup and before storing new subtitles.
  • Makes subtitle-sync metadata helpers tolerate missing language objects, malformed score thresholds, and non-callable callbacks.

Why this is separate

Sync/profile/upgrade/cache data is upstream of wanted-search state. Making these readers defensive first keeps the later normalized wanted-state PR focused on storage and query performance, while ensuring malformed external payloads do not poison the state that wanted search consumes.

Tests

The added tests cover malformed Radarr/Sonarr profile payloads, non-list API payloads, malformed movie-file metadata, mixed valid/malformed sync entries, bad upgrade language/profile data, missing audio-language rows, missing history scores, notification results without messages, malformed subtitle-cache entries, malformed subtitle-sync score thresholds, and callback safety. The tests use the shared transactional fixture layer so valid entries can be asserted alongside skipped malformed entries.

@mjc
mjc marked this pull request as draft June 12, 2026 05:26
@mjc

mjc commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Stack note for review: this branch is intentionally based on the PR #3383 test harness branch in my fork, but the upstream PR targets development so maintainers can review it in the normal upstream queue. If PR #3383 lands first, this should flatten down to only the sync/profile metadata parsing and subtitle-upgrade hardening changes from this PR.

@mjc
mjc force-pushed the mjc/wanted-search-sync-upgrade branch 2 times, most recently from 33b59d9 to 18e403d Compare June 14, 2026 17:31
@mjc
mjc force-pushed the mjc/wanted-search-sync-upgrade branch from 18e403d to e33e821 Compare June 19, 2026 17:50
mjc and others added 16 commits June 19, 2026 12:11
Align requests floor with vendored runtime
Add shared wanted-search fixture layer

Cover wanted-search fixtures with focused tests
(cherry picked from commit faa01ff)

test: add mass-download fixture support

test: complete mass-download fixture support

test: add shared wanted database binder

test: simplify wanted kind inference

test: register wanted fixtures at top level

test: drop unused transactional tmp fixture

test: bind profile lookup to wanted test database

test: bind profile list lookup to wanted fixtures
- add sync profile tests for malformed language/profile API payloads
- guard sync and profile parsing code against None/non-dict/non-string values
- make subtitle sync/cache helpers resilient to malformed or missing language fields

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- add upgrade path tests for malformed profile/audio payloads and safe notification behavior
- guard database profile helpers against malformed profile dictionaries and invalid audio payloads
- make upgrade language/profile handling tolerant of missing keys and invalid item shapes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- keep get_video sceneName sentinel behavior by ignoring None values
- return empty code3 fallback for missing subtitle language metadata
- skip upgrade attempts when parsed language is empty
- preserve explicit subsync threshold value 0 during conversion
- guard Sonarr/Radarr profile parsing against non-dict entries
- harden Sonarr/Radarr sync loops against malformed series/movie payloads
- guard profile cutoff iteration when items is non-list
- remove dead nested test class and add regressions for non-dict profiles and empty parsed language

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mjc
mjc force-pushed the mjc/wanted-search-sync-upgrade branch from e33e821 to 71e721c Compare June 19, 2026 18:15
@mjc
mjc marked this pull request as ready for review June 19, 2026 18:20
@morpheus65535 morpheus65535 added the PR on Standby PR waiting an action before it get merged. label Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PR on Standby PR waiting an action before it get merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants