Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,9 @@ has seen the raw output. Triage into new roadmap passes after.
- `players` is INTEGER. Scraper/AI ranges like `"1-4"` are normalized to the max (4) before saving.
- Sort title is auto-generated during scraping and AI Fill via `generate_sort_title()` in `services/game_utils.py`.
- When a system has curated default controllers in the DB, they always override scraped/AI controller values.
- `units_sold` (INTEGER, migration 023) is filled only by the AI — scrape gap-fill and AI Fill; `scrape_ai.parse_units_sold()` turns "2.4 million" etc. into an int.
- `release_dates` (TEXT, migration 024) is a JSON object of region code → date (`{"JP": "2002-11-17", "NA": "2002-11"}`), filled only by the AI — scrape gap-fill and AI Fill. `services/release_dates.py` parses, stores and displays it; `release_date` stays the row's own release.
- RPCS3 compatibility (PS3) is never stored: `services/rpcs3_compat.py` reads the local RPCS3's `GuiConfigs/compat_database.dat` (re-read when it changes) and looks games up by `games.disc_serial`. Local ROMs only, never imported games.
- Regional versions (PASS-59-106): only `services/version_groups.py` links rows through `games.version_group_id`. `_build_games_query` shows one row per group unless `group_versions=False`. Contract: `docs/specs/PASS-59-106-regional-versions.md`.

### Media handling
Expand All @@ -116,6 +119,7 @@ Moved to `.claude/rules/ratings.md`, which loads when a matching file is read. R

### Page wiring
- Both `/games` (Library) and `/system/<id>` use `AllGamesController` — keep them in sync.
- The game page's Previous / Next links follow the list it was opened from (`?from=library`, else its console). `AllGamesController` saves each list's filter and sort params per tab via `RetroDB.saveGameNavContext`; the game page sends them to `/api/games/neighbors`.

---

Expand Down
4 changes: 2 additions & 2 deletions config.example.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,8 +129,8 @@

# Application metadata
APP_NAME = "RetroDB"
APP_VERSION = "3.33.0"
APP_LAST_UPDATE = "2026-10-23"
APP_VERSION = "3.35.0"
APP_LAST_UPDATE = "2026-10-27"
APP_DESCRIPTION = "Retro Gaming ROM Library Manager"

# Supported image extensions for boxart/screenshots
Expand Down
101 changes: 101 additions & 0 deletions data/changelog.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,104 @@
- version: 3.35.0
date: 2026-10-27
tags:
- type: feature
label: Release dates for every region
- type: feature
label: Previous and next game on the game page
- type: feature
label: Uploaded covers and screenshots are processed like scraped ones
- type: fix
label: RetroAchievements shows on the game page for every game that has it
- type: fix
label: The Systems page's unscraped count matches the Unscraped filter
body: |
<ul>
<li><strong>Release dates for every region.</strong> When the AI
fills gaps during a scrape, and when you press AI Fill, it is now
also asked for the date the game came out in each region, such as
North America, Europe and Japan. The game page and the quick view
list them, earliest first, with dates and region names in your
language. A game's own release date is unchanged.</li>
<li><strong>Previous and next game on the game page.</strong> Next
to the path at the top of a game page are Previous and Next links
and where the game sits in the list. They follow the page you came
from: opened from the Library they step through the Library, and
from a console page through that console, in the same order and
with the same filters and search you had there.</li>
<li><strong>Uploaded covers and screenshots are processed like
scraped ones.</strong> A cover, 3D box or screenshot you upload in
the edit window is now saved in the same image format as scraped
images, resized to the same size, and covers get the smaller
copies the Library uses. A screenshot that looks the same as one
the game already has is not added, and you are told how many were
left out. Uploading a new cover also removes the file it replaces.</li>
<li><strong>RetroAchievements shows on the game page for every game
that has it.</strong> Some games had the RA badge in the Library
but no RetroAchievements card on their page, such as Metroid Prime
and Metroid Prime 2 on GameCube. The page now uses the same
RetroAchievements link as the badge, and loads faster because it
no longer asks RetroAchievements each time.</li>
<li><strong>The Systems page's unscraped count matches the Unscraped
filter.</strong> The count included bonus discs, which the
console's game list never shows, so a console could say 2
unscraped while its Unscraped filter showed none.</li>
</ul>
- version: 3.34.0
date: 2026-10-27
tags:
- type: feature
label: See how well each PS3 game runs in RPCS3
- type: feature
label: Sales figures for your games
- type: fix
label: The Franchise / Series switch works on the game page
- type: fix
label: Every Achievements tab is the same width
- type: fix
label: RetroDB finds RPCS3's settings on Windows and macOS
- type: fix
label: AI Fill works for games with a score but no age rating
- type: fix
label: Game card labels follow your language
body: |
<ul>
<li><strong>See how well each PS3 game runs in RPCS3.</strong> PS3
games in the Library, on the PlayStation 3 page and on the game
page show RPCS3's compatibility status: Playable, Ingame, Intro,
Loadable or Nothing, with the date it was last tested. RetroDB
reads it from the RPCS3 on your computer, so in RPCS3 choose
Manage &gt; Databases &gt; Download Compatibility Database once,
and again whenever you want newer results. Only games with a ROM
on disk are shown; imported games are not. RetroDB now also reads
the game ID of PSN and installed game folders, and of folders and
files whose name carries it, such as [BLUS30443].</li>
<li><strong>Sales figures for your games.</strong> When the AI fills
gaps during a scrape, and when you press AI Fill, it is now also
asked how many copies the game sold on its console, from free
published figures such as the publisher's reports, platform
best-seller lists, Famitsu, Wikipedia and VGChartz. The game page
shows the number. A game with no published figure stays blank
rather than getting a guess.</li>
<li><strong>The Franchise / Series switch works on the game
page.</strong> In the game page's edit window, clicking the
Franchise / Series switch of a game marked Standalone did nothing,
so it could not be put in a series. It now switches, and so does
the Platform Availability switch.</li>
<li><strong>Every Achievements tab is the same width.</strong>
RetroAchievements, RPCS3, PSN, Steam and Xbox each sized the page
to their own cards, so the page changed width as you moved between
tabs. They now share one width.</li>
<li><strong>RetroDB finds RPCS3's settings on Windows and
macOS.</strong> On Windows RPCS3 keeps its settings beside
rpcs3.exe, and RetroDB now looks there and in a portable folder,
so per-game settings and the compatibility list are found.</li>
<li><strong>AI Fill works for games with a score but no age
rating.</strong> AI Fill could fail with an error for such a
game while working out its age rating.</li>
<li><strong>Game card labels follow your language.</strong> The
completion status and score tooltips on Library cards were always
in English.</li>
</ul>
- version: 3.33.0
date: 2026-10-23
tags:
Expand Down
8 changes: 7 additions & 1 deletion docs/specs/api-contracts.md
Original file line number Diff line number Diff line change
Expand Up @@ -428,7 +428,13 @@ shown rows only.
trust the client. New paginated routes must do the same — `min(value, ceiling)`
inline, not after a "validate" call that might be skipped.

Bulk-id endpoints (`/api/games/ids`, `/api/games/card-data`) use a different
`/api/games/neighbors?id=N&<the /api/games filters and sort>` returns
`{found, position, total, prev, next}` for the game page's Previous / Next
links, `prev` / `next` being `{id, title}` or null. A version the list folds
into its group's card is placed at that card; `found` is false when the game
is not in the list.

Bulk-id endpoints (`/api/games/ids`, `/api/games/neighbors`, `/api/games/card-data`) use a different
cap pattern — see Pass 25.8: `LIMIT 20 * MAX_LIST_ROWS` (default 10 000) on
id-only fetches, and a hard `[1, 50]` range on card-data ids. Match the
relevant pattern by route shape; the rule is "always cap", not "always paginate".
Expand Down
2 changes: 2 additions & 0 deletions docs/specs/migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -538,6 +538,8 @@ your migration — not in the runner.
| 020 | `020_player.py` | Add `emulators.is_player`; insert the bundled player's row (`RetroDB Player`, `retrodb-player`) with no `system_emulators` rows. | PASS-59-64 §4.4. Additive. The row's `args_template` is unused: the resolver builds the player's argv. |
| 021 | `021_official_names.py` | Add `games.title_locked` (default 0); create `title_proposals`; trigger `trg_games_delete_title_proposal` removes a deleted game's proposal. | PASS-59-159 §4.1. Additive. |
| 022 | `022_player_name.py` | Rename the player's row from migration 020's placeholder `RetroDB Player` to `RetroArch (RetroDB)`. | PASS-59-139. Renames only the placeholder, and only when the new name is free, so a name the user chose is kept. |
| 023 | `023_games_units_sold.py` | Add `games.units_sold` (INTEGER, lifetime copies sold on the row's platform). | Additive. Filled only by the AI (scrape gap-fill and AI Fill); NULL = not known. |
| 024 | `024_games_release_dates.py` | Add `games.release_dates` (TEXT, JSON object of region code → date). | Additive. Filled only by the AI (scrape gap-fill and AI Fill); NULL = not known. |

---

Expand Down
6 changes: 5 additions & 1 deletion docs/specs/scrapers.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,11 @@ Text-only. AI never returns images, screenshots, videos, or manuals — `AI_FILL

**`PLATFORM_ALIASES`** maps non-canonical platform names the AI may produce (`ps1` → `PlayStation`, `playstation portable` → `PSP`).

**Integer-column trap (Pass 45.3):** AI Fill writes through `routes/games_ai.py`. The integer columns (`players`, `critic_score`, `critic_score_count`, `user_score`, `user_score_count`) coerce via `int(float(value))`. If the AI returns `"0"`, that becomes `0` — and a bare `field = ?` UPDATE then clobbers a curated `players=4` with `0`. Fix: any integer coerce to `0` is **skipped** entirely from the UPDATE clause (chosen over `COALESCE(NULLIF(?, 0), col)` because the existing `should_apply` filter already guarantees we only reach the int-coerce on values worth writing). Regression tests in `tests/test_pass45_security.py::TestPass45_3*`.
**Integer-column trap (Pass 45.3):** AI Fill writes through `routes/games_ai.py`. The integer columns (`players`, `critic_score`, `critic_score_count`, `user_score`, `user_score_count`, `units_sold`) coerce via `int(float(value))`. If the AI returns `"0"`, that becomes `0` — and a bare `field = ?` UPDATE then clobbers a curated `players=4` with `0`. Fix: any integer coerce to `0` is **skipped** entirely from the UPDATE clause (chosen over `COALESCE(NULLIF(?, 0), col)` because the existing `should_apply` filter already guarantees we only reach the int-coerce on values worth writing). Regression tests in `tests/test_pass45_security.py::TestPass45_3*`.

**Sales (`units_sold`):** only the AI fills it, in the scrape's AI gap-fill and in AI Fill; no API scraper publishes sales. The prompt asks for lifetime copies sold on the row's platform from free published sources and for null when only a multi-platform total exists. `parse_units_sold()` turns what a model writes (`"2.4 million"`, `"~850k"`, `"2,400,000"`) into an integer and drops anything above 500 million.

**Release dates by region (`release_dates`):** only the AI fills it, in the scrape's AI gap-fill and in AI Fill, fill-only like every other field. The prompt asks for an object keyed by region code (`NA`, `EU`, `JP`, `AU`, `KR`, `CN`, `TW`, `BR`, `AS`, `WW`) with `YYYY-MM-DD`, `YYYY-MM` or `YYYY` dates. `services/release_dates.parse_release_dates()` also takes region names (`"Japan"`, `"PAL"`) and a list of `{region, date}`, drops unknown regions and impossible dates, and the column stores compact JSON in region order. `release_date` stays the date of the row's own release.

**Circuit breaker:** `_ai_breaker` (5 failures / 120 s recovery) wraps `fetch_ai_metadata` calls from `hybrid_scraper` so a repeatedly-failing provider doesn't burn a quota on every gap-fill attempt.

Expand Down
Loading
Loading