Let admins update RetroDB with one click from GitHub Releases

.claude/rules/distribution.md

@@ -28,7 +28,7 @@ python3 build_dist.py --standalone --cpu-only  # CPU-only standalone (~600 MB, -
 ```
 
 - Output: `STAGING_DIR` constant in `build_dist.py` — defaults to `/mnt/Games/Scripts/Linux/Staging_Area/RetroDB`; override with the `RETRODB_STAGING_DIR` env var (`release.yml` does, to `/tmp/staging`). The default must name a mounted, writable drive: `build_standalone` calls `os.makedirs(STAGING_DIR)`, so a retired drive fails the documented build command with `PermissionError`.
-- Filename: `RetroDB-v{VERSION}-{Platform}.zip` (source) or `RetroDB-v{VERSION}-{Platform}-Standalone.zip`
+- Filename: `RetroDB-v{VERSION}-{Platform}.zip` (source) or `RetroDB-v{VERSION}-{Platform}-Standalone.zip`. The self-updater (PASS-59-169, `services/updater.py::asset_names`) finds a release's download by these names, its checksum in `SHA256SUMS.txt` / `<zip>.sha256`, and only sees a release once it is **published** — so publishing the draft is what offers the update to users.
 - What ships in a source ZIP: `collect_files` enumerates **`git ls-files`** — the allow-list `.gitignore` already maintains — and only then applies the deny-lists (Pass 59.13). Every generated artifact the ZIP needs (`main.min.css`, both bundles, `translations/**/*.mo`) is committed, so nothing shipped depends on the walk. Without git (a tarball, not a checkout) it falls back to the filesystem walk, which is why the deny-lists stay. They are `EXCLUDE_FILES`, `EXCLUDE_DIRS`, `EXCLUDE_EXTENSIONS` and the `INCLUDE_IMAGE_DIRS` whitelist in `build_dist.py`, and they remove the tracked-but-maintainer-only files. Between them this excludes user config (`config.py`, `data/{settings,scraper_settings,rom_tools_config,psn_tokens,xbox_tokens}.json`, `data/.secret_key`), the runtime DB (the whole `database/` dir — covers `database/roms.db`; `data/retrodb.db` is also listed as a legacy path, and all `.db*` / `.log` files are excluded by extension), and scraped media (`static/videos/`, plus every `static/images/<dir>/` that is not in `INCLUDE_IMAGE_DIRS = {'hardware','ratings','systems','avatars'}`). The one exception is `INCLUDE_IMAGE_FILES`: the app's trophy and platform icons, which share `static/images/trophies/` with each user's downloaded PSN trophy art. They are named file by file in `build_dist.py`, `.gitignore` and `retrodb.spec`. `tests/test_pass59_packaging.py::TestReferencedImagesShip` fails if an image the templates or scripts name stops shipping. Per-platform: only that platform's start script (`start.sh` / `start.command` / `start.bat`).
 - Standalone build is driven by `retrodb.spec` (PyInstaller onedir mode). Spec whitelists static subdirs explicitly to avoid sweeping in scraped media; new pip deps that PyInstaller's static analyser can't follow (string-imported via `importlib`) must be added to `HIDDEN_IMPORTS` in the spec.
 - `--cpu-only` (Pass 46.3 Part 3) builds the standalone inside an isolated venv under `STAGING_DIR/.cpu-build-venv` (vanilla CPU `onnxruntime` from `requirements.txt` + PyInstaller), so it produces the ~600 MB CPU bundle even on the maintainer's AMD box where `onnxruntime-rocm` is installed system-wide (the two flavours share the `onnxruntime` import name and can't coexist). Artifacts get a `-CPU` filename suffix. Plain `--standalone` inherits whatever onnxruntime is in the current env (GPU/ROCm locally; CPU on a clean checkout, since `requirements.txt` pins vanilla).

.gitignore

@@ -25,6 +25,11 @@ data/m3u_staging/
 # HLTB dataset (large CSV, downloaded on demand)
 data/hltb_dataset.csv
 
+# Self-updater state and work folder (PASS-59-169)
+data/update_check.json
+data/update_status.json
+.update/
+
 # PSN credentials
 docs/psn-npsso.env
 

CLAUDE.md

@@ -20,7 +20,7 @@ genuinely diverges.
 ## Mandatory Workflow
 
 ### After Every Code Change
-1. **Bundle changes into releases** (user, 2026-10-17): a release with one changelog item is too small. A release carries 5 to 10 changelog items (user, 2026-10-23): release once the open entry reaches 10, or sooner at a natural stop once it has 5. While the version at the top of `data/changelog.yaml` has no `v<version>` git tag, it is still open: add the change to that entry and keep the version. Otherwise bump `config.py` AND `config.example.py` (`APP_VERSION` + `APP_LAST_UPDATE`) and open a new entry. Keep `APP_LAST_UPDATE` current either way.
+1. **Bundle changes into releases** (user, 2026-10-17): a release with one changelog item is too small. A release carries at least 5 changelog items, with no upper limit (user, 2026-10-27: the floor only stops a release going out with one item; an open entry may grow past 10). While the version at the top of `data/changelog.yaml` has no `v<version>` git tag, it is still open: add the change to that entry and keep the version. Otherwise bump `config.py` AND `config.example.py` (`APP_VERSION` + `APP_LAST_UPDATE`) and open a new entry. Keep `APP_LAST_UPDATE` current either way.
 2. Add or extend the changelog entry at top of `data/changelog.yaml`. For a real release, also translate the **recent** entry (current line onward; deep history stays English) into each per-locale `data/changelog.<locale>.yaml` (the human-translation locales in `docs/specs/i18n.md` §9 — a different delivery path from the installed UI-catalog set `services/i18n.py::available_locales()` derives, §3). Repeat `version`, `date`, and every tag verbatim — the `/changelog` route swaps the whole entry in by version, so anything omitted is dropped, not inherited (Pass 43.6). Skip for internal/no-release bumps.
 3. Rebuild CSS if any `static/css/**.css` changed: `python3 build_css.py`
 4. Rebuild JS if any bundled `static/js/*.js` changed: `python3 build_js.py`
@@ -114,6 +114,11 @@ has seen the raw output. Triage into new roadmap passes after.
 ### Multi-rating system (9 systems)
 Moved to `.claude/rules/ratings.md`, which loads when a matching file is read. Read it before touching ratings, the edit modals or the scrapers.
 
+### Self-updater (PASS-59-169)
+- The block at the top of `app.py` hands `--apply-update` to `updater_apply.py` before any other import; keep it first (the rest of `app.py` opens and migrates the database). `updater_apply.py` stays standard-library only: the standalone binary runs it inside its own bundle.
+- Install runs the NEW version's helper, so a change to the plan file (`plan_version`) or to which paths an update may write (`updater_apply.PROTECTED`) must stay readable from the previous release's `services/updater.py`.
+- The updater picks release assets by name (`RetroDB-v{V}-{Platform}.zip` + `SHA256SUMS.txt`, `RetroDB-v{V}-{Platform}[-CPU]-Standalone.zip` + `.sha256`); renaming them in `build_dist.py` / `release.yml` stops updates. Spec: `docs/specs/PASS-59-169-self-updater.md`.
+
 ### Page wiring
 - Both `/games` (Library) and `/system/<id>` use `AllGamesController` — keep them in sync.
 

app.py

@@ -5,6 +5,14 @@
 # Features: ROM scanning, metadata scraping, beautiful cyberpunk UI
 # =============================================================================
 
+import sys
+if len(sys.argv) > 1 and sys.argv[1] == '--apply-update':
+    # PASS-59-169 — the standalone binary is also the update helper. Hand
+    # over before anything else is imported: the rest of this module opens
+    # and migrates the database, which the helper must not do.
+    import updater_apply
+    sys.exit(updater_apply.main(sys.argv[1:]))
+
 from werkzeug.exceptions import HTTPException
 from flask import Flask, render_template, request, redirect, url_for, jsonify, session, g, send_from_directory, abort
 import gzip
@@ -342,6 +350,7 @@ def serve_static_image(filepath):
 from routes.launch import bp as launch_bp
 from routes.emulators import bp as emulators_bp
 from routes.launch_settings import bp as launch_settings_bp
+from routes.updates import bp as updates_bp
 
 app.register_blueprint(clz_import_bp)
 app.register_blueprint(controllers_bp)
@@ -365,6 +374,7 @@ def serve_static_image(filepath):
 app.register_blueprint(platform_import_bp)
 app.register_blueprint(steam_achievements_bp)
 app.register_blueprint(xbox_achievements_bp)
+app.register_blueprint(updates_bp)
 app.register_blueprint(game_imports_bp)
 app.register_blueprint(launch_bp)
 app.register_blueprint(emulators_bp)
@@ -1019,11 +1029,17 @@ def _get_collector_rank_safe():
     trophies module / table isn't ready yet (e.g. first boot before the
     table has been populated). The sidebar badge reads this on every
     render, so it must never raise."""
+    # inject_config runs for every render_template call in a request, so the
+    # rank is read once per request and reused.
+    if '_collector_rank' in g:
+        return g._collector_rank
     try:
         from routes.collector_trophies import get_current_rank
-        return get_current_rank()
+        rank = get_current_rank()
     except Exception:
-        return None
+        rank = None
+    g._collector_rank = rank
+    return rank
 
 
 # =============================================================================
@@ -1337,6 +1353,11 @@ def dashboard():
             jtype = rj['job_type']
             status = rj['status']
 
+            # A job the user may neither resume nor dismiss is not offered:
+            # the banner would show buttons that only answer 403.
+            if not _may_manage_job(jtype, params):
+                continue
+
             # Determine if this job had real progress (interrupted mid-run)
             # vs was just queued (no work done yet)
             has_progress = progress.get('current', 0) > 0
@@ -1409,6 +1430,16 @@ def dashboard():
 }
 
 
+def _may_manage_job(job_type, params):
+    """PASS-59-118 — resuming or dismissing a recoverable job needs the
+    rights starting it needed: the start route's role, and a per-user job
+    belongs to its user (admins excepted)."""
+    role = g.user['role']
+    owner = (params or {}).get('user_id')
+    return (role in _RESUME_ROLES.get(job_type, ('admin',))
+            and (owner is None or owner == g.user['id'] or role == 'admin'))
+
+
 @app.route('/api/jobs/resume/<int:job_id>', methods=['POST'])
 @login_required
 @handle_api_errors
@@ -1450,12 +1481,7 @@ def resume_job(job_id):
     if not handler or not hasattr(handler, 'resume_from_params'):
         return api_error(f"No handler for job type '{job['job_type']}'", 400)
 
-    # PASS-59-118 — resuming needs the rights starting needed: the start
-    # route's role, and a per-user job belongs to its user (admins excepted).
-    role = g.user['role']
-    need = _RESUME_ROLES.get(job['job_type'], ('admin',))
-    owner = (job.get('params') or {}).get('user_id')
-    if role not in need or (owner is not None and owner != g.user['id'] and role != 'admin'):
+    if not _may_manage_job(job['job_type'], job.get('params')):
         return api_error('You do not have permission to perform this action', 403)
 
     resumed = handler.resume_from_params(job['params'], progress=job.get('progress'))
@@ -1471,8 +1497,19 @@ def resume_job(job_id):
 @login_required
 @handle_api_errors
 def dismiss_job_route(job_id):
-    """Dismiss a recoverable job so it no longer appears on the dashboard."""
+    """Dismiss a recoverable job so it no longer appears on the dashboard.
+
+    The same rights as resuming it: dismissing throws the job away.
+    """
     from services.jobs.base import dismiss_job
+    row = query("SELECT job_type, params FROM job_queue WHERE id = ?", (job_id,), one=True)
+    if row:
+        try:
+            params = json.loads(row['params']) if row['params'] else {}
+        except (TypeError, ValueError):
+            params = {}
+        if not _may_manage_job(row['job_type'], params if isinstance(params, dict) else {}):
+            return api_error('You do not have permission to perform this action', 403)
     dismiss_job(job_id)
     return api_success()
 
@@ -1890,12 +1927,16 @@ def should_open_browser(environ, frozen):
 
     The standalone bundle always does (only the server knows the resolved
     port, Pass 59.10); a source install when start.sh / start.command /
-    start.bat set RETRODB_OPEN_BROWSER=1. Never under LocalWebServerManager
+    start.bat set RETRODB_OPEN_BROWSER=1; RETRODB_OPEN_BROWSER=0 turns it off
+    for both (the update helper's restart, PASS-59-169). Never under LocalWebServerManager
     (LWSM_MANAGED=1), which has its own Open button.
     scripts/retrodb_launcher.py opens the browser itself.
     """
     if environ.get('LWSM_MANAGED') == '1':
         return False
+    # '0' — the update helper restarting RetroDB: the open tab reloads itself.
+    if environ.get('RETRODB_OPEN_BROWSER') == '0':
+        return False
     return bool(frozen) or environ.get('RETRODB_OPEN_BROWSER') == '1'
 
 

build_css.py

@@ -51,6 +51,7 @@
     'features/filters.css',
     'features/hltb.css',
     'features/manual-reader.css',
+    'features/updates.css',
 
     # 5. Pages
     'pages/game-list.css',

build_dist.py

@@ -68,6 +68,8 @@
     'data/settings.json',
     'data/rom_tools_config.json',
     'data/psn_tokens.json',
+    'data/update_check.json',    # self-updater state (PASS-59-169)
+    'data/update_status.json',
     'data/xbox_tokens.json',
     'data/psn_tokens.json.migrated-015',
     'data/xbox_tokens.json.migrated-015',
@@ -84,6 +86,7 @@
     '__pycache__', '.claude', '.git', '.vscode', '.idea',
     'demo', 'dist', 'venv', '.venv',
     'database', 'logs',  # .gitkeep files added explicitly
+    '.update',  # self-updater work folder (PASS-59-169)
 }
 
 # .db-shm joins .db-wal/.db-journal — WAL mode (v3.22.1) leaves all three behind.

build_js.py

@@ -58,6 +58,7 @@
     'museum.js',                # Page-specific
     'versions.js',              # Page-specific (game page + Regional Versions)
     'official-names.js',        # Page-specific (ROM Tools > Official Names)
+    'updates.js',               # Page-specific (Settings > Updates + dashboard banner)
 ]
 
 BUNDLES = [

config.example.py

@@ -130,7 +130,7 @@
 # Application metadata
 APP_NAME = "RetroDB"
 APP_VERSION = "3.33.0"
-APP_LAST_UPDATE = "2026-10-23"
+APP_LAST_UPDATE = "2026-10-27"
 APP_DESCRIPTION = "Retro Gaming ROM Library Manager"
 
 # Supported image extensions for boxart/screenshots

data/changelog.yaml

@@ -1,12 +1,42 @@
 - version: 3.33.0
-  date: 2026-10-23
+  date: 2026-10-27
   tags:
+  - type: feature
+    label: RetroDB updates itself
   - type: feature
     label: Get RetroArch from inside RetroDB
   - type: fix
     label: Next disk works for games played in RetroDB
-  body: |
-    <ul>
+  - type: fix
+    label: Game cards line up on a phone and after resizing the window
+  - type: fix
+    label: Scraped ratings and player counts are more accurate
+  - type: fix
+    label: ScreenScraper keeps its details when it is the main source
+  - type: fix
+    label: Clear scraped data puts the file name back as the title
+  - type: fix
+    label: RetroAchievements refresh for one console works again
+  - type: fix
+    label: Screenshot duplicate finder works in the downloaded app
+  - type: fix
+    label: ScreenScraper passwords are hidden, and the last admin stays admin
+  - type: fix
+    label: Interrupted jobs are only offered to people allowed to run them
+  - type: performance
+    label: Faster trophies, library imports and maintenance tools
+  body: |
+    <ul>
+      <li><strong>RetroDB updates itself.</strong> When a new version is
+        out, admins see it on the dashboard and in Settings &gt; System &gt;
+        Updates, with what is new. Download fetches it and checks it is
+        exactly the published file; Install and restart backs up your
+        database, puts the new version in place and restarts. If the new
+        version does not start, RetroDB goes back to the old one by itself
+        and says why. Your games, settings, images and logs are not touched.
+        RetroDB looks for updates once a day; you can turn that off. A copy
+        run from git, or by a process manager, is told about updates but
+        not updated.</li>
       <li><strong>Get RetroArch from inside RetroDB.</strong> Settings &gt;
         Emulators has a new Play in RetroDB section. Press Get RetroArch
         and RetroDB downloads the official RetroArch from libretro, checks
@@ -20,6 +50,48 @@
         own RetroArch with no setup. RetroArch listens for RetroDB's
         commands only while such a game runs, and on Windows and macOS
         while any game runs.</li>
+      <li><strong>Game cards line up on a phone and after resizing the
+        window.</strong> On a phone the Library and console pages showed one
+        card and then a long empty gap; cards now stack one under another.
+        Making the window narrower or wider re-arranges the cards instead of
+        leaving them in the old columns.</li>
+      <li><strong>Scraped ratings and player counts are more accurate.</strong>
+        Age ratings from IGDB for CERO, USK, GRAC, ClassInd and ACB are now
+        read correctly, so a game rated CERO Z or USK 18 gets that rating.
+        TheGamesDB, RAWG, IGDB and ES-DE no longer guess "1 player" when
+        they don't know; an ES-DE import no longer overwrites a player count
+        or game modes you set. TheGamesDB no longer saves a number in place
+        of a publisher or developer name after a failed lookup.</li>
+      <li><strong>ScreenScraper keeps its details when it is the main
+        source.</strong> Its title, description, date, developer, publisher
+        and players are kept on every scrape, not only the first few
+        minutes after a search. Downloaded videos and manuals get a proper
+        file name.</li>
+      <li><strong>Clear scraped data puts the file name back as the
+        title.</strong> It cleared the details but left the scraped
+        title.</li>
+      <li><strong>RetroAchievements refresh for one console works
+        again.</strong> It failed with an error every time.</li>
+      <li><strong>Screenshot duplicate finder works in the downloaded
+        app.</strong> It found no screenshots there, and deleting a
+        duplicate left the file on disk.</li>
+      <li><strong>ScreenScraper passwords are hidden, and the last admin
+        stays admin.</strong> Settings now shows your ScreenScraper
+        passwords masked, like your other keys. The only admin account can
+        no longer be switched to editor or turned off, which would have
+        left nobody able to manage RetroDB.</li>
+      <li><strong>Interrupted jobs are only offered to people allowed to run
+        them.</strong> The dashboard's interrupted-job banner now shows a
+        job only to someone allowed to start it, for example a bulk scrape
+        only to editors and admins. Other accounts no longer see it and can
+        no longer dismiss it.</li>
+      <li><strong>Faster trophies, library imports and maintenance
+        tools.</strong> The Trophies page, Steam, Xbox and PlayStation
+        library imports, collector trophies, Calculate storage and the
+        unused-media scan all do far less work on a large library. Job
+        progress pop-ups keep updating when the server is busy, and the
+        HowLongToBeat and alternate-title jobs no longer hold up other
+        changes while they run.</li>
     </ul>
 - version: 3.32.0
   date: 2026-10-23

docs/design.md

@@ -92,6 +92,12 @@ Themes are [`specs/themes.md`](specs/themes.md).
 **Translation.** Flask-Babel catalogs, plus whole per-locale files for the
 help manual and changelog. That is [`specs/i18n.md`](specs/i18n.md).
 
+**Updates.** RetroDB checks GitHub Releases for a newer version, and an
+admin installs it with one click: the download is checked against the
+release's SHA-256, the database is backed up, and the new version's helper
+swaps the files, restarts, and rolls back if the new version does not come
+up. That is [`specs/PASS-59-169-self-updater.md`](specs/PASS-59-169-self-updater.md).
+
 ### Subsystems without a spec
 
 Recorded under PASS-59-66 (2026-10-17) so the next audit does not decide this