fix(state): cap retry attempts for pending downloads - #49
Open
rfsbraz wants to merge 1 commit into
Open
Conversation
Contributor
Testing this PROption 1 — Docker Compose override: Create a services:
telegram-downloader:
build: https://github.com/rfsbraz/telegram-downloader.git#refs/pull/49/headThen run: docker compose up --buildOption 2 — Direct build and run: docker build https://github.com/rfsbraz/telegram-downloader.git#refs/pull/49/head -t telegram-downloader:pr-49
docker run --rm -it telegram-downloader:pr-49 |
There was a problem hiding this comment.
Pull request overview
This PR adds bounded retry/dead-letter behavior for the persistent pending download queue so a permanently failing message ID can’t cause run_check to spin indefinitely.
Changes:
- Add an
attemptscolumn (with auto-migration) topending_downloads, plus APIs to increment attempts and count exhausted entries. - Update
run_checkto exclude exhausted entries from scheduling and to increment attempts for failed downloads. - Introduce
max_download_attemptsconfiguration (default 5) and add unit tests for attempt accounting and legacy DB migration.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/state/pending.py |
Adds attempts column + migration, filters get_oldest/count by attempt budget, and introduces increment_attempts/count_exhausted. |
src/main.py |
Uses attempt budget when pulling pending batches, increments attempts for failed IDs, and reports exhausted entries. |
src/config/schema.py |
Adds max_download_attempts config option with validation. |
tests/unit/state/test_pending.py |
Adds unit tests covering attempt counting, exclusion at limit, isolation, and legacy schema migration. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+65
to
+69
| self._connection.execute( | ||
| "ALTER TABLE pending_downloads ADD COLUMN attempts INTEGER NOT NULL DEFAULT 0" | ||
| ) | ||
| except sqlite3.OperationalError: | ||
| pass # Column already exists |
Comment on lines
+378
to
+381
| log.warning( | ||
| f"{len(failed_ids)} download(s) failed, will retry " | ||
| f"(up to {cfg.max_download_attempts} attempts per message)" | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
run_check: failed downloads stayed in the pending queue, and the download loop immediately pulled the exact same IDs back out withget_oldest. A single permanently failing message (deleted media, persistent server error) made the check iteration spin forever, hammering Telegram through the full retry/backoff budget on every pass and never letting the check complete.attemptscolumn topending_downloadswith automatic schema migration for existing databases (ALTER TABLE, no manual steps).get_oldestandcountexclude entries that reached the limit, so each failing message is retried a bounded number of times and then dead-lettered.max_download_attemptsconfig option (default 5, each attempt still carries the existing per-download retry budget).