What happens
already_tried() treats failed as a definitive verdict, so a source that once returned failed is skipped for that ref forever. But two of Anna's most common outcomes are explicitly transient:
annas_headful_optin | failed | budget_exhausted_retry_later — the reason literally says retry later
annas_archive_optin | failed | md5_found_but_no_dl — the fingerprint was found, only the download slot was rationed
Once either is recorded, both Anna's routes report skipped_already_tried on every later pass. run --retry-exhausted lifts the ref-level cascade_exhausted_needs_manual lock but not this source-level memory, so the ref is re-planned and then skips the only sources that could serve it.
Observed on 0.4.0
Three books re-run today with run --ref … --retry-exhausted, full 16-source cascade:
manual_oa_url no_source no_oa_url_in_frontmatter
crossref_oa no_source no_doi
arxiv no_source no_arxiv_id
openalex_oa no_source no_oa_url
unpaywall no_source no_doi
hal no_source hal_no_match_or_no_file
core no_source no_core_match
publisher_doi no_source no_doi
archive_org no_source ia_no_match_above_threshold
publisher_headful no_source no_doi_and_no_manual_url
scihub_optin no_source no_doi
annas_archive_optin skipped_already_tried
libgen_optin no_source no_doi
annas_scidb_optin no_source no_doi
annas_headful_optin skipped_already_tried
websearch no_source websearch_already_queued
Result: planned=3 done=0 blocked=3. The browser starts fine and the shared session works (profile created, Chromium alive throughout) — it is simply never asked to do anything.
Second observation, same run: the new routes are DOI-gated
publisher_doi, publisher_headful, libgen_optin and annas_scidb_optin all return no_doi on books. libgen.py searches Libgen by DOI only, though Libgen's main strength is monographs — a title+author search would make it the book route the release notes describe. As it stands, for a reference without a DOI (any book), 0.4.0 adds no reachable source.
Suggested fix
- Split the verdict: keep
failed definitive, add a transient verdict (or let already_tried() re-open attempts whose reason matches retry_later / no_dl / no_slot).
- Have
--retry-exhausted also clear source-level transient attempts, not just the ref lock.
- Give
libgen_optin a title+author search path when no DOI is present.
Environment
paper-trail 0.4.0, Debian container, xvfb-run, RESEARCH_ENABLE_SHADOW_LIBS=1, RESEARCH_BROWSER_PROFILE on a mounted volume, RESEARCH_MIN_BOOK_PAGES=30.
What happens
already_tried()treatsfailedas a definitive verdict, so a source that once returnedfailedis skipped for that ref forever. But two of Anna's most common outcomes are explicitly transient:annas_headful_optin | failed | budget_exhausted_retry_later— the reason literally says retry laterannas_archive_optin | failed | md5_found_but_no_dl— the fingerprint was found, only the download slot was rationedOnce either is recorded, both Anna's routes report
skipped_already_triedon every later pass.run --retry-exhaustedlifts the ref-levelcascade_exhausted_needs_manuallock but not this source-level memory, so the ref is re-planned and then skips the only sources that could serve it.Observed on 0.4.0
Three books re-run today with
run --ref … --retry-exhausted, full 16-source cascade:Result:
planned=3 done=0 blocked=3. The browser starts fine and the shared session works (profile created, Chromium alive throughout) — it is simply never asked to do anything.Second observation, same run: the new routes are DOI-gated
publisher_doi,publisher_headful,libgen_optinandannas_scidb_optinall returnno_doion books.libgen.pysearches Libgen by DOI only, though Libgen's main strength is monographs — a title+author search would make it the book route the release notes describe. As it stands, for a reference without a DOI (any book), 0.4.0 adds no reachable source.Suggested fix
faileddefinitive, add atransientverdict (or letalready_tried()re-open attempts whose reason matchesretry_later/no_dl/no_slot).--retry-exhaustedalso clear source-level transient attempts, not just the ref lock.libgen_optina title+author search path when no DOI is present.Environment
paper-trail 0.4.0, Debian container,
xvfb-run,RESEARCH_ENABLE_SHADOW_LIBS=1,RESEARCH_BROWSER_PROFILEon a mounted volume,RESEARCH_MIN_BOOK_PAGES=30.