Skip to content

Transient Anna's failures are recorded as definitive: the only book-capable source is never retried #11

Description

@roomi-fields

What happens

already_tried() treats failed as a definitive verdict, so a source that once returned failed is skipped for that ref forever. But two of Anna's most common outcomes are explicitly transient:

  • annas_headful_optin | failed | budget_exhausted_retry_later — the reason literally says retry later
  • annas_archive_optin | failed | md5_found_but_no_dl — the fingerprint was found, only the download slot was rationed

Once either is recorded, both Anna's routes report skipped_already_tried on every later pass. run --retry-exhausted lifts the ref-level cascade_exhausted_needs_manual lock but not this source-level memory, so the ref is re-planned and then skips the only sources that could serve it.

Observed on 0.4.0

Three books re-run today with run --ref … --retry-exhausted, full 16-source cascade:

manual_oa_url        no_source   no_oa_url_in_frontmatter
crossref_oa          no_source   no_doi
arxiv                no_source   no_arxiv_id
openalex_oa          no_source   no_oa_url
unpaywall            no_source   no_doi
hal                  no_source   hal_no_match_or_no_file
core                 no_source   no_core_match
publisher_doi        no_source   no_doi
archive_org          no_source   ia_no_match_above_threshold
publisher_headful    no_source   no_doi_and_no_manual_url
scihub_optin         no_source   no_doi
annas_archive_optin  skipped_already_tried
libgen_optin         no_source   no_doi
annas_scidb_optin    no_source   no_doi
annas_headful_optin  skipped_already_tried
websearch            no_source   websearch_already_queued

Result: planned=3 done=0 blocked=3. The browser starts fine and the shared session works (profile created, Chromium alive throughout) — it is simply never asked to do anything.

Second observation, same run: the new routes are DOI-gated

publisher_doi, publisher_headful, libgen_optin and annas_scidb_optin all return no_doi on books. libgen.py searches Libgen by DOI only, though Libgen's main strength is monographs — a title+author search would make it the book route the release notes describe. As it stands, for a reference without a DOI (any book), 0.4.0 adds no reachable source.

Suggested fix

  • Split the verdict: keep failed definitive, add a transient verdict (or let already_tried() re-open attempts whose reason matches retry_later / no_dl / no_slot).
  • Have --retry-exhausted also clear source-level transient attempts, not just the ref lock.
  • Give libgen_optin a title+author search path when no DOI is present.

Environment

paper-trail 0.4.0, Debian container, xvfb-run, RESEARCH_ENABLE_SHADOW_LIBS=1, RESEARCH_BROWSER_PROFILE on a mounted volume, RESEARCH_MIN_BOOK_PAGES=30.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions