paperfetch

Find, download, and verify open-access academic-paper PDFs — legally.

Most "get the PDF" tools hand you a URL and stop. paperfetch goes further: it downloads the file, confirms it's a real PDF, and confirms it's actually the paper you asked for (title/author match) — so a plausible-but-wrong open PDF is never saved. It pulls only from legitimately-open sources: arXiv, the OA aggregators (Unpaywall / OpenAlex / Semantic Scholar / Crossref), gold-OA publishers (MDPI), open conference proceedings (OpenReview / ACL / AAAI), Europe PMC, and author/repository copies found by an optional web search.

Install

pip install paperfetch                 # core (curl_cffi + pymupdf)
pip install "paperfetch[scrape]"       # + nicer landing-page link extraction (bs4)
pip install "paperfetch[browser]"      # + BrightData Scraping-Browser fallback (playwright)

Quickstart

CLI:

paperfetch --arxiv 2502.07868 --out papers
paperfetch --doi 10.3390/math12010001 --title "…" --authors "…"
paperfetch --title "ArchetypeTrader: Reinforcement Learning …" --authors "Bo An" --search
paperfetch --doi 10.1145/3772326.3774721 --json

Library:

from paperfetch import fetch_paper

r = fetch_paper(doi="10.3390/math12010001", title="…", authors="…", out="papers")
if r.ok:
    print(r.path, "verified=", r.verified, "via", r.source_url)

Custom storage (S3, a DB, …) — implement the tiny Storage protocol:

from paperfetch import fetch_paper, Storage
class MyStore:  # has(key)->bool, save(key,data,meta)->str, record_failure(key,reason)->None
    ...
fetch_paper(doi="…", title="…", storage=MyStore())

How it resolves (legal order; first verified PDF wins)

1. Direct-URL constructors for known open hosts (arXiv / OpenReview / ACL / AAAI).
2. OA aggregators by DOI — Unpaywall, OpenAlex, Semantic Scholar, Crossref (keyless).
3. arXiv title search.
4. Landing-page scrape of the citation_pdf_url meta tag.
5. Europe PMC / MDPI gold-OA.
6. Web search (--search, optional) for alternative mirrors — author pages, ResearchGate, repositories — via OpenRouter (perplexity/sonar).

Candidate URLs are deduped, paywalled-publisher hosts are dropped, and direct-PDF / arXiv / repository URLs are tried first. Every kept PDF is title-verified (unless --no-verify).

Optional integrations (off by default)

• --email you@example.com — sent to the OA APIs' polite pool (recommended).
• --search — needs OPENROUTER_API_KEY.
• --brightdata / --use-sb — fetch an already-open PDF past anti-bot/Cloudflare via BrightData Web Unlocker / Scraping Browser (needs BRIGHTDATA_WU_* / BRIGHTDATA_SB_*). No-ops without creds.

Legal & ethics

paperfetch only retrieves content that is already legally open: arXiv, OA-aggregator locations, gold-OA publishers, open proceedings, Europe PMC, and author/repository copies. It does not include Sci-Hub or any other paywall-bypass, and it never circumvents access controls. The optional BrightData fallbacks exist solely to fetch a PDF a host already serves openly when anti-bot defenses block a plain download — not to access paywalled material. Please use a real contact email for the OA APIs and respect each source's terms of use.

Credit

Extracted from the winterchill papers pipeline. MIT-licensed.