Skip to content

Make CLI search defaults faster - #71

Open
Fr4nzz wants to merge 1 commit into
openags:mainfrom
Fr4nzz:codex/search-default-speed
Open

Make CLI search defaults faster#71
Fr4nzz wants to merge 1 commit into
openags:mainfrom
Fr4nzz:codex/search-default-speed

Conversation

@Fr4nzz

@Fr4nzz Fr4nzz commented May 7, 2026

Copy link
Copy Markdown

Summary

This PR makes the CLI paper-search search path faster by default while preserving broad coverage as an explicit opt-in.

Changes:

  • Adds fast and fastest source presets.
  • Makes fast the default source preset: OpenAlex, Crossref, arXiv, PubMed, Europe PMC.
  • Adds Semantic Scholar to fast only when PAPER_SEARCH_MCP_SEMANTIC_SCHOLAR_API_KEY / SEMANTIC_SCHOLAR_API_KEY is configured.
  • Keeps exhaustive broad search available with --exhaustive -s all.
  • Lazy-initializes only selected searchers, avoiding setup/warning overhead for unused sources.
  • Auto-adds Unpaywall to generic search only when the query contains a DOI.
  • Makes anonymous Semantic Scholar 429s fail fast instead of sleeping/retrying.

Why

Some sources are excellent for retrieval or exhaustive discovery, but are poor defaults for a broad topic query:

  • Unpaywall is DOI-centric and returned no topic-search results in testing, while adding latency.
  • Anonymous Semantic Scholar can spend time retrying 429s; if a key is configured, it is valuable and included in fast.
  • CORE/PMC/Google Scholar can be useful but are slower/noisier as default topic-search sources.
  • arXiv remains in the default because it can contribute unique useful papers.

This keeps the CLI responsive for default agent workflows without removing any source.

Verification

uv run pytest tests/test_fallback.py tests/test_semantic.py -q
# 10 passed, 6 skipped

Local benchmark on a broad SDM query showed the old path could spend ~16s when anonymous Semantic Scholar hit 429s, while the curated fast set avoids that retry delay. Source overlap checks also showed arXiv contributed unique results, so it stays in the default fast set.

HagiaHaya added a commit to HagiaHaya/paper-search-mcp that referenced this pull request Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant