Add Serply web and Google Scholar retrievers

[googio]

Add Serply retrievers (web + Google Scholar)

This adds two optional retrievers backed by the Serply SERP API, following the existing per-provider pattern (one directory under gpt_researcher/retrievers/, registered in retrievers/__init__.py, retrievers/utils.py, and actions/retriever.py):

• RETRIEVER=serply - Google web results, mirroring the serper retriever it sits next to.
• RETRIEVER=serply_scholar - Google Scholar results. GPT Researcher already has arXiv, Semantic Scholar, PubMed Central, and OpenAlex, but no Google Scholar, which spans publishers, theses, and citations across every field. This gives the research flow that coverage.

Both read SERPLY_API_KEY from the environment and normalize responses to the standard {title, href, body} shape. Like the sibling retrievers, search() always returns a list, never None.

Why two

The repo already pairs same-vendor SERP wrappers (serper and serpapi) and ships four academic retrievers, so both variants fit the existing surface. Each is small and self-contained; the file count comes from the per-provider scaffold applied twice, not from a large change to any one file.

Optional and non-breaking

Nothing changes for anyone who does not set RETRIEVER=serply or serply_scholar. No new dependency is added (requests is already required). Note: Serply sits behind Cloudflare, which rejects the default requests User-Agent, so both retrievers send an explicit one.

Testing

• Live-tested both retrievers against api.serply.io, confirming real web and scholar results normalize to the expected shape.
• Added tests/test_serply_returns_list.py, mirroring test_serper_returns_list.py, covering the return-a-list error paths with the network mocked.

Docs: added both entries to docs/docs/gpt-researcher/search-engines/search-engines.md. More at serply.io/docs.

Disclosure: I work with Serply. Happy to adjust scope, naming, or split the web and scholar retrievers into separate PRs if you prefer.

[assafelovic]

Reviewed properly — thanks, this is a clean PR.

It applies to current main and the full suite passes with it (392 tests, 0 failures). Tests, docs and error handling are all present, and adding both the web and Scholar variants in one PR is the right granularity.

---

Why this is not merged yet, and it isn't code quality.

main already ships 22 retrievers. Six open PRs (#1814, #1850, #1867, #2015, #2048, #2068) would add seven more, and most come from the vendor whose API is being added. Each one is a few hundred lines the project maintains indefinitely, behind an API key CI cannot hold, against a service whose uptime and response shape nobody here controls — and the recently-merged work included roughly twenty separate fixes for retrievers returning payload shapes their parsers did not expect.

That is a scope decision for @assafelovic, not a verdict on this patch. The sustainable answer is probably a documented plugin path — an entry-point group so a vendor ships and versions their own package — rather than accepting or rejecting each of these case by case, which is exactly why they have all been sitting.

Not closing it. Flagging it so the decision gets made once instead of six times.