Skip to content

Latest commit

 

History

4 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 

Repository files navigation

Google News Scraper API

google-news-scraper-api

pypi python npm node license

Google News has no public API. The usual workaround is a news scraper API that fetches the results page and returns parsed articles, which is what this repository demonstrates in Python, Node and cURL.

Every request and response here was run against the live endpoint on 2026-08-25.

Contents

The call

The news surface is the Google endpoint with search_type=news:

curl -G "https://app.scrapingbee.com/api/v1/google" \
     -H "Authorization: Bearer YOUR-API-KEY" \
     --data-urlencode "search=openai" \
     --data-urlencode "search_type=news" \
     --data-urlencode "country_code=us"

Auth goes in the header. The api_key query parameter still works but is marked deprecated.

What an article looks like

The response puts everything under news_results. A real item carries nine keys:

{
  "title": "OpenAI Claims Its New Chips Can Outperform Nvidia",
  "source": "Bloomberg",
  "domain": "bloomberg.com",
  "link": "https://www.bloomberg.com/...",
  "snippet": "...",
  "date": "2026-08-25T13:59:52.966Z",
  "relative_date": "3 hours ago",
  "position": 1,
  "page": 1
}

Sort on date, display relative_date. Filter on domain when you keep a publisher allow-list, because source is a display name and changes formatting more often than the domain does.

Scrape Google News results in Python

A working news scraper in Python is about fifteen lines:

import requests

ENDPOINT = "https://app.scrapingbee.com/api/v1/google"

def google_news(query, country="us", page=1):
    response = requests.get(
        ENDPOINT,
        headers={"Authorization": "Bearer YOUR-API-KEY"},
        params={"search": query, "search_type": "news",
                "country_code": country, "page": page},
        timeout=140,
    )
    response.raise_for_status()
    print("credits:", response.headers.get("Spb-cost"))
    return response.json().get("news_results", [])

for article in google_news("openai"):
    print(article["date"], article["source"], article["title"])

Two parameters that decide what you get

nfpr=true stops Google's automatic spelling correction. This is the single most useful flag for brand monitoring: Google rewrites unfamiliar company names to whatever it thinks you meant, and the job silently starts tracking a different word.

date_range narrows recency, which keeps a daily monitor from re-reading last month.

curl -G "https://app.scrapingbee.com/api/v1/google" \
     -H "Authorization: Bearer YOUR-API-KEY" \
     --data-urlencode "search=acme corp" \
     --data-urlencode "search_type=news" \
     --data-urlencode "nfpr=true" \
     --data-urlencode "date_range=d"

One documented limitation: search_type=news is unavailable with device=mobile.

Deduplication is not optional

Wire copy gets syndicated. The same story appears under several publishers and again across neighbouring queries, so any monitor that skips this step over-reports badly. Dedupe on link:

function dedupe(articles) {
  const seen = new Set();
  return articles.filter(a => {
    const key = a.link || `${a.source}|${a.title}`;
    if (seen.has(key)) return false;
    seen.add(key);
    return true;
  });
}

In testing, three related queries at two pages each collapsed by roughly half after dedupe.

Cost and scheduling

Call Credits
One news query 10 observed in testing, 15 documented standard
/api/v1/usage 0
HTTP 500 0

Failures are not charged, so retries are free. Before scheduling a sweep, check what is left:

curl "https://app.scrapingbee.com/api/v1/usage" -H "Authorization: Bearer YOUR-API-KEY"

Ten queries paged three deep, run hourly, is roughly 7,200 credits a day. Plan tiers are on the pricing page.

Ready-made packages

pip install google-news-scraper-api
npm install google-news-scraper-api
from google_news_scraper_api import GoogleNewsScraper

scraper = GoogleNewsScraper("YOUR-API-KEY")
articles = list(scraper.paginate("climate policy", pages=3, country_code="us"))
print(len(GoogleNewsScraper.deduplicate(articles)))

Both ship paging that stops on the first empty page and a static deduplicate helper.

Related surfaces

Same endpoint, different search_type: Google search . Shopping . Images . Lens . Ads . AI Mode

Also useful for news work: Google News RSS API and Fast Search.

Scope

Everything here targets publicly served search results. Credentialed access is off limits under the ScrapingBee terms, which prohibit scraping behind a login. Treat the API key as a secret and never paste it into an AI coding assistant.

License

MIT. See LICENSE.