Similarweb Scraper: export to CSV or JSON with Python, Node.js or curl

Scrape Similarweb top websites rankings by category: global rank, domain, category, rank change, average visit duration, pages per visit, and bounce rate. Up to 50 results per category page.

This repository has ready scripts for the Similarweb Scraper: Top Websites by Category | $1.70 per 1,000 Actor on Apify. The Actor does the fetching, paging and cleaning; the scripts here run it with your input and save the results as a CSV or JSON file.

Quick start

1. Create an Apify account and copy your API token from Console > Settings > Integrations.
2. Install the client and run the script with the example input:

pip install "apify-client>=2"
export APIFY_TOKEN=your_token
python run.py input.json -o results.csv --max-items 50

Edit input.json for your own run. You can also run the Actor in the browser on its Apify page, schedule it there, and connect its output to Google Sheets, Slack, Zapier or Make.

Input

The example input (input.json):

{
  "categorySlugs": [
    "",
    "computers-electronics-and-technology",
    "arts-and-entertainment"
  ],
  "maxItems": 20
}

Field	Type	What it does
categorySlugs	array	Similarweb category path slugs from similarweb.com/top-websites/.
maxItems	integer	The run stops after this many result rows.

Output

One row per result. A real row from a run of the example input (more in examples/sample-output.json):

{
  "rank": 1,
  "domain": "google.com",
  "category": "Computers Electronics and Technology > Search Engines",
  "categorySlug": "computers-electronics-and-technology/search-engines",
  "rankChange": 0,
  "avgVisitDuration": "00:10:00",
  "avgVisitDurationSeconds": 600,
  "pagesPerVisit": 8.68,
  "bounceRate": 28.4,
  "rankingList": "global",
  "url": "https://www.similarweb.com/top-websites/"
}

Fields: rank, domain, category, categorySlug, rankChange, avgVisitDuration, avgVisitDurationSeconds, pagesPerVisit, bounceRate, rankingList, url.

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('v0iddo/similarweb-scraper').call({"categorySlugs": ["", "computers-electronics-and-technology", "arts-and-entertainment"], "maxItems": 20});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

Full example: examples/node/run.mjs.

Any language: one HTTP request

curl -X POST "https://api.apify.com/v2/acts/v0iddo~similarweb-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" -H 'Content-Type: application/json' \
  -d '{"categorySlugs": ["", "computers-electronics-and-technology", "arts-and-entertainment"], "maxItems": 20}'

The response is the JSON array of results.

Cost

The Actor is paid per result: $1.70 per 1,000 results, nothing per run. maxItems caps the rows, and so the cost, of one run. A new Apify account gets $5 of platform credit every month.

License

MIT for the code in this repository. Made by vøiddo.