Cloudflare Turnstile solver for Crawl4AI

Solve Cloudflare Turnstile (and the 5-second challenge) inside Crawl4AI crawls. When Crawl4AI hits a Turnstile-protected page it stalls on the widget or gets a 403 — this shows the clean pattern: solve the token out-of-band via the Peak API, inject it, and let the crawl continue.

No headless-browser CAPTCHA plugin, no clicking. One POST /solve returns a valid cf-turnstile-response token in about a second; you inject it and Crawl4AI reads the unlocked page.

The problem

crawl4ai → https://target.example/  →  Cloudflare Turnstile  →  page never resolves / 403

Crawl4AI drives a real browser, but a real browser still can't pass Turnstile on a datacenter IP — the widget just spins. You need a valid token.

The fix (install)

pip install crawl4ai requests

Get a key at peak.fo (1,000 free solves, no card).

Working example

import asyncio, os, requests
from crawl4ai import AsyncWebCrawler, CrawlerRunConfig

PEAK_KEY = "pk_your_api_key"
PEAK_APP_ID = os.getenv("PEAK_APP_ID")  # optional, see "Earn with your app ID" below

def solve_turnstile(url: str, sitekey: str, proxy: str | None = None, app_id: str | None = None) -> str:
    """Return a valid cf-turnstile-response token from Peak."""
    body = {"url": url, "sitekey": sitekey}
    if proxy:
        body["task_type"] = "turnstiletask"
        body["proxy"] = proxy
    else:
        body["task_type"] = "TurnstileTaskProxyLess"  # Peak supplies the IP
    app = app_id or PEAK_APP_ID
    if app:
        body["appId"] = app  # optional revenue share; does not change the solve
    r = requests.post(
        "https://api.peak.fo/solve",
        headers={"X-API-Key": PEAK_KEY},
        json=body, timeout=60,
    )
    r.raise_for_status()
    return r.json()["data"]["token"]

async def crawl(url: str, sitekey: str, proxy: str | None = None, app_id: str | None = None):
    token = solve_turnstile(url, sitekey, proxy, app_id)

    # Inject the token into the widget's response field and fire its callback,
    # then let Crawl4AI extract the now-unlocked page.
    inject = f"""
    (() => {{
      const token = "{token}";
      document
        .querySelectorAll('[name="cf-turnstile-response"], #cf-turnstile-response, .cf-turnstile textarea')
        .forEach(el => {{ el.value = token; }});
      // Fire the site's success callback if it registered one via data-callback.
      const w = document.querySelector('.cf-turnstile');
      const cb = w && w.getAttribute('data-callback');
      if (cb && typeof window[cb] === 'function') window[cb](token);
    }})();
    """

    config = CrawlerRunConfig(js_code=[inject], wait_for="css:body")
    async with AsyncWebCrawler() as crawler:
        result = await crawler.arun(url=url, config=config)
        print(result.markdown[:500])
        return result

if __name__ == "__main__":
    # The sitekey is the data-sitekey attribute on the target page.
    asyncio.run(crawl("https://target.example/", "0x4AAAAAAA..."))

How it works

1. Read the sitekey — it's the data-sitekey on the .cf-turnstile element of the page you're crawling.
2. Solve out-of-band — POST https://api.peak.fo/solve with the sitekey + URL returns { "success": true, "data": { "token": "0.xxx..." }, "cost": 0.0009 }. Pay only for solves that land.
3. Inject + continue — set cf-turnstile-response and fire the site's data-callback, then Crawl4AI reads the unlocked content.

Proxyless mode (TurnstileTaskProxyLess) lets Peak supply the exit IP so you don't have to. For targets that bind the token to your IP, pass your own proxy with task_type: "turnstiletask".

Earn with your app ID

Pass your Peak app id and you earn 5% of every solve this tool makes, paid as solve credit. It is optional and only changes who gets credited — never the solve result or speed. Create an app id at peak.fo/dashboard/developer; details at peak.fo/earn.

# Set once via env: export PEAK_APP_ID=app_your_id ... or pass it explicitly:
token = solve_turnstile(url, sitekey, app_id="app_your_id")

Why Peak

• One synchronous call, token back in ~1s — no submit-then-poll.
• Pay per success — failed solves are free.
• Cloudflare specialist — Turnstile + the 5-second challenge, from $0.90/1k.
• Native SDKs for Python, Node, Go, Rust, C#, plus drop-in wrappers for Scrapy, Playwright, Selenium, cloudscraper, and curl_cffi.

Docs: peak.fo/docs/turnstile · Pricing: peak.fo

---

For legitimate automation, QA, and scraping of data you're authorized to access. Respect each site's Terms of Service and robots.txt.