Real browsers. Structured extraction. Screenshots and automation.

JavaScript / TypeScript SDK

Turn a URL into rendered HTML, readable Markdown, structured data, or a PNG screenshot. When your workflow needs interaction, connect a remote browser with Playwright or Puppeteer.

Website · API documentation · Get an API key · 中文

• TypeScript types and editor autocomplete.
• Node.js 20+, ESM and CommonJS.
• Zero runtime dependencies; uses native fetch.
• Explicit deadlines, cancellation, and typed errors.
• No automatic retries of metered requests or browser creation.

Install

npm install adscrawl

Your first request

Create an account and create an API key in the dashboard. Set ADSCRAWL_API_KEY in your server environment, then:

Each example below includes its own imports and client initialization. Run examples as ES modules or in your TypeScript project; the CommonJS example uses require().

import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const markdown = await client.markdown({
  url: 'https://www.adscrawl.net',
  waitUntil: 'domcontentloaded',
});
console.log(markdown);

You can also pass new AdsCrawl({ apiKey: process.env.ADSCRAWL_API_KEY }). Use this SDK on your server; do not expose your API key in frontend bundles.

CommonJS:

const { AdsCrawl } = require('adscrawl');

const client = new AdsCrawl();

async function main() {
  const markdown = await client.markdown({
    url: 'https://www.adscrawl.net',
    waitUntil: 'domcontentloaded',
  });
  console.log(markdown);
}

main().catch((error) => {
  console.error(error);
  process.exitCode = 1;
});

Rendered content and screenshots

import { writeFile } from 'node:fs/promises';
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const html = await client.html({ url: 'https://www.adscrawl.net' });
console.log(html);
const article = await client.article({ url: 'https://www.adscrawl.net' });
console.log(article.title, article.textContent);

const png = await client.screenshot({
  url: 'https://www.adscrawl.net',
  viewport: { width: 1440, height: 900 },
  fullPage: true,
  waitUntil: 'load',
});
await writeFile('page.png', png);

html() returns a string for contentMode: 'html' (the default) or 'markdown', and an Article for 'json'. markdown() and article() are conveniences for those modes. screenshot() returns a Uint8Array containing a PNG.

Pass selector to extract an element or capture its screenshot. Page operations also accept locale, timezoneId, geolocation, cookies, fingerprint, and User-Agent settings. Use a custom proxy or managed countryCode, such as US or GLOBAL; they are mutually exclusive.

import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const markdown = await client.markdown({
  url: 'https://www.adscrawl.net',
  countryCode: 'US',
  userAgentMode: 'random',
  userAgentOs: 'windows',
});
console.log(markdown);

Proxy and fingerprint settings: BrowserScan screenshot

AdsCrawl renders pages in a real browser with configurable proxy routing and browser fingerprints. Randomized settings are generated as a coherent profile across the operating system, GPU, hardware, fonts, and related signals. This browser workflow has been verified to access and render BrowserScan, Pixelscan, and IPhey and return page screenshots, so you can inspect the simulated browser environment yourself.

This complete example opens BrowserScan and saves the returned PNG as browserscan.png. It works with ADSCRAWL_API_KEY and managed GLOBAL routing by default. To demonstrate your own proxy, set ADSCRAWL_PROXY_SERVER to an http:// or socks5:// proxy URL with an explicit port; optionally set both ADSCRAWL_PROXY_USERNAME and ADSCRAWL_PROXY_PASSWORD. Credentials stay in environment variables.

import { writeFile } from 'node:fs/promises';
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const server = process.env.ADSCRAWL_PROXY_SERVER;
const username = process.env.ADSCRAWL_PROXY_USERNAME;
const password = process.env.ADSCRAWL_PROXY_PASSWORD;
if (Boolean(username) !== Boolean(password)) {
  throw new Error('Set both proxy username and password, or neither.');
}
if (!server && (username || password)) {
  throw new Error('Set ADSCRAWL_PROXY_SERVER when supplying proxy credentials.');
}
const routing = server
  ? { proxy: username && password ? { server, username, password } : { server } }
  : { countryCode: 'GLOBAL' };

const png = await client.screenshot({
  url: 'https://www.browserscan.net/',
  ...routing,
  viewport: { width: 1440, height: 900 },
  fullPage: true,
  waitUntil: 'networkidle',
  timeoutMs: 60_000,
  userAgentMode: 'random',
  userAgentOs: 'windows',
  fingerprint: {
    webRtc: 'forward',
    webGl: 'random',
    webGpu: 'random',
    webGlImage: 'random',
    canvas: 'random',
    audioContext: 'random',
    clientRects: 'random',
    speechVoices: 'random',
    fonts: 'random',
    hardware: 'random',
    doNotTrack: 'random',
  },
}, { timeoutMs: 75_000 });

const output = 'browserscan.png';
await writeFile(output, png);
console.log(`Saved fingerprint-check screenshot to ${output}`);

webRtc: 'forward' uses the proxy exit address. random selects fingerprint settings; it does not blindly randomize each signal independently. Leave locale and timezone unset in this example instead of hardcoding a region that may differ from your proxy. Custom proxy and managed countryCode cannot be supplied together.

From this repository, run the same demo after setting your API key:

npm ci
npm run build
node examples/fingerprint.mjs

You can also capture the other diagnostic sites:

node examples/fingerprint.mjs https://pixelscan.net/ pixelscan.png
node examples/fingerprint.mjs https://iphey.com/ iphey.png

The demo shows real-browser access and screenshot rendering on fingerprint-checking sites. Open the saved PNG to inspect the simulated browser environment. Some sites require a click to start their full scan; use a CDP session when interaction is needed. Successful access and rendering are separate from the fingerprint score shown by the site.

Structured extraction

List available templates and their parameters:

import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const { templates } = await client.spa.templates();
console.log(templates);

const result = await client.spa.extract({
  template: 'google-trends-explore',
  keyword: 'playwright,puppeteer',
});
console.log(result.data);

For your own page, specify DOM or network fields. A generic describes the expected output; it does not validate your custom data at runtime. Always check missingFields.

import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const result = await client.spa.extract<{ title: string }>({
  url: 'https://www.adscrawl.net',
  fields: {
    title: { source: 'dom', selector: 'h1', value: 'text', required: true },
  },
});
console.log(result.data.title);
console.log(result.missingFields);

const inspection = await client.spa.inspect({ url: 'https://www.adscrawl.net' });
console.log(inspection.candidates);

Use actions for clicks, input, scrolling, and waits, and waitFor for a visible selector or text. See the API reference for template-specific requirements.

Remote browsers with Playwright

Install your preferred automation library separately:

npm install adscrawl playwright-core

import { chromium } from 'playwright-core';
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const session = await client.cdp.create({
  idleTimeoutMs: 600_000,
  maxSessionMs: 3_600_000,
  browserSettings: { viewport: { width: 1440, height: 900 } },
});

try {
  const browser = await chromium.connectOverCDP(session.cdpBaseUrl);
  const context = browser.contexts()[0] ?? await browser.newContext();
  const page = context.pages()[0] ?? await context.newPage();
  await page.goto('https://www.adscrawl.net');
  console.log(await page.title());
} finally {
  await client.cdp.close(session.sessionId);
}

The creation response contains sessionId, expiresAt, and cdpBaseUrl. It does not contain a WebSocket URL. For Puppeteer, use discovery:

npm install adscrawl puppeteer-core

import puppeteer from 'puppeteer-core';
import AdsCrawl from 'adscrawl';

const client = new AdsCrawl();
const session = await client.cdp.create();
try {
  const version = await client.cdp.getVersion(session);
  const browser = await puppeteer.connect({
    browserWSEndpoint: version.webSocketDebuggerUrl,
  });
  try {
    const pages = await browser.pages();
    const page = pages[0] ?? await browser.newPage();
    await page.goto('https://www.adscrawl.net');
    console.log(await page.title());
  } finally {
    browser.disconnect();
  }
} finally {
  await client.cdp.close(session.sessionId);
}

cdp.list() returns { ok, data }. cdp.liveToken(sessionId) returns a single-use live-control URL valid for 30 seconds. Treat all connection URLs as secrets and do not log them.

Persistent cloud browsers

Cloud browser profiles retain their configuration after stop. API-key starts require an explicit, top-level custom proxy on every start, even when a proxy was saved in the profile.

Set ADSCRAWL_PROXY_SERVER to your HTTP or SOCKS5 proxy URL with an explicit port. If authentication is needed, also set ADSCRAWL_PROXY_USERNAME and ADSCRAWL_PROXY_PASSWORD.

import { setTimeout as delay } from 'node:timers/promises';
import AdsCrawl, {
  AdsCrawlAPIError,
  AdsCrawlConnectionError,
  AdsCrawlTimeoutError,
} from 'adscrawl';

const client = new AdsCrawl();
const server = process.env.ADSCRAWL_PROXY_SERVER;
if (!server) throw new Error('Set ADSCRAWL_PROXY_SERVER to your proxy URL.');
const username = process.env.ADSCRAWL_PROXY_USERNAME;
const password = process.env.ADSCRAWL_PROXY_PASSWORD;
if (Boolean(username) !== Boolean(password)) {
  throw new Error('Set both proxy username and password, or neither.');
}
const proxy = username && password ? { server, username, password } : { server };

async function stopAndWait(id: string) {
  const end = Date.now() + 120_000;
  while (Date.now() < end) {
    try {
      const stopped = await client.cloudBrowsers.stop(id, {
        timeoutMs: Math.max(1, Math.min(10_000, end - Date.now())),
      });
      if (stopped.runtime.status === 'stopped') return;
    } catch (error) {
      const retryable = error instanceof AdsCrawlTimeoutError
        || error instanceof AdsCrawlConnectionError
        || (error instanceof AdsCrawlAPIError
          && (error.code === 'CDP_SESSION_STARTING' || error.status >= 500));
      if (!retryable) throw error;
    }
    const remaining = end - Date.now();
    if (remaining > 0) await delay(Math.min(2000, remaining));
  }
  throw new Error(`Stop unconfirmed for ${id}; inspect the profile and retry stop.`);
}

const { id } = await client.cloudBrowsers.create({ remark: 'My workflow' });
try {
  await client.cloudBrowsers.start(id, { proxy });
  const profile = await client.cloudBrowsers.get(id);
  console.log({ id: profile.id, status: profile.runtime.status });
  // profile.runtime.connectUrl opens the interactive browser in an
  // authenticated browser signed in as the profile owner.
} finally {
  await stopAndWait(id);
}

cloudBrowsers.launch({ proxy, tabs?, cookies?, fingerprint? }) creates a persistent profile and starts it in one request. Failed launches can return a profile id in AdsCrawlAPIError.id; inspect and stop that profile. If no id was received, inspect cloudBrowsers.list() before repeating the launch.

cloudBrowsers.list({ page?, pageSize? }) includes pagination and saved/running quotas. A stop response with runtime.status: 'stopping' means shutdown is pending and quota is still reserved. Closing the viewer does not stop billing. The SDK covers API-key endpoints; profile configuration PATCH/DELETE and viewer authentication require a dashboard session and are not included.

Configuration, deadlines, and cancellation

import AdsCrawl from 'adscrawl';

const client = new AdsCrawl({
  apiKey: process.env.ADSCRAWL_API_KEY,
  baseURL: 'https://api.adscrawl.net',
  timeoutMs: 90_000,
  // fetch: customFetch,
});

const controller = new AbortController();
const result = await client.markdown(
  { url: 'https://www.adscrawl.net', timeoutMs: 60_000 }, // Server task timeout.
  { timeoutMs: 75_000, signal: controller.signal }, // HTTP deadline / cancellation.
);
console.log(result);

The API key defaults to ADSCRAWL_API_KEY. The API origin defaults to ADSCRAWL_BASE_URL, then ADSCRAWL_API_URL, then https://api.adscrawl.net. HTTP deadlines cover both the request and reading its response. Ordinary calls default to 90 seconds; cloudBrowsers.launch() defaults to 195 seconds to allow startup and server cleanup unless you set a client or per-call deadline. For long tasks, set an HTTP deadline greater than the server task timeout. Cancellation or a deadline does not prove remote work stopped.

Errors

import AdsCrawl, { AdsCrawlAPIError, AdsCrawlTimeoutError } from 'adscrawl';

const client = new AdsCrawl();

try {
  const markdown = await client.markdown({ url: 'https://www.adscrawl.net' });
  console.log(markdown);
} catch (error) {
  if (error instanceof AdsCrawlAPIError) {
    console.error(error.status, error.code, error.traceId);
    // error.body contains the service response with credentials redacted.
  } else if (error instanceof AdsCrawlTimeoutError) {
    console.error('Request timed out. Inspect sessions/profiles before retrying browser creation.');
  } else {
    throw error;
  }
}

API errors preserve HTTP status, stable code, redacted body, traceId, and requestId when available. Network failures use AdsCrawlConnectionError; invalid/empty responses use AdsCrawlResponseError. Caller cancellation preserves the AbortSignal reason. Requests are never automatically retried. See official status/error descriptions.

Develop and release

npm ci
npm run check
npm pack --dry-run

Tests use fake credentials, mocked API responses, and a local HTTP server. They do not consume AdsCrawl credits. Runnable examples are in examples/. See RELEASING.md for first publish and GitHub Actions trusted publishing.

License

MIT