Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions box/guides/web-scraping-playwright.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ title: "Scrape Dynamic Websites with Playwright"

In this guide, we use Upstash Box to run [Playwright](https://playwright.dev) against a JavaScript-heavy site, scrape structured data from it, and pull the results back to our own server. Because a Box is a real Linux container rather than a restricted serverless runtime, Chromium and its system dependencies install and run exactly like they would on your laptop.

<Note>
Boxes also ship a managed, pre-installed browser. Create one with `browser:
true` and drive Chromium through the SDK, or [connect Playwright over
CDP](/box/overall/browser/connect) with no install step. This guide shows
the manual route, where the box's agent installs and controls Playwright
itself. See [Browser](/box/overall/browser/overview) for the built-in one.
Comment on lines +7 to +12
</Note>

---

## 1. Installation
Expand Down
113 changes: 113 additions & 0 deletions box/overall/browser/ai-actions.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
---
title: "AI Actions"
---

Beyond reading pages, a tab can act. A DOM-aware browser agent runs inside the box and resolves natural-language instructions against the live page. It can find elements, execute single actions, or complete multi-step tasks on its own.

<Note>
AI actions use an LLM and are metered. They need an API key for the model's
provider (Anthropic, OpenAI, OpenRouter, Vercel, or OpenCode) on the box or
your account. Every method accepts a provider-prefixed `model` override such
as `"openai/gpt-4o"`. Without an override, the call uses the model the box
was configured with. If the box has no model, it falls back to
`anthropic/claude-sonnet-4-5`.
</Note>

## Observe

`observe()` finds actionable elements matching an instruction. Use it to check the page before acting, or to build your own action loop:

<CodeGroup>
```typescript box.ts
const { elements } = await tab.observe("find the login and signup buttons")

for (const el of elements) {
console.log(el.description, el.selector)
}
```

```python box.py
result = tab.observe("find the login and signup buttons")

for el in result.elements:
print(el.description, el.selector)
```
</CodeGroup>

## Act

`act()` resolves and executes exactly one action described in natural language:

<CodeGroup>
```typescript box.ts
const action = await tab.act("click the primary call-to-action")

console.log(action.success, action.actionDescription)
console.log(action.inputTokens, action.outputTokens)
```

```python box.py
action = tab.act("click the primary call-to-action")

print(action.success, action.action_description)
print(action.input_tokens, action.output_tokens)
```
</CodeGroup>

The result reports what was done (`actions` with the resolved selectors), whether it succeeded, and the token usage of the call.

## Run

`run()` is the autonomous mode. The agent reads the page, acts, and repeats until the task is complete or it hits the step limit. Pass a schema to get structured data back at the end:

<CodeGroup>
```typescript box.ts
import { z } from "zod"

const { data, completed, steps } = await tab.run(
"Find the pricing page and summarize the free tier",
{
schema: z.object({ summary: z.string() }),
maxSteps: 15,
// model: "openai/gpt-4o", // any provider you hold a key for
},
)

console.log(completed, data.summary)
for (const step of steps) {
console.log(step.step, step.action, step.url)
}
```

```python box.py
from pydantic import BaseModel

class Summary(BaseModel):
summary: str

result = tab.run(
"Find the pricing page and summarize the free tier",
schema=Summary,
max_steps=15,
# model="openai/gpt-4o", # any provider you hold a key for
)

print(result.completed, result.data.summary)
for step in result.steps:
print(step.step, step.action, step.url)
```
</CodeGroup>

- `maxSteps`: defaults to `15`, capped at `30`.
- `schema`: optional. Without it, `run` returns its findings as text in `result`.
- The result includes `completed`, a step-by-step trace in `steps` (each with the action taken, its reasoning, and the URL), and total token usage.

## Which one to use

| Method | Does | Best for |
|---|---|---|
| `observe` | Finds elements, executes nothing | Inspecting a page, building custom loops |
| `act` | Executes one action | Flows where your code decides each step |
| `run` | Executes a whole task autonomously | Open-ended or navigation-heavy tasks |
Comment on lines +107 to +111

For fully scripted control with no LLM in the loop, [connect over CDP](/box/overall/browser/connect) with Playwright or Puppeteer instead. Both drive the same tabs, so you can mix scripted steps with AI steps. To watch or replay what the agent did, see [Live View](/box/overall/browser/live-view) and [Recordings](/box/overall/browser/recordings).
81 changes: 81 additions & 0 deletions box/overall/browser/connect.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
---
title: "Connect over CDP"
---

The box browser is a real Chromium, and you can drive it with the tools you already use. `cdpUrl()` returns an authenticated Chrome DevTools Protocol WebSocket URL that Playwright, Puppeteer, or Stagehand can connect to directly. There is no browser to install and nothing to manage.

<CodeGroup>
```typescript box.ts
const cdpUrl = await box.browser.cdpUrl()
// wss://…?token=…
```

```python box.py
cdp_url = box.browser.cdp_url()
# wss://…?token=…
```
</CodeGroup>

Like [live view](/box/overall/browser/live-view) URLs, the CDP URL carries its auth token in the URL. Anyone who has it gets full control of the browser, so treat it as a secret.

## Playwright

`playwright-core` is enough here, since you connect to the box's Chromium instead of launching one locally:

<CodeGroup>
```typescript scrape.ts
import { chromium } from "playwright-core"

const browser = await chromium.connectOverCDP(cdpUrl)

const context = browser.contexts()[0] ?? (await browser.newContext())
const page = context.pages()[0] ?? (await context.newPage())

await page.goto("https://upstash.com")
console.log(await page.title())
```

```python scrape.py
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
browser = p.chromium.connect_over_cdp(cdp_url)
context = browser.contexts[0] if browser.contexts else browser.new_context()
page = context.pages[0] if context.pages else context.new_page()
page.goto("https://upstash.com")
print(page.title())
```
</CodeGroup>

## Puppeteer

```typescript scrape.ts
import puppeteer from "puppeteer-core"

const browser = await puppeteer.connect({ browserWSEndpoint: cdpUrl })

const page = (await browser.pages())[0] ?? (await browser.newPage())
await page.goto("https://upstash.com")
```

## Stagehand

[Stagehand](https://www.stagehand.dev) can use the box browser as its local browser:

```typescript agent.ts
import { Stagehand } from "@browserbasehq/stagehand"

const stagehand = new Stagehand({
env: "LOCAL",
localBrowserLaunchOptions: { cdpUrl },
})

await stagehand.init()
await stagehand.act("click the first link")
```

## Mixing CDP and SDK control

CDP clients and the SDK drive the same browser and the same tabs. A page opened by Playwright shows up in `box.browser.listTabs()`, and a tab created by the SDK is visible to Playwright. You can script the predictable steps like login and pagination with Playwright, hand the tab to [`act` or `run`](/box/overall/browser/ai-actions) for the steps that are easier to describe in natural language, and watch either through [Live View](/box/overall/browser/live-view).

As a rule of thumb: use CDP when you want precise, repeatable scripting with no LLM in the loop. Use [AI Actions](/box/overall/browser/ai-actions) when describing the task is easier than scripting it.
48 changes: 48 additions & 0 deletions box/overall/browser/live-view.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
title: "Live View"
---

Every tab can produce a live view URL: a shareable link that streams the tab in real time. Open it in any browser, or embed it in an `<iframe>` to show your users what their agent is doing.

<CodeGroup>
```typescript box.ts
const liveUrl = await tab.liveViewUrl()

console.log(liveUrl)
// https://<box>-9223.<domain>/screencast?token=…&tab=…
```

```python box.py
live_url = tab.live_view_url()

print(live_url)
# https://<box>-9223.<domain>/screencast?token=…&tab=…
```
</CodeGroup>

The URL is self-contained. Authentication is a token embedded in the URL itself, so it works anywhere a browser can load a page. The viewing side needs no API key and no SDK.

## Embedding

```html
<iframe
src="https://<box>-9223.<domain>/screencast?token=…&tab=…"
style="width: 100%; aspect-ratio: 16 / 10; border: 0;"
></iframe>
```

A common pattern is to start a [`tab.run()`](/box/overall/browser/ai-actions) task and render the live view next to it, so users can watch the agent work in real time.

## View-only

The live view streams frames out only. Viewers cannot click, type, or otherwise interact with the page. To drive the browser, use the SDK or [connect over CDP](/box/overall/browser/connect).

If you need an interactive viewer inside your own product, you can build one over CDP by streaming screencast frames out and sending input events back. This is how the console's browser tab works.

<Warning>
Anyone with the URL can watch the tab, since the access token is part of the
URL. Treat live view URLs like secrets and share them only with people who
should see the session.
</Warning>

For an interactive version of the same view, open the **Browser** tab on your box's page in the [Upstash Console](https://console.upstash.com). If you want a replayable video instead of a live stream, see [Recordings](/box/overall/browser/recordings).
88 changes: 88 additions & 0 deletions box/overall/browser/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
---
title: "Browser"
---

**Every box can come with its own browser.** Create a box with `browser: true` to get a managed, headless Chromium that you control through the SDK. You can open tabs, read pages, take screenshots, extract structured data, run AI agents on the live DOM, record sessions, and connect Playwright directly over CDP.

Everything works headless. There is no desktop, no VNC, and nothing to install. Chromium is provisioned with the box and boots on first use.

## Create a box with a browser

<CodeGroup>
```typescript box.ts
import { Box } from "@upstash/box"

const box = await Box.create({
runtime: "node",
browser: true,
})

// Open a tab (boots Chromium on first use)
const tab = await box.browser.tab.create("https://news.ycombinator.com")

// Navigate the same tab and read the result
const page = await tab.goto("https://upstash.com/docs")
console.log(page.title)
```

```python box.py
from upstash_box import Box

box = Box.create(runtime="node", browser=True)

# Open a tab (boots Chromium on first use)
tab = box.browser.tab.create("https://news.ycombinator.com")

# Navigate the same tab and read the result
page = tab.goto("https://upstash.com/docs")
print(page.title)
```
</CodeGroup>

<Note>
The browser can only be provisioned when the box is created. It cannot be
enabled on an existing box. If you need a browser on a box that does not have
one, create a new box with `browser: true`.
</Note>

## How it's organized

`box.browser` manages the browser itself: opening and listing tabs, recordings, and the CDP endpoint. Page-level operations live on a **Tab** handle, the object returned by `tab.create`, `listTabs`, or `getTab`. A tab handle is addressed by its Chrome DevTools Protocol target id, so it stays valid across navigations. You navigate, read, screenshot, and run AI tasks through the same handle.

## What you can do

<CardGroup cols={2}>
<Card title="Tabs & Navigation" href="/box/overall/browser/tabs">
Open tabs, navigate, list and re-attach to them, and close them.
</Card>

<Card title="Reading Pages" href="/box/overall/browser/reading-pages">
Page text and links, PNG screenshots, and schema-validated data extraction.
</Card>

<Card title="AI Actions" href="/box/overall/browser/ai-actions">
Natural-language actions and autonomous multi-step tasks on the live DOM.
</Card>

<Card title="Live View" href="/box/overall/browser/live-view">
A shareable, view-only live stream of any tab that you can embed in an iframe.
</Card>

<Card title="Recordings" href="/box/overall/browser/recordings">
Record browser sessions to replayable video with chapter markers.
</Card>

<Card title="Connect over CDP" href="/box/overall/browser/connect">
Drive the same browser with Playwright, Puppeteer, or Stagehand.
</Card>
</CardGroup>

<Note>
The AI-powered operations use an LLM and are metered:
[`extract`](/box/overall/browser/reading-pages) and [`observe`, `act`,
`run`](/box/overall/browser/ai-actions). They need an API key for the model's
provider (Anthropic, OpenAI, OpenRouter, Vercel, or OpenCode) on the box or
your account.
</Note>

You can also watch and control the browser from the **Browser** tab on your box's page in the [Upstash Console](https://console.upstash.com). It shows the live view, runs AI tasks, and includes the SDK snippet for everything you do there.
Loading