Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 13 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,39 +22,10 @@ The browser can run locally, on a remote server, or as an existing browser with
## Embedded experience

<div align="center">
<table width="100%">
<tr>
<td colspan="3"><strong>◉ BrowserKit</strong> <code>Agent Workspace</code></td>
<td align="right">● LIVE</td>
</tr>
<tr>
<td align="center">← ↻</td>
<td colspan="2"><code>https://example.com/workspace</code></td>
<td align="center"><strong>Take control</strong></td>
</tr>
<tr>
<td colspan="3" width="70%" valign="top">
<h3>Live browser surface</h3>
<p>Rendered by Chrome / Chromium and streamed over WebSocket.</p>
<blockquote>The agent is inspecting the page and interacting with a target…</blockquote>
<p><code>Mouse</code> <code>Touch</code> <code>Keyboard</code> <code>IME</code></p>
</td>
<td width="30%" valign="top">
<strong>Diagnostics</strong><br><br>
Console Network<br><br>
<code>GET /api/items 200</code><br>
<code>DOMContentLoaded</code><br>
<code>0 errors</code>
</td>
</tr>
<tr>
<td colspan="2">Tab: Workspace</td>
<td colspan="2" align="right">1920 × 1080 Controller connected</td>
</tr>
</table>
<img src="docs/images/browser-embedded.svg" width="100%" alt="BrowserView embedded in an application, with tabs, address bar, live page, and the F12 Network panel">
</div>

This repository-native HTML mockup represents the embedded layout. The actual `BrowserView` renders the live browser stream, tabs, address bar, input controls, and diagnostics directly where it is mounted.
`BrowserView` renders the live browser stream, tabs, address bar, input controls, and diagnostics directly inside the host application's container.

<a id="capabilities"></a>

Expand Down Expand Up @@ -119,7 +90,7 @@ Run build commands from the repository root and dependency installation commands

## 1. CLI and Skill

The installer installs both the client and the Agent Skill:
The installer builds the client and configures its Agent Skill. Codex is the default target:

```sh
./install.sh
Expand All @@ -128,8 +99,16 @@ browserkit --version
```

The default client directory is `~/.local/bin`.
The default Skill directory is `${CODEX_HOME:-~/.codex}/skills/browserkit-cli`.
Override them with `BROWSERKIT_BIN_DIR` and `BROWSERKIT_SKILL_DIR`.
The default Codex Skill directory is `${CODEX_HOME:-~/.codex}/skills/browserkit-cli`.

Configure the same Skill for Pi Agent, or for both agents:

```sh
./install.sh --agent pi
./install.sh --agent all
```

Pi installs the Skill under `${PI_CODING_AGENT_DIR:-~/.pi/agent}/skills/browserkit-cli`; start a new Pi session and use `/skill:browserkit-cli` to invoke it explicitly. Override destinations with `BROWSERKIT_BIN_DIR`, `BROWSERKIT_CODEX_SKILL_DIR`, or `BROWSERKIT_PI_SKILL_DIR`. The legacy `BROWSERKIT_SKILL_DIR` override remains supported for a selected target.
Agent usage instructions are in the [Skill](skills/browserkit-cli/SKILL.md).

### Attach to an existing browser
Expand Down
45 changes: 12 additions & 33 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,39 +21,10 @@ Agent 可通过 CLI 或 Go 函数读取页面、操作元素和管理 Tab;用
## 嵌入效果

<div align="center">
<table width="100%">
<tr>
<td colspan="3"><strong>◉ BrowserKit</strong> <code>Agent Workspace</code></td>
<td align="right">● LIVE</td>
</tr>
<tr>
<td align="center">← ↻</td>
<td colspan="2"><code>https://example.com/workspace</code></td>
<td align="center"><strong>接管操作</strong></td>
</tr>
<tr>
<td colspan="3" width="70%" valign="top">
<h3>浏览器实时画面</h3>
<p>由 Chrome / Chromium 渲染,通过 WebSocket 实时传输。</p>
<blockquote>Agent 正在检查页面内容并操作目标元素……</blockquote>
<p><code>鼠标</code> <code>触摸</code> <code>键盘</code> <code>中文输入法</code></p>
</td>
<td width="30%" valign="top">
<strong>开发诊断</strong><br><br>
Console Network<br><br>
<code>GET /api/items 200</code><br>
<code>DOMContentLoaded</code><br>
<code>0 errors</code>
</td>
</tr>
<tr>
<td colspan="2">Tab:Workspace</td>
<td colspan="2" align="right">1920 × 1080 控制方已连接</td>
</tr>
</table>
<img src="docs/images/browser-embedded.svg" width="100%" alt="BrowserView 嵌入应用后的效果,包含 Tab、地址栏、实时页面与 F12 Network 面板">
</div>

这是基于 README 原生 HTML 的界面示意;实际 `BrowserView` 会在调用位置嵌入浏览器图流、Tab、地址栏、输入控制与诊断面板。
`BrowserView` 会直接在宿主应用的容器内渲染浏览器图流、Tab、地址栏、输入控制与诊断面板。

<a id="capabilities"></a>

Expand Down Expand Up @@ -118,7 +89,7 @@ go mod download

## 1. CLI 与 Skill

安装脚本同时安装 client 和 Agent Skill:
安装脚本会构建 client 并配置 Agent Skill,默认目标为 Codex:

```sh
./install.sh
Expand All @@ -127,7 +98,15 @@ browserkit --version
```

默认 client 目录为 `~/.local/bin`,Skill 目录为 `${CODEX_HOME:-~/.codex}/skills/browserkit-cli`。
用 `BROWSERKIT_BIN_DIR` 和 `BROWSERKIT_SKILL_DIR` 修改安装位置。

配置给 Pi Agent,或同时配置给两个 Agent:

```sh
./install.sh --agent pi
./install.sh --agent all
```

Pi Skill 默认安装到 `${PI_CODING_AGENT_DIR:-~/.pi/agent}/skills/browserkit-cli`;新建 Pi 会话后可用 `/skill:browserkit-cli` 显式调用。使用 `BROWSERKIT_BIN_DIR`、`BROWSERKIT_CODEX_SKILL_DIR` 和 `BROWSERKIT_PI_SKILL_DIR` 修改对应位置;旧的 `BROWSERKIT_SKILL_DIR` 仍可覆盖所选目标的 Skill 目录。
Agent 操作约定见 [Skill](skills/browserkit-cli/SKILL.md)。

### 接管已有浏览器
Expand Down
11 changes: 11 additions & 0 deletions attach.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,17 @@ func attachPage(ctx context.Context, browser Session) (Page, error) {
if err != nil {
return nil, err
}
if detector, ok := browser.(ActiveTabSession); ok {
targetID, err := detector.ActiveTab(ctx, listed)
if err != nil {
return nil, err
}
page, err := tabs.PageForTab(ctx, targetID)
if err != nil {
return nil, fmt.Errorf("绑定激活 Tab: %w", err)
}
return page, nil
}
return activePage(ctx, tabs, listed)
}

Expand Down
37 changes: 36 additions & 1 deletion attach_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package browserkit
import (
"context"
"encoding/json"
"errors"
"testing"
)

Expand Down Expand Up @@ -103,12 +104,46 @@ func (p *focusPage) Evaluate(context.Context, string, EvaluateOptions) (Evaluate

type focusSession struct {
*fakeViewportSession
pages map[string]*focusPage
pages map[string]*focusPage
pageCalls []string
}

func (s *focusSession) PageForTab(_ context.Context, id string) (Page, error) {
s.pageCalls = append(s.pageCalls, id)
return s.pages[id], nil
}

type activeTargetSession struct {
*focusSession
activeTarget string
probeCalls int
}

func (s *activeTargetSession) ActiveTab(_ context.Context, listed []TabInfo) (string, error) {
s.probeCalls++
if len(listed) == 0 {
return "", errors.New("missing tabs")
}
return s.activeTarget, nil
}

func TestAttachBindsOnlyProbedActiveTab(t *testing.T) {
a := &focusPage{fakePage: &fakePage{info: PageInfo{URL: "https://example.com/a"}}}
b := &focusPage{fakePage: &fakePage{info: PageInfo{URL: "https://example.com/b"}}}
legacy := &focusSession{
fakeViewportSession: &fakeViewportSession{fakeSession: &fakeSession{}, tabs: []TabInfo{{TargetID: "a", Type: "page"}, {TargetID: "b", Type: "page"}}},
pages: map[string]*focusPage{"a": a, "b": b},
}
session := &activeTargetSession{focusSession: legacy, activeTarget: "b"}
page, err := attachPage(context.Background(), session)
if err != nil {
t.Fatal(err)
}
if page != b || session.probeCalls != 1 || len(session.pageCalls) != 1 || session.pageCalls[0] != "b" {
t.Fatalf("page=%p probes=%d bindings=%v", page, session.probeCalls, session.pageCalls)
}
}

func TestAttachActiveTab(t *testing.T) {
for _, tc := range []struct {
name string
Expand Down
166 changes: 166 additions & 0 deletions docs/images/browser-embedded.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading