Skip to content

Commit 3159edb

Browse files
s-JoLclaude
andcommitted
fix(gateway): add WebSocket auto-reconnect and RPC retry logic
WebSocket connections now automatically reconnect with exponential backoff on unexpected disconnects, and syncHistory catches up on missed events. RPC calls retry transient network errors (up to 2 attempts) while immediately surfacing auth/business errors via NonRetryableError. Also replaces the example teach skill with the latest teach output and updates all references. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 10416fe commit 3159edb

9 files changed

Lines changed: 283 additions & 237 deletions

File tree

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ Teach a task by demonstrating it once. Understudy learns the **intent**, not the
113113

114114
> *Demo flow: search for a photo → open in Pixelmator Pro → remove background → send via Telegram. Taught once, replayed autonomously.*
115115
116-
> *The published skill artifact from this demo is available at [examples/published-skills/taught-person-photo-cutout-bc88ec/SKILL.md](./examples/published-skills/taught-person-photo-cutout-bc88ec/SKILL.md).*
116+
> *The published skill artifact from this demo is available at [examples/published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md](./examples/published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md).*
117117
118118
### Example Workspaces
119119

@@ -172,7 +172,7 @@ The published SKILL.md is a three-layer abstraction: intent procedure (natural l
172172

173173
The draft/publish pipeline is no longer limited to one artifact shape. The current schema can publish `skill`, `worker`, and `playbook` workspace artifacts, although teach-by-demonstration most commonly produces reusable skills today.
174174

175-
A real published output example is available at [examples/published-skills/taught-person-photo-cutout-bc88ec/SKILL.md](./examples/published-skills/taught-person-photo-cutout-bc88ec/SKILL.md). It is kept under `examples/` on purpose, so the repo documents the artifact format without auto-loading it as a real workspace skill.
175+
A real published output example is available at [examples/published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md](./examples/published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md). It is kept under `examples/` on purpose, so the repo documents the artifact format without auto-loading it as a real workspace skill.
176176

177177
For the full teach pipeline, evidence pack construction, and validation details, see [Product Design](./docs/Product_Design.md).
178178

README.zh-CN.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ Understudy 使用你已有的消息应用:Telegram、Discord、Slack、WhatsAp
113113

114114
> *演示流程:搜索照片 → 在 Pixelmator Pro 中打开 → 去除背景 → 通过 Telegram 发送。教一次,自主重放。*
115115
116-
> *此演示生成的已发布 skill 产物:[examples/published-skills/taught-person-photo-cutout-bc88ec/SKILL.md](./examples/published-skills/taught-person-photo-cutout-bc88ec/SKILL.md)*
116+
> *此演示生成的已发布 skill 产物:[examples/published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md](./examples/published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md)*
117117
118118
### 示例工作区
119119

@@ -172,7 +172,7 @@ Planner 决定每一步使用哪条路线。一个任务可能浏览网站、运
172172

173173
Draft / publish 管线现在也不只支持一种产物。当前 schema 可以发布 `skill``worker``playbook` 这几类 workspace artifact,不过演示教学最常见的产物仍然是可复用 skill。
174174

175-
仓库里放了一个真实发布产物示例:[examples/published-skills/taught-person-photo-cutout-bc88ec/SKILL.md](./examples/published-skills/taught-person-photo-cutout-bc88ec/SKILL.md)。它故意放在 `examples/` 下,只用于展示产物格式,不会被当成真正的 workspace skill 自动加载。
175+
仓库里放了一个真实发布产物示例:[examples/published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md](./examples/published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md)。它故意放在 `examples/` 下,只用于展示产物格式,不会被当成真正的 workspace skill 自动加载。
176176

177177
完整的 teach 管线、证据包构建和验证细节见 [产品设计](./docs/Product_Design.zh-CN.md)
178178

apps/cli/src/commands/chat-gateway-session.ts

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -945,6 +945,8 @@ export async function createGatewayBackedInteractiveSession(
945945
};
946946

947947
const close = async () => {
948+
wsIntentionallyClosed = true;
949+
if (wsReconnectTimer) clearTimeout(wsReconnectTimer);
948950
gatewayState.isStreaming = false;
949951
gatewayState.currentTurn = undefined;
950952
gatewayState.socket?.removeAllListeners();
@@ -1291,6 +1293,10 @@ export async function createGatewayBackedInteractiveSession(
12911293
getGatewaySessionId: () => string | undefined;
12921294
};
12931295

1296+
let wsIntentionallyClosed = false;
1297+
let wsReconnectAttempt = 0;
1298+
let wsReconnectTimer: ReturnType<typeof setTimeout> | undefined;
1299+
12941300
const connectEventStream = () => {
12951301
const socketUrl = buildGatewaySocketUrl(
12961302
options.gatewayUrl,
@@ -1299,6 +1305,24 @@ export async function createGatewayBackedInteractiveSession(
12991305
try {
13001306
const socket = new WebSocket(socketUrl);
13011307
gatewayState.socket = socket;
1308+
1309+
socket.on("open", () => {
1310+
wsReconnectAttempt = 0;
1311+
// Catch up on any events missed while disconnected.
1312+
syncHistory().catch(() => {});
1313+
});
1314+
1315+
socket.on("close", () => {
1316+
if (wsIntentionallyClosed) return;
1317+
scheduleReconnect();
1318+
});
1319+
1320+
socket.on("error", () => {
1321+
// The "close" event always follows "error", so reconnect
1322+
// is handled there. Just make sure the socket is cleaned up.
1323+
try { socket.close(); } catch { /* already closing */ }
1324+
});
1325+
13021326
socket.on("message", (raw) => {
13031327
try {
13041328
const parsed = JSON.parse(raw.toString()) as { type?: string; data?: Record<string, unknown> };
@@ -1387,9 +1411,21 @@ export async function createGatewayBackedInteractiveSession(
13871411
});
13881412
} catch {
13891413
gatewayState.socket = null;
1414+
if (!wsIntentionallyClosed) scheduleReconnect();
13901415
}
13911416
};
13921417

1418+
const scheduleReconnect = () => {
1419+
if (wsIntentionallyClosed) return;
1420+
if (wsReconnectTimer) clearTimeout(wsReconnectTimer);
1421+
// Exponential backoff: 1s, 2s, 4s, capped at 10s.
1422+
const delay = Math.min(1000 * Math.pow(2, wsReconnectAttempt), 10_000);
1423+
wsReconnectAttempt++;
1424+
wsReconnectTimer = setTimeout(() => {
1425+
if (!wsIntentionallyClosed) connectEventStream();
1426+
}, delay);
1427+
};
1428+
13931429
try {
13941430
releaseSessionManagerOverride = registerGatewaySessionManagerOverride({
13951431
list: async (cwd, _sessionDir, onProgress) =>

apps/cli/src/rpc-client.ts

Lines changed: 72 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ export interface RpcClientOptions {
1414

1515
export interface RpcCallOptions {
1616
timeout?: number;
17+
/** Max retry attempts on transient network errors (default: 2). */
18+
retries?: number;
1719
}
1820

1921
export class GatewayRpcClient {
@@ -40,64 +42,95 @@ export class GatewayRpcClient {
4042
params: Record<string, unknown> = {},
4143
options: RpcCallOptions = {},
4244
): Promise<T> {
43-
const id = randomUUID();
44-
const headers: Record<string, string> = { "Content-Type": "application/json" };
45-
if (this.token) {
46-
headers["Authorization"] = `Bearer ${this.token}`;
47-
}
45+
const maxRetries = options.retries ?? 2;
46+
let lastError: unknown;
4847

49-
const controller = new AbortController();
50-
const timeoutMs = options.timeout ?? this.timeout;
51-
const timer = setTimeout(() => controller.abort(), timeoutMs);
48+
for (let attempt = 0; attempt <= maxRetries; attempt++) {
49+
const id = randomUUID();
50+
const headers: Record<string, string> = { "Content-Type": "application/json" };
51+
if (this.token) {
52+
headers["Authorization"] = `Bearer ${this.token}`;
53+
}
5254

53-
try {
54-
const response = await fetch(`${this.baseUrl}/rpc`, {
55-
method: "POST",
56-
headers,
57-
body: JSON.stringify({ id, method, params }),
58-
signal: controller.signal,
59-
});
55+
const controller = new AbortController();
56+
const timeoutMs = options.timeout ?? this.timeout;
57+
const timer = setTimeout(() => controller.abort(), timeoutMs);
6058

61-
if (response.status === 401) {
62-
throw new Error("Authentication failed. Check UNDERSTUDY_GATEWAY_TOKEN or --token.");
63-
}
64-
if (response.status === 429) {
65-
throw new Error("Rate limited. Try again later.");
66-
}
67-
if (!response.ok) {
68-
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
69-
}
59+
try {
60+
const response = await fetch(`${this.baseUrl}/rpc`, {
61+
method: "POST",
62+
headers,
63+
body: JSON.stringify({ id, method, params }),
64+
signal: controller.signal,
65+
});
66+
67+
if (response.status === 401) {
68+
throw new NonRetryableError("Authentication failed. Check UNDERSTUDY_GATEWAY_TOKEN or --token.");
69+
}
70+
if (response.status === 429) {
71+
throw new NonRetryableError("Rate limited. Try again later.");
72+
}
73+
if (!response.ok) {
74+
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
75+
}
7076

71-
const result = await response.json() as { id: string; result?: T; error?: { code: number; message: string } };
72-
if (result.error) {
73-
throw new Error(`RPC error ${result.error.code}: ${result.error.message}`);
77+
const result = await response.json() as { id: string; result?: T; error?: { code: number; message: string } };
78+
if (result.error) {
79+
throw new NonRetryableError(`RPC error ${result.error.code}: ${result.error.message}`);
80+
}
81+
return result.result as T;
82+
} catch (error) {
83+
if (error instanceof NonRetryableError) throw error;
84+
lastError = error;
85+
if (attempt < maxRetries) {
86+
await sleep(1000 * (attempt + 1));
87+
}
88+
} finally {
89+
clearTimeout(timer);
7490
}
75-
return result.result as T;
76-
} finally {
77-
clearTimeout(timer);
7891
}
92+
throw lastError;
7993
}
8094

8195
/** Fetch the health endpoint */
8296
async health(): Promise<Record<string, unknown>> {
83-
const controller = new AbortController();
84-
const timer = setTimeout(() => controller.abort(), this.timeout);
85-
try {
86-
const response = await fetch(`${this.baseUrl}/health`, { signal: controller.signal });
87-
if (!response.ok) {
88-
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
97+
return this.fetchWithRetry(`${this.baseUrl}/health`);
98+
}
99+
100+
private async fetchWithRetry<T>(url: string, retries = 2): Promise<T> {
101+
let lastError: unknown;
102+
for (let attempt = 0; attempt <= retries; attempt++) {
103+
const controller = new AbortController();
104+
const timer = setTimeout(() => controller.abort(), this.timeout);
105+
try {
106+
const response = await fetch(url, { signal: controller.signal });
107+
if (!response.ok) {
108+
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
109+
}
110+
return await response.json() as T;
111+
} catch (error) {
112+
lastError = error;
113+
if (attempt < retries) {
114+
await sleep(1000 * (attempt + 1));
115+
}
116+
} finally {
117+
clearTimeout(timer);
89118
}
90-
return await response.json() as Record<string, unknown>;
91-
} finally {
92-
clearTimeout(timer);
93119
}
120+
throw lastError;
94121
}
95122

96123
get url(): string {
97124
return this.baseUrl;
98125
}
99126
}
100127

128+
class NonRetryableError extends Error {}
129+
130+
function sleep(ms: number): Promise<void> {
131+
return new Promise((resolve) => setTimeout(resolve, ms));
132+
}
133+
101134
/** Create an RPC client with default config/env settings */
102135
export function createRpcClient(options?: RpcClientOptions): GatewayRpcClient {
103136
const port = options?.port ?? parseInt(process.env.UNDERSTUDY_GATEWAY_PORT ?? "23333", 10);

docs/index.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1100,7 +1100,7 @@ <h3>Show once, publish, replay later</h3>
11001100
Video has been sped up. Full-length original:
11011101
<a href="https://drive.google.com/file/d/1VJfodHJD_RSnb8g_vj2bId48tu_1vwHK/view?usp=sharing" target="_blank" rel="noreferrer">Google Drive</a>.
11021102
Published skill artifact:
1103-
<a href="https://github.com/understudy-ai/understudy/blob/main/examples/published-skills/taught-person-photo-cutout-bc88ec/SKILL.md" target="_blank" rel="noreferrer">SKILL.md</a>.
1103+
<a href="https://github.com/understudy-ai/understudy/blob/main/examples/published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md" target="_blank" rel="noreferrer">SKILL.md</a>.
11041104
</div>
11051105
</div>
11061106

docs/zh-CN/index.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1100,7 +1100,7 @@ <h3>演示一次,发布后再重放</h3>
11001100
以上视频经过加速。完整原速视频:
11011101
<a href="https://drive.google.com/file/d/1VJfodHJD_RSnb8g_vj2bId48tu_1vwHK/view?usp=sharing" target="_blank" rel="noreferrer">Google Drive</a>
11021102
已发布的技能产物:
1103-
<a href="https://github.com/understudy-ai/understudy/blob/main/examples/published-skills/taught-person-photo-cutout-bc88ec/SKILL.md" target="_blank" rel="noreferrer">SKILL.md</a>
1103+
<a href="https://github.com/understudy-ai/understudy/blob/main/examples/published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md" target="_blank" rel="noreferrer">SKILL.md</a>
11041104
</div>
11051105
</div>
11061106

examples/demo-teach/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Demonstrates Understudy's distinctive learn-from-demonstration stack: teach once
2020
## Published Skill Example
2121

2222
The published SKILL.md from this demo is available at:
23-
[`../published-skills/taught-person-photo-cutout-bc88ec/SKILL.md`](../published-skills/taught-person-photo-cutout-bc88ec/SKILL.md)
23+
[`../published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md`](../published-skills/taught-create-a-background-removed-portrait-for-a-requested-person-and-send-it-in-telegram-cd861a/SKILL.md)
2424

2525
## Prerequisites
2626

0 commit comments

Comments
 (0)