Skip to content

Commit 21f5804

Browse files
leeyyileeyyi
committed
FEAT: 在模型请求失败时能够自动重试
1 parent 32d7af9 commit 21f5804

21 files changed

Lines changed: 1367 additions & 113 deletions

‎examples/llmagent_with_model_retry/.env‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Copy this file or edit it in place before running the example.
2+
# The example uses an OpenAI-compatible endpoint.
3+
4+
TRPC_AGENT_API_KEY=
5+
TRPC_AGENT_BASE_URL=
6+
TRPC_AGENT_MODEL_NAME=
7+
8+
# Optional model retry tuning.
9+
# num_retries is the number of retry attempts in addition to the initial call.
10+
TRPC_AGENT_MODEL_RETRY_NUM_RETRIES=2
11+
TRPC_AGENT_MODEL_RETRY_INITIAL_BACKOFF=1.0
12+
TRPC_AGENT_MODEL_RETRY_MAX_BACKOFF=8.0
13+
TRPC_AGENT_MODEL_RETRY_BACKOFF_MULTIPLIER=2.0
14+
TRPC_AGENT_MODEL_RETRY_JITTER=true
15+
TRPC_AGENT_MODEL_RETRY_RESPECT_RETRY_AFTER=true

‎examples/llmagent_with_model_retry/README.md‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
# LLM Agent 模型重试示例
2+
3+
本示例演示如何在模型构造时传入 `ModelRetryConfig`，让业务代码不用自己实现重试逻辑；当 LLM 请求遇到限流、超时、网络波动等瞬时错误时，SDK 会自动重试。
4+
5+
## 关键特性
6+
7+
- **按需开启重试**：只有显式传入 `ModelRetryConfig` 的模型才会启用 SDK 托管重试。
8+
- **一套配置，多种模型可用**：同一个 `ModelRetryConfig` 可以传给 OpenAI、Anthropic 等模型实现，统一控制重试行为。
9+
- **只重试临时问题**：限流、超时、服务端内部错误、连接异常会在次数限制内重试。
10+
- **避免重复输出内容**：流式输出已经产生内容后如果再失败，错误会直接透出，不会重试并重复输出内容。
11+
- **优先使用服务端等待时间**：默认会读取 `Retry-After` / `retry-after-ms`，有服务端等待时间时优先使用。
12+
13+
## Agent 层级结构说明
14+
15+
本例是单 Agent 示例，重试配置绑定在模型上：
16+
17+
```text
18+
weather_retry_agent (LlmAgent)
19+
├── model: OpenAIModel(..., model_retry_config=ModelRetryConfig(...))
20+
├── tool: get_weather_report(city)
21+
└── runner: 无自定义重试逻辑
22+
```
23+
24+
关键文件：
25+
26+
- [examples/llmagent_with_model_retry/agent/agent.py](./agent/agent.py)：创建模型并注入 `ModelRetryConfig`
27+
- [examples/llmagent_with_model_retry/agent/config.py](./agent/config.py)：读取模型连接与重试环境变量
28+
- [examples/llmagent_with_model_retry/agent/tools.py](./agent/tools.py)：天气工具实现
29+
- [examples/llmagent_with_model_retry/agent/prompts.py](./agent/prompts.py)：提示词
30+
- [examples/llmagent_with_model_retry/run_agent.py](./run_agent.py)：运行入口，展示业务层无需手写重试
31+
32+
## 关键代码解释
33+
34+
### 1) 创建重试配置
35+
36+
`agent/config.py` 从环境变量构造：
37+
38+
```python
39+
from trpc_agent_sdk.configs import ExponentialBackoffConfig
40+
from trpc_agent_sdk.configs import ModelRetryConfig
41+
42+
ModelRetryConfig(
43+
num_retries=2,
44+
backoff=ExponentialBackoffConfig(
45+
initial_backoff=1.0,
46+
max_backoff=8.0,
47+
multiplier=2.0,
48+
jitter=True,
49+
respect_retry_after=True,
50+
),
51+
)
52+
```
53+
54+
### 2) 注入模型
55+
56+
`agent/agent.py` 将配置传给模型构造器：
57+
58+
```python
59+
OpenAIModel(
60+
model_name=model_name,
61+
api_key=api_key,
62+
base_url=base_url,
63+
model_retry_config=retry_config,
64+
)
65+
```
66+
67+
### 3) Runner 不需要重试逻辑
68+
69+
`run_agent.py` 仍然只调用：
70+
71+
```python
72+
async for event in runner.run_async(...):
73+
...
74+
```
75+
76+
如果模型调用在产出内容前遇到可重试错误，SDK 会按 `ModelRetryConfig` 自动重试。
77+
78+
## 会重试和不会重试的场景
79+
80+
### 会重试
81+
82+
- `429` 限流
83+
- `408` / `409` 超时或模型服务锁等待超时
84+
- `5xx` 服务端内部错误
85+
- 超时异常
86+
- 连接或传输异常
87+
88+
### 不会重试
89+
90+
- `400` 错误请求
91+
- `401` / `403` 认证或权限错误
92+
- 未识别错误类别
93+
- 重试次数已耗尽
94+
- 流式输出已经产生内容后才发生的错误
95+
96+
## 环境与运行
97+
98+
### 环境要求
99+
100+
- Python 3.10+
101+
102+
### 安装步骤
103+
104+
```bash
105+
git clone https://github.com/trpc-group/trpc-agent-python.git
106+
cd trpc-agent-python
107+
python3 -m venv .venv
108+
source .venv/bin/activate
109+
pip3 install -e .
110+
```
111+
112+
### 环境变量要求
113+
114+
在 [examples/llmagent_with_model_retry/.env](./.env) 中配置（或通过 `export` 设置）：
115+
116+
```bash
117+
# 必填：模型连接配置
118+
TRPC_AGENT_API_KEY=<your-api-key>
119+
TRPC_AGENT_BASE_URL=<openai-compatible-base-url>
120+
TRPC_AGENT_MODEL_NAME=<model-name>
121+
122+
# 可选：模型重试配置；不设置时使用示例默认值
123+
TRPC_AGENT_MODEL_RETRY_NUM_RETRIES=2
124+
TRPC_AGENT_MODEL_RETRY_INITIAL_BACKOFF=1.0
125+
TRPC_AGENT_MODEL_RETRY_MAX_BACKOFF=8.0
126+
TRPC_AGENT_MODEL_RETRY_BACKOFF_MULTIPLIER=2.0
127+
TRPC_AGENT_MODEL_RETRY_JITTER=true
128+
TRPC_AGENT_MODEL_RETRY_RESPECT_RETRY_AFTER=true
129+
```
130+
131+
### 运行命令
132+
133+
```bash
134+
cd examples/llmagent_with_model_retry
135+
python3 run_agent.py
136+
```
137+
138+
## 运行结果示例
139+
140+
```text
141+
Model retry enabled: {'num_retries': 2, 'rules': {'retryable_error_codes': ['408', '409', '429', '500', '502', '503', '504'], 'non_retryable_error_codes': ['400', '401', '403'], 'retryable_exception_class_name_parts': ['Timeout', 'Connection', 'Transport']}, 'backoff': {'type': 'exponential', 'initial_backoff': 1.0, 'max_backoff': 8.0, 'multiplier': 2.0, 'jitter': True, 'respect_retry_after': True}}
142+
Session ID: fdb9e370...
143+
SDK-managed retry is configured on the model; no retry loop is needed in this runner.
144+
User: What's the current weather in Beijing?
145+
Assistant:
146+
Invoke Tool: get_weather_report({'city': 'Beijing'})
147+
Tool Result: {'temperature': '25°C', 'condition': 'Sunny', 'humidity': '60%'}
148+
149+
# 当模型服务返回可重试错误时，SDK 会在模型层自动重试：
150+
[WARNING] Model call failed (category=rate_limit); retrying in 0.46s (attempt 1/2).
151+
```
152+
153+
## 适用场景建议
154+
155+
- 在模型服务限流，或其他瞬时错误时自动重试
156+
- 需要结合服务端 `Retry-After` 控制退避等待时间。

‎examples/llmagent_with_model_retry/agent/__init__.py‎

Whitespace-only changes.

‎examples/llmagent_with_model_retry/agent/agent.py‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Agent module for the model retry example."""
7+
8+
from trpc_agent_sdk.agents import LlmAgent
9+
from trpc_agent_sdk.models import LLMModel
10+
from trpc_agent_sdk.models import OpenAIModel
11+
from trpc_agent_sdk.tools import FunctionTool
12+
13+
from .config import get_model_config
14+
from .config import get_model_retry_config
15+
from .prompts import INSTRUCTION
16+
from .tools import get_weather_report
17+
18+
19+
def _create_model() -> LLMModel:
20+
"""Create an OpenAI-compatible model with SDK-managed retry enabled."""
21+
api_key, base_url, model_name = get_model_config()
22+
retry_config = get_model_retry_config()
23+
print(f"Model retry enabled: {retry_config.model_dump()}")
24+
return OpenAIModel(
25+
model_name=model_name,
26+
api_key=api_key,
27+
base_url=base_url,
28+
model_retry_config=retry_config,
29+
)
30+
31+
32+
def create_agent() -> LlmAgent:
33+
"""Create a weather agent that uses model-level retry."""
34+
return LlmAgent(
35+
name="weather_retry_agent",
36+
description="A weather assistant with SDK-managed model retry enabled.",
37+
model=_create_model(),
38+
instruction=INSTRUCTION,
39+
tools=[FunctionTool(get_weather_report)],
40+
)
41+
42+
43+
root_agent = create_agent()

‎examples/llmagent_with_model_retry/agent/config.py‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Configuration helpers for the model retry example."""
7+
8+
import os
9+
10+
from trpc_agent_sdk.configs import ExponentialBackoffConfig
11+
from trpc_agent_sdk.configs import ModelRetryConfig
12+
13+
14+
def _get_bool(name: str, default: bool) -> bool:
15+
value = os.getenv(name)
16+
if value is None or value == "":
17+
return default
18+
return value.strip().lower() in {"1", "true", "yes", "y", "on"}
19+
20+
21+
def _get_float(name: str, default: float) -> float:
22+
value = os.getenv(name)
23+
if value is None or value == "":
24+
return default
25+
return float(value)
26+
27+
28+
def _get_int(name: str, default: int) -> int:
29+
value = os.getenv(name)
30+
if value is None or value == "":
31+
return default
32+
return int(value)
33+
34+
35+
def get_model_config() -> tuple[str, str, str]:
36+
"""Get model connection settings from environment variables."""
37+
api_key = os.getenv("TRPC_AGENT_API_KEY", "")
38+
base_url = os.getenv("TRPC_AGENT_BASE_URL", "")
39+
model_name = os.getenv("TRPC_AGENT_MODEL_NAME", "")
40+
if not api_key or not base_url or not model_name:
41+
raise ValueError("TRPC_AGENT_API_KEY, TRPC_AGENT_BASE_URL, and "
42+
"TRPC_AGENT_MODEL_NAME must be set in environment variables")
43+
return api_key, base_url, model_name
44+
45+
46+
def get_model_retry_config() -> ModelRetryConfig:
47+
"""Build the opt-in SDK-managed model retry configuration."""
48+
return ModelRetryConfig(
49+
num_retries=_get_int("TRPC_AGENT_MODEL_RETRY_NUM_RETRIES", 2),
50+
backoff=ExponentialBackoffConfig(
51+
initial_backoff=_get_float("TRPC_AGENT_MODEL_RETRY_INITIAL_BACKOFF", 1.0),
52+
max_backoff=_get_float("TRPC_AGENT_MODEL_RETRY_MAX_BACKOFF", 8.0),
53+
multiplier=_get_float("TRPC_AGENT_MODEL_RETRY_BACKOFF_MULTIPLIER", 2.0),
54+
jitter=_get_bool("TRPC_AGENT_MODEL_RETRY_JITTER", True),
55+
respect_retry_after=_get_bool("TRPC_AGENT_MODEL_RETRY_RESPECT_RETRY_AFTER", True),
56+
),
57+
)

‎examples/llmagent_with_model_retry/agent/prompts.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Prompts for the model retry example agent."""
7+
8+
INSTRUCTION = """You are a practical weather assistant.
9+
10+
When the user asks for weather, identify the city and call get_weather_report.
11+
If the city is missing, ask one short clarification question.
12+
After receiving tool results, summarize the weather clearly and mention the retry configuration only if the user asks about reliability.
13+
"""

‎examples/llmagent_with_model_retry/agent/tools.py‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
2+
#
3+
# Copyright (C) 2026 Tencent. All rights reserved.
4+
#
5+
# tRPC-Agent-Python is licensed under Apache-2.0.
6+
"""Tools for the model retry example agent."""
7+
8+
9+
def get_weather_report(city: str) -> dict:
10+
"""Get weather information for the specified city."""
11+
weather_data = {
12+
"Beijing": {
13+
"temperature": "25°C",
14+
"condition": "Sunny",
15+
"humidity": "60%",
16+
},
17+
"Shanghai": {
18+
"temperature": "28°C",
19+
"condition": "Cloudy",
20+
"humidity": "70%",
21+
},
22+
"Guangzhou": {
23+
"temperature": "32°C",
24+
"condition": "Thunderstorm",
25+
"humidity": "85%",
26+
},
27+
"Shenzhen": {
28+
"temperature": "30°C",
29+
"condition": "Light rain",
30+
"humidity": "78%",
31+
},
32+
}
33+
return weather_data.get(
34+
city,
35+
{
36+
"temperature": "Unknown",
37+
"condition": "Data not available",
38+
"humidity": "Unknown",
39+
},
40+
)

0 commit comments

Comments
 (0)