Skip to content

fix(provider): 补充独立模型调用统计 - #9692

Open
RhoninSeiei wants to merge 8 commits into
AstrBotDevs:masterfrom
RhoninSeiei:fix/provider-stats-call-coverage
Open

fix(provider): 补充独立模型调用统计#9692
RhoninSeiei wants to merge 8 commits into
AstrBotDevs:masterfrom
RhoninSeiei:fix/provider-stats-call-coverage

Conversation

@RhoninSeiei

@RhoninSeiei RhoninSeiei commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

问题背景

AstrBot 原有统计由常规 Agent 执行过程写入 provider_stats。以下调用方式没有经过该统计过程:

  • 插件通过 Context.llm_generate() 发起的单次模型请求。
  • 插件通过 Context.tool_loop_agent() 创建的独立工具循环。
  • 定时任务创建的 Agent。

这些请求虽然能够正常返回,但 Dashboard 中的模型调用次数和 Token 总量会低于实际值。请求抛出异常、返回 role="err" 或被取消时,也可能缺少对应状态。启用后备 Provider 时,失败请求的 Token 还可能归到最终使用的 Provider,造成 Provider 归属和用量重复计算。

脱敏后的现象示例:

# 修改前:SDK 请求成功,但 provider_stats 中没有对应记录
agent_type=provider rows=0

# 修改后:请求状态和 Token 用量均有独立记录
agent_type=provider status=completed input_other=<n> input_cached=<n> output=<n>

修改内容

  • 为单次模型请求、独立工具循环和定时任务统一写入 Provider 调用统计。
  • 在成功、异常、错误响应和取消结束时记录相应状态。
  • 失败响应没有 Token 用量时,仍然保留零 Token 的失败记录和对应耗时。
  • 后备模型按实际参与请求的 Provider 分段保存 Token 用量,最终 Provider 只保存剩余用量。
  • Dashboard 汇总 internalprovider 两类调用,并且仅将 completed 计入成功次数。
  • 数据库参数只包含 input_otherinput_cachedoutput 三个公开 Token 字段。

修改后行为

  • SDK 和插件发起的独立模型请求会计入调用次数、Token 总量、响应时间和成功率。
  • 异常请求保存为 error,取消请求保存为 aborted
  • 返回 role="err" 的响应保留服务端返回的 Token 用量。
  • 后备模型的失败请求与最终成功请求分别归属实际 Provider,且不会重复计算 Token。

验证

  • 覆盖 SDK 成功、异常和错误响应统计。
  • 覆盖工具循环、定时任务、取消处理和后备模型分段统计。
  • 覆盖 Dashboard 对独立 Provider 调用的汇总。
  • 重点测试通过,Ruff 检查和 Python 语法检查通过。

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. labels Aug 14, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/provider/stats.py" line_range="82-91" />
<code_context>
+) -> None:
+    """Persist stats for one direct provider request."""
+    try:
+        usage = response.usage if response and response.usage else TokenUsage()
+        await db.insert_provider_stat(
+            umo=umo,
+            conversation_id=conversation_id,
+            provider_id=_provider_id(provider),
+            provider_model=provider.get_model(),
+            status=_response_status(response),
+            stats={
+                "token_usage": usage.__dict__.copy(),
+                "start_time": start_time,
+                "end_time": end_time,
+                "time_to_first_token": 0.0,
+            },
+            agent_type=agent_type,
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Clarify and stabilize the shape of stored token usage stats

Here `usage.__dict__.copy()` is persisted directly, so any future changes to `TokenUsage` (new internal fields, non-serializable attrs) will silently alter the stored schema or break serialization. To keep the stored shape stable and explicit, either expose and use a `to_dict()` on `TokenUsage`, or construct the dict from the specific fields you intend to persist (input/output/total tokens, etc.).

Suggested implementation:

```python
            stats={
                "token_usage": usage.to_dict(),
                "start_time": start_time,
                "end_time": end_time,
                "time_to_first_token": 0.0,
            },

```

You’ll need to implement a `to_dict()` method on the `TokenUsage` class (wherever it is defined) that returns a stable, explicit shape, e.g.:

- Only include the fields you intend to persist (for example: `input_tokens`, `output_tokens`, `total_tokens`, `cached_tokens`, etc.).
- Avoid non-serializable attributes or internal fields that might change over time.

For example:

```python
@dataclass
class TokenUsage:
    prompt_tokens: int = 0
    completion_tokens: int = 0
    total_tokens: int = 0

    def to_dict(self) -> dict[str, int]:
        return {
            "prompt_tokens": self.prompt_tokens,
            "completion_tokens": self.completion_tokens,
            "total_tokens": self.total_tokens,
        }
```

Adjust the exact fields to match your current `TokenUsage` definition and what you want to persist long-term.
</issue_to_address>

### Comment 2
<location path="tests/unit/test_star_context.py" line_range="150-159" />
<code_context>
         assert config.provider_settings is provider_settings
         assert config.provider_settings["fallback_chat_models"] == ["fallback-provider"]

+    @pytest.mark.asyncio
+    async def test_woke_main_agent_persists_one_aggregated_provider_stat(
+        self,
</code_context>
<issue_to_address>
**suggestion (testing):** Add tests for error scenarios in `llm_generate` to validate `ProviderStat.status` and usage when the provider fails or returns an error response.

Since stats are recorded in a `finally` block and status is derived from the LLM response, it would be good to cover non-happy paths too. Please add at least:

1. A case where `StatsProvider.text_chat` raises (e.g. `RuntimeError`), asserting via `pytest.raises` that:
   - the exception is propagated,
   - a `ProviderStat` is still persisted,
   - it has `status == "error"` and zero (or default) token usage.

2. A case where `StatsProvider.text_chat` returns an `LLMResponse` with `role="err"`, asserting that:
   - `llm_generate` returns this response,
   - the `ProviderStat` has `status == "error"` while preserving the usage values.

This will help ensure stats remain correct for failure and error-response scenarios, which downstream reporting depends on.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread astrbot/core/provider/stats.py
Comment thread tests/unit/test_star_context.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant