Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
148 changes: 148 additions & 0 deletions examples/tool_safety/DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# Tool Script Safety Guard 设计说明

## 目标与边界

Safety Guard 是执行前治理层。它对脚本、命令、参数、工作目录、环境变量名称和 tool 元数据做静态检查,输出 `allow`、`deny` 或 `needs_human_review`。它不能替代进程隔离、只读文件系统、最小权限、网络出口控制和运行时资源限制。

数据流如下:

```text
Tool / MCP Tool / Skill / CodeExecutor 请求
|
v
SafetyScanRequest(不保留环境变量值)
|
v
ToolSafetyScanner + ToolSafetyPolicy + SafetyRule[]
|
v
SafetyFinding[] --聚合--> SafetyReport
| |
v v
JSONL AuditEvent Filter / Guard 执行决策
|
v
OpenTelemetry tool.safety.* 属性
```

扫描发生在真实 handler 或 executor delegate 之前。`deny` 必须阻断;`needs_human_review` 是否阻断由 `block_on_review` 决定。人工批准应由上层审批系统显式记录,不能把 review 自动降级为 allow。

## 接入示例

### Tool 与 MCP Tool Filter

Tool filter 从实际 tool 上下文获取名称,只接收参数中的脚本字段,不要求模型伪造 `tool_name`:

```python
from trpc_agent_sdk.tools import BashTool
from trpc_agent_sdk.tools.safety import JsonlAuditSink
from trpc_agent_sdk.tools.safety import ToolSafetyFilter, ToolSafetyGuard, ToolSafetyPolicy

policy = ToolSafetyPolicy.from_yaml("tool_safety_policy.yaml")
safety_guard = ToolSafetyGuard(policy, audit_sink=JsonlAuditSink("tool_safety_audit.jsonl"))
safety_filter = ToolSafetyFilter(safety_guard)
tool = BashTool()
tool.add_one_filter(safety_filter)
```

Safety Filter 带有最终授权标记,框架会让其他参数转换 Filter 和 tool callback 先运行,再在最靠近真实 handler 的位置扫描最终参数,避免下游原地修改已扫描内容。

Filter 会在扫描前合并可信的 tool 固定 override、默认 timeout 和默认 cwd。以示例策略的 `max_timeout_seconds: 120` 为例,`BashTool` 缺省的 300 秒 timeout 会被拒绝,调用方必须显式请求不超过策略的值。参数转换、scanner 或报告聚合异常会生成脱敏的 `SCAN-INPUT` / `SCAN-ERROR` deny 报告并记录一次审计事件,不会以普通异常形式绕过审计。

`StreamingProgressTool` 在启动用户 async generator 之前运行同一套有序 Filter 和 tool callback;被拒绝时只返回结构化阻断结果,generator 不会开始执行。最终授权只针对完整组装后的 tool call,不应对早期参数分片做放行判断。

MCP Tool 应在本地代理真正发出 MCP 调用前应用同一个 Filter。远端 MCP server 仍需独立鉴权、最小权限和审计,因为本地静态扫描无法证明远端实现的实际行为。

### Skill

`skill_run` 的 `command`、`cwd`、`env` 名称和 timeout 都应在 workspace runner 启动进程前进入 Filter:

```python
from trpc_agent_sdk.skills.tools import SkillRunTool
from trpc_agent_sdk.tools.safety import ToolSafetyFilter

skill_tool = SkillRunTool(repository=repository, filters=[ToolSafetyFilter(safety_guard)])
```

Skill 内容可能在扫描后被更新,因此还要固定 skill 版本或内容摘要,并在执行环境中限制挂载、网络和凭据。

### CodeExecutor wrapper

CodeExecutor 使用委托包装,逐个保留代码块语言,再聚合报告:

```python
from trpc_agent_sdk.code_executors import UnsafeLocalCodeExecutor
from trpc_agent_sdk.tools.safety import SafetyGuardedCodeExecutor

delegate = UnsafeLocalCodeExecutor(timeout=30)
executor = SafetyGuardedCodeExecutor(inner=delegate, guard=safety_guard)
```

包装器需要镜像 delegate 的 `stateful`、workspace runtime、delimiter 和重试配置,阻断时不得调用 delegate。

## 规则与策略

Python 使用 AST 提取调用、常量路径和数据流信号;Bash 使用不执行命令的 token/语法模式扫描。每个 finding 至少包含:

- `rule_id` 和风险分类;
- `risk_level` 与局部决策;
- 已脱敏、长度受限的 `evidence`;
- 可执行的 `recommendation`;
- 可选行列位置和不含秘密值的 metadata。

自定义规则通过 scanner 的 `rules` 参数注入,不修改内置 scanner:

```python
from trpc_agent_sdk.tools.safety import SafetyDecision, SafetyFinding
from trpc_agent_sdk.tools.safety import RiskCategory, RiskLevel, ToolSafetyScanner

class DenyInternalBinaryRule:
rule_id = "CUSTOM-INTERNAL-BINARY"

def scan(self, context, policy):
if "/internal/bin/" not in context.request.script:
return []
return [SafetyFinding(
rule_id=self.rule_id,
category=RiskCategory.PROCESS_EXECUTION,
risk_level=RiskLevel.HIGH,
decision=SafetyDecision.DENY,
evidence="/internal/bin/<redacted>",
recommendation="Use an explicitly approved command.",
)]

scanner = ToolSafetyScanner(policy, rules=[DenyInternalBinaryRule()])
```

自定义规则必须是纯静态、确定性且无副作用;异常由 scanner 按 policy 的 fail-closed 行为处理。

## 审计与监控

审计事件至少包含 `tool_name`、`decision`、`risk_level`、`rule_ids`、扫描耗时、`redacted`、`blocked`、人工批准状态、脚本 SHA-256 和策略版本。不得写入脚本文本、环境变量值、Authorization header 或私钥正文。

当前 span 应设置:

- `tool.safety.decision`
- `tool.safety.risk_level`
- `tool.safety.rule_id`
- `tool.safety.rule_ids`
- `tool.safety.duration_ms`
- `tool.safety.redacted`
- `tool.safety.blocked`

策略版本和脚本 SHA-256 只写入审计事件,不写入当前 span,以控制 telemetry 基数。

监控系统可以按 deny/review 比例、rule id、tool name 和扫描失败率告警,但不能把 telemetry 成功视为安全放行条件。

## 已知限制

- **误报**:Bash 的复杂引用、合法管道、安全的 subprocess 和测试夹具可能触发 review;通过窄化白名单或单条 rule action 调整,不应关闭整个 guard。
- **漏报**:编码、压缩、别名、间接导入、反射、动态属性和多阶段下载执行可能绕过静态模式。
- **动态代码**:`eval`、`exec`、运行时拼接 URL/路径和下载后的脚本无法在首次扫描时完全解析,应阻断或人工复核,并在下一执行边界重新扫描。
- **TOCTOU**:扫描后文件、Skill、cwd、符号链接或策略可能变化。执行端应校验内容摘要,固定版本,并尽量在隔离 workspace 内完成扫描和执行。
- **MCP**:本地只能分析请求参数,无法验证远端 tool 是否执行了额外命令、访问其他数据或正确实施资源限制。
- **Streaming**:分片参数在结束前可能不是完整语法。只能在完整 tool call 组装后做最终授权,不能因早期分片看似安全而提前执行。
- **资源限制**:静态规则只能识别明显循环、sleep、fork bomb 和大写入信号;CPU、内存、进程数、磁盘、输出和墙钟时间必须由 sandbox/runtime 强制限制。
- **运行时注入**:Guard 会检查调用参数、固定 tool override 和可见默认值,但 workspace/repository 在 handler 内部追加的可信环境或文件仍需由运行时白名单、固定配置和 sandbox 约束。

因此生产部署应组合:Safety Guard + Container/Cube sandbox + 最小凭据 + 出网白名单 + 资源限制 + 不可篡改审计。
53 changes: 53 additions & 0 deletions examples/tool_safety/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Tool Script Safety Guard 示例

本目录提供一套可重复运行的公开验收样本,用于验证 Tool、Skill 和 CodeExecutor 在执行 Python 或 Bash 内容前的静态安全检查。样本只会被读取和扫描,测试不会执行其中的脚本。

## 快速运行

在仓库根目录执行:

```bash
python scripts/tool_safety_check.py examples/tool_safety/samples \
--policy examples/tool_safety/tool_safety_policy.yaml \
--report /tmp/tool_safety_report.json \
--audit /tmp/tool_safety_audit.jsonl \
--tool-name public_sample_scan
```

CLI 接受任意数量的 `.py`、`.sh`、`.bash` 文件或目录。目录会递归扫描,结果按路径稳定排序。标准输出和 `--report` 都是结构化 JSON;`--audit` 每个文件追加一条已脱敏 JSONL 事件。

仓库同时保留一份由上述 12 个样本真实生成的 [`tool_safety_report.json`](tool_safety_report.json) 和 [`tool_safety_audit.jsonl`](tool_safety_audit.jsonl)。报告不含原始脚本或环境变量值,审计文件恰好每个样本一条事件。

退出码遵循最严格决策:

| 退出码 | 决策 |
| --- | --- |
| `0` | `allow` |
| `1` | `deny` |
| `2` | `needs_human_review` |

多文件扫描按 `deny > needs_human_review > allow` 聚合。策略读取失败、输入不可读或扫描基础设施异常时,CLI 采用 fail-closed 并返回 `deny`。

## 公开样本

[`samples/manifest.yaml`](samples/manifest.yaml) 是唯一真值源,列出恰好 12 个样本的预期决策和至少应命中的规则。场景包括安全 Python、危险删除、读取 SSH 私钥、非白名单与白名单网络请求、subprocess、shell 注入、依赖安装、无限循环、敏感信息输出、Bash 管道和人工复核。

测试会逐个调用 CLI,并校验:

- 高危样本检出率不低于 90%;
- 安全样本误报率不高于 10%;
- 危险删除、读取密钥、非白名单外连检出率均为 100%;
- JSON 报告和 JSONL 审计可以被标准解析器读取。

## 策略修改

[`tool_safety_policy.yaml`](tool_safety_policy.yaml) 采用严格字段校验。未知字段、错误类型和非法阈值会直接失败,避免拼写错误造成静默放行。修改以下字段不需要改代码:

- `allowed_domains`:精确域名和子域名白名单;
- `allowed_commands`:允许的可执行文件 basename 或精确路径;basename 白名单不会放行 `./git` 等相对路径;
- `denied_paths`:禁止路径及 glob;
- `max_timeout_seconds`、`max_output_bytes`、`max_script_bytes`:资源上限;
- `long_sleep_seconds`、`max_concurrency`:可疑资源使用阈值;
- `rule_actions`:按 rule id 覆盖默认决策。

完整处理流程、接入方式和安全边界见 [`DESIGN.md`](DESIGN.md)。
4 changes: 4 additions & 0 deletions examples/tool_safety/samples/01_safe_python.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Static scanner sample; this file is not executed by the example suite.

values = [1, 2, 3]
print(sum(values))
4 changes: 4 additions & 0 deletions examples/tool_safety/samples/02_dangerous_delete.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/sh
# Static scanner sample; do not execute.

rm -rf /
6 changes: 6 additions & 0 deletions examples/tool_safety/samples/03_read_ssh_key.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Static scanner sample; do not execute.

from pathlib import Path

private_key = Path("~/.ssh/id_rsa").read_text(encoding="utf-8")
print(len(private_key))
5 changes: 5 additions & 0 deletions examples/tool_safety/samples/04_non_allowlisted_network.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Static scanner sample; do not execute.

import requests

requests.post("https://collector.external.example/upload", data={"status": "ok"}, timeout=5)
6 changes: 6 additions & 0 deletions examples/tool_safety/samples/05_allowlisted_network.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Static scanner sample; this file is not executed by the example suite.

import requests

response = requests.get("https://api.example.com/health", timeout=5)
print(response.status_code)
5 changes: 5 additions & 0 deletions examples/tool_safety/samples/06_subprocess_call.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Static scanner sample; do not execute.

import subprocess

subprocess.run(["git", "status"], check=True)
6 changes: 6 additions & 0 deletions examples/tool_safety/samples/07_shell_injection.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Static scanner sample; do not execute.

import subprocess

user_value = input("value: ")
subprocess.run("echo " + user_value, shell=True, check=True)
4 changes: 4 additions & 0 deletions examples/tool_safety/samples/08_dependency_install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/sh
# Static scanner sample; do not execute.

python -m pip install untrusted-example-package
4 changes: 4 additions & 0 deletions examples/tool_safety/samples/09_infinite_loop.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Static scanner sample; do not execute.

while True:
pass
3 changes: 3 additions & 0 deletions examples/tool_safety/samples/10_sensitive_output.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Static scanner sample; do not execute.

print("API_KEY=example_sensitive_value_123456")
4 changes: 4 additions & 0 deletions examples/tool_safety/samples/11_bash_pipeline.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/sh
# Static scanner sample; do not execute.

printf '%s\n' alpha beta | grep beta
6 changes: 6 additions & 0 deletions examples/tool_safety/samples/12_human_review.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Static scanner sample; do not execute.

import requests

target_url = input("target URL: ")
requests.get(target_url, timeout=5)
87 changes: 87 additions & 0 deletions examples/tool_safety/samples/manifest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
version: "1"
policy: ../tool_safety_policy.yaml
samples:
- file: 01_safe_python.py
scenario: safe_python
category: safe
safe: true
required_detection: false
expected_decision: allow
expected_rules: []
- file: 02_dangerous_delete.sh
scenario: dangerous_delete
category: dangerous_file_operation
safe: false
required_detection: true
expected_decision: deny
expected_rules: [FILE-DANGEROUS-DELETE]
- file: 03_read_ssh_key.py
scenario: read_secret_file
category: dangerous_file_operation
safe: false
required_detection: true
expected_decision: deny
expected_rules: [FILE-DENIED-PATH]
- file: 04_non_allowlisted_network.py
scenario: non_allowlisted_network
category: network_access
safe: false
required_detection: true
expected_decision: deny
expected_rules: [NET-NON-WHITELISTED]
- file: 05_allowlisted_network.py
scenario: allowlisted_network
category: safe
safe: true
required_detection: false
expected_decision: allow
expected_rules: []
- file: 06_subprocess_call.py
scenario: subprocess_call
category: process_execution
safe: false
required_detection: true
expected_decision: needs_human_review
expected_rules: [PROC-SUBPROCESS]
- file: 07_shell_injection.py
scenario: shell_injection
category: process_execution
safe: false
required_detection: true
expected_decision: deny
expected_rules: [PROC-SHELL-INJECTION]
- file: 08_dependency_install.sh
scenario: dependency_install
category: dependency_installation
safe: false
required_detection: true
expected_decision: deny
expected_rules: [DEP-INSTALL]
- file: 09_infinite_loop.py
scenario: infinite_loop
category: resource_abuse
safe: false
required_detection: true
expected_decision: deny
expected_rules: [RES-INFINITE-LOOP]
- file: 10_sensitive_output.py
scenario: sensitive_output
category: sensitive_data_exposure
safe: false
required_detection: true
expected_decision: deny
expected_rules: [SECRET-EXPOSURE]
- file: 11_bash_pipeline.sh
scenario: bash_pipeline
category: process_execution
safe: false
required_detection: true
expected_decision: needs_human_review
expected_rules: [PROC-PIPELINE]
- file: 12_human_review.py
scenario: human_review
category: network_access
safe: false
required_detection: true
expected_decision: needs_human_review
expected_rules: [NET-DYNAMIC-TARGET]
12 changes: 12 additions & 0 deletions examples/tool_safety/tool_safety_audit.jsonl
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{"timestamp":"2026-07-12T16:16:57.488210Z","tool_name":"public_sample_scan","decision":"allow","risk_level":"low","rule_id":null,"rule_ids":[],"duration_ms":0.6008749987813644,"redacted":true,"blocked":false,"human_review_approved":false,"script_sha256":"b1efc1f8a3d7bd0b92834054b90e6ec4050e1b860517a6a94615c2221ae1dc2a","policy_version":"1"}
{"timestamp":"2026-07-12T16:16:57.488336Z","tool_name":"public_sample_scan","decision":"deny","risk_level":"critical","rule_id":"FILE-DANGEROUS-DELETE","rule_ids":["FILE-DANGEROUS-DELETE","POLICY-ARGV-COMMAND"],"duration_ms":0.2756249959929846,"redacted":true,"blocked":true,"human_review_approved":false,"script_sha256":"e561b4bbf19161ca31d57774ac207ebeb9063913551bde1d80f0a5cfe2a42478","policy_version":"1"}
{"timestamp":"2026-07-12T16:16:57.488379Z","tool_name":"public_sample_scan","decision":"deny","risk_level":"critical","rule_id":"FILE-DENIED-PATH","rule_ids":["FILE-DENIED-PATH","SECRET-EXPOSURE"],"duration_ms":0.3285410057287663,"redacted":true,"blocked":true,"human_review_approved":false,"script_sha256":"4cfacd0db5bf443bf11a0b554ab909be6428e63c081351187858413e4f4dc9cf","policy_version":"1"}
{"timestamp":"2026-07-12T16:16:57.488414Z","tool_name":"public_sample_scan","decision":"deny","risk_level":"high","rule_id":"NET-NON-WHITELISTED","rule_ids":["NET-NON-WHITELISTED"],"duration_ms":0.20820800273213536,"redacted":true,"blocked":true,"human_review_approved":false,"script_sha256":"33f11abee7dd0655103ed15f3c27677666bb04f797c0c0b7e31be0048dd821b5","policy_version":"1"}
{"timestamp":"2026-07-12T16:16:57.488446Z","tool_name":"public_sample_scan","decision":"allow","risk_level":"low","rule_id":null,"rule_ids":[],"duration_ms":0.19500000053085387,"redacted":true,"blocked":false,"human_review_approved":false,"script_sha256":"53a3e8a2dced493d5d0118207a338b71e46f3e01a6952a6aa4477df06f87a980","policy_version":"1"}
{"timestamp":"2026-07-12T16:16:57.488477Z","tool_name":"public_sample_scan","decision":"needs_human_review","risk_level":"high","rule_id":"PROC-SUBPROCESS","rule_ids":["PROC-SUBPROCESS"],"duration_ms":0.15474999963771552,"redacted":true,"blocked":true,"human_review_approved":false,"script_sha256":"415aa546b6024bf3b0e36b4af4b1d88d971eb8e02d756533e9be44f536914984","policy_version":"1"}
{"timestamp":"2026-07-12T16:16:57.488506Z","tool_name":"public_sample_scan","decision":"deny","risk_level":"critical","rule_id":"PROC-SHELL-INJECTION","rule_ids":["PROC-SUBPROCESS","PROC-SHELL-INJECTION"],"duration_ms":0.20270900131436065,"redacted":true,"blocked":true,"human_review_approved":false,"script_sha256":"576e95fadeb1284ea9a27816bf32d944fa0fc2bf58d36139670830e7d8e1b419","policy_version":"1"}
{"timestamp":"2026-07-12T16:16:57.488541Z","tool_name":"public_sample_scan","decision":"deny","risk_level":"high","rule_id":"DEP-INSTALL","rule_ids":["DEP-INSTALL"],"duration_ms":0.11441600508987904,"redacted":true,"blocked":true,"human_review_approved":false,"script_sha256":"028b5fb0b3f97323bec90c8648729335cb66fa56eb81a9b662a70cbb4d055309","policy_version":"1"}
{"timestamp":"2026-07-12T16:16:57.488570Z","tool_name":"public_sample_scan","decision":"deny","risk_level":"critical","rule_id":"RES-INFINITE-LOOP","rule_ids":["RES-INFINITE-LOOP"],"duration_ms":0.0832090008771047,"redacted":true,"blocked":true,"human_review_approved":false,"script_sha256":"a878581bd711f1d7f73f3678df35ae2d6e79ca1e70b942e0b2bad1a5e7b82a96","policy_version":"1"}
{"timestamp":"2026-07-12T16:16:57.488599Z","tool_name":"public_sample_scan","decision":"deny","risk_level":"critical","rule_id":"SECRET-EXPOSURE","rule_ids":["SECRET-EXPOSURE"],"duration_ms":0.09045899787452072,"redacted":true,"blocked":true,"human_review_approved":false,"script_sha256":"a002bc42f64ba0dbc4950c75076e7635a00fd9f61ead9ab1f723c7b3e64341ac","policy_version":"1"}
{"timestamp":"2026-07-12T16:16:57.488626Z","tool_name":"public_sample_scan","decision":"needs_human_review","risk_level":"medium","rule_id":"PROC-PIPELINE","rule_ids":["PROC-PIPELINE"],"duration_ms":0.11808300041593611,"redacted":true,"blocked":true,"human_review_approved":false,"script_sha256":"526a9e5468c01bb720b27879fb2a58b3a4934cda91cf0d6de8a3a6e0c4e8730c","policy_version":"1"}
{"timestamp":"2026-07-12T16:16:57.488672Z","tool_name":"public_sample_scan","decision":"needs_human_review","risk_level":"medium","rule_id":"NET-DYNAMIC-TARGET","rule_ids":["NET-DYNAMIC-TARGET"],"duration_ms":0.1784170017344877,"redacted":true,"blocked":true,"human_review_approved":false,"script_sha256":"64cb6a650c3a3ae9a519752127380a7080d2c8b4fa750ada03c65998b7635de9","policy_version":"1"}
Loading
Loading