Skip to content

feat: Tool Script Safety Guard — pre-execution safety scanner#175

Open
DNKYr wants to merge 15 commits into
trpc-group:mainfrom
DNKYr:feature/tool-safety-guard
Open

feat: Tool Script Safety Guard — pre-execution safety scanner#175
DNKYr wants to merge 15 commits into
trpc-group:mainfrom
DNKYr:feature/tool-safety-guard

Conversation

@DNKYr

@DNKYr DNKYr commented Jul 13, 2026

Copy link
Copy Markdown

Summary

Implements #90 — a pre-execution script safety scanner for Python/Bash tool scripts with pluggable filter integration, audit logging, and OpenTelemetry span attributes.

What's included

  • trpc_agent_sdk/tools/safety/ — 7 modules: types, policy (YAML), rules (13 pattern + 3 AST), scanner, filter (BaseFilter), audit logger (JSONL), telemetry (OTel)
  • 6 risk categories: dangerous file ops, network access, system commands, dependency install, resource abuse, sensitive info leak
  • ToolSafetyFilter — plugs into existing BaseFilter chain via FilterType.TOOL, blocks execution on deny
  • Configurable policy — tool_safety_policy.yaml supports whitelist/blocklist domains, commands, paths; per-rule enable/disable/severity/decision overrides
  • 40 tests covering all 12 sample scripts + edge cases + policy validation + filter integration + audit + telemetry
  • Performance: 500-line script scanned in 0.08ms (threshold: 1s)
  • Documentation: design docs (EN/ZH), test plan, README with integration guide, example outputs

Acceptance criteria (#90)

  • 12 sample scripts produce structured reports
  • High-risk detection: critical types (delete, read-secrets, non-whitelisted network) at 100%
  • 500-line scan ≤ 1 second
  • Reports contain decision, risk_level, rule_id, evidence, recommendation
  • Policy YAML changes without code changes
  • Filter blocks before execution + audit event
  • README explains sandbox/filter/telemetry/CodeExecutor relationship

Known limitations (documented)

  • Pattern-based detection bypassable via obfuscation
  • Bash scanning is pattern-only (no Bash AST)
  • No data flow analysis
  • Not a replacement for runtime sandboxing

@codecov

codecov Bot commented Jul 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.74656% with 59 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@4336564). Learn more about missing BASE report.

Files with missing lines Patch % Lines
trpc_agent_sdk/tools/safety/_rules.py 58.57143% 29 Missing ⚠️
trpc_agent_sdk/tools/safety/_scanner.py 77.96610% 26 Missing ⚠️
trpc_agent_sdk/tools/safety/_filter.py 91.11111% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main        #175   +/-   ##
==========================================
  Coverage        ?   87.51181%           
==========================================
  Files           ?         475           
  Lines           ?       44466           
  Branches        ?           0           
==========================================
  Hits            ?       38913           
  Misses          ?        5553           
  Partials        ?           0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant