agent-guard

A security gateway for AI agents — prompt-injection defense, risk-tiered action policy, sandboxed execution, human-in-the-loop approval, and forensic audit logging. Pure Python 3.10+, zero runtime dependencies, mapped to the OWASP Top 10 for LLM Applications.

Sister project of VoxAgent (TypeScript voice-agent platform): VoxAgent ships an in-process guard; agent-guard generalizes the idea into a standalone, language-agnostic policy gateway.

Why no "confidence scores"?

Most agent-security specs demand things like "block if intent_alignment < 85%". That number is theater: an LLM does not emit calibrated confidence, and a regex engine cannot produce a meaningful 0–100 score. agent-guard refuses to invent one. Instead every decision is built from two auditable inputs:

1. Evidence — the exact detection rules that matched, with excerpts.
2. Declared impact — each tool registers an ActionTier (READ / WRITE / MONEY / SYSTEM).

The "user asked to summarize but the agent clicked buy" scenario is handled structurally: MONEY-tier actions always stop for a human, no scoring involved.

Architecture

flowchart LR
    A[Agent proposes action] --> G{AgentGuard.evaluate}
    G --> P["prompt_guard<br/>(LLM01: injection, EN+VI)"]
    G --> W["web_guard<br/>(LLM02: SQLi/XSS/cmdi + schema)"]
    P --> M[merge risk]
    W --> M
    M --> PE["policy engine<br/>(LLM08: tier × risk table)"]
    PE -->|ALLOW| X[caller executes]
    PE -->|NEEDS_APPROVAL| Q[ApprovalQueue<br/>timeout ⇒ deny]
    PE -->|DENY| D[blocked]
    Q -->|approved, SYSTEM tier| S[SandboxExecutor<br/>rlimits + timeout]
    G --> L[(AuditLog JSONL)]

Loading

Decision table (deny by default)

Tier	Input risk	Decision
(unregistered tool)	any	DENY
any	HIGH	DENY
READ	NONE / LOW	ALLOW
READ	MEDIUM	NEEDS_APPROVAL
WRITE (allowlisted)	NONE / LOW	ALLOW
WRITE (allowlisted)	MEDIUM	NEEDS_APPROVAL
WRITE (not allowlisted)	any	DENY
MONEY / SYSTEM	any	NEEDS_APPROVAL

OWASP LLM Top 10 mapping

OWASP	Threat	agent-guard module
LLM01	Prompt injection (override, exfiltration, delimiter abuse, jailbreak — English and Vietnamese)	prompt_guard.py
LLM02	Insecure output handling (agent output becomes SQL/HTML/shell input)	web_guard.py — pattern screen + positive schema validation
LLM08	Excessive agency	policy.py (tiered decision table) + approval.py (human-in-the-loop)
—	Forensics & incident response	audit.py (JSONL, torn-write tolerant, replayable)
—	Blast-radius containment	sandbox.py (rlimits, wall timeout, no shell)

Quickstart

python3 -m venv .venv && .venv/bin/pip install -e .[dev]
.venv/bin/pytest            # 52 tests, includes attack corpus + latency budget
.venv/bin/python examples/demo.py

from agent_guard import (ActionRequest, ActionTier, AgentGuard, Field, ToolPolicy)

guard = AgentGuard(audit_path="audit.jsonl")
guard.register_tool(ToolPolicy("search_reviews", ActionTier.READ))
guard.register_tool(ToolPolicy("purchase", ActionTier.MONEY),
                    schema={"product_id": Field(int), "quantity": Field(int)})

result = guard.evaluate(ActionRequest(
    agent_id="assistant-1",
    tool="purchase",                          # agent wants to buy...
    user_input="summarize product reviews",   # ...but user asked to summarize
    args={"product_id": 500, "quantity": 2}))

print(result.decision)        # Decision.NEEDS_APPROVAL — parked for a human
print(result.explain())       # full evidence + latency
guard.approvals.approve(result.ticket.ticket_id, approver="alice")

HTTP sidecar (use from any language)

The gateway also runs out-of-process, so non-Python agent runtimes can enforce the same policy. Stdlib-only, zero dependencies:

python -m agent_guard.server --port 8788

Endpoint	Purpose
POST /evaluate	{agent_id?, tool, user_input?, args?} → {decision, reason, risk, matched_rules, ticket_id, latency_ms}
GET /health	liveness + registered tools
GET /approvals	pending human-approval tickets
POST /approvals/<id>	{action: approve|deny, approver, note?}

Reference integration: VoxAgent (TypeScript) calls /evaluate before every /chat turn and fails closed if the sidecar stops answering — verified end-to-end with EN + VI injection attempts blocked and evidence ([PI001, PI010]) surfaced to the client.

Testing & quality

• 52 tests (pytest), no network, no API keys — includes real HTTP round-trips against the sidecar on an ephemeral port.
• Attack corpus: 21 real injection strings (EN+VI) — all must be flagged; 15 benign strings — measured 0% false-positive rate on the corpus.
• Latency budget test: average gateway overhead asserted < 100 ms (measured ~0.03 ms per call on a laptop).
• Sandbox tests use real subprocesses: timeout kill, memory-limit kill, no-shell-interpretation proof.
• CI: GitHub Actions runs the suite on Python 3.10 → 3.14.

Honest limitations (read before trusting any guard)

• Pattern-based detection is one layer, not a solution. Novel or obfuscated injections (translation tricks, encodings, many-shot) can evade regexes. Production stacks add an LLM-based classifier and — most importantly — the structural defenses here (tier policy, approvals), which hold even when detection misses.
• SandboxExecutor is a process sandbox, not a container. It enforces CPU, memory, wall-time, and no-shell, but cannot cut network or filesystem access. It is the policy seam: swap in docker run --network none, gVisor, or Firecracker behind the same interface for hostile code.
• The approval queue is in-memory — a deliberate seam; back it with a DB and a pager/Slack integration in production.
• This gateway validates inputs and actions; it does not verify the truthfulness of model outputs.

Layout

src/agent_guard/
├── models.py        # Risk, ActionTier, Decision, verdicts (evidence-carrying)
├── prompt_guard.py  # LLM01 — prompt-injection rules, EN + VI
├── web_guard.py     # LLM02 — SQLi/XSS/cmdi/path-traversal + schema allowlist
├── policy.py        # LLM08 — tier × risk decision table, deny by default
├── approval.py      # human-in-the-loop queue, expiry ⇒ deny, audit trail
├── sandbox.py       # rlimit + timeout subprocess executor
├── audit.py         # JSONL audit log + forensic reader/replay
├── gateway.py       # AgentGuard facade wiring it all together
└── server.py        # stdlib HTTP sidecar — policy gateway for any language

License

MIT