Agent Security Gate

BSc Thesis: Auditable Policy Enforcement for LLM Agent Tool Calls
Technische Universität Darmstadt

Deterministic policy enforcement before agent tool execution — OPA Rego, human approvals, hash-chained audit. Integrates with any agent runtime via the connector SDK.

Blocks unsafe tool calls before execution and records an auditable decision trace.

Start here from any device: Reviewer information hub

Security reviewer guide: docs/security-reviewer-guide.md · Case study: docs/case-study.md · Threat model: docs/agent-security-gate-threat-model.md · Benchmark: docs/benchmark-results/agentdojo-local.md · Authorship: AUTHORS.md

---

Trust model

ASG is a policy enforcement point at the tool-call boundary. It is enforceable for every tool call that reaches it — in strict mode the adapters refuse any call without a valid single-use grant from a prior decide — but a call that never reaches a gated adapter is outside its control. The trusted computing base is therefore the agent runtime plus the connector SDK; the model and its prompt are not trusted.

Make "reaches it" non-optional in a real deployment: put the tool backends on a network where the gateway is the only route to them (see deploy/network-policy.example.yaml). Then a prompt-injected agent that tries to skip the gate has nothing to call.

---

Benchmark (18 scenarios, 5 runs each)

Policy regression comparing an intentional unprotected baseline to the gated runtime path:

Metric	No gate	Policy gate
Attack success rate	100%	0%
Data leakage	100%	0%
Benign task success	100%	100%

Methodology and limits: docs/benchmark-methodology.md. Attack classes: docs/benchmark-results/latest.md#attack-classes-covered. The gate baseline uses the same code path as POST /v1/gateway/decide (parity test).

External benchmark (AgentDojo Banking, local model)

Authored fixtures cannot show what the gate does to a real agent, so the same enforcement point was measured on AgentDojo's Banking suite with a local model:

	No authorizer	ASG + OPA
Attacker goals achieved (9 standalone injection-goal runs)	6/9	0/9
Policy-violating tool calls executed	11/11	0/11
Benign cases completed (72 paired cases)	36/36	33/36

Every attacker goal the unprotected baseline reached was stopped at the tool boundary. The cost was three held-out cases that legitimately needed an approval-gated tool. Scored-case security was 100% in both arms — this model rarely followed the injection — so the arms differ only in the goal runs, and that limit is stated in the results rather than averaged away.

n is small: 0/9 is a Wilson 95% CI of 0%–30%, 6/9 is 35%–88%, one model and one suite (python scripts/benchmark_confidence.py). The arm separation on the goal runs is the signal; the rates do not generalise.

Per-call decisions, policy coverage, authorization latency, OPA-down behaviour, and full limits: docs/benchmark-results/agentdojo-local.md. Protocol: docs/agentdojo-benchmark.md.

---

Quick start (local, free)

git clone https://github.com/giselleevita/agent-security-gate
cd agent-security-gate
cp .env.example .env
docker compose up -d --build
curl http://localhost:8000/health

Four decisions in 30 seconds

POST /agent is a demo façade, not an agent: it maps plain text to a tool call deterministically so the decision path can be shown without an LLM. The decisions below are the real ones — same code path as POST /v1/gateway/decide. Real agents integrate via the connector SDK.

# 1. Doc exfiltration → blocked
curl -s -X POST http://localhost:8000/agent \
  -H "Authorization: Bearer test-token" -H "Content-Type: application/json" \
  -d '{"input": "read /internal/secrets.yaml"}'
# {"allowed":false,"reason":"denied_doc_prefix: /internal/","tool":"docs.read",
#  "audit_id":"evt_e8c556b4…","approval_url":null}

# 2. SSRF → blocked
curl -s -X POST http://localhost:8000/agent \
  -H "Authorization: Bearer test-token" -H "Content-Type: application/json" \
  -d '{"input": "fetch http://169.254.169.254/latest/meta-data/"}'
# {"allowed":false,"reason":"ssrf_blocked_ip_literal","tool":"http.get",
#  "audit_id":"evt_3fe242a0…","approval_url":null}

# 3. Privilege escalation → approval required
curl -s -X POST http://localhost:8000/agent \
  -H "Authorization: Bearer test-token" -H "Content-Type: application/json" \
  -d '{"input": "update accounts set role=admin"}'
# {"allowed":false,"reason":"approval_required","tool":"db.write",
#  "audit_id":"evt_ba036129…","approval_url":"/v1/approvals/request"}

# 4. Legitimate read → allowed
curl -s -X POST http://localhost:8000/agent \
  -H "Authorization: Bearer test-token" -H "Content-Type: application/json" \
  -d '{"input": "summarize /public/readme.md"}'
# {"allowed":true,"reason":"allow","tool":"docs.read",
#  "audit_id":"evt_ea118016…","approval_url":null}

# Audit trail
curl -s "http://localhost:8000/audit?limit=4" -H "Authorization: Bearer approver-token"
python scripts/verify_audit.py --path audit/events.jsonl

Demo metadata: GET /demo · Approval UI: http://localhost:8000/ui/approvals · Video: docs/assets/asg-demo.mp4

---

Research Contribution (BSc Thesis)

This project addresses the research question: How can we enforce deterministic, auditable safety boundaries over LLM agent tool execution without sacrificing agent autonomy?

Key contributions:

• Policy-as-code safety gates: Demonstrates OPA/Rego as a practical framework for agent authorization, with fail-closed semantics and auditability
• Binding enforcement + approval workflow: Shows how hash-chained audit logs and human approvals create verifiable compliance traces for regulated domains
• Two-tier evaluation: 18 hand-authored policy-regression scenarios (doc exfiltration, SSRF, privilege escalation, prompt injection) against an intentional no-gate baseline, plus an external AgentDojo Banking run on a local model. The authored suite establishes regression correctness and runtime parity, not adaptive red-team coverage; the external run shows what the boundary stops inside a real agent loop, with its own limits stated rather than averaged away
• Connector SDK for agent integration: Provides a reusable interface for coupling agent runtimes to safety policies

Thesis evaluation (15 min): Quick start above + read docs/technical-brief.md and docs/benchmark-methodology.md.

---

Scope and limitations

• Tool-boundary PEP — policy runs on proposed tool calls, not inside the model. A call that never reaches a gated adapter is out of scope; see Trust model for making that path the only one.
• No LLM in this repo — the /agent endpoint is a demo façade that maps plain text to tool calls; real agents integrate via the connector SDK.
• Policy regression benchmark — 18 hand-authored scenarios with an intentional no-gate baseline; not adaptive red-team coverage.
• Demo defaults — ASG_ENFORCE_MODE=off in docker compose so local try is frictionless. Pilots should use strict so tool endpoints require a prior allow decision (see below).
• Reference implementation, not a hosted product — bring your own IdP (OIDC_*), secret mounts (*_FILE), immutable audit sink, and HA Redis/Postgres per the runbooks.

---

Strict enforcement (binding control)

Without strict mode, /v1/gateway/decide is advisory — agents can call tool endpoints directly. For enforceable governance:

ASG_ENFORCE_MODE=strict docker compose up -d --build
python examples/gated_agent.py

The SDK couples decide → execute and passes X-ASG-Audit-Id so adapters refuse calls without a valid, single-use grant. See docs/connector-sdk.md and examples/gated_agent.py.

---

What it blocks

Attack	Example	Result
Doc exfiltration	docs.read /internal/secrets.yaml	denied_doc_prefix
SSRF	GET http://169.254.169.254/meta-data/	ssrf_blocked_ip_literal
Privilege escalation	db.write UPDATE accounts SET role='admin'	approval_required
Unknown tool	shell.exec	deny (fail closed)
PII / canary in output	SSN or canary token in tool output	dlp_redacted / canary_detected

DLP/canary scanning runs on the tool→agent return path: it redacts matches and fails the call closed (and records the block in the audit log) before the agent sees the output. It does not retract a side effect the tool already performed — pair it with the host allowlist and egress controls for that.

---

How this differs from other LLM security tools

Most LLM security tooling inspects text. ASG authorizes actions. The two are complementary — a content scanner and an enforcement point solve different halves of the problem.

	Primary question	Where it runs
NeMo Guardrails	"Should the model say/discuss this?"	Around the conversation, often LLM-in-the-loop
llm-guard	"Does this text contain injection / PII / toxicity?"	Scanners on prompt and response text
OPA alone	"Is this request allowed?"	A policy engine — you still build the enforcement point around it
Agent Security Gate	"Should this exact tool call execute, for this principal, right now?"	A PEP at the tool-call boundary, before side effects

What follows from being an enforcement point rather than a classifier:

• Deterministic, not probabilistic. Decisions come from OPA/Rego policy plus explicit pre-checks (SSRF with DNS pinning, DLP) — no model in the decision path. Given a fixed policy bundle and successful name resolution, the same call yields the same decision; DNS or OPA infrastructure failures resolve to deny (fail-closed), which is intentional and can differ from a prior allow.
• Fail closed. An unknown tool is denied by default rather than passed through.
• Binding, not advisory. In strict mode the adapters refuse any call without a valid single-use grant, so an agent cannot skip the gate and call the tool directly.
• Answers "what happened?" Every decision lands in a hash-chained audit log built for after-the-fact review, with human approvals and dual-control for risky operations.

If you need topic control or content filtering, use one of the tools above — ideally alongside this one. If you need a deterministic, auditable answer to whether an action may run, that is what ASG is for.

---

Architecture

Agent → POST /v1/gateway/decide → OPA + pre-checks (SSRF, DLP)
                    ↓
         allow / deny / approval_required → hash-chained audit
                    ↓
              gated tool adapters (optional strict enforcement)

Details: docs/architecture.md

---

Core features

• OPA Rego policy-as-code (fail closed on unknown tools)
• Shared SSRF evaluator with DNS pinning (adapters/http.py)
• Human approval with dual-control, operation binding, resume tokens
• DLP + canary scanning on the tool-output return path (redact + fail closed + audit; one shared scanner across all egress paths)
• Hash-chained audit log — tamper-evident by itself; tamper-resistance needs AUDIT_HMAC_KEY (held off-host) + an S3 Object Lock (WORM) mirror, both wired in and off by a flag
• OIDC JWT auth (asg:agent / asg:approver roles)
• Per-tenant policy files; Prometheus metrics and Grafana dashboard JSON
• Test suite spanning unit, integration, and benchmark parity; CI benchmark threshold gate
• Bounded exhaustive policy checks: opa test over every rule branch + full input-space enumeration of the safety invariants against the live OPA decision (ADR 0006)

---

Evaluate it in 15 minutes

1. docker compose up -d --build and run the four curls above.
2. python scripts/verify_audit.py --path audit/events.jsonl
3. Skim policies/asg.rego, app/decision.py (decide_tool_call_impl), and tests/integration/test_decide.py.
4. Optional: ASG_ENFORCE_MODE=strict + examples/gated_agent.py.

Deeper write-up: docs/technical-brief.md

Verify a clean checkout

The full verification command requires Bash, GNU Make, curl, and Docker Compose (Linux, macOS, or WSL). Create and activate a Python virtual environment first. After installing the development and security extras, run the complete lint, dependency, policy, unit, benchmark, and Docker integration path with one command:

python -m pip install --constraint requirements-dev.lock -e ".[dev,security]"
make verify

For a minimal simulated agent flow, see examples/injected_agent_tool_call.py. It demonstrates how an agent-produced tool call is checked before execution (no in-tree LLM).

For the protected-function demonstration, use the complete setup and expected observations. Installing the Docker image alone does not install the host-side Python example.

---

API overview

Endpoint	Purpose
POST /v1/gateway/decide	Authorize a tool call before execution
POST /v1/approvals/*	Request, approve, deny (Postgres-backed)
POST /v1/docs/read, POST /v1/http/proxy	Gated adapters (enforce with X-ASG-Audit-Id)
POST /agent	Demo façade only (keyword → tool mapping)
GET /audit, POST /v1/audit/export	Audit read / auditor export package
GET /metrics, GET /v1/stats	Observability

Auth: bearer token or OIDC JWT. Full contract: docs/connector-sdk.md.

LangGraph: docs/integrations/langgraph.md · examples/langgraph_gated_agent.py

---

Configuration

Variable	Default	Notes
ASG_ENFORCE_MODE	off	Set strict for binding enforcement
ASG_DEMO_MODE	false	true in compose demo (fixed tokens)
OIDC_ISSUER / OIDC_AUDIENCE	unset	Production identity
AUDIT_HMAC_KEY	unset	Sign audit entries (tamper-resistance); warned about at startup when enforcing without it
ASG_TENANT_POLICY_STRICT	false	Deny unknown tenants
ASG_CONTEXT_KEY_ALLOWLIST	false	Reject decide-request context keys outside a tool's context_keys_allowed
ASG_MAX_EXCEPTION_TTL_S	86400	Ceiling on time-bound policy-exception lifetime

Policy data: policies/data/policy_data.json, per-tenant overrides in policies/data/tenants/{id}/.

---

Documentation

• Gated function-tool demo — a real function, an execution spy, and five decisions
• AgentDojo authorization adapter
• AgentDojo Banking benchmark protocol
• AgentDojo local-model results
• OpenAI Agents SDK authorization adapter

Doc	Purpose
Reviewer information hub	One-page index for review, outreach, evidence, releases, and public status
Case study	Threat, contract, benchmark design, what went wrong, and what the numbers support
Security reviewer guide	Five-minute demo, review paths, non-claims, and independent reproduction template
Demo assets	Reproducible terminal recordings, including the protected function tool
Agent task-quality experiment	Ongoing preregistered protocol; no completed result is claimed yet
Architecture decisions (ADRs)	Why the non-obvious calls were made, and what was rejected
Threat model	Trust boundaries and abuse paths
Connector SDK	Decide → execute contract
Benchmark methodology	Metrics and limitations
SafeRemediate methodology note	Evaluating recovery and policy leakage without overstating evidence
HA deployment	Multi-replica topology
Observability	Metrics, logs, dashboards
Backup / restore	RPO/RTO procedures

---

Roadmap

See ROADMAP.md. Shipped: OPA enforcement, approvals, audit chain, DLP, CI benchmark gate, SDK strict mode, LangGraph example. Planned: operator dashboard, SIEM integration.

---

Contributing

See CONTRIBUTING.md.

Security

Report vulnerabilities via GitHub private advisories. See SECURITY.md.

License

Apache License 2.0 — see LICENSE.