LLM security · AI red teaming · prompt-injection & jailbreak testing · MCP / agent / RAG security · OWASP LLM Top 10
The Armor for AI Security. AgentArmor is an open-source scanner that continuously tests, audits, and red-teams your AI stack — LLM APIs, chatbots, tool-calling agents, MCP servers, and RAG pipelines — for prompt injection, jailbreaks, system-prompt leaks, data exfiltration, and unsafe tool use.
Every scan runs a multi-agent analysis that scores findings on a 0–100 risk model, maps them to the OWASP LLM Top 10, explains in plain English what went wrong and how to fix it, and exports reports for developers and security teams (SARIF, HTML, PDF, CSV).
Latest release: v1.4.3 · Changelog
🔍 Who it's for: AI engineers shipping LLM features, red teamers and pentesters testing GenAI apps, and security teams that need OWASP-mapped, audit-ready reports.
| Capability | Description |
|---|---|
| Endpoint scanning | Test any OpenAI-compatible or auto-detected chat API URL |
| Cloud providers | OpenAI, Anthropic, Gemini, and more via LiteLLM |
| Local models | Offline scanning of .gguf and HuggingFace models |
| Agent / MCP / RAG | Security probes for tool-calling agents, MCP servers, and retrieval pipelines |
| L0 adaptive attacks | 100+ mutation variants per attack goal (jailbreak, prompt leak, exfiltration) |
| Self-play red teaming | Attacker → Target → Judge loop to find vulns static probes miss |
| Benchmarking | Compare models and tools (AgentArmor vs PyRIT, Garak, Promptfoo, Inspect AI) |
| Marketplace | Install community probes and OWASP packs |
| Monitoring | Scheduled rescans with drift detection |
| CI/CD | GitHub Action + agentarmor gate for pipeline security gates |
End users on Windows: download the release installer, double-click, and run scans — no Python, Rust, or Node install needed.
Developers / CLI users: need Python 3.10+. Rust is only for compiling the optional native signature engine or building the Tauri GUI yourself.
- Download
AgentArmor_1.4.3_x64-setup.exeor.msifrom Releases - Run the installer
- Open AgentArmor → choose scan type (API, Local Model, Agent, MCP, RAG, Benchmark)
- Configure target → run scan → review findings → export reports
Portable mode: place a file named PORTABLE next to the executable; data is stored in ./data/.
pip install agentarmor
# Optional: offline local model scanning
pip install agentarmor[local]
# One-time detection model download
agentarmor models downloaddocker build -f docker/Dockerfile -t agentarmor/agentarmor .
docker run -p 8787:8787 agentarmor/agentarmor# OpenAI-compatible chat API (use the POST URL from DevTools, not the HTML page)
agentarmor scan --url http://localhost:8000/v1/chat/completions
# Cloud provider
export OPENAI_API_KEY=sk-...
agentarmor scan --provider openai
# Multi-agent analysis runs on every scan (offline mode was removed in v1.4.0).
# Provide an analysis API key via AGENTARMOR_ANALYSIS_API_KEY or --analysis-api-key.
export AGENTARMOR_ANALYSIS_API_KEY=sk-...
agentarmor scan --url https://api.example.com/v1/chat/completions \
--self-play-enabledagentarmor scan --agent crewai --agent-config agent.toml
agentarmor scan --mcp ./filesystem-mcp
agentarmor scan --rag ./corpus --embedder bgepip install agentarmor[local]
agentarmor scan --model llama-3.gguf
agentarmor scan --model ./models/qwen3# Model leaderboard
agentarmor benchmark --providers openai,anthropic,gemini --suite owasp
# Tools comparison
agentarmor benchmark tools --suite owasp-llm01 --targets corpusagentarmor marketplace list
agentarmor marketplace install roleplay-injection
agentarmor monitor add "Daily API" --url https://api.example.com/v1/chat/completions --cron daily
agentarmor monitor run <schedule-id>
agentarmor dataset export -o research.jsonl --anonymizeagentarmor scan --url http://localhost:8000/v1/chat/completions --format html,sarif,pdf -o ./reports/
agentarmor gate --sarif ./reports/findings.sarif --fail-on HIGH,CRITICALagentarmor scan --config AgentArmor.tomlSee AgentArmor.toml for endpoint profiles, L0 attack goals, detection mode, and plugin directories.
The Tauri v2 GUI includes:
| Screen | Purpose |
|---|---|
| Home | Scan type picker + benchmark shortcut |
| Chatbot wizard | Guided API scan with multi-agent analysis |
| Scan progress | Live SSE probe stream |
| Findings | Risk scores, attack chains, evidence graph |
| Reports | Export HTML, SARIF, PDF, CSV, JSON |
| Benchmark | Model leaderboard + tools comparison |
| Marketplace | Install community probes |
| Monitoring | Schedules, drift alerts, dataset export |
| Settings | L0, self-play, detection, API keys |
For GUI development only (not required for end users):
agentarmor serve --port 8787
cd gui && npm install && npm run dev # http://localhost:1420See gui/README.md for Tauri build instructions.
agentarmor serve --port 8787| Endpoint | Description |
|---|---|
GET /health |
Sidecar status |
POST /v1/scans |
Start a scan |
GET /v1/scans/{id} |
Scan status + metadata |
GET /v1/scans/{id}/events |
SSE live progress |
GET /v1/findings |
List findings |
POST /v1/benchmarks |
Run benchmark |
GET /v1/marketplace/rules |
List marketplace packs |
POST /v1/monitoring/schedules |
Create monitor schedule |
POST /v1/datasets/export |
Export research JSONL |
name: AI Security Scan
on: [push, pull_request]
jobs:
agentarmor:
runs-on: ubuntu-latest
steps:
- uses: YashvantHange/AgentArmor/action@v1
with:
url: https://api.example.com/v1/chat/completions
fail-on: HIGH,CRITICALSee action/action.yml.
| Provider | Environment variable |
|---|---|
| OpenAI | OPENAI_API_KEY |
| Anthropic | ANTHROPIC_API_KEY |
| Gemini | GEMINI_API_KEY |
| Mistral | MISTRAL_API_KEY |
| Groq | GROQ_API_KEY |
| Together | TOGETHER_API_KEY |
| OpenRouter | OPENROUTER_API_KEY |
| Azure | AZURE_API_KEY, AZURE_API_BASE |
| Bedrock | AWS_* credentials |
As of v1.4.0 the offline analysis mode was removed — AgentArmor runs multi-agent (cloud) analysis on every scan: capability-aware attack-graph planning, an LLM judge, and confidence scoring. This requires an analysis provider API key. Set AGENTARMOR_ANALYSIS_API_KEY (or a provider key such as OPENAI_API_KEY), or configure it in GUI Settings / AgentArmor.toml. Scans started without a key are rejected with a clear error.
Author probes with the SDK and publish to your local marketplace:
agentarmor marketplace validate probes/custom_probe.py
agentarmor marketplace publish probes/custom_probe.py --id my.probe --name "My Probe"See probes/custom_probe.py and agentarmor.sdk.probe_sdk.
No native Mac .app yet (Windows desktop first). Use PyPI + CLI or dev GUI:
pip install agentarmor
agentarmor models download
agentarmor serve --port 8787See docs/MAC.md.
git clone https://github.com/YashvantHange/AgentArmor.git
cd AgentArmor
pip install -e ".[dev]"
pytestOptional native L1 engine (requires Rust):
cd native/l1_signatures && pip install maturin && maturin develop --releaseWithout Rust, the Python L1 fallback is used automatically.
Windows installer build (maintainers):
powershell -File packaging/build-installer.ps1See repository license file.