An AI-powered Security Operations Center (SOC) platform that autonomously generates, validates, and deploys new security tools through natural language requests for adapting tools to evolving workflows. Built with LangChain, Ollama, and a modular sub-agent pipeline.
- Project Overview
- Evaluation Criteria
- Architecture
- Prerequisites
- Setup Instructions
- Running the Application
- Using the Builder Agent (Co-Creation Workflow)
- Walkthrough: Building the Asset Email Drafting Tool
- Project Structure
- Testing
- Source Declarations
- Documentation Index
The SOC Platform Builder Agent is a multi-agent system designed for cybersecurity operations teams. It provides three core capabilities:
-
Query Builder — Generates LEQL (Log Entry Query Language) queries for security event investigation using RAG (Retrieval-Augmented Generation) over a curated dataset of query examples.
-
Triage Agent — Performs automated alert triage using a ReAct agent with OSINT tools (VirusTotal, WHOIS, DuckDuckGo, Nmap) to investigate security alerts and produce structured reports.
-
Builder Agent — The central innovation of this project. Users describe a new capability in natural language (e.g., "build a tool that drafts emails for offline assets"), and the builder agent:
- Introspects the live codebase to extract valid import patterns and architecture
- Scrapes library documentation for correct API usage
- Generates complete, production-ready Python code
- Validates the code through AST safety analysis and stub detection
- Deploys to an isolated git-worktree sandbox for testing
- Promotes to main via blue-green deployment on approval
The key technical contributions of this project are:
- Modular sub-agent pipeline for grounded code generation (introspection → documentation → validation)
- 25-pattern stub/placeholder detection ensuring LLM-generated code is complete, not abbreviated
- Blue-green sandbox deployment using git worktrees for safe, reversible code deployment
- Adaptive context budgeting that dynamically allocates token budget based on the model's context window
- Co-creation workflow where the builder generates, validates, and iteratively repairs code through developer feedback
This project is evaluated on the following criteria as defined in the assignment:
| Criterion | Description | How This Project Addresses It |
|---|---|---|
| Technical Contribution | Novel technical approach or methodology | Modular sub-agent pipeline with AST validation, stub detection, and sandbox deployment — a complete autonomous code generation and deployment system |
| Functionality | Working, demonstrable software | Three operational agents (Query Builder, Triage, Builder) accessible through a unified Dash web UI |
| User Experience | Intuitive and responsive interface | Terminal-style Dash UI with real-time response streaming, session management, and document context switching |
| Documentation | Clear instructions, organized files | This README, setup guide, sub-agent pipeline docs, session report, and co-building design document |
- All files properly organized and documented — Modular package structure (
fusion_assistant_*), docstrings in all modules, organizeddocs/directory - README.md with clear instructions — This file, covering setup, run modes, and the full co-creation workflow
- Source declarations — See Source Declarations section below
┌─────────────────────────────────────────────────────────────┐
│ Dash Web UI (port 8050) │
│ dash_ui_ReAct.py │
└──────────┬──────────────────┬──────────────────┬────────────┘
│ │ │
┌──────▼──────┐ ┌──────▼──────┐ ┌──────▼──────┐
│ Query │ │ Triage │ │ Builder │
│ Builder │ │ Agent │ │ Agent │
│ (ReAct) │ │ (ReAct) │ │ (Generate) │
└──────┬──────┘ └──────┬──────┘ └──────┬──────┘
│ │ │
┌──────▼──────┐ ┌──────▼──────┐ ┌──────▼──────────────┐
│ FAISS │ │ OSINT Tools │ │ Sub-Agent Pipeline │
│ Vector │ │ VirusTotal │ │ ┌─────────────────┐ │
│ Store │ │ WHOIS │ │ │ Introspection │ │
│ (RAG) │ │ DuckDuckGo │ │ │ Doc Scraper │ │
│ │ │ Nmap │ │ │ Lib Version │ │
└─────────────┘ └─────────────┘ │ │ Validation │ │
│ └────────┬────────┘ │
│ ┌────────▼────────┐ │
│ │ Sandbox Manager │ │
│ │ (git worktree) │ │
│ └─────────────────┘ │
└─────────────────────┘
│
┌────────▼────────┐
│ Ollama LLM │
│ gpt-oss:20b │
│ (DGX Spark) │
└─────────────────┘
| Requirement | Version | Purpose |
|---|---|---|
| Python | 3.11+ | Runtime |
| Git | Any recent | Sandbox worktree deployment |
| Ollama | Latest | LLM inference backend |
| OpenSSH | Any | Remote mode tunnel (optional) |
- Local mode: Any machine with Ollama installed and a model pulled (e.g.,
ollama pull llama3) - Remote mode: Requires SSH access to a host running Ollama (e.g., a GPU server, cloud instance, or DGX system)
git clone https://github.com/YOUR_ORG/SOC_Platform_Builder_Agent.git
cd SOC_Platform_Builder_AgentThe project includes a cross-platform bootstrap script that creates the virtual environment and installs all dependencies.
macOS / Linux:
chmod +x scripts/bootstrap_env.sh
./scripts/bootstrap_env.shWindows:
.\scripts\bootstrap_env.batCross-platform (direct):
python bootstrap_env.pyThis installs from requirements-direct.txt by default. For the exact frozen dependency set:
python bootstrap_env.py --requirements frozenThe run scripts auto-detect the virtual environment, but for interactive use:
macOS / Linux:
source .venv/bin/activateWindows:
.\.venv\Scripts\activateLocal mode — Ensure Ollama is running on localhost:11434:
ollama serve
ollama pull <your-model> # e.g., llama3, mistral, codellamaRemote mode — Ensure SSH access to a host running Ollama:
# Verify SSH connectivity to your remote GPU host
ssh <your-user>@<your-host>
# The launcher will create the tunnel automatically when started with --mode remoteCreate a .env file in the project root with your settings:
# Required for local mode
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=<your-model-name> # e.g., llama3, mistral, codellama
# Required for remote mode
SPARK_USER=<your-ssh-username>
SPARK_HOST=<your-remote-host-ip>
# Optional
VT_API_KEY=<your-virustotal-api-key> # For triage agent OSINT toolspython run_app.py --mode local # Local Ollama on localhost
python run_app.py --mode remote # Remote Ollama via SSH tunnel
python run_app.py --mode demo # Remote without builder agentThen open http://localhost:8050 in your browser.
| Mode | Command | Description |
|---|---|---|
| Local | python run_app.py --mode local |
Ollama on localhost:11434 |
| Capable | python run_app.py --mode capable |
Local Ollama, full model stack |
| Remote | python run_app.py --mode remote |
SSH tunnel to remote GPU host |
| Demo | python run_app.py --mode demo |
Remote, builder agent disabled |
python run_app.py --mode remote --spark-user <user> --spark-host <host-ip>
python run_app.py --mode remote --port 8051 # Alternate UI port
python run_app.py --mode remote --skip-tunnel-check # Skip connectivity testPre-configured scripts are provided in scripts/wrappers/:
| Platform | Local | Remote | Demo |
|---|---|---|---|
| macOS/Linux | ./scripts/wrappers/run_local.sh |
./scripts/wrappers/run_remote.sh |
./scripts/wrappers/run_demo_no_builder.sh |
| Windows | scripts\wrappers\run_local.bat |
scripts\wrappers\run_remote.bat |
scripts\wrappers\run_demo_no_builder.bat |
The Builder Agent enables co-creation between the user and the LLM. This is the core workflow for generating new security tools:
Type a natural language description of the tool you want in the Dash UI prompt:
"The purpose of this tool is to draft emails based on whether or not assets listed in a provided JSONL file are online or offline"
The builder agent generates a structured proposal with 7 sections:
- Capability Summary — What the tool does
- Design Decisions — Architecture choices
- File Manifest — Files to create/modify
- User Interaction Flow — How users trigger the capability from the UI
- Generated Code — Complete, runnable Python code
- Integration Steps — Deployment checklist
- Routing Update — How the tool connects to the existing request flow
Review the proposal. If issues are found, describe what needs to change:
"Fix the LCEL chain construction — use the template in the chain, not the formatted output. Escape {hostname} with double braces."
The builder regenerates the complete proposal with fixes applied.
When satisfied, type "done" to trigger:
- Sandbox creation — isolated git worktree on a dedicated branch
- Code extraction — parses code blocks from the proposal
- Stub detection — 25 regex patterns reject placeholder code
- AST validation — checks for dangerous calls, import violations, empty bodies
- Promotion — merges to main, auto-reloads the app
If the deployed tool has issues:
./scripts/rollback.shOr use git directly:
git revert <sandbox-commit-hash>The pipeline enforces 6 safety layers on all generated code:
| Layer | Check | Location |
|---|---|---|
| 1 | Stub/placeholder pattern scan (25 patterns) | sandbox.py |
| 2 | Line-count ratio check (≥25% of original) | sandbox.py |
| 3 | AST static analysis (blocked calls, import allowlist) | validation.py |
| 4 | Stub comment regex scan | validation.py |
| 5 | Syntax validation (py_compile) |
sandbox.py |
| 6 | 30-second execution timeout | validation.py |
This section documents the complete co-creation workflow from the April 27, 2026 test session. For full details, see docs/SESSION_REPORT_20260427.md.
User prompt in Dash UI:
"The purpose of this tool is to draft emails based on whether or not assets listed in a provided JSONL file are online or offline"
The builder generated a 3-file proposal. Two files were complete, but app.py contained stub markers:
# ... existing constants ...
# ... existing routing logic ...
# ... rest of the application ...Problem: These placeholder comments would overwrite the real app.py with a partial excerpt.
Action taken: Hardened the stub detection system from 6 to 25 regex patterns and added explicit prompt rules against excerpts and abbreviations.
After hardening, the second output had zero stubs but contained 2 runtime bugs:
- LangChain LCEL chain bug:
_EMAIL_PROMPT.format()returns a string, which can't be piped with LangChain's LCEL|operator - Template variable:
{hostname}in the system prompt was treated as a LangChain variable instead of a literal
Fix the following bugs in asset_email_agent.py:
1. LCEL chain construction is wrong. The correct pattern is:
chain = _EMAIL_PROMPT | llm | StrOutputParser()
email_text = chain.invoke({"asset_json": asset_json})
2. Escape {hostname} with double braces {{hostname}}
3. Remove the unused MessagesPlaceholder import.
All 3 bugs fixed. No regressions. Code deployed to sandbox, validated, and promoted to main.
$ OLLAMA_BASE_URL=http://localhost:11435 python3 -c "
from fusion_assistant_asset_email.asset_email_agent import run_asset_email_agent
emails = run_asset_email_agent('test_assets.jsonl')
print(f'Generated {len(emails)} emails')
"
Generated 2 email(s) (expected: 2 — offline + stale, skipping online)
The co-creation process revealed that the builder produced a working module but not a complete capability — the tool had no user-facing interaction path from the Dash UI. This led to adding a required "User Interaction Flow" section to the design template.
SOC_Platform_Builder_Agent/
├── README.md # This file
├── run_app.py # Cross-platform launcher with SSH tunnel
├── run_tests.py # Test suite runner
├── dash_ui_ReAct.py # Dash web UI (port 8050)
├── bootstrap_env.py # Cross-platform environment setup
├── prompts.py # Shared prompt templates
├── embeddings_oss.py # HuggingFace embedding configuration
├── requirements-direct.txt # Portable dependency list
├── requirements.txt # Frozen dependency set
│
├── fusion_assistant_query_builder/ # Query Builder agent
│ ├── app.py # Main routing and orchestration
│ ├── react_agent.py # ReAct agent implementation
│ ├── groups.py # Group chat system
│ ├── agents/ # LEQL agent implementations
│ ├── llm/ # LLM model configuration
│ ├── retrieval/ # FAISS vector store and retrievers
│ ├── io/ # Storage paths and file I/O
│ └── persistence/ # Chat history management
│
├── fusion_assistant_triage/ # Triage Agent
│ ├── triage_agent.py # ReAct triage with OSINT tools
│ ├── app.py # Triage orchestration
│ └── tools/ # VirusTotal, WHOIS, DuckDuckGo, Nmap
│
├── fusion_assistant_builder/ # Builder Agent (core innovation)
│ ├── builder_agent.py # Code generation with design template
│ ├── detection.py # Builder request detection
│ └── sub_agents/ # Modular pipeline
│ ├── introspection.py # Live codebase scanning
│ ├── doc_scraper.py # Library documentation retrieval
│ ├── lib_version.py # LangChain version awareness
│ └── validation.py # AST safety analysis + stub detection
│
├── fusion_assistant_sandbox/ # Sandbox Manager
│ └── sandbox.py # Git worktree blue-green deployment
│
├── config/ # Configuration
│ └── model_config.yaml # LLM provider and model settings
│
├── docs/ # Project documentation
│ ├── SETUP_AND_RUN_GUIDE.md # Detailed platform-specific setup
│ ├── CHANGES.md # Full changelog
│ ├── SUBAGENT_PIPELINE.md # Sub-agent pipeline developer guide
│ ├── SESSION_REPORT_20260427.md # Test session report (April 27)
│ └── CO_BUILDING_DESIGN.md # Co-creation design document
│
├── scripts/ # Utility and wrapper scripts
│ ├── bootstrap_env.bat / .sh # Platform-specific bootstrap wrappers
│ ├── rollback.sh # Sandbox rollback utility
│ └── wrappers/ # Pre-configured run scripts
│
├── tests/ # Automated test suite
├── test_data/ # Test alert fixtures
└── data/ # Runtime data (mostly gitignored)
└── test_fixtures/ # Sample data files
python run_tests.py # Cross-platform
./run_tests.sh # macOS/Linux
run_tests.bat # Windows- Environment bootstrap selection logic
- Launcher behavior and remote tunnel setup
- Sandbox code block extraction and stub detection
- Cross-platform restart script generation
Query Builder:
- Open http://localhost:8050
- Type: "I need a LEQL query for active directory failed logins"
- Verify the agent returns a properly formatted LEQL query
Triage Agent:
- Paste a security alert into the prompt
- Verify the agent performs OSINT lookups and generates a triage report
Builder Agent:
- Type: "Build a tool that [describe capability]"
- Review the 7-section proposal
- Iterate with revision prompts if needed
- Type "done" to deploy to sandbox
| Component | Author | Notes |
|---|---|---|
| Project architecture, Dash UI, query builder, triage agent, builder agent framework | Francis Hahn + Kritan Banstola | Original implementation |
| Sub-agent pipeline (introspection, doc scraper, lib version, validation) | Francis Hahn + Antigravity (AI assistant) | Co-created during development sessions. The pipeline design, prompt engineering, stub detection patterns, and safety layer architecture were developed through iterative co-building between the developer and the AI assistant. See docs/CO_BUILDING_DESIGN.md and docs/SESSION_REPORT_20260427.md for detailed records of this process. |
| Sandbox manager (blue-green deployment) | Francis Hahn + Antigravity | Git worktree-based sandbox with auto-reload mechanism |
| Generated code (e.g., asset email agent) | gpt-oss:20b (via Builder Agent) | Code generated by the LLM through the builder agent pipeline, validated by AST analysis and stub detection, reviewed by the developer |
All third-party dependencies are listed in requirements-direct.txt. Key libraries:
| Library | License | Purpose |
|---|---|---|
| LangChain | MIT | Agent framework, prompts, chains |
| Dash | MIT | Web UI framework |
| FAISS | MIT | Vector similarity search |
| Ollama | MIT | Local LLM inference |
| spaCy | MIT | NLP processing |
| sentence-transformers | Apache 2.0 | Embedding models |
| python-whois | MIT | WHOIS lookups (triage) |
| vt-py | Apache 2.0 | VirusTotal API (triage) |
| duckduckgo-search | MIT | Web search (triage, doc scraper) |
| BeautifulSoup | MIT | HTML parsing (doc scraper) |
| Tool | Usage |
|---|---|
| Antigravity (Google DeepMind) | AI coding assistant used for co-building the sub-agent pipeline, debugging dependency issues, and iterative code review. All interactions are documented in docs/SESSION_REPORT_20260427.md and docs/CO_BUILDING_DESIGN.md. |
| gpt-oss:20b (Ollama) | The LLM running on DGX Spark used by the builder agent to generate new tool code at runtime. Generated code is validated through the AST safety pipeline before deployment. |
| Document | Location | Description |
|---|---|---|
| README.md | Root | This file — setup, usage, and submission documentation |
| SETUP_AND_RUN_GUIDE.md | docs/ |
Detailed platform-specific setup instructions |
| CHANGES.md | docs/ |
Full changelog of all modifications |
| SUBAGENT_PIPELINE.md | docs/ |
Technical guide to the sub-agent pipeline architecture |
| SESSION_REPORT_20260427.md | docs/ |
Complete test session report with timeline, bugs found, and fixes |
| CO_BUILDING_DESIGN.md | docs/ |
Design document for internalizing the co-building workflow |
python3.11 bootstrap_env.py --recreate
# or on Windows:
py -3.11 bootstrap_env.py --recreate- Verify VPN or network connectivity to your remote host
- Test SSH manually:
ssh <your-user>@<your-host> - Verify Ollama is running on the remote host:
curl http://<your-host>:11434/api/tags - Check that local port 11435 is not occupied:
lsof -i :11435
# Find and kill the existing process
lsof -iTCP:8050 -sTCP:LISTEN
kill <PID>
# Then restart
python run_app.py --mode remotepython bootstrap_env.py --recreate