agent-muni

A local, offline coding agent -- built from scratch to understand how tools like Cursor and Kiro actually work under the hood. Runs entirely on a CPU-only laptop, no GPU, no API key, no per-token cost.

Why

Most "I built an AI agent" projects wrap a cloud API. This one runs the full loop -- reasoning, tool-calling, memory, retrieval -- on local hardware via Ollama (https://ollama.com), to understand the real engineering tradeoffs: which local models actually support reliable tool-calling, how session state has to be structured to survive multi-turn tool use, and where a zero-cost local stack genuinely holds up versus where it doesn't.

Features

• Interactive agent session (agent-muni) -- a persistent, Kiro/Cursor-style REPL. Type a task, watch it reason and call tools, keep chatting in the same context until /quit.
• Real tool-calling, not prompted JSON-in-text -- file read/write, directory listing, code search (grep), and a small allowlisted shell command set.
• Persistent memory, two layers:
  • SQLite (~/.ai-memory-vault/global_brain.db) -- the source of truth, fast keyword (FTS5) search.
  • ChromaDB + sentence-transformers (all-MiniLM-L6-v2) -- semantic/meaning-based search over the same memories, fully local.
• mem update -- a trigger phrase that reviews the current session and saves confirmed findings back to memory automatically.
• Web search fallback (DuckDuckGo via ddgs, no API key) -- used automatically when local memory has nothing relevant.
• Dynamic model routing -- routes to the model that's actually proven reliable for tool-calling on this hardware (see Known Limitations below).

Architecture

User (interactive session)
     |
     v
LangGraph ReAct agent  ----  Ollama (llama3.1:8b)
     |
     |--> read_file_tool / write_file_tool / list_dir_tool
     |--> search_code_tool (grep)
     |--> run_shell_tool (allowlisted commands only)
     |--> memory_search_tool -----------> SQLite + FTS5
     |--> memory_semantic_search_tool --> ChromaDB (sentence-transformers)
     |--> memory_save_tool --------------> SQLite (source of truth)
     |--> memory_sync_embeddings_tool --> re-indexes SQLite into ChromaDB
     |--> web_search_tool ---------------> DuckDuckGo (fallback only)

Tech Stack

Layer	Choice	Why
LLM runtime	Ollama	Zero-cost, CPU-friendly, easy local model swapping
Agent orchestration	LangGraph	ReAct pattern, real tool-calling, checkpointable state
Structured memory	SQLite + FTS5	Zero dependency, fast, permanent source of truth
Semantic memory	ChromaDB + sentence-transformers	Local embeddings, no API key, meaning-based recall
Web fallback	DuckDuckGo (ddgs)	No API key required
Package management	uv	Fast, modern Python dependency management

Setup

Requires Ollama (https://ollama.com) and uv (https://docs.astral.sh/uv/) installed.

git clone https://github.com/munikumar-pulikanti/hearthagent.git
cd hearthagent
uv sync

ollama pull llama3.1:8b
ollama pull nomic-embed-text

uv run python main.py

Usage

$ agent-muni
you> list the files in this project and tell me what's here
you> read main.py and explain what it does
you> remember that: FINDING. save with scope 'x', type 'known-bug'
you> mem update
you> /quit

One-shot mode (no session, single task):

uv run python main.py "your task here"

Known Limitations

• Tool-calling reliability varies significantly by model, even among models that advertise tools support in Ollama. qwen2.5-coder:14b produces excellent code but frequently prints a well-formed tool call as JSON text instead of triggering an actual function call -- llama3.1:8b has proven far more reliable for this project's tool-heavy workload. This is the single biggest practical finding from building this: test tool-calling reliability directly, don't assume it from a capability badge.
• CPU-only inference is slow for larger models (14B+); expect multi-minute responses for heavier tasks.
• The allowlisted shell command set is intentionally narrow -- this is a personal dev tool, not a sandboxed production system.

Roadmap

• Iteration/loop limits to prevent runaway agent loops
• Containerize + add OpenTelemetry tracing
• Evaluate devstral / qwen3-coder for more reliable code-writing tool calls

License

MIT

Privacy and security notes

• web_search_tool sends your query text to DuckDuckGo. Nothing else in this project makes network calls.
• run_shell_tool executes real shell commands, restricted to an allowlist (see agent/tools.py). Review the allowlist before running this on a machine you care about.
• The metrics database logs a snippet of every task you send the agent, stored locally at ~/.ai-memory-vault/agent_metrics.db. It never leaves your machine, but avoid pasting secrets into prompts if that matters to you.
• No API keys or credentials are used or stored anywhere in this project.