ReasoningBank

A minimal, self-learning agent loop: an LLM agent that distills lessons from its own trajectories and retrieves them by semantic similarity for future tasks. No fine-tuning, no RL — just embed, store, retrieve, inject.

Loosely inspired by the ReasoningBank paradigm: agents accumulate reusable strategies (and pitfalls) from their own past, indexed by meaning rather than recency.

---

Why this exists

Most LLM agents are stateless — they make the same mistake forever. ReasoningBank gives an agent a persistent memory of past lessons, retrieved by semantic similarity, and injected into the next prompt. Empirically: with a populated bank, the agent uses fewer reasoning tokens and converges on correct approaches faster.

task ── search bank ── inject lessons ──► LLM solve ──► trajectory
                                                            │
                                                            ▼
                            ◄── store ◄── distill ◄── judge SUCCESS / FAILURE

Three roles, three models (defaults):

Role	Model	Job
Agent	claude-sonnet-4-6	Solver. Reads top-k lessons. Produces trajectory.
Judge	claude-haiku-4-5	Verdict. One token: SUCCESS or FAILURE.
Distiller	claude-haiku-4-5	Extracts a reusable strategy (success) or pitfall (failure).

---

Quickstart

Requirements

• Python 3.10+
• Anthropic API key

Install

git clone https://github.com/awaemmanuel/reasoningbank.git reasoningbank
cd reasoningbank
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
export ANTHROPIC_API_KEY=sk-ant-...

Run the demo

python run.py

Output (truncated):

Starting bank: 0 patterns stored

=== Task 1/3 ===
Write a Python function that returns the nth Fibonacci number using memoization.

Memories used: 0
Verdict: SUCCESS
Distilled lesson:
  Use functools.lru_cache on the recursive helper to memoize subproblems...
Stored as id=1

=== Task 2/3 ===
...

Re-run and the bank persists across sessions (reasoningbank.db).

CLI flags

python run.py --help

Flag	Meaning
--db PATH	SQLite path (default reasoningbank.db).
-k N	Memories to retrieve per task (default 3).
--no-persist	Run tasks without storing new lessons.
--no-mix	Disable verdict-balanced retrieval (top-k by score only).
--clear	Wipe the bank before running.
--list	Print bank contents and exit.
--no-prewarm	Skip embed model prewarm (slower first call).
[tasks ...]	Positional tasks override the built-in demo set.

# fresh demo
python run.py --clear

# inspect what's stored
python run.py --list

# your own tasks
python run.py "implement Dijkstra's shortest path" "detect a cycle in a linked list"

# wider retrieval, no verdict mixing
python run.py -k 5 --no-mix

---

Architecture

┌──────────────┐    embed     ┌──────────────────┐
│   task str   │ ────────────►│  query vector    │
└──────────────┘              └────────┬─────────┘
                                       │ matmul
                                       ▼
                              ┌──────────────────┐
                              │  bank matrix     │   in-memory float32 ndarray
                              │  (lazy load)     │   rebuilt on dirty flag
                              └────────┬─────────┘
                                       │ top-k mixed (SUCCESS + FAILURE)
                                       ▼
                              ┌──────────────────┐
                              │  agent prompt    │
                              │  + lessons block │
                              └────────┬─────────┘
                                       │ Sonnet
                                       ▼
                              ┌──────────────────┐    judge
                              │   trajectory     │ ──────────► SUCCESS | FAILURE
                              └────────┬─────────┘    Haiku
                                       │ distill (Haiku)
                                       ▼
                              ┌──────────────────┐
                              │  lesson string   │
                              └────────┬─────────┘
                                       │ embed + dedup (cosine ≥ 0.95)
                                       ▼
                              ┌──────────────────┐
                              │  SQLite WAL      │
                              │  patterns table  │
                              └──────────────────┘

File layout

File	Role
src/embed.py	BGE-small-en-v1.5 via fastembed, L2-normalized 384-dim. lru_cache(2048).
src/bank.py	SQLite + in-memory ndarray. store, search, search_mixed, count, clear. WAL, indexes, dedup.
src/distill.py	judge + distill LLM calls. _call wraps with retry + exponential backoff.
src/agent.py	run_task — one full loop iteration.
src/types.py	TypedDicts: Memory, TaskResult, Verdict.
run.py	CLI driver.
tests/	pytest suite. Anthropic mocked.

Storage schema

CREATE TABLE patterns (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    task TEXT NOT NULL,        -- original task string
    pattern TEXT NOT NULL,     -- distilled lesson
    verdict TEXT NOT NULL,     -- SUCCESS | FAILURE
    embedding BLOB NOT NULL,   -- 384-dim float32 of pattern
    created_at REAL NOT NULL,
    metadata TEXT              -- JSON
);
CREATE INDEX idx_patterns_created ON patterns(created_at DESC);
CREATE INDEX idx_patterns_verdict ON patterns(verdict);

Embeddings are stored as raw float32 bytes. On bank open they are stacked into a single ndarray; search is one matrix @ query_vec matmul.

Verdict-mixed retrieval

Naive top-k can return only successes, hiding pitfalls. search_mixed(k) returns ⌈k/2⌉ SUCCESS lessons + ⌊k/2⌋ FAILURE lessons (highest-scored from each side), then re-sorts by score. The agent sees both do this and avoid that.

Dedup on store

Before insert, the new lesson's embedding is compared against the full matrix. If max cosine ≥ 0.95 (configurable per-bank), the insert is skipped and store() returns None. Prevents lesson bloat across repeated runs.

API retry

Anthropic calls go through _call() which catches APIError/APIStatusError, retries 3× with exponential backoff (0.75s → 1.5s → 3.0s), and skips retry on non-retryable 4xx (400/401/403/404/etc.).

---

Public API

from src import ReasoningBank, run_task

with ReasoningBank(db_path="reasoningbank.db") as bank:
    result = run_task(
        "write a function to detect a cycle in a linked list",
        bank,
        k=3,                # how many memories to retrieve
        persist=True,       # store distilled lesson back
        mix_verdicts=True,  # use search_mixed
    )
    print(result["verdict"], result["pattern"])

Low-level bank ops:

bank.store(task, pattern, verdict, metadata={"model": "claude-sonnet-4-6"})
bank.search(query, k=5, verdict=None)        # verdict optional filter
bank.search_mixed(query, k=4)                # SUCCESS + FAILURE balanced
bank.all()                                   # full dump
bank.count()
bank.clear()

---

Environment variables

Variable	Default	Purpose
ANTHROPIC_API_KEY	—	Required. Anthropic API auth.
RB_AGENT_MODEL	claude-sonnet-4-6	Solver model.
RB_MODEL	claude-haiku-4-5-20251001	Judge + distiller model.

---

Development

pip install -r requirements.txt
python -m pytest -q

Tests mock the Anthropic client so they run without an API key. The first test invocation will download the BGE-small-en-v1.5 model (~30MB) into ~/.cache/fastembed; subsequent runs hit the cache.

---

Design notes

Why SQLite + ndarray, not a vector DB? This is a demo. At ~100s of patterns, brute matmul over 384-dim vectors is microseconds. HNSW / DiskANN / FAISS only pays off above ~10k rows.

Why BGE-small, not OpenAI/Voyage embeddings? Local, free, 384-dim, ~30MB ONNX model, good enough for English. Zero round-trip latency. No PyTorch dependency.

Why TypedDict, not dataclass? A Memory is naturally a dict shape coming out of the DB row. TypedDict gives type hints with no runtime cost.

Why anchored regex in judge? Trajectories can contain the word "success" in prose. \b(SUCCESS|FAILURE)\b over uppercased output binds the verdict to the model's literal one-token reply.

Why prewarm? The first embed() call loads the ONNX model from disk (~3s). prewarm() hides that latency before the task loop starts.

---

Extending

Want	Approach
Scale to 10k+ patterns	Swap brute matmul for HNSW (e.g. hnswlib) or DiskANN.
Forget useless lessons	Track retrieval count + age; prune low-utility (EWC++ style).
Hierarchical memory	Cluster patterns, store abstract + specific tiers.
Multi-agent learning	Share a single bank across agents → swarm-level skill transfer.
Different LLMs	Swap Anthropic() client; agent/judge/distill prompts are model-agnostic.
Trajectory analytics	Persist full trajectory + tokens; do offline analysis on success patterns.

---

Contributing

Issues and pull requests welcome. Please:

1. Open an issue first for non-trivial changes so we can align on scope.
2. Keep PRs focused — one logical change per PR.
3. Run python -m pytest -q locally before pushing; CI will run it too.
4. Match the existing style (type hints, no docstrings unless non-obvious, no comment fluff).

License

MIT © 2026 ReasoningBank Contributors