Holoken-RL: CPU-native Topological Memory for Multi-Agent RL

Lightweight associative memory for RL agents. No GPUs. No vector DBs. Pure topology.

---

The Problem

Current RL agents rely on flat vector databases (FAISS, Annoy) or fixed context windows for memory. This is:

• Slow — brute-force similarity search scales linearly with the number of experiences.
• Heavy — FAISS alone can add 500 MB+ to your runtime and requires AVX2/SIMD optimized builds.
• Flat — cosine similarity treats all dimensions equally, ignoring the hierarchical structure of agent states (e.g., position → energy → high-level intent).

The result: agents forget, duplicate effort, and struggle to transfer knowledge across similar situations.

---

The Solution

Holoken-RL projects high-dimensional agent states onto a 2D Poincaré disk using a hierarchical tensor-network substrate. Instead of brute-force cosine similarity, agents retrieve past experiences using hyperbolic geodesic distances on the boundary.

Agent States (16D)
       ↓
PCA / Random Projection
       ↓
Poincaré Disk (2D hyperbolic coordinates)
       ↓
Pentagonal Tensor Network {5,4} — hierarchical bulk
       ↓
Boundary Field — concatenated feature vectors
       ↓
Geodesic Similarity Search (hyperbolic distance)

Why this works

1. Hierarchical clustering for free — In hyperbolic space, similar states naturally cluster. The distance between two points encodes both semantic similarity and hierarchical depth.
2. Dirty-subgraph updates — When one agent moves, only the nodes on the path from its bulk position to the boundary are recomputed. The rest of the network stays frozen.
3. Zero external dependencies for inference — Everything runs on NumPy. No CUDA drivers, no 500 MB index files, no compilation step.

---

Performance

Metric	Value
Throughput	> 400,000 agent updates / sec (MacBook Air M2, single core)
Latency @ 5,000 agents	12.26 ms / step
Memory footprint	< 50 MB for 5,000 agents
Cold start	No index building — network is created in < 1 ms
Dependencies	numpy, scikit-learn (PCA only)

Benchmarked on Apple M2, Python 3.12, NumPy 1.26.

Run the benchmark yourself:

python benchmarks/incremental_benchmark.py

---

Quick Start

from holoken_rl import CognitiveMemoryBridge
import numpy as np

# 1. Create memory substrate for 1,000 agents with 16D states
memory = CognitiveMemoryBridge(n_agents=1000, state_dim=16)

# 2. Ingest a batch of agent states
states = np.random.randn(100, 16).astype(np.float32)
boundary = memory.ingest(states)

# 3. Find agents most similar to agent #0
similar = memory.find_similar(agent_id=0, k=5)
# [(agent_id, hyperbolic_distance), ...]

🧬 The Genesis Experiment (demos/genesis_v4.py)

We test Holoken's topological memory in a brutal, resource-constrained simulation:

• Environment: 64×64 field with 3% lethal trap density.
• Agent Perception: High-dimensional 5×5 spatial patches (126D raw state), externally compressed via sklearn.decomposition.PCA to 16D, then projected to the Poincaré disk.
• Mortality Rate: 84% (Extreme selection pressure).
• Result: Agents equipped with Holoken-RL show a +5.0% survival advantage over agents relying on pure reactive reflexes. Memory of spatial traps crystallizes directly into the hyperbolic topology.

---

Project Structure

holoken_rl/
├── holoken_rl/           # Core memory substrate
│   ├── poincare.py       # Poincaré disk geometry & geodesics
│   ├── hyperbolic.py     # 3D AdS volume model (optional depth)
│   ├── tensor_network.py # Pentagonal tiling {5,4} graph
│   ├── incremental.py    # HierarchicalFeatureNetwork + DynamicAgentSimulator
│   ├── cognitive_memory.py  # Main API: CognitiveMemoryBridge
│   └── neuropl_bridge.py    # Agent dataclass + decision support
├── demos/
│   └── demo_virtual_world.py   # 100-agent survival colony
├── benchmarks/
│   └── incremental_benchmark.py # Throughput & latency report
├── tests/
│   └── test_cognitive_memory.py # pytest suite
├── WHITE_PAPER.md
├── setup.py
└── README.md

---

How it differs from FAISS / Vector DBs

	FAISS / Annoy	Holoken-RL
Search primitive	Cosine / L2 distance	Hyperbolic geodesic distance
Index build	O(N log N) — minutes for millions	O(1) — network topology is fixed
Update cost	Rebuild index or maintain HNSW graph	O(log N) dirty subgraph only
Memory	200–1000 MB for 1M vectors	< 50 MB for 5,000 agents (scales with network layers, not agents)
Hierarchy	Flat	Native — bulk → boundary reflects state abstraction
Hardware	AVX2 / GPU preferred	Runs on a Raspberry Pi

---

Roadmap

• PyPI package (pip install holoken-rl)
• Gymnasium integration — HolokenMemory wrapper for any Env
• Ray RLlib connector for distributed multi-agent training
• GPU backend (JAX) for >1M agent scales
• Persistent disk serialization of boundary fields

---

Citation

If you use Holoken-RL in research, please cite:

@software{holokenrl2024,
  title={Holoken-RL: CPU-native Topological Memory for Multi-Agent RL},
  author={Holoken Research},
  year={2024},
  url={https://github.com/holoken/holoken-rl}
}

---

License

MIT — free for academic and commercial use.

---

Built with pure NumPy and a lot of coffee.