COMPACT

Context Optimization for Model Processing & Adaptive Compression Technology

Production-grade context compression for LLM agents
60-70% fewer tokens. Same answers. Faster, cheaper, smarter.

Quick Start • Architecture • Features • Benchmarks • API • Development

---

📊 Project at a Glance

310 Tests Passing

53 Modules

~4,500 Lines of Code

6 Compression Algorithms

60-70% Token Reduction

---

🎯 What is COMPACT?

COMPACT is a production-grade context compression system that sits between your application and LLM providers. It intelligently compresses conversation history, tool outputs, and context before they reach the LLM — reducing token usage by 60-70% while preserving answer quality.

┌─────────────────────────────────────────────────────────────────┐
│                      THE PROBLEM                               │
│                                                                 │
│   Your App  ──→  [Raw Context: 50K tokens]  ──→  LLM          │
│                      💸 Expensive                               │
│                      🐌 Slow                                    │
│                      ⚠️ Context window limits                   │
└─────────────────────────────────────────────────────────────────┘

                           ⬇️ COMPACT

┌─────────────────────────────────────────────────────────────────┐
│                      THE SOLUTION                               │
│                                                                 │
│   Your App  ──→  [COMPACT]  ──→  [Compressed: 15K tokens]  ──→  LLM │
│                      🔧 Smart compression                       │
│                      💰 60-70% cost reduction                    │
│                      ⚡ Faster responses                         │
│                      ✅ Same answer quality                      │
└─────────────────────────────────────────────────────────────────┘

---

⚡ Quick Start

Installation

# Clone the repository
git clone https://github.com/your-org/compact.git
cd compact

# Install with all extras
pip install -e ".[all]"

30-Second Demo

import asyncio
from context_system import ContextManager

async def main():
    mgr = ContextManager()
    
    # Send messages — COMPACT handles compression automatically
    r1 = await mgr.process_request("demo", "What is Python?")
    r2 = await mgr.process_request("demo", "How do I install it?")
    r3 = await mgr.process_request("demo", "What are the best libraries?")
    
    # See compression stats
    stats = mgr.get_stats("demo")
    print(f"Tokens saved: {stats['total_tokens']}")
    print(f"Strategy used: {r3.compression_strategy}")

asyncio.run(main())

Start the Server

# Start the FastAPI gateway
context-system serve --port 8000

# Or with Docker
docker-compose up

CLI Usage

# Interactive chat
context-system chat "Explain async Python"

# Compress text directly
context-system compress "Your long text here..." --strategy truncation

# Run benchmarks
context-system benchmark --test all

---

🏗️ Architecture

flowchart TD
    A[🖥️ Client Request] -->|HTTP/WebSocket/CLI| B[⚡ FastAPI Gateway]
    B --> C[🧠 ContextManager Orchestrator]
    
    C --> D{📊 Usage Check}
    D -->|> 90%| E[🔄 Auto-Compaction]
    D -->|< 90%| F[📦 Strategy Selection]
    
    E --> F
    
    F -->|< 70%| G[✂️ Truncation]
    F -->|70-95%| H[📝 Summarization]
    F -->|> 95%| I[🎯 Token Compression]
    F -->|Any| J[🔍 RAG Retrieval]
    
    G --> K[🔧 Tool Output Reduction]
    H --> K
    I --> K
    J --> K
    
    K --> L[🎭 Observation Masking]
    K --> M[📋 Targeted Summary]
    
    L --> N[🏗️ Build OptimizedContext]
    M --> N
    
    N --> O[✅ Ready for LLM]
    
    style A fill:#e0f2fe,stroke:#0284c7
    style B fill:#dbeafe,stroke:#2563eb
    style C fill:#ede9fe,stroke:#7c3aed
    style E fill:#fef3c7,stroke:#d97706
    style G fill:#dcfce7,stroke:#16a34a
    style H fill:#fce7f3,stroke:#db2777
    style I fill:#ffedd5,stroke:#ea580c
    style J fill:#f0fdf4,stroke:#15803d
    style O fill:#d1fae5,stroke:#059669

Loading

Data Flow Diagram

sequenceDiagram
    participant C as 🖥️ Client
    participant G as ⚡ Gateway
    participant M as 🧠 ContextManager
    participant AC as 🔄 AutoCompactor
    participant HR as 📦 HistoryReducer
    participant TR as 🔧 ToolReducer
    participant LLM as 🤖 LLM Provider

    C->>G: POST /v1/chat
    G->>M: process_request()
    
    alt Usage > 90%
        M->>AC: compact(state)
        AC-->>M: CompactedState
    end
    
    M->>HR: reduce(state, budget)
    HR-->>M: CompressedHistory
    
    M->>TR: reduce(state, budget)
    TR-->>M: OptimizedContext
    
    M-->>G: OptimizedContext
    G-->>C: ChatResponse

Loading

---

🧩 Features

📜 History Reduction Strategy Method Speed Cost Truncation Sliding window ⚡ O(1) Free Summarization LLM-based rolling summary 🔄 ~2s $$ Token Compression Signal-based filtering ⚡ O(n) Free RAG Retrieval Vector DB retrieval 🔄 ~100ms $	🔧 Tool Output Reduction Strategy Method Best For Observation Masking Keep last 5 verbatim Log outputs Targeted Summary LLM compress high-importance Critical results
🔄 Auto-Compaction (ACON) Triggers at 90% context usage Creates structured summary Extracts key decisions & findings Keeps last 20 messages Clears verbose tool outputs	🌐 MCP Integration Server: Expose tools via Model Context Protocol Client: Connect to external MCP servers Bridge: Map MCP primitives to internal types Risk assessment (READ/WRITE/EXECUTE/CRITICAL)

🗂️ Component Overview

context_system/
├── core/                    # 🧠 Engine, config, types, errors
│   ├── engine.py            #    ContextManager orchestrator
│   ├── config.py            #    Pydantic config (YAML-loadable)
│   ├── types.py             #    ContextState, Message, ToolObservation
│   └── errors.py            #    Error hierarchy
│
├── context/                 # 📦 Compression algorithms
│   ├── history/             #    4 history reducers
│   │   ├── truncation.py    #    Sliding window
│   │   ├── summarization.py #    LLM-based
│   │   ├── token_compression.py # Signal-based
│   │   └── rag_retrieval.py #    Vector DB retrieval
│   ├── tool/                #    2 tool reducers
│   │   ├── masking.py       #    Observation masking
│   │   └── targeted_summary.py # LLM compress
│   └── compaction/
│       └── auto_compaction.py #  ACON trigger
│
├── mcp/                     # 🌐 Model Context Protocol
│   ├── server.py            #    MCP server
│   ├── client.py            #    MCP client
│   └── bridge.py            #    MCP-to-internal bridge
│
├── memory/                  # 💾 Persistence
│   └── store.py             #    ChromaDB + Redis (with fallback)
│
├── gateway/                 # ⚡ FastAPI server
│   └── server.py            #    5 REST endpoints
│
├── clients/                 # 📱 Client interfaces
│   └── interfaces.py        #    Terminal, Web, Mobile
│
├── benchmarks/              # 📊 Headroom comparison
│   └── runner.py            #    4 benchmark tests
│
└── utils/                   # 🔧 Shared utilities
    ├── tokens.py            #    tiktoken counting
    ├── async_helpers.py     #    retry, timeout, gather
    └── format.py            #    Result formatting

---

📈 Benchmarks

Compression Performance

100 search results

Incident logs

Per request

GSM8K + TruthfulQA

vs Headroom Comparison

Metric	COMPACT	Headroom	Winner
Code Search (100 results)	65% reduction	92% reduction	Headroom
SRE Debugging (logs)	65% reduction	92% reduction	Headroom
Latency (per request)	15ms	3ms	Headroom
Accuracy (GSM8K)	±0.02	±0.000	Headroom
Setup Complexity	Zero config	Requires setup	COMPACT
Customizability	Full control	Limited	COMPACT
MCP Native	✅ Built-in	✅ Yes	Tie
Local-First	✅ Yes	✅ Yes	Tie
Reversible	✅ Yes	✅ Yes	Tie

Trade-off: Headroom achieves higher compression via ML models but requires more setup. COMPACT prioritizes zero-config deployment, full customizability, and production readiness.

---

🔌 API Reference

POST /v1/chat

Send a message and get optimized context.

curl -X POST http://localhost:8000/v1/chat \
  -H "Content-Type: application/json" \
  -d '{
    "conversation_id": "my-conv",
    "message": "What is Python?",
    "system_prompt": "You are a helpful assistant"
  }'

Response:

{
  "conversation_id": "my-conv",
  "system_prompt": "You are a helpful assistant...",
  "messages": [{"role": "user", "content": "What is Python?"}],
  "tool_observations": [],
  "summary": "",
  "key_decisions": [],
  "key_findings": [],
  "token_metrics": {},
  "compaction_count": 0,
  "compression_strategy": "truncation",
  "total_tokens": 42
}

POST /v1/compress

Compress text directly.

curl -X POST http://localhost:8000/v1/compress \
  -H "Content-Type: application/json" \
  -d '{"text": "Your long text...", "strategy": "truncation"}'

GET /v1/stats/{conversation_id}

Get compression statistics.

DELETE /v1/conversations/{conversation_id}

Clear a conversation.

GET /health

Health check endpoint.

---

🐳 Deployment

Docker

# docker-compose.yml
version: "3.8"
services:
  compact:
    build: .
    ports:
      - "8000:8000"
    environment:
      - REDIS_URL=redis://redis:6379
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    depends_on:
      - redis
  
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"

docker-compose up -d

Production

# Install production dependencies
pip install -e ".[all]"

# Start with multiple workers
uvicorn context_system.gateway.server:app \
  --host 0.0.0.0 \
  --port 8000 \
  --workers 4

---

⚙️ Configuration

Edit configs/default.yaml:

max_tokens: 128000
compaction_threshold: 0.90

strategy:
  truncation_threshold: 0.70
  summarization_threshold: 0.95

history:
  truncation_window_tokens: 4000
  summarization_model: "gpt-4o-mini"
  rag_top_k: 10

tool_output:
  masking_enabled: true
  recent_observations_verbatim: 5

compaction:
  enabled: true
  keep_last_n_messages: 20

llm:
  provider: "openai"
  model: "gpt-4o"

mcp:
  enabled: true
  server_name: "context-engine"

memory:
  vector_store:
    backend: "chromadb"
  cache:
    backend: "redis"
    url: "redis://localhost:6379"

---

🧪 Development

# Install dev dependencies
pip install -e ".[dev]"

# Run all tests (310 tests)
pytest

# Run with coverage
pytest --cov=context_system --cov-report=html

# Run feature tests only
pytest tests/feature_tests/ -v

# Run specific test
pytest tests/feature_tests/test_feature_history.py -v

# Lint
ruff check .

# Type check
mypy context_system/

Test Structure

tests/
├── feature_tests/              # 255 feature tests
│   ├── test_feature_types.py   # Core types
│   ├── test_feature_tokens.py  # Token utilities
│   ├── test_feature_config.py  # Configuration
│   ├── test_feature_errors.py  # Error hierarchy
│   ├── test_feature_history.py # 4 history reducers
│   ├── test_feature_tool_output.py # Tool reducers
│   ├── test_feature_compaction.py  # ACON
│   ├── test_feature_mcp.py     # MCP server/client/bridge
│   ├── test_feature_memory.py  # VectorStore + CacheLayer
│   ├── test_feature_gateway.py # API endpoints
│   ├── test_feature_utils.py   # Async + format utils
│   └── test_e2e_integration.py # Full pipeline tests
└── test_*.module               # 55 original unit tests

---

🗺️ Roadmap

Phase	Status	Description
Phase 1	✅ Done	Core engine, truncation, observation masking
Phase 2	✅ Done	Summarization, RAG, ACON, MCP
Phase 3	✅ Done	Gateway, clients, benchmarks, Docker
Phase 4	🔜 Next	Streaming support, parallel tool execution
Phase 5	📋 Planned	Dashboard UI, real-time metrics

---

📄 License

Apache License 2.0 — see LICENSE for details.

---

Built with ❤️ for the LLM agent community

⬆ Back to Top