Skip to content

Latest commit

 

History

History
440 lines (368 loc) · 19.7 KB

File metadata and controls

440 lines (368 loc) · 19.7 KB

ADR-002: Context Window Management Strategy

Status

Proposed

Context

LobeHub v2 operates within strict context window constraints:

Model Context Window Practical Limit Latency Threshold
GPT-4 128K tokens ~40K tokens >40K degrades
Claude 3 200K tokens ~50K tokens >50K degrades
Claude 3.5 200K tokens ~60K tokens >60K degrades

Problems observed:

  • Large CSV files (>100KB raw) quickly consume available context
  • Loading full datasets into conversation history is wasteful
  • Repeated queries on same data re-send entire datasets
  • Complex multi-step analysis accumulates token usage exponentially
  • Visualization data (base64 images) is token-expensive

Decision

Implement a tiered context management system with intelligent compression, prioritization, and incremental loading.

Core Principles

  1. Token Budgeting: Pre-allocate token budgets by component
  2. Progressive Disclosure: Load data incrementally based on relevance
  3. Compression Strategies: Multiple techniques to minimize token usage
  4. Caching at Context Level: Avoid re-sending unchanged information
  5. Smart Eviction: Remove low-priority content before high-priority

Context Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                          Context Window (e.g., 50k tokens)                   │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │ TIER 1: IMMUTABLE (Always Present)                                  │   │
│  │ ┌─────────────────────────────────────────────────────────────────┐ │   │
│  │ │ System Prompt + Core Instructions                               │ │   │
│  │ │ Budget: 500 tokens    Priority: CRITICAL    Eviction: NEVER     │ │   │
│  │ └─────────────────────────────────────────────────────────────────┘ │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │ TIER 2: CONVERSATION (Managed by LobeHub)                           │   │
│  │ ┌─────────────────────────────────────────────────────────────────┐ │   │
│  │ │ Recent Messages + Tool Results                                  │ │   │
│  │ │ Budget: 2000 tokens   Priority: HIGH        Eviction: LRU       │ │   │
│  │ └─────────────────────────────────────────────────────────────────┘ │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │ TIER 3: SCHEMA (Controlled by MCP)                                  │   │
│  │ ┌─────────────────────────────────────────────────────────────────┐ │   │
│  │ │ Active Dataset Schemas                                          │ │   │
│  │ │ Budget: 300 tokens    Priority: HIGH        Eviction: LFU       │ │   │
│  │ └─────────────────────────────────────────────────────────────────┘ │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │ TIER 4: SAMPLES (On-Demand Loading)                                 │   │
│  │ ┌─────────────────────────────────────────────────────────────────┐ │   │
│  │ │ Representative Data Samples                                     │ │   │
│  │ │ Budget: 800 tokens    Priority: MEDIUM      Eviction: FIFO      │ │   │
│  │ └─────────────────────────────────────────────────────────────────┘ │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │ TIER 5: RESULTS (Dynamic Content)                                   │   │
│  │ ┌─────────────────────────────────────────────────────────────────┐ │   │
│  │ │ Query Results + Aggregations + Visualizations                   │ │   │
│  │ │ Budget: 1500 tokens   Priority: MEDIUM      Eviction: AGE       │ │   │
│  │ └─────────────────────────────────────────────────────────────────┘ │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │ TIER 6: BUFFER (Reserved)                                           │   │
│  │ ┌─────────────────────────────────────────────────────────────────┐ │   │
│  │ │ Reserved for Response Generation                                │ │   │
│  │ │ Budget: 1000 tokens   Priority: LOW         Eviction: N/A       │ │   │
│  │ └─────────────────────────────────────────────────────────────────┘ │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │ TIER 7: EVICTABLE (Compress/Remove First)                           │   │
│  │ ┌─────────────────────────────────────────────────────────────────┐ │   │
│  │ │ Old Results + Large Visualizations + Cached Queries             │ │   │
│  │ │ Budget: Variable      Priority: LOW         Eviction: AGGRESSIVE│ │   │
│  │ └─────────────────────────────────────────────────────────────────┘ │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Token Budget Allocation

Tier Component Budget (tokens) % of Total Eviction Strategy
1 System Prompt 500 10% Never
2 Conversation 2000 40% LRU (Last Recent Use)
3 Schema 300 6% LFU (Least Frequently Used)
4 Samples 800 16% FIFO (First In, First Out)
5 Results 1500 30% Age-based
6 Buffer 1000 20% Reserved
Total Reserved ~6100 ~122% Dynamic adjustment

Note: Percentages sum >100% because tiers are dynamically balanced based on current needs

Compression Strategies

1. Schema Compression

Before: Full column details with types, samples, stats

{
  "columns": [
    {"name": "customer_id", "type": "int64", "nullable": false, "unique": 15000},
    {"name": "revenue", "type": "float64", "nullable": true, "min": 0.99, "max": 9999.99}
  ]
}

After: Abbreviated schema

{"schema": "customer_id:int!,revenue:float[0.99-9999.99]"}

Compression ratio: 5-10x

2. Tabular Data Compression

Before: Full JSON array

[
  {"category": "Electronics", "revenue": 150000, "orders": 523},
  {"category": "Clothing", "revenue": 89000, "orders": 1205}
]

After: Markdown table

| Category | Revenue | Orders |
|----------|---------|--------|
| Electronics | 150K | 523 |
| Clothing | 89K | 1.2K |

Compression ratio: 2-3x

3. Numeric Compression

Before: Full precision

{"value": 1234567.8912345}

After: Human-readable

{"value": "1.23M"}

Compression ratio: Variable

4. Sample Row Reduction

Strategy: Reduce sample rows when context pressure increases

  • Normal: 20 rows
  • Pressure: 10 rows
  • Critical: 5 rows

5. Column Selection

Strategy: Include only referenced columns

# User asks about "revenue by category"
# Only include: category, revenue (not: id, timestamp, metadata, etc.)

Context State Management

interface ContextState {
  // Current token usage by tier
  usage: {
    tier1_system: number;
    tier2_conversation: number;
    tier3_schema: number;
    tier4_samples: number;
    tier5_results: number;
    tier6_buffer: number;
  };
  
  // Active datasets in context
  activeDatasets: Map<string, DatasetContext>;
  
  // Compression level (0-3)
  compressionLevel: 0 | 1 | 2 | 3;
  
  // Last access times for eviction
  accessLog: Array<{
    tier: number;
    component: string;
    timestamp: Date;
    tokenCount: number;
  }>;
}

interface DatasetContext {
  filePath: string;
  schema: Schema;
  sample: any[];
  lastQuery: string;
  lastResult: any;
  tokenCount: number;
  priority: 'high' | 'medium' | 'low';
  lastAccessed: Date;
}

Adaptive Compression Algorithm

def manage_context_pressure(context_state):
    """
    Adjust compression and eviction based on current token usage
    """
    total_used = sum(context_state.usage.values())
    window_size = get_context_window_size()
    
    usage_ratio = total_used / window_size
    
    if usage_ratio < 0.5:
        # Low pressure: minimal compression
        context_state.compression_level = 0
        
    elif usage_ratio < 0.7:
        # Medium pressure: light compression
        context_state.compression_level = 1
        compress_schemas(aggressive=False)
        
    elif usage_ratio < 0.85:
        # High pressure: aggressive compression
        context_state.compression_level = 2
        compress_schemas(aggressive=True)
        reduce_samples(target_rows=10)
        compress_numbers(precision=2)
        
    else:
        # Critical pressure: maximum compression + eviction
        context_state.compression_level = 3
        reduce_samples(target_rows=5)
        compress_numbers(precision=1)
        evict_low_priority_items()
        truncate_old_results(max_age=5)

Incremental Loading Pattern

User: "Analyze this large dataset"
    │
    ▼
Step 1: Load Schema Only
    ├─ Send: Column names, types, row count
    ├─ Tokens: ~200
    └─ Context: "Dataset has 5M rows, 25 columns..."
    │
    ▼
Step 2: Load Sample (if requested)
    ├─ Send: 20 representative rows
    ├─ Tokens: ~800
    └─ Context: "Sample data: [...]"
    │
    ▼
Step 3: Execute Query
    ├─ Process: Full aggregation in Code-Server
    ├─ Send: Only aggregated results
    ├─ Tokens: ~500
    └─ Context: "Results: {aggregates}"
    │
    ▼
Step 4: Drill-down (if needed)
    ├─ Send: Filtered subset
    ├─ Tokens: ~600
    └─ Context: "Filtered results: [...]"

Smart Eviction Policies

1. Dataset Eviction

When new dataset is loaded and space is needed:

eviction_priority = [
    # Lowest priority: Remove samples first
    lambda ds: (ds.lastAccessed > 5min_ago, ds.samples),
    
    # Medium priority: Remove old results
    lambda ds: (ds.lastQuery != current_query, ds.lastResult),
    
    # High priority: Remove schema (rarely)
    lambda ds: (ds.priority == 'low', ds.schema),
]

2. Result Eviction

def should_evict_result(result):
    # Evict if:
    # 1. Result is older than 10 minutes
    # 2. Result size > 1000 tokens and newer results exist
    # 3. Result not referenced in recent queries
    return (
        result.age > 600 or
        (result.token_count > 1000 and newer_results_exist()) or
        result.reference_count == 0
    )

Context-Aware API Design

All MCP tools return context_tokens_used field:

{
  "result": {...},
  "context_tokens_used": 450,
  "context_metadata": {
    "compression_applied": "schema_abbreviated",
    "rows_included": 15,
    "columns_included": 5,
    "eviction_recommendation": "none"
  }
}

LobeHub Integration

// In LobeHub agent logic
async function handleAnalyticsRequest(userQuery: string) {
  const currentContext = getCurrentContextSize();
  const availableTokens = CONTEXT_WINDOW - currentContext;
  
  if (availableTokens < 2000) {
    // Compress existing context
    await compressContext();
  }
  
  // Determine optimal query strategy
  const strategy = selectStrategy(userQuery, availableTokens);
  
  switch(strategy) {
    case 'schema_only':
      return await mcp.profile_dataset({compress: 'maximum'});
      
    case 'sample_and_query':
      const sample = await mcp.stream_sample({rows: 10});
      const result = await mcp.execute_query({return_limit: 20});
      return {sample, result};
      
    case 'direct_query':
      return await mcp.execute_query({return_limit: 50});
  }
}

Consequences

Positive

  1. 90%+ Token Reduction: Large datasets no longer overwhelm context
  2. Faster Responses: Less context = faster LLM processing
  3. Cost Savings: Reduced token usage = lower API costs
  4. Better UX: Users can work with large data iteratively
  5. Predictable Behavior: Clear budgets prevent context overflow

Negative

  1. Complexity: Additional layer of state management
  2. Information Loss: Compression may lose nuance
  3. Extra Round-trips: Incremental loading requires multiple MCP calls
  4. State Synchronization: Must keep LobeHub and MCP context in sync

Neutral

  1. Learning Curve: Users need to understand "ask for schema first" pattern
  2. Tool Awareness: Agent must know when to request more/less data

Implementation Phases

Phase 1: Basic Budgeting (MVP)

  • Implement token counting for all responses
  • Add context_tokens_used to all tool outputs
  • Static budget allocation
  • Simple compression (markdown tables)

Phase 2: Adaptive Management

  • Dynamic compression based on pressure
  • Smart eviction policies
  • Incremental loading workflows
  • Priority-based content retention

Phase 3: Advanced Optimization

  • Predictive loading (pre-load likely-needed data)
  • Semantic compression (LLM-based summarization)
  • Cross-query result deduplication
  • Context usage analytics

Success Metrics

Metric Before Target Measurement
Avg tokens/dataset 50,000+ <2,000 Per-tool telemetry
Context overflow errors Frequent Zero Error tracking
Query latency Degrades Stable Timing logs
User satisfaction Low High Feedback
Cost per session High Reduced 80% Token usage logs

Related Decisions

  • ADR-001: Analytics Architecture
  • ADR-003: Caching and Query Optimization
  • MCP-SPECIFICATION: Detailed tool specifications

References

  • LobeHub Issue #3279: Context length performance
  • OpenAI Tokenizer Guidelines
  • Anthropic Context Management Best Practices
  • "Attention Is All You Need" (Transformer architecture)

Decision Record

Date Author Decision Rationale
2026-03-17 Sisyphus Adopt tiered context management with adaptive compression Enables large-scale analytics within LobeHub constraints

Status Legend:

  • Proposed: Under review
  • Accepted: Approved for implementation
  • Deprecated: Replaced by newer ADR
  • Superseded: See referenced ADR