|
| 1 | +# PyVectorHound Architecture & PyStreamMCP Integration |
| 2 | + |
| 3 | +## Mission |
| 4 | + |
| 5 | +**Debug & Optimize Retrieval** |
| 6 | + |
| 7 | +Core Question: Why did this retrieval work (or fail)? How do we improve it? |
| 8 | + |
| 9 | +## Core Responsibility |
| 10 | + |
| 11 | +PyVectorHound is **exclusively responsible** for: |
| 12 | + |
| 13 | +- **Retrieval Debugging** — Understanding why queries returned specific results |
| 14 | +- **Relevance Analysis** — Scoring retrieved documents for relevance |
| 15 | +- **Failure Analysis** — Root-cause analysis of retrieval failures |
| 16 | +- **Replay & Forensics** — Replaying queries to debug issues |
| 17 | +- **Quality Scoring** — Measuring retrieval quality |
| 18 | +- **Attribution Analysis** — Tracing why a result was retrieved |
| 19 | + |
| 20 | +## What We Do NOT Own |
| 21 | + |
| 22 | +### ❌ Query Optimization (PyStreamMCP) |
| 23 | +- Query planning |
| 24 | +- Source discovery |
| 25 | +- Token optimization |
| 26 | +- Streaming retrieval |
| 27 | +- Cost estimation |
| 28 | + |
| 29 | +**Our role:** Debug & analyze queries that PyStreamMCP produces. |
| 30 | + |
| 31 | +### ❌ Data Validation (StatGuardian) |
| 32 | +- Schema validation |
| 33 | +- Data freshness |
| 34 | +- Drift detection |
| 35 | + |
| 36 | +**Our role:** Assume data is valid, focus on why it was/wasn't retrieved. |
| 37 | + |
| 38 | +## Critical: Use PyStreamMCP, Don't Rebuild It |
| 39 | + |
| 40 | +**IMPORTANT:** PyVectorHound must integrate with PyStreamMCP rather than rebuilding query functionality. |
| 41 | + |
| 42 | +### Architecture Pattern |
| 43 | + |
| 44 | +``` |
| 45 | +Query |
| 46 | + ↓ |
| 47 | +PyStreamMCP (Optimization & Execution) |
| 48 | + ↓ |
| 49 | +Retrieval Results |
| 50 | + ↓ |
| 51 | +PyVectorHound (Debug & Analyze) |
| 52 | + ↓ |
| 53 | +Root Cause Analysis & Insights |
| 54 | + ↓ |
| 55 | +Feedback Loop to PyStreamMCP |
| 56 | +``` |
| 57 | + |
| 58 | +### Integration Example |
| 59 | + |
| 60 | +```python |
| 61 | +# ✅ CORRECT: Use PyStreamMCP's query execution |
| 62 | +from pystreammcp import Agent, QueryExecutor |
| 63 | +from pyvectorhound import RetrievalDebugger |
| 64 | + |
| 65 | +# Create query with PyStreamMCP |
| 66 | +agent = Agent(agent_id="debug_query") |
| 67 | +query_result = agent.query("customer data") |
| 68 | + |
| 69 | +# Debug why this query returned these results |
| 70 | +debugger = RetrievalDebugger(query_result) |
| 71 | +analysis = debugger.analyze() |
| 72 | +# - Why were these documents retrieved? |
| 73 | +# - What sources were queried? |
| 74 | +# - What was the token cost? |
| 75 | +# - Could we have done better? |
| 76 | + |
| 77 | +# Get root causes |
| 78 | +root_causes = analysis.root_causes() |
| 79 | +# - Low relevance due to source X |
| 80 | +# - Token budget exceeded, missing source Y |
| 81 | +# - Query optimization missed source Z |
| 82 | + |
| 83 | +# Recommend improvements |
| 84 | +improvements = analysis.recommend_improvements() |
| 85 | +# - Use token_efficient strategy |
| 86 | +# - Include source Z in next run |
| 87 | +# - Adjust relevance threshold |
| 88 | +``` |
| 89 | + |
| 90 | +### What NOT to Do |
| 91 | + |
| 92 | +❌ **WRONG: Don't rebuild PyStreamMCP's functionality** |
| 93 | + |
| 94 | +```python |
| 95 | +# ❌ DO NOT DO THIS |
| 96 | +class QueryOptimizer: |
| 97 | + def discover_sources(self): # ← PyStreamMCP owns this |
| 98 | + pass |
| 99 | + |
| 100 | + def optimize_for_tokens(self): # ← PyStreamMCP owns this |
| 101 | + pass |
| 102 | + |
| 103 | + def estimate_cost(self): # ← PyStreamMCP owns this |
| 104 | + pass |
| 105 | +``` |
| 106 | + |
| 107 | +Instead, use PyStreamMCP's QueryExecutor: |
| 108 | +```python |
| 109 | +# ✅ DO THIS |
| 110 | +from pystreammcp import QueryExecutor |
| 111 | + |
| 112 | +executor = QueryExecutor() |
| 113 | +result = executor.execute_query(optimized_query) |
| 114 | + |
| 115 | +# Then debug the result |
| 116 | +debugger = RetrievalDebugger(result) |
| 117 | +``` |
| 118 | + |
| 119 | +### Why? |
| 120 | + |
| 121 | +PyStreamMCP already provides: |
| 122 | +- ✓ Query planning & optimization |
| 123 | +- ✓ Context discovery (6+ framework integrations) |
| 124 | +- ✓ 7 optimization techniques |
| 125 | +- ✓ Cost tracking & estimation |
| 126 | +- ✓ Token efficiency (60-75% reduction) |
| 127 | +- ✓ Streaming execution |
| 128 | +- ✓ Multi-agent optimization |
| 129 | + |
| 130 | +Rebuilding this in PyVectorHound would: |
| 131 | +- Duplicate 1000+ lines of code |
| 132 | +- Miss learned optimizations |
| 133 | +- Create maintenance burden |
| 134 | +- Break integration with PyStreamMCP |
| 135 | + |
| 136 | +## Focused Responsibility |
| 137 | + |
| 138 | +PyVectorHound excels at what it's built for: |
| 139 | + |
| 140 | +✓ **Why did retrieval fail?** |
| 141 | +- Which sources had no matches? |
| 142 | +- Why were expected documents missing? |
| 143 | +- Did token budget cause truncation? |
| 144 | +- Was relevance threshold too high? |
| 145 | + |
| 146 | +✓ **Can we do better?** |
| 147 | +- Which optimization would help most? |
| 148 | +- Should we use different sources? |
| 149 | +- Can we adjust token allocation? |
| 150 | +- What does the replay show? |
| 151 | + |
| 152 | +✓ **Root Cause Analysis** |
| 153 | +- 8-failure taxonomy |
| 154 | +- Automatic classification |
| 155 | +- Replay debugging |
| 156 | +- Forensic analysis |
| 157 | + |
| 158 | +## Integration Points |
| 159 | + |
| 160 | +### With PyStreamMCP (Primary Integration) |
| 161 | + |
| 162 | +```python |
| 163 | +from pystreammcp import QueryExecutor, Agent |
| 164 | +from pyvectorhound import RetrievalDebugger, RootCauseAnalyzer |
| 165 | + |
| 166 | +# Execute query using PyStreamMCP |
| 167 | +executor = QueryExecutor() |
| 168 | +result = executor.execute_query( |
| 169 | + query="customer data", |
| 170 | + strategy="token_efficient" |
| 171 | +) |
| 172 | + |
| 173 | +# Debug the result |
| 174 | +debugger = RetrievalDebugger(result) |
| 175 | +analysis = debugger.analyze() |
| 176 | + |
| 177 | +# Get root causes using PyVectorHound's forensics |
| 178 | +analyzer = RootCauseAnalyzer() |
| 179 | +causes = analyzer.analyze(analysis) |
| 180 | + |
| 181 | +# Example causes: |
| 182 | +# - TokenBudgetExceeded: "Missing source X due to token limit" |
| 183 | +# - LowRelevanceScore: "Document below 0.5 threshold" |
| 184 | +# - SourceNotQueried: "Source Y not discovered by PyStreamMCP" |
| 185 | +``` |
| 186 | + |
| 187 | +### With StatGuardian |
| 188 | + |
| 189 | +```python |
| 190 | +from statguardian import ValidationGate |
| 191 | +from pyvectorhound import RetrievalDebugger |
| 192 | + |
| 193 | +# Check if data is valid before analyzing |
| 194 | +validation = ValidationGate.check(source="customer_data") |
| 195 | + |
| 196 | +if validation.is_valid(): |
| 197 | + # Safe to debug |
| 198 | + debugger = RetrievalDebugger(result) |
| 199 | +else: |
| 200 | + # Data quality issue, not a retrieval issue |
| 201 | + print(f"Data quality issue: {validation.error}") |
| 202 | +``` |
| 203 | + |
| 204 | +## Module Structure |
| 205 | + |
| 206 | +``` |
| 207 | +src/ |
| 208 | +├── debugger.rs # Core retrieval debugging |
| 209 | +├── analyzer.rs # Root cause analysis |
| 210 | +├── failure_taxonomy.rs # 8 failure types |
| 211 | +├── replay.rs # Query replay for debugging |
| 212 | +├── forensics/ # Forensic analysis |
| 213 | +│ ├── source_analysis.rs |
| 214 | +│ ├── relevance_analysis.rs |
| 215 | +│ └── token_analysis.rs |
| 216 | +└── storage/ # Debug data persistence |
| 217 | +``` |
| 218 | + |
| 219 | +## Philosophy |
| 220 | + |
| 221 | +PyVectorHound is to retrieval debugging what a debugger is to code execution. |
| 222 | + |
| 223 | +- A code debugger doesn't write programs, it debugs them |
| 224 | +- A database profiler doesn't optimize queries, it analyzes them |
| 225 | +- PyVectorHound doesn't optimize retrieval, it debugs it |
| 226 | + |
| 227 | +PyStreamMCP handles optimization. PyVectorHound explains why it worked or didn't. |
| 228 | + |
| 229 | +## Outcome |
| 230 | + |
| 231 | +When properly integrated: |
| 232 | + |
| 233 | +``` |
| 234 | +Query Submitted |
| 235 | + ↓ |
| 236 | +PyStreamMCP executes with optimizations |
| 237 | + ↓ |
| 238 | +Results Retrieved |
| 239 | + ↓ |
| 240 | +PyVectorHound analyzes the result |
| 241 | + ↓ |
| 242 | +Root cause identified |
| 243 | + ↓ |
| 244 | +Feedback to PyStreamMCP for next run |
| 245 | + ↓ |
| 246 | +Continuous Improvement |
| 247 | +``` |
| 248 | + |
| 249 | +PyStreamMCP optimizes. PyVectorHound debugs. Together they form a complete retrieval intelligence system. |
0 commit comments