Skip to content

Commit 7e49bee

Browse files
Mullasseryclaude
andcommitted
docs: Add PyStreamMCP integration guidance - enforce boundaries
- Add comprehensive ARCHITECTURE.md - Clarify: Debug PyStreamMCP results, don't rebuild query logic - Explain: Why NOT to reimplement query optimization - Document: Integration patterns for retrieval debugging - Show: Forensic analysis on top of PyStreamMCP output PyVectorHound debugs retrieval. PyStreamMCP optimizes it. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
1 parent a23aa5c commit 7e49bee

1 file changed

Lines changed: 249 additions & 0 deletions

File tree

ARCHITECTURE.md

Lines changed: 249 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,249 @@
1+
# PyVectorHound Architecture & PyStreamMCP Integration
2+
3+
## Mission
4+
5+
**Debug & Optimize Retrieval**
6+
7+
Core Question: Why did this retrieval work (or fail)? How do we improve it?
8+
9+
## Core Responsibility
10+
11+
PyVectorHound is **exclusively responsible** for:
12+
13+
- **Retrieval Debugging** — Understanding why queries returned specific results
14+
- **Relevance Analysis** — Scoring retrieved documents for relevance
15+
- **Failure Analysis** — Root-cause analysis of retrieval failures
16+
- **Replay & Forensics** — Replaying queries to debug issues
17+
- **Quality Scoring** — Measuring retrieval quality
18+
- **Attribution Analysis** — Tracing why a result was retrieved
19+
20+
## What We Do NOT Own
21+
22+
### ❌ Query Optimization (PyStreamMCP)
23+
- Query planning
24+
- Source discovery
25+
- Token optimization
26+
- Streaming retrieval
27+
- Cost estimation
28+
29+
**Our role:** Debug & analyze queries that PyStreamMCP produces.
30+
31+
### ❌ Data Validation (StatGuardian)
32+
- Schema validation
33+
- Data freshness
34+
- Drift detection
35+
36+
**Our role:** Assume data is valid, focus on why it was/wasn't retrieved.
37+
38+
## Critical: Use PyStreamMCP, Don't Rebuild It
39+
40+
**IMPORTANT:** PyVectorHound must integrate with PyStreamMCP rather than rebuilding query functionality.
41+
42+
### Architecture Pattern
43+
44+
```
45+
Query
46+
47+
PyStreamMCP (Optimization & Execution)
48+
49+
Retrieval Results
50+
51+
PyVectorHound (Debug & Analyze)
52+
53+
Root Cause Analysis & Insights
54+
55+
Feedback Loop to PyStreamMCP
56+
```
57+
58+
### Integration Example
59+
60+
```python
61+
# ✅ CORRECT: Use PyStreamMCP's query execution
62+
from pystreammcp import Agent, QueryExecutor
63+
from pyvectorhound import RetrievalDebugger
64+
65+
# Create query with PyStreamMCP
66+
agent = Agent(agent_id="debug_query")
67+
query_result = agent.query("customer data")
68+
69+
# Debug why this query returned these results
70+
debugger = RetrievalDebugger(query_result)
71+
analysis = debugger.analyze()
72+
# - Why were these documents retrieved?
73+
# - What sources were queried?
74+
# - What was the token cost?
75+
# - Could we have done better?
76+
77+
# Get root causes
78+
root_causes = analysis.root_causes()
79+
# - Low relevance due to source X
80+
# - Token budget exceeded, missing source Y
81+
# - Query optimization missed source Z
82+
83+
# Recommend improvements
84+
improvements = analysis.recommend_improvements()
85+
# - Use token_efficient strategy
86+
# - Include source Z in next run
87+
# - Adjust relevance threshold
88+
```
89+
90+
### What NOT to Do
91+
92+
**WRONG: Don't rebuild PyStreamMCP's functionality**
93+
94+
```python
95+
# ❌ DO NOT DO THIS
96+
class QueryOptimizer:
97+
def discover_sources(self): # ← PyStreamMCP owns this
98+
pass
99+
100+
def optimize_for_tokens(self): # ← PyStreamMCP owns this
101+
pass
102+
103+
def estimate_cost(self): # ← PyStreamMCP owns this
104+
pass
105+
```
106+
107+
Instead, use PyStreamMCP's QueryExecutor:
108+
```python
109+
# ✅ DO THIS
110+
from pystreammcp import QueryExecutor
111+
112+
executor = QueryExecutor()
113+
result = executor.execute_query(optimized_query)
114+
115+
# Then debug the result
116+
debugger = RetrievalDebugger(result)
117+
```
118+
119+
### Why?
120+
121+
PyStreamMCP already provides:
122+
- ✓ Query planning & optimization
123+
- ✓ Context discovery (6+ framework integrations)
124+
- ✓ 7 optimization techniques
125+
- ✓ Cost tracking & estimation
126+
- ✓ Token efficiency (60-75% reduction)
127+
- ✓ Streaming execution
128+
- ✓ Multi-agent optimization
129+
130+
Rebuilding this in PyVectorHound would:
131+
- Duplicate 1000+ lines of code
132+
- Miss learned optimizations
133+
- Create maintenance burden
134+
- Break integration with PyStreamMCP
135+
136+
## Focused Responsibility
137+
138+
PyVectorHound excels at what it's built for:
139+
140+
**Why did retrieval fail?**
141+
- Which sources had no matches?
142+
- Why were expected documents missing?
143+
- Did token budget cause truncation?
144+
- Was relevance threshold too high?
145+
146+
**Can we do better?**
147+
- Which optimization would help most?
148+
- Should we use different sources?
149+
- Can we adjust token allocation?
150+
- What does the replay show?
151+
152+
**Root Cause Analysis**
153+
- 8-failure taxonomy
154+
- Automatic classification
155+
- Replay debugging
156+
- Forensic analysis
157+
158+
## Integration Points
159+
160+
### With PyStreamMCP (Primary Integration)
161+
162+
```python
163+
from pystreammcp import QueryExecutor, Agent
164+
from pyvectorhound import RetrievalDebugger, RootCauseAnalyzer
165+
166+
# Execute query using PyStreamMCP
167+
executor = QueryExecutor()
168+
result = executor.execute_query(
169+
query="customer data",
170+
strategy="token_efficient"
171+
)
172+
173+
# Debug the result
174+
debugger = RetrievalDebugger(result)
175+
analysis = debugger.analyze()
176+
177+
# Get root causes using PyVectorHound's forensics
178+
analyzer = RootCauseAnalyzer()
179+
causes = analyzer.analyze(analysis)
180+
181+
# Example causes:
182+
# - TokenBudgetExceeded: "Missing source X due to token limit"
183+
# - LowRelevanceScore: "Document below 0.5 threshold"
184+
# - SourceNotQueried: "Source Y not discovered by PyStreamMCP"
185+
```
186+
187+
### With StatGuardian
188+
189+
```python
190+
from statguardian import ValidationGate
191+
from pyvectorhound import RetrievalDebugger
192+
193+
# Check if data is valid before analyzing
194+
validation = ValidationGate.check(source="customer_data")
195+
196+
if validation.is_valid():
197+
# Safe to debug
198+
debugger = RetrievalDebugger(result)
199+
else:
200+
# Data quality issue, not a retrieval issue
201+
print(f"Data quality issue: {validation.error}")
202+
```
203+
204+
## Module Structure
205+
206+
```
207+
src/
208+
├── debugger.rs # Core retrieval debugging
209+
├── analyzer.rs # Root cause analysis
210+
├── failure_taxonomy.rs # 8 failure types
211+
├── replay.rs # Query replay for debugging
212+
├── forensics/ # Forensic analysis
213+
│ ├── source_analysis.rs
214+
│ ├── relevance_analysis.rs
215+
│ └── token_analysis.rs
216+
└── storage/ # Debug data persistence
217+
```
218+
219+
## Philosophy
220+
221+
PyVectorHound is to retrieval debugging what a debugger is to code execution.
222+
223+
- A code debugger doesn't write programs, it debugs them
224+
- A database profiler doesn't optimize queries, it analyzes them
225+
- PyVectorHound doesn't optimize retrieval, it debugs it
226+
227+
PyStreamMCP handles optimization. PyVectorHound explains why it worked or didn't.
228+
229+
## Outcome
230+
231+
When properly integrated:
232+
233+
```
234+
Query Submitted
235+
236+
PyStreamMCP executes with optimizations
237+
238+
Results Retrieved
239+
240+
PyVectorHound analyzes the result
241+
242+
Root cause identified
243+
244+
Feedback to PyStreamMCP for next run
245+
246+
Continuous Improvement
247+
```
248+
249+
PyStreamMCP optimizes. PyVectorHound debugs. Together they form a complete retrieval intelligence system.

0 commit comments

Comments
 (0)