diff --git a/README.md b/README.md new file mode 100644 index 0000000..16394d4 --- /dev/null +++ b/README.md @@ -0,0 +1,199 @@ +# AXON + +> A personal AI assistant with persistent, encrypted, semantically-searchable memory β€” built on a LangGraph supervisor agent, FastAPI, and Next.js. + +AXON is a full-stack AI assistant that goes beyond a stateless chatbot. It remembers what you tell it (encrypted at rest, PII-masked before embedding), can ingest your documents, summarise YouTube videos, and synthesise speech β€” all behind your own Supabase auth. + +--- + +## ✨ Features + +- πŸ’¬ **Streaming chat** over Server-Sent Events with any model on [OpenRouter](https://openrouter.ai) +- 🧠 **Encrypted long-term memory** with semantic search (pgvector + HNSW, cosine similarity) +- πŸ“„ **File ingestion** β€” upload `.txt`, `.md`, `.pdf`, `.docx`; auto-chunked and embedded +- πŸ“Ί **YouTube agent** β€” paste a link, get a hierarchical LLM summary saved to your library +- πŸ”Š **Text-to-speech** via OpenRouter, stored in a private Supabase bucket +- πŸ” **Privacy by design** β€” Microsoft Presidio masks PII before embedding; Fernet (AES-128-CBC + HMAC) encrypts stored plaintext +- 🚦 **Per-user rate limiting** with SlowAPI +- πŸ”‘ **Supabase Auth** (Google OAuth) with row-level security on every table + +--- + +## πŸ—οΈ Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Next.js 16 β”‚ HTTPS β”‚ FastAPI + LangGraph β”‚ HTTPS β”‚ OpenRouter β”‚ +β”‚ (App Router) β”‚ ──────► β”‚ Supervisor Agent β”‚ ──────► β”‚ LLMs / TTS / β”‚ +β”‚ Server Functionsβ”‚ SSE β”‚ + sub-agents β”‚ β”‚ Embeddings β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ β”‚ + β”‚ JWT β”‚ pgvector / Storage / RLS + β–Ό β–Ό +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Supabase (Postgres + Auth + Storage) β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +### LangGraph supervisor + +``` +START β†’ supervisor ──► memory_agent β†’ supervisor β†’ END + ──► youtube_agent β†’ supervisor β†’ END + ──► save_transcript β†’ supervisor β†’ END + ──► generate_tts β†’ supervisor β†’ END + ──► (direct reply) ─────────────────► END +``` + +The **supervisor** is a tool-calling LLM that decides whether to delegate. Sub-agents return `ToolMessage`s that feed back into a second supervisor pass which streams the final answer to the user. + +--- + +## 🧱 Tech Stack + +### Backend (`backend/`) + +| Layer | Tool | +| ------------- | ------------------------------------------- | +| Framework | FastAPI (async) | +| Agents | LangGraph + LangChain (supervisor pattern) | +| LLM gateway | OpenRouter | +| Database | Supabase Postgres + pgvector (HNSW) | +| Storage | Supabase Storage (private `audio` bucket) | +| Auth | Supabase JWT (HS256) | +| Encryption | Fernet (AES-128-CBC + HMAC-SHA256) | +| PII masking | Microsoft Presidio + spaCy `en_core_web_sm` | +| Rate limiting | SlowAPI | +| Package mgr | [uv](https://github.com/astral-sh/uv) | + +### Frontend (`frontend/`) + +| Layer | Tool | +| ------------- | ---------------------------------- | +| Framework | Next.js 16 (App Router, Turbopack) | +| UI | React 19 + Tailwind v4 + shadcn/ui | +| Data fetching | TanStack Query v5 | +| Validation | Zod v4 | +| SSE parsing | `eventsource-parser` | +| Auth client | `@supabase/ssr` | +| Package mgr | pnpm | + +--- + +## πŸ“‚ Repository Layout + +``` +axon/ +β”œβ”€β”€ backend/ +β”‚ └── app/ +β”‚ β”œβ”€β”€ agents/ # LangGraph orchestrator, supervisor, sub-agents +β”‚ β”œβ”€β”€ core/ # config, security, crypto, privacy, limiter +β”‚ β”œβ”€β”€ db/ # Supabase client + embeddings +β”‚ └── features/ # auth Β· chat Β· memory Β· models Β· youtube Β· audio +β”œβ”€β”€ frontend/ +β”‚ β”œβ”€β”€ app/ # Next.js App Router (routing only) +β”‚ β”œβ”€β”€ features/ # audio Β· auth Β· chat Β· memory Β· youtube +β”‚ β”œβ”€β”€ components/ui/ # shadcn primitives +β”‚ └── lib/ # api client, Supabase server/client helpers +└── supabase/ + └── migrations/ # Schema, RLS, pgvector, RPCs +``` + +Both halves follow a **feature-based** architecture β€” each feature owns its router, service, schemas, components, and hooks. + +--- + +## πŸš€ Getting Started + +### Prerequisites + +- Python β‰₯ 3.10, [uv](https://github.com/astral-sh/uv) +- Node.js β‰₯ 20, [pnpm](https://pnpm.io) +- A [Supabase](https://supabase.com) project (free tier works) +- An [OpenRouter](https://openrouter.ai) API key + +### 1. Supabase + +1. Create a project, copy the URL + anon key + service-role key. +2. Run the migrations in `supabase/migrations/` (in order) via the SQL editor or the Supabase CLI. +3. Enable Google OAuth and set the redirect URL. + +### 2. Backend + +```bash +cd backend +uv sync +uv run python -m spacy download en_core_web_sm +cp .env.example .env # fill in keys +uv run uvicorn app.main:app --reload +``` + +Open http://localhost:8000/docs for the OpenAPI explorer. + +### 3. Frontend + +```bash +cd frontend +pnpm install +cp .env.example .env.local # fill in Supabase + backend URL +pnpm dev +``` + +Open http://localhost:3000. + +--- + +## πŸ”‘ Environment Variables (overview) + +**Backend** (`backend/.env`) + +``` +SUPABASE_URL= +SUPABASE_ANON_KEY= +SUPABASE_SERVICE_ROLE_KEY= +SUPABASE_JWT_SECRET= +OPENROUTER_API_KEY= +ENCRYPTION_KEY= # Fernet key (urlsafe base64, 32 bytes) +EMBEDDING_MODEL=openai/text-embedding-3-large +EMBEDDING_DIMENSIONS=1536 +``` + +**Frontend** (`frontend/.env.local`) + +``` +NEXT_PUBLIC_SUPABASE_URL= +NEXT_PUBLIC_SUPABASE_ANON_KEY= +BACKEND_URL=http://localhost:8000 +``` + +--- + +## πŸ›‘οΈ Security Notes + +- Embeddings are computed on **PII-masked** text; stored plaintext is **Fernet-encrypted**. +- The user ID is **always derived from a validated JWT** server-side β€” never trusted from the client. +- Every table has **row-level security** enforced in Postgres; the backend uses the service-role key but consistently filters by `user_id`. +- Rate limits are keyed on the authenticated user (per-JWT), not IP. + +--- + +## πŸ“‹ API Surface (selected) + +| Method | Path | Notes | +| ------ | ---------------------------- | ------------------------------- | +| GET | `/api/v1/health` | Liveness, public | +| GET | `/api/v1/auth/me` | JWT-protected user info | +| GET | `/api/v1/models` | Cached OpenRouter catalogue | +| POST | `/api/v1/chat/stream` | SSE stream from the agent | +| GET | `/api/v1/chat/conversations` | List conversations | +| POST | `/api/v1/memory` Β· `/search` | CRUD + semantic search | +| POST | `/api/v1/memory/upload` | Multipart file ingestion | +| GET | `/api/v1/youtube` | Saved YouTube video transcripts | + +See `backend/BACKEND.md` for the full reference. + +--- + +## πŸ“œ License + +This is a personal project. No license granted; all rights reserved by the author. diff --git a/frontend/lib/api.ts b/frontend/lib/api.ts index 4e32ba3..5204b37 100644 --- a/frontend/lib/api.ts +++ b/frontend/lib/api.ts @@ -250,10 +250,13 @@ export function renameAudioEntry( id, title, }); - return apiFetch(`/api/v1/audio/${encodeURIComponent(validId)}`, { - method: 'PATCH', - body: JSON.stringify({ title: validTitle }), - }); + return apiFetch( + `/api/v1/audio/${encodeURIComponent(validId)}`, + { + method: 'PATCH', + body: JSON.stringify({ title: validTitle }), + }, + ); } export function deleteAudioEntry(id: string): Promise { diff --git a/implementationPlan.MD b/implementationPlan.MD index f42ff77..21ff9fd 100644 --- a/implementationPlan.MD +++ b/implementationPlan.MD @@ -548,6 +548,150 @@ User: "yes" --- +## PHASE 11 β€” Deep Search Agent + +> **Architecture decision log:** +> +> - **Search provider:** Tavily API β€” purpose-built for LLM/agent workflows; `search_depth="advanced"` returns full page content (not just snippets), structured results, and optionally a pre-generated answer. Free tier: 1 000 searches/month. Uses its own `TAVILY_API_KEY` β€” existing `OPENROUTER_API_KEY` unchanged. +> - **Agent pattern:** Same supervisor handoff as YouTube β€” `transfer_to_deepsearch_agent` β†’ `deepsearch_agent_node` β†’ supervisor. No new graph topology needed. +> - **Save flow:** After presenting results, supervisor asks "Want to save this research?" β†’ user confirms β†’ `save_deep_search` tool node β†’ INSERT into `deep_searches` table + embed `query + summary` β†’ INSERT into `memory_entries` (so memory_agent can recall it in future conversations). +> - **Delete:** Must remove from BOTH `deep_searches` table AND the corresponding `memory_entries` row. `memory_entry_id` FK stored directly in the row (cleaner than pattern-matching). +> - **Edit:** `title` change β€” free (no re-embed). `summary` change β€” re-embed β†’ update `memory_entries.embedding` in place. + +### Overall flow + +```text +User: "deep search quantum computing breakthroughs 2025" + β†’ supervisor detects research intent β†’ calls transfer_to_deepsearch_agent + β†’ deepsearch_agent: + 1. Extract query from last HumanMessage + 2. TavilyClient.search(query, search_depth="advanced", max_results=10) via asyncio.to_thread + 3. Build synthesis context (each result capped at 2000 chars, total 20 000) + 4. LLM call β†’ synthesize summary + key_findings: list[str] + 5. Build sources: [{title, url, snippet: content[:300]}] + 6. Store JSON in state["deepsearch_context"]; return ToolMessage + β†’ supervisor: presents summary + key findings + numbered source list + asks "Want me to save this research?" + +User: "yes" + β†’ supervisor calls save_deep_search() + β†’ save_deep_search node: + INSERT deep_searches; embed(query + "\n\n" + summary) β†’ INSERT memory_entries + UPDATE deep_searches.memory_entry_id = + β†’ supervisor: "Research saved βœ“ β€” also indexed in your memory for future recall" + +User later: "what did I find about quantum computing?" + β†’ memory_agent β†’ vector search β†’ returns saved summary +``` + +--- + +### 11A β€” Dependencies + Config + +- [ ] `uv add tavily-python` β€” official Tavily Python client +- [ ] `app/core/config.py`: add `tavily_api_key: str` + `deepsearch_model: str = "google/gemini-2.5-flash"` + `deepsearch_max_results: int = 10` +- [ ] `backend/.env.example`: add `TAVILY_API_KEY=` + +### 11B β€” Database (`supabase/migrations/`) + +- [ ] New migration: create `deep_searches` table: + ```sql + CREATE TABLE public.deep_searches ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + user_id UUID NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE, + query TEXT NOT NULL, + title TEXT NOT NULL, -- editable; defaults to query + summary TEXT NOT NULL, -- AI-synthesized answer + key_findings JSONB NOT NULL DEFAULT '[]', -- string[] + sources JSONB NOT NULL DEFAULT '[]', -- [{title, url, snippet}] + memory_entry_id UUID REFERENCES memory_entries(id) ON DELETE SET NULL, + created_at TIMESTAMPTZ NOT NULL DEFAULT now() + ); + CREATE INDEX deep_searches_user_created_idx + ON public.deep_searches (user_id, created_at DESC); + ``` +- [ ] RLS: `user_id = (select auth.uid())` on SELECT / INSERT / DELETE / UPDATE β€” subquery form (same pattern as all other tables) +- [ ] `memory_entry_id ON DELETE SET NULL` β€” deep search row survives if memory entry is independently cleaned up + +### 11C β€” Backend feature layer (`app/features/deepsearch/`) + +- [ ] `__init__.py` +- [ ] `schemas.py`: `SourceItem(title, url, snippet)` Β· `DeepSearchOut(id, query, title, summary, key_findings, sources, created_at)` Β· `DeepSearchCreate(query, title: str = Field(..., max_length=200), summary: str = Field(..., max_length=10000), key_findings, sources)` Β· `DeepSearchPatch(title: str | None = Field(None, max_length=200), summary: str | None = Field(None, max_length=10000))` with `model_validator` requiring at least one field; service layer must accept these schemas for create/update so direct API calls cannot bypass the limits +- [ ] `service.py`: + - `save_deep_search(user_id, payload: dict) -> DeepSearchOut` β€” INSERT row; attempt embed `f"[Deep Search] {query}: {summary}"` β†’ INSERT `memory_entries` β†’ UPDATE row with `memory_entry_id`; if embedding or `memory_entries` INSERT fails, log the error with context (deep_search id, query, exception) and continue with `memory_entry_id = NULL` (non-fatal, same resilience pattern as YouTube) + - `list_deep_searches(user_id, limit=50, offset=0) -> list[DeepSearchOut]` β€” ordered by `created_at DESC` + - `get_deep_search(id, user_id) -> DeepSearchOut | None` + - `delete_deep_search(id, user_id) -> bool` β€” fetch `memory_entry_id` β†’ `delete_memory()` if set β†’ DELETE row + - `update_deep_search(id, user_id, patch) -> DeepSearchOut | None` β€” if `summary` changed: re-embed β†’ update `memory_entries.embedding`; if `memory_entry_id` is NULL attempt to create a new memory entry and set it; if that fails log and keep the row updated without `memory_entry_id`; UPDATE row; all log messages must reference `update_deep_search` and include `memory_entry_id` context +- [ ] `router.py`: + - `GET /api/v1/deepsearch` β€” list paginated; `@limiter.limit("60/minute")` + - `GET /api/v1/deepsearch/{id}` β€” single; `@limiter.limit("60/minute")` + - `DELETE /api/v1/deepsearch/{id}` β€” 204; `@limiter.limit("30/minute")` + - `PATCH /api/v1/deepsearch/{id}` β€” edit title/summary; `@limiter.limit("30/minute")` + +### 11D β€” Agent subagent (`app/agents/subagents/deepsearch_agent.py`) + +- [ ] `deepsearch_agent_node(state: AxonState) -> dict`: + 1. Extract query from last `HumanMessage` (strip intent phrases: "deep search", "research", "find out about") + 2. `asyncio.to_thread(TavilyClient(api_key=settings.tavily_api_key).search, query, search_depth="advanced", max_results=settings.deepsearch_max_results)` + 3. Build synthesis context: each result capped at 2 000 chars; total cap 20 000 chars + 4. LLM call (`settings.deepsearch_model` via OpenRouter) β†’ `summary` + `key_findings: list[str]` + 5. Build `sources`: `[{title, url, snippet: content[:300]}]` + 6. Validate with Pydantic `DeepSearchPayload`; fallback to raw concat on parse error + 7. Store JSON in `state["deepsearch_context"]`; return ToolMessage closing handoff tool call +- [ ] Error handling: + - `TavilyAPIError` / network errors β†’ log internally, return descriptive error ToolMessage; never expose raw API errors or tracebacks + - LLM synthesis call (`settings.deepsearch_model` via OpenRouter): wrap in `try/except`; catch timeouts, rate-limit errors, and HTTP errors; apply exponential backoff with 2 retries; on persistent failure return a descriptive error ToolMessage without raw exception details + - Malformed/unparseable LLM response (including `JSONDecodeError`): log internally; fall back to raw concatenation strategy (step 6) instead of crashing + - All internal exceptions must be logged with context (query, model) but must NOT be leaked to the returned ToolMessage + +### 11E β€” Save tool (`app/features/deepsearch/tool.py`) + +- [ ] `save_deep_search_node(state: AxonState) -> dict` β€” LangGraph node (same pattern as `save_transcript_node`): + - Read + parse `state["deepsearch_context"]`; validate required fields + - Call `deepsearch_service.save_deep_search(user_id, payload)` + - Return confirmation ToolMessage; clear `state["deepsearch_context"]` +- [ ] `save_deep_search` stub `@tool` (body never executes) + +### 11F β€” State + Supervisor + Orchestrator wiring + +- [ ] `app/agents/state.py`: add `deepsearch_context: str` (empty string default) +- [ ] `app/agents/supervisor.py`: + - Add `transfer_to_deepsearch_agent` + `save_deep_search` stubs to `bind_tools` + - System prompt: trigger phrases β†’ `transfer_to_deepsearch_agent`; after ToolMessage present summary + ask to save; on confirmation call `save_deep_search`; do NOT re-call agent if `deepsearch_context` already populated +- [ ] `app/agents/orchestrator.py`: add `deepsearch_agent` node + `save_deep_search_node`; extend `_should_continue` routing; add edge `deepsearch_agent β†’ supervisor` +- [ ] Mount deepsearch router in `app/main.py` + +### 11G β€” Frontend Deep Search Library (`features/deepsearch/`) + +- [ ] `features/deepsearch/types.ts` β€” `SourceItem`, `DeepSearchOut` +- [ ] `features/deepsearch/hooks/useDeepSearch.ts` β€” `useDeepSearches()` (user-scoped key `['deep-searches', userId]`) Β· `useDeleteDeepSearch()` Β· `useUpdateDeepSearch()` (optimistic title) +- [ ] `features/deepsearch/components/DeepSearchCard.tsx`: + - Inline-editable title (`useOptimistic` β€” same pattern as `AudioCard`) + - Summary: `line-clamp-4` + expand/collapse (same pattern as `VideoTranscriptCard`) + - Summary edit mode: click β†’ textarea β†’ Save/Cancel β†’ `useUpdateDeepSearch` (re-embed server-side on summary change) + - Key findings: bullet list, first 4 visible + expand + - Sources: collapsed accordion ("N sources") β†’ numbered `` links + - Delete dialog: warns that the linked memory entry will also be removed +- [ ] `features/deepsearch/components/DeepSearchLibrary.tsx` β€” grid, skeleton, empty state +- [ ] `app/(dashboard)/deepsearch/page.tsx` β€” back-to-chat header +- [ ] `features/chat/components/ChatWindow.tsx` β€” add `Search` icon link to `/deepsearch` in toolbar +- [ ] `lib/api.ts`: `listDeepSearches()` Β· `deleteDeepSearch(id)` (Zod UUID) Β· `updateDeepSearch(id, patch)` (Zod: title max 200, summary max 10 000, at least one field) + +### 11H β€” Verification + +- [ ] "deep search latest AI models 2025" β†’ deepsearch_agent calls Tavily β†’ summary + key findings + sources in chat +- [ ] "yes save it" β†’ `save_deep_search` called β†’ entry visible in `GET /api/v1/deepsearch` +- [ ] `/deepsearch` page β†’ cards with summary, findings, collapsible source links +- [ ] Edit title β†’ optimistic update β†’ persisted +- [ ] Edit summary β†’ `PATCH` β†’ re-embed triggered β†’ memory_agent finds updated content in future chat +- [ ] Delete β†’ row removed from `deep_searches` + `memory_entries` β†’ UI updates +- [ ] Future chat "what did I research?" β†’ memory_agent returns saved summary +- [ ] Tavily 429/down β†’ graceful error message, no crash +- [ ] `PATCH` with empty body β†’ 422 + +--- + ## End-to-End Verification Checklist - [ ] Google OAuth β†’ JWT validated on `/api/v1/auth/me` β†’ 200