Summary
POST /api/personal/add_directory runs the entire indexing job synchronously inside the event loop, making the whole app (UI + API) unresponsive until indexing completes. On a real-world directory this means tens of minutes of apparent hang with no progress feedback — users reasonably conclude the app crashed (ours did).
Where
routes/personal_routes.py — add_directory_to_rag is declared async def but directly calls rag.index_personal_documents(directory, owner=owner), which is fully synchronous: os.walk + file reads + per-chunk embedding HTTP calls + per-chunk Chroma inserts (src/rag_vector.py::index_personal_documents). Because the handler is async def, FastAPI runs it on the event loop rather than the threadpool, so every other request queues behind it.
Impact observed
Indexing a mounted documentation vault (159 markdown files + inadvertently-included plugin JS, see companion issue) froze the UI for 25+ minutes at ~2–4 chunks/sec (each chunk = 2 embedding lanes). The only signals of life were container logs.
Suggested fix (any of, roughly in order of effort)
- Drop the
async (plain def handler) so FastAPI moves it to the threadpool — one-word change, UI stays responsive.
await run_in_executor(...) around index_personal_documents.- Proper fix: run as a background job (FastAPI
BackgroundTasks or the existing task system) and return a job id; expose progress (files done / total) so the Knowledge UI and /rag add can show status instead of hanging the browser tab.
Environment
Odysseus v1.0.2 (docker compose, main); code paths verified unchanged on dev HEAD 2026-07-15. Embeddings via external OpenAI-compatible endpoint (llama.cpp) + fastembed fallback lane, both active.
Summary
POST /api/personal/add_directoryruns the entire indexing job synchronously inside the event loop, making the whole app (UI + API) unresponsive until indexing completes. On a real-world directory this means tens of minutes of apparent hang with no progress feedback — users reasonably conclude the app crashed (ours did).Where
routes/personal_routes.py—add_directory_to_ragis declaredasync defbut directly callsrag.index_personal_documents(directory, owner=owner), which is fully synchronous:os.walk+ file reads + per-chunk embedding HTTP calls + per-chunk Chroma inserts (src/rag_vector.py::index_personal_documents). Because the handler isasync def, FastAPI runs it on the event loop rather than the threadpool, so every other request queues behind it.Impact observed
Indexing a mounted documentation vault (159 markdown files + inadvertently-included plugin JS, see companion issue) froze the UI for 25+ minutes at ~2–4 chunks/sec (each chunk = 2 embedding lanes). The only signals of life were container logs.
Suggested fix (any of, roughly in order of effort)
async(plaindefhandler) so FastAPI moves it to the threadpool — one-word change, UI stays responsive.await run_in_executor(...)aroundindex_personal_documents.BackgroundTasksor the existing task system) and return a job id; expose progress (files done / total) so the Knowledge UI and/rag addcan show status instead of hanging the browser tab.Environment
Odysseus v1.0.2 (docker compose, main); code paths verified unchanged on
devHEAD 2026-07-15. Embeddings via external OpenAI-compatible endpoint (llama.cpp) + fastembed fallback lane, both active.