Supercells#25
Conversation
Signed-off-by: Roberto Luna-Rojas <roberto.luna.rojas@gmail.com>
Signed-off-by: Roberto Luna-Rojas <roberto.luna.rojas@gmail.com>
Signed-off-by: Roberto Luna-Rojas <roberto.luna.rojas@gmail.com>
Signed-off-by: Roberto Luna-Rojas <roberto.luna.rojas@gmail.com>
- voice_router.py: FastAPI routes for voice enroll, verify, and status - tools/voice_tools.py: voice processing utilities - VoiceAuth, VoiceChat components and pages (React frontend) - Integrated voice router into main.py - Updated App.js with voice page routing - Added voice-related dependencies to requirements.txt Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
| Filename | Overview |
|---|---|
| backend/main.py | FastAPI entrypoint using Google ADK runner; /api/agent/chat has no server-side auth check, allowing any caller to bypass voice verification entirely. |
| backend/agent.py | Alternate direct-Groq orchestrator that is never imported by main.py (dead code); contains an unbounded while-True tool-calling loop with no iteration cap. |
| backend/tools/order_tools.py | Order placement has a TOCTOU race between stock check and JSON.NUMINCRBY decrement; concurrent orders can oversell and push stock negative. |
| backend/tools/voice_tools.py | ECAPA-TDNN speaker verification via SpeechBrain; voice embeddings are stored without a TTL so they accumulate indefinitely in Valkey. |
| backend/tools/cart_tools.py | Cart CRUD backed by Valkey hashes with TTL; session identity is pulled from ADK ToolContext using a private _invocation_context attribute which is fragile. |
| backend/orchestrator.py | Google ADK multi-agent orchestrator that wraps four specialist agents as AgentTools; sets GROQ_API_KEY into the process environment. |
| backend/voice_router.py | FastAPI router for voice enroll/verify endpoints; no rate limiting on /verify makes brute-force audio attempts feasible. |
| frontend/src/components/VoiceAuth.jsx | Voice enrollment/verification UI; API_BASE hardcoded to localhost:8000 and recorded audio blob URLs are never revoked, causing memory leaks. |
| frontend/src/components/VoiceChat.jsx | Voice/text chat UI with Web Speech API recognition; API_BASE hardcoded to localhost:8000 making deployment non-functional. |
| frontend/src/components/VoiceAIAssistant.jsx | Global floating voice assistant component mounted on every page; also hardcodes API_BASE to localhost:8000. |
| backend/seed_data.py | Seeds products, coupons, and vector embeddings into Valkey; imports bcrypt which is unused and would cause an ImportError if not installed. |
| backend/tools/search_tools.py | Semantic KNN search and filter-based product lookup via Valkey FT.SEARCH; model is lazily loaded once and reused correctly. |
Sequence Diagram
sequenceDiagram
actor User
participant FE as React Frontend
participant VR as /api/voice/verify
participant Chat as /api/agent/chat
participant Orch as ADK Orchestrator
participant Agent as Specialist Agent
participant Valkey as Valkey
User->>FE: Speak verification phrase
FE->>VR: POST audio + user_id
VR->>Valkey: "GET voice:embedding:{user_id}"
Valkey-->>VR: stored embedding
VR-->>FE: "{verified: true, score: 0.72}"
Note over FE: Navigates to /voice-chat (frontend only — no token issued)
User->>FE: Add Samsung A54 to cart
FE->>Chat: "POST {message, session_id} — no auth token"
Note over Chat: ⚠️ Any caller can reach this endpoint
Chat->>Orch: runner.run_async(message)
Orch->>Agent: cart_agent(add product:01-samsung-a54)
Agent->>Valkey: "HSET cart:{session_id} product:01-samsung-a54 1"
Valkey-->>Agent: OK
Agent-->>Orch: "{status: added}"
Orch-->>Chat: Added Samsung A54 to your cart!
Chat-->>FE: "{response: Added Samsung A54...}"
FE-->>User: Display response
Reviews (1): Last reviewed commit: "Merge branch 'main' of https://github.co..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| data = r.get(f"conversation:{session_id}") | ||
| if data: | ||
| return json.loads(data) | ||
| return [] | ||
|
|
||
|
|
||
| def save_conversation_history(session_id: str, history: list): | ||
| r.set(f"conversation:{session_id}", json.dumps(history), ex=SESSION_TTL) | ||
|
|
||
|
|
||
| def run_agent(message: str, session_id: str) -> dict: | ||
| history = get_conversation_history(session_id) | ||
| history.append({"role": "user", "content": message}) | ||
|
|
||
| messages = [{"role": "system", "content": SYSTEM_PROMPT}] + history | ||
|
|
||
| while True: | ||
| response = client.chat.completions.create( | ||
| model="llama-3.3-70b-versatile", | ||
| messages=messages, | ||
| tools=TOOLS, | ||
| tool_choice="auto", | ||
| max_tokens=512, | ||
| temperature=0.3, | ||
| ) | ||
|
|
||
| choice = response.choices[0] | ||
|
|
||
| if not choice.message.tool_calls: | ||
| final_text = choice.message.content | ||
| history.append({"role": "assistant", "content": final_text}) | ||
| save_conversation_history(session_id, history) | ||
| return {"response": final_text, "session_id": session_id} | ||
|
|
||
| messages.append(choice.message) | ||
|
|
||
| for tool_call in choice.message.tool_calls: | ||
| result = _run_tool( | ||
| tool_call.function.name, | ||
| json.loads(tool_call.function.arguments), | ||
| session_id | ||
| ) | ||
| messages.append({ | ||
| "role": "tool", | ||
| "tool_call_id": tool_call.id, |
There was a problem hiding this comment.
while True loop with no iteration cap
run_agent loops forever until the LLM emits a response with no tool calls. If the model repeatedly returns only tool calls (e.g., a routing misconfiguration keeps re-delegating, or every sub-agent call returns an error that still triggers another call), the request hangs permanently. Add a max-iteration guard and raise an error when it's exceeded so callers get a meaningful 500 instead of a stalled connection.
| "product_id": item["product_id"], | ||
| "product_name": item["name"], | ||
| "available": int(product.get("stock", 0)), | ||
| "requested": item["quantity"], | ||
| } | ||
|
|
||
| created_at = int(time.time()) | ||
| order_seq = r.incr("order_counter") | ||
| order_id = f"order:{order_seq:05d}-{uuid.uuid4().hex[:6]}" | ||
|
|
||
| order_doc = { | ||
| "id": order_id, | ||
| "user_id": user_id, | ||
| "session_id": session_id, | ||
| "items": totals["items"], | ||
| "subtotal": totals["subtotal"], | ||
| "discount": totals["discount"], | ||
| "total": totals["total"], | ||
| "coupon": totals["coupon"], | ||
| "status": "confirmed", | ||
| "created_at": created_at, | ||
| } | ||
|
|
||
| r.execute_command("JSON.SET", order_id, "$", json.dumps(order_doc)) | ||
| r.zadd(f"user_orders:{user_id}", {order_id: created_at}) | ||
|
|
||
| for item in totals["items"]: | ||
| r.execute_command( | ||
| "JSON.NUMINCRBY", item["product_id"], "$.stock", -item["quantity"] | ||
| ) | ||
|
|
||
| clear_cart(tool_context) | ||
|
|
||
| return { |
There was a problem hiding this comment.
place_order reads stock for each item to verify availability, then later decrements every item with JSON.NUMINCRBY. Between those two phases, a concurrent order for the same product can pass its own stock check and also decrement — resulting in negative stock and overselling. The check and the decrement must be atomic; a Lua script executed via r.eval (which Valkey runs atomically) or a WATCH/MULTI/EXEC transaction would eliminate the window.
|
|
||
| @app.post("/api/agent/chat", response_model=ChatResponse) | ||
| async def chat(req: ChatRequest): | ||
| if not req.message.strip(): | ||
| raise HTTPException(status_code=400, detail="Message cannot be empty") | ||
| try: | ||
| await session_service.create_session( | ||
| app_name="voicecart", | ||
| user_id=req.session_id, | ||
| session_id=req.session_id, | ||
| state={"voicecart_session_id": req.session_id}, | ||
| ) | ||
| except Exception: | ||
| pass # session may already exist | ||
|
|
||
| try: | ||
| content = Content(role="user", parts=[Part(text=req.message.strip())]) | ||
| response_text = "" | ||
|
|
||
| async for event in runner.run_async( | ||
| user_id=req.session_id, | ||
| session_id=req.session_id, | ||
| new_message=content, | ||
| ): | ||
| if event.is_final_response(): | ||
| response_text = event.content.parts[0].text | ||
|
|
||
| return ChatResponse(response=response_text, session_id=req.session_id) | ||
| except Exception as e: |
There was a problem hiding this comment.
/api/agent/chat accepts any caller-supplied session_id and routes it straight to the orchestrator. Voice verification is enforced only in the React frontend, so any HTTP client that skips the UI can POST directly to this endpoint with an arbitrary session and gain full access to cart and order operations for that session. At minimum, the backend should validate that the given session_id has a verified voice token (e.g. a short-lived signed token set by the /api/voice/verify handler on success).
| key = ENROLL_KEY.format(user_id=user_id) | ||
| r.set(key, json.dumps(embedding.tolist())) |
There was a problem hiding this comment.
cart:*, conversation:*) use explicit expiry. Consider aligning the enrollment key with a long but bounded TTL (e.g. SESSION_TTL from config, or a dedicated VOICE_ENROLL_TTL).
| key = ENROLL_KEY.format(user_id=user_id) | |
| r.set(key, json.dumps(embedding.tolist())) | |
| key = ENROLL_KEY.format(user_id=user_id) | |
| from config import SESSION_TTL | |
| r.set(key, json.dumps(embedding.tolist()), ex=SESSION_TTL) |
| import React, { useState, useRef, useEffect } from 'react'; | ||
| import { Link, useNavigate } from 'react-router-dom'; | ||
|
|
||
| const API_BASE = 'http://localhost:8000'; |
There was a problem hiding this comment.
API_BASE is hardcoded to localhost in three components
VoiceAuth.jsx, VoiceChat.jsx, and VoiceAIAssistant.jsx all define const API_BASE = 'http://localhost:8000'. This value is baked into the production bundle, so the app will silently fail for any user not running the backend on their own machine. Use a build-time environment variable (e.g. process.env.REACT_APP_API_BASE) with localhost:8000 as the development default.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| const blob = new Blob(chunks, { type: 'audio/webm' }); | ||
| const url = URL.createObjectURL(blob); | ||
| if (type === 'enroll') { | ||
| setEnrollAudio(blob); | ||
| setEnrollAudioUrl(url); | ||
| setEnrollResult(null); | ||
| setEnrollRecording(false); | ||
| } else { | ||
| setVerifyAudio(blob); | ||
| setVerifyAudioUrl(url); | ||
| setVerifyResult(null); | ||
| setVerifyRecording(false); | ||
| } |
There was a problem hiding this comment.
URL.createObjectURL blobs are never revoked
A new object URL is created each time the user stops recording (both in the enroll and verify flows), but URL.revokeObjectURL(url) is never called. Each dangling blob URL holds a reference to the recorded audio data in memory until the page unloads. Call revokeObjectURL on the previous URL before setting the new one, and again when the component unmounts.
| import json | ||
| from groq import Groq | ||
| from config import GROQ_API_KEY | ||
| from valkey_client import r | ||
| from config import SESSION_TTL | ||
|
|
||
| client = Groq(api_key=GROQ_API_KEY) | ||
|
|
||
| SYSTEM_PROMPT = """You are VoiceCart, a friendly voice shopping assistant for an Indian e-commerce store. | ||
|
|
There was a problem hiding this comment.
agent.py is dead code alongside orchestrator.py
main.py imports and uses orchestrator (Google ADK pipeline), so agent.py's direct-Groq run_agent function is never called. Having two complete, parallel orchestration implementations increases maintenance burden and makes it unclear which one is the source of truth. This file should either be removed or clearly documented as a fallback/reference.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| import bcrypt | ||
| from sentence_transformers import SentenceTransformer |
There was a problem hiding this comment.
bcrypt is imported but never used in this file — the seeding script stores only product JSON and coupon data, not password hashes. This unused import will cause an ImportError if bcrypt is not installed and misleads readers into thinking passwords are hashed here.
| import bcrypt | |
| from sentence_transformers import SentenceTransformer | |
| from sentence_transformers import SentenceTransformer |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
3 participants
VoiceCart AI
Overview
VoiceCart AI is an Indian AI-powered e-commerce voice shopping assistant that enables users to browse products, manage carts, and place orders entirely through natural spoken language. The platform combines conversational AI, semantic product search, and voice biometric authentication to deliver a hands-free shopping experience without relying on traditional passwords.
Key Features
1. Voice-Based Authentication
Users enroll their voice during signup.
Returning users can securely log in by speaking a verification phrase.
Authentication is powered by speaker recognition using the SpeechBrain ECAPA-TDNN model.
Eliminates the need for passwords while improving convenience and security.
2. AI-Powered Voice Shopping
Users can interact naturally with the system using spoken commands such as:
“Show me boAt headphones under ₹2000”
“Add the first product to my cart”
“Remove the Nike shoes from my cart”
The AI understands user intent and performs actions in real time.
3. Smart Semantic Product Search
Uses vector embeddings to understand product meaning instead of exact keyword matching.
Supports intelligent searches even when users do not know exact product names.
Example: Searching for “gaming headphones” can still return relevant wireless headset products.
4. Complete Voice-Based Cart & Order Management
Users can:
Add or remove products from the cart
Apply coupons and discounts
View totals and pricing
Place orders completely through voice commands
Technology Stack
Layer | Technology -- | -- AI Agents | Google ADK multi-agent framework (Orchestrator + 4 specialist agents) LLM | Groq LLaMA 3.3 70B Versatile Voice Authentication | SpeechBrain ECAPA-TDNN speaker recognition Database | Valkey (Redis-compatible) Vector Search | Sentence Transformers (all-MiniLM-L6-v2) + Valkey FT.SEARCH KNN Backend | FastAPI (Python) Frontend | React Audio Processing | WebM/Opus browser recording → ffmpeg conversion → 16kHz WAVSystem Architecture
Agent Responsibilities
Search Agent
Handles semantic product discovery, filtering, and ranking.
Cart Agent
Manages cart operations such as adding, removing, and updating product quantities.
Order Agent
Processes checkout, billing, and order placement workflows.
Discovery Agent
Provides trending products, recommendations, and personalized suggestions.
Product Catalog
The platform currently supports:
Electronics (smartphones, headphones, TVs, monitors)
Footwear
Brands include:
Samsung
Apple
Sony
boAt
Nike
Adidas
Each product contains:
Price
Ratings
Stock availability
Category tags
Semantic vector embeddings for intelligent search
Workflow
User speaks a command through the web interface.
Audio is recorded in WebM/Opus format.
ffmpeg converts audio into 16kHz WAV.
Voice authentication verifies the speaker.
The orchestrator agent detects user intent.
Specialist agents execute tasks such as search, cart updates, or checkout.
Results are returned to the user in real time.
Highlights
Passwordless voice biometric authentication
Natural language shopping experience
Multi-agent AI architecture
Real-time semantic product search
Fully voice-controlled e-commerce workflow
Fast inference using Groq-hosted LLaMA 3.3 70B