Vyoom — AI Property Tour Video Platform

Generate multilingual, avatar-led real-estate tour videos from a PDF brochure + DWG/DXF floor plan. Vyoom pipelines the work across 9 specialised AI agents — intake, fact validation, strategy, script, localisation, cast anchoring, assembly, QA, and publishing — each calling Gemini via a thin wrapper that falls back to deterministic mocks in development.

---

Stack

Layer	Tech
Backend	FastAPI · Python 3.11 · Motor (async MongoDB)
AI	Google Gemini (google-genai) via llm_client.py
Storage	AWS S3 ap-south-1 (Mumbai — DPDP) · local-disk fallback
TTS	Sarvam AI Bulbul V3 (SARVAM_API_KEY) · mock FFmpeg synth
Video	Veo 3.1 · Runway Gen 4 · HeyGen avatar (HEYGEN_API_KEY) · mock FFmpeg renders
Rendering	Shotstack
Frontend	React 19 · Vite · Tailwind CSS · shadcn/ui
Auth	Hand-rolled JWT (python-jose HS256) · passlib bcrypt
Rate limiting	slowapi

---

Project layout

Ai-Video/
├── backend/
│   ├── server.py              # FastAPI app factory (~830 lines)
│   ├── auth.py                # JWT auth module
│   ├── llm_client.py          # google-genai wrapper (replaces LlmChat)
│   ├── storage.py             # S3 + local-disk object storage
│   ├── common.py              # Shared models & helpers
│   ├── routers/               # One APIRouter per domain
│   │   ├── auth_router.py     # /auth/register, /auth/login, /auth/me
│   │   ├── projects.py
│   │   ├── intake.py
│   │   ├── facts.py
│   │   ├── strategy.py
│   │   ├── script.py
│   │   ├── assets.py
│   │   ├── localization.py
│   │   ├── assembly.py
│   │   ├── qa.py
│   │   └── publish.py
│   ├── intake_agent.py        # Agent 1 — brochure + CAD extraction
│   ├── fact_validator.py      # Agent 2 — compliance fact check
│   ├── strategy_script.py     # Agent 3 — video strategy
│   ├── generation.py          # Agent 4/5 — scene scripts
│   ├── cast_anchor.py         # Agent 6/7 — photoreal cast still
│   ├── audio_gen.py           # Agent 9 — dub audio (Sarvam / mock)
│   ├── video_gen.py           # Agent 8 — clip gen (Veo / Runway / HeyGen / mock)
│   ├── shotstack.py           # Assembly render (Shotstack)
│   ├── ken_burns.py           # DXF → Ken Burns MP4
│   ├── qa_guardrail.py        # Claims diff check
│   ├── publish_pkg.py         # Publish package generation
│   ├── scripts/
│   │   └── migrate_add_org_id.py   # One-time MongoDB backfill
│   ├── .env.example
│   ├── requirements.txt
│   └── requirements-dev.txt
├── frontend/                  # Vite + React
│   ├── src/
│   │   ├── lib/api.js         # Axios client (VITE_BACKEND_URL)
│   │   ├── pages/
│   │   │   ├── ProjectsList.js
│   │   │   └── ProjectDetail.js
│   │   └── components/        # UI components + shadcn/ui
│   ├── vite.config.js         # @ alias + dev proxy → :8001
│   └── .env.example
├── scripts/                   # Utility scripts
└── tests/

---

Quick start

Prerequisites

• Python 3.11+
• Node.js 18+
• MongoDB (local or Atlas)
• FFmpeg (for mock audio/video; imageio-ffmpeg pip package provides a static build)

1. Backend

cd backend

# Copy and fill env vars
cp .env.example .env
# At minimum set: MONGO_URL, JWT_SECRET, GEMINI_API_KEY

# Create a virtual environment
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt  # production
pip install -r requirements-dev.txt  # + dev tools

# Start
uvicorn server:app --reload --port 8001

API docs available at http://localhost:8001/docs

2. Frontend

cd frontend

cp .env.example .env.local
# VITE_BACKEND_URL=http://localhost:8001

npm install
npm run dev     # → http://localhost:3000

3. Existing database migration (first run only)

If you have data from before multi-tenancy was added, backfill org_id:

cd backend
python scripts/migrate_add_org_id.py

---

Environment variables

See backend/.env.example for the full list. Key variables:

Variable	Required	Description
MONGO_URL	✅	MongoDB connection string
JWT_SECRET	✅	Random secret for JWT signing — change in prod
GEMINI_API_KEY	✅	Google AI Studio key — agents fall back to mock without it
AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / S3_BUCKET_NAME	Optional	S3 storage — falls back to ./local_storage/
AWS_REGION	Optional	Default: ap-south-1 (Mumbai — DPDP data-residency)
SARVAM_API_KEY	Optional	Enable Sarvam AI Bulbul V3 TTS
HEYGEN_API_KEY	Optional	Enable HeyGen avatar video rendering
SHOTSTACK_API_KEY	Optional	Enable Shotstack assembly render
CORS_ORIGINS	Optional	Comma-separated allowed origins (default: http://localhost:3000)

Frontend (frontend/.env.local):

Variable	Description
VITE_BACKEND_URL	Backend base URL (default: http://localhost:8001)

---

Mock mode

Every external provider has a mock fallback that activates automatically when the corresponding API key is absent. Mocks produce real output (actual MP4s, MP3s, valid JSON) so the full pipeline can run end-to-end in development without any paid keys:

Provider	Mock behaviour
Gemini (LLM)	RuntimeError / ValueError — UI shows "mock mode" banner
S3	Files written to backend/local_storage/
Sarvam AI (TTS)	FFmpeg sine-wave MP3 with per-language duration bias
Veo / Runway / HeyGen	FFmpeg testsrc2 MP4 with exact scene duration
Shotstack	Placeholder render response

---

Auth

Registration creates an org + admin user in one step:

# Register
curl -X POST http://localhost:8001/auth/register \
  -H "Content-Type: application/json" \
  -d '{"email":"you@example.com","password":"secret","org_name":"Acme Realty"}'

# Login → token
curl -X POST http://localhost:8001/auth/login \
  -d "username=you@example.com&password=secret"

# Use token
curl http://localhost:8001/api/projects \
  -H "Authorization: Bearer <token>"

---

Architecture

                    ┌─────────────────────────────────────────┐
                    │               FastAPI (server.py)        │
                    │  Auth · CORS · Rate limit · Routers      │
                    └────────────┬───────────────┬────────────┘
                                 │               │
              ┌──────────────────┘               └──────────────────┐
              ▼                                                       ▼
  ┌─────────────────────┐                            ┌──────────────────────┐
  │  MongoDB (motor)    │                            │  llm_client.py        │
  │  users / orgs       │                            │  google-genai SDK     │
  │  projects / scenes  │                            │  (Gemini 2.0)         │
  │  artifacts / assets │                            └──────────┬───────────┘
  └─────────────────────┘                                       │
                                         ┌─────────────────────┘
                    ┌────────────────────▼──────────────────────────────┐
                    │             AI Agent pipeline                      │
                    │  intake → facts → strategy → script →             │
                    │  localization → cast-anchor → assembly → QA →     │
                    │  publish                                           │
                    └────────────────────┬──────────────────────────────┘
                                         │
                    ┌────────────────────▼──────────────────────────────┐
                    │             Provider layer (mocked in dev)         │
                    │  storage.py  →  S3 ap-south-1 / local_storage/    │
                    │  audio_gen   →  Sarvam AI / FFmpeg mock            │
                    │  video_gen   →  Veo / Runway / HeyGen / FFmpeg     │
                    │  shotstack   →  Shotstack render                   │
                    └───────────────────────────────────────────────────┘

---

Development

# Lint + format (backend)
cd backend
black .
isort .
flake8 .

# Type check
mypy .

# Tests
pytest

# Vite build check (frontend)
cd frontend
npm run build

---

Data residency

All S3 operations default to ap-south-1 (AWS Mumbai) to satisfy India's DPDP Act requirements for Indian real-estate customer data. Do not change AWS_REGION to a non-India region without legal review.