VISION is an advanced, multimodal AI learning assistant built for students, educators, and researchers. It bridges the gap between passive consumption and active recall by transforming long-form content — YouTube video lectures, PDFs, and raw text — into structured summaries and context-aware interactive quizzes.
.✨ Key FeaturesFeatureDescriptionMulti-Input IngestionNative support for scraping YouTube audio, parsing PDF binary streams, or processing direct text strings.Automated Audio TranscriptionExtracts and converts lecture audio into high-fidelity text transcripts using OpenAI's Whisper model.Smart SummarizationLeverages a fine-tuned T5 pipeline to generate dense, hierarchical bullet points from lengthy source text.Interactive Contextual Q&ADynamically synthesizes customizable study quizzes using LLaMA 3, focusing on active recall.Persistent Session HistorySecurely maps data outputs and processing metadata into MongoDB Atlas for historic retrieval.Analytics-Ready StateBackend architecture decouples state tracking to easily feed data dashboards in future builds.
🛠️ Tech StackLayerTechnologyFrontend UIStreamlit (Python-native reactive UI framework) Backend APIFastAPI, Pydantic v2 (Async, type-safe execution) OrchestrationLangChain, Groq Cloud API EngineSpeech-to-TextOpenAI WhisperNLP SummarizationT5 (Text-to-Text Transfer Transformer)LLM ReasoningLLaMA 3 (8B/70B variants) DatabaseMongoDB Atlas (Cloud NoSQL) 📁 Project StructurePlaintextVISION/ ├── backend/ # FastAPI core routes, schemas, and API controllers ├── frontend/ # Streamlit UI layouts, pages, and web components ├── models/ # Isolated pipeline scripts (T5 configurations, Whisper wrapper, LLaMA 3 prompts) ├── .env # Local hardware/API keys (Environment secrets) ├── requirements.txt # Frozen Python dependency map └── README.md # Documentation
🚀 Getting StartedPrerequisitesPython 3.10 or higher installed locally.An active MongoDB Atlas cluster instance connection string.A registered Groq Cloud API Key.1. Clone the RepositoryBashgit clone https://github.com/Priyankshu-07/VISION.git cd VISION 2. Environment VirtualizationCreate and activate an isolated virtual sandbox environment:Bashpython -m venv venv
venv\Scripts\activate
source venv/bin/activate
3. Dependencies InstallationBashpip install -r requirements.txt
4. Configuration EnvironmentGenerate a .env file within the project's root folder:Code snippetMONGO_URL=your_mongodb_atlas_connection_string
GROQ_API_KEY=your_groq_api_key
🔮 Future Development Roadmap[ ] Advanced Analytics Dashboard: Integrating custom charting components to visualize learning progression metrics over time.[ ] Model Evaluation Matrices: Rigorous benchmarking evaluating local T5 metrics against leading cloud LLMs regarding summary density.[ ] Infrastructure Containerization: Packaging configurations inside lightweight Docker containers for immediate deployment to Render, AWS, or Railway.[ ] RAG Architecture Integration: Deploying Retrieval-Augmented Generation across historical summary databases for fast semantic search queries across older notes.
🤝 ContributingContributions push the developer community forward. To submit changes:Fork the repository layout. Spin up your feature branch (git checkout -b feature/AmazingFeature). Commit development benchmarks (git commit -m 'Add some AmazingFeature'). Push directly to your branch origin (git push origin feature/AmazingFeature). Open an official, descriptive Pull Request.
VISION follows a modular, three-layer pipeline:
[ Ingestion Layer ] [ Processing Layer ] [ Persistence Layer ]
• YouTube URLs → • Audio Extraction (Whisper) → • MongoDB Atlas
• PDF Uploads • Summarization (T5) (Metadata, Transcripts,
• Raw Text • Q&A Generation (LLaMA 3/Groq) Summaries, Quiz State)
Ingestion Layer — Handles diverse input types: video URLs, document streams, and raw text.
Processing Layer — Runs a multi-model pipeline. Video audio is transcribed via OpenAI Whisper; text is compressed into hierarchical summaries using T5; interactive quizzes are generated dynamically using LLaMA 3 via the Groq API.
Persistence Layer — A schema-flexible MongoDB Atlas layer caches processing states, user history, and compiled study materials for fast, asynchronous retrieval.
| Feature | Description |
|---|---|
| Multi-Input Ingestion | Supports YouTube audio scraping, PDF parsing, and raw text input |
| Automated Transcription | Converts lecture audio to high-fidelity text using OpenAI Whisper |
| Smart Summarization | Generates dense, hierarchical bullet-point summaries using a fine-tuned T5 pipeline |
| Interactive Q&A | Dynamically synthesizes customizable study quizzes via LLaMA 3 for active recall |
| Persistent Session History | Maps outputs and metadata to MongoDB Atlas for historical retrieval |
| Analytics-Ready State | Decoupled state tracking designed to feed data dashboards in future builds |
| Layer | Technology |
|---|---|
| Frontend UI | Streamlit |
| Backend API | FastAPI, Pydantic v2 |
| Orchestration | LangChain, Groq Cloud API |
| Speech-to-Text | OpenAI Whisper |
| NLP Summarization | T5 (Text-to-Text Transfer Transformer) |
| LLM Reasoning | LLaMA 3 (8B / 70B) |
| Database | MongoDB Atlas |
VISION/
├── backend/ # FastAPI routes, schemas, and API controllers
├── frontend/ # Streamlit UI layouts, pages, and components
├── models/ # Pipeline scripts (T5, Whisper wrapper, LLaMA 3 prompts)
├── .env # Environment secrets (API keys, DB connection)
├── requirements.txt # Python dependency list
└── README.md
- Python 3.10 or higher
- An active MongoDB Atlas cluster and connection string
- A Groq Cloud API key
git clone https://github.com/Priyankshu-07/VISION.git
cd VISIONpython -m venv venv
# Windows
venv\Scripts\activate
# macOS / Linux
source venv/bin/activatepip install -r requirements.txtCreate a .env file in the project root:
MONGO_URL=your_mongodb_atlas_connection_string
GROQ_API_KEY=your_groq_api_key
⚠️ Never commit your.envfile. Make sure it's listed in.gitignore.
Start the services in this order. Open two separate terminal windows:
Terminal 1 — Backend:
uvicorn backend.main:app --reloadTerminal 2 — Frontend:
streamlit run frontend/app.pyThe app will be available at http://localhost:8501.
- Analytics Dashboard — Visualize learning progression metrics over time
- Model Benchmarking — Evaluate local T5 summaries against cloud LLMs for quality and density
- Docker Support — Containerize the stack for deployment to Render, AWS, or Railway
- RAG Integration — Enable semantic search across historical summaries using Retrieval-Augmented Generation
Contributions are welcome! To get started:
- Fork the repository
- Create a feature branch:
git checkout -b feature/YourFeature - Commit your changes:
git commit -m "Add YourFeature" - Push to your branch:
git push origin feature/YourFeature - Open a Pull Request with a clear description of your changes
Distributed under the MIT License. See LICENSE for details.