StudentSync

The AI-powered study agent that actually gets the work done.

Quick Start • Features • Architecture • Configuration • Commands • Contributing

🎯 What is StudentSync?

StudentSync is a messaging-first AI agent platform built for college students. Send a natural-language prompt, a PDF, a voice note, or a screenshot to your Telegram or WhatsApp chat — and StudentSync handles the rest: parsing, solving, code execution, LaTeX compilation, PDF generation, and even auto-submission to Google Classroom.

Under the hood, it's a modular TypeScript agent runtime with a plugin-based tool system. The LLM is the router — no keyword matching, no handler switches. Adding a new capability is literally a single file.

  You: "Solve Q3 from this PDF and submit it to OS class"
  
  StudentSync:
    ├─ 📄 Parsed PDF → extracted Q3
    ├─ 🧠 Classified as "Coding + Theory" → built execution plan
    ├─ 💻 Executed Python solution in sandbox
    ├─ 📝 Formatted answer with LaTeX
    ├─ 📋 Rendered as professional PDF
    └─ 🎓 Uploaded to Google Classroom → turned in ✅

⚡ Quick Start

One-Line Install (Global + Daemon)

curl -fsSL https://raw.githubusercontent.com/Ayush-Vish/studentsync/main/install.sh | bash

This installs studentsync globally via npm, sets up PM2, and starts the daemon. Then run the setup wizard:

studentsync onboard

Manual Setup

# 1. Clone
git clone https://github.com/Ayush-Vish/studentsync.git
cd studentsync

# 2. Install dependencies
npm install

# 3. Provision runtime toolchains (Python venv, Chrome, Tectonic)
npm run setup

# 4. Build
npm run build

# 5. Run interactive onboarding
npm start onboard

# 6. Launch
npm start

Prerequisites: Node.js 20+, npm 9+. Optional: ffmpeg (voice notes), Python 3.10+ (code sandbox).

✨ Features

📱 Multi-Platform Messaging

Platform	Integration	Auth
Telegram	Full-featured via Telegraf	Bot token from @BotFather
WhatsApp	Multi-device via Baileys	QR scan in terminal
Slack	Placeholder for future expansion	—

Session persistence ensures automatic reconnection across restarts.

🎓 Academic Intelligence

Voice Notes — Send a voice message on Telegram/WhatsApp; transcribed locally with whisper.cpp (free, on-device, no API keys)
PDF & Image Parsing — Extract assignment context from PDFs (pdf-parse) and images (tesseract.js OCR)
Adaptive Task Classification — Classifies tasks as Theory, MCQ, Coding, Research, or Mixed, then builds custom execution plans
Agentic Problem Solving — Iterative agent loop with tool calls for complex multi-step assignments
Follow-up Support — Full conversation history for revisions and clarifications

🛠️ Tool Plugin System

The core of StudentSync. Every capability is a self-describing ToolPlugin that the LLM can invoke dynamically:

Category	Tools
Code Execution	Python, Node.js, Bash, C, C++, Java sandboxes
Document Rendering	PDF generation, LaTeX compilation (Tectonic), code screenshots
Web Research	Real-time web search via DuckDuckGo
File Operations	Full CRUD on the agent workspace
Shell Integration	Controlled shell command execution
Text Processing	Grep and sed for advanced search/replace
Resume Builder	LaTeX resume generation from Overleaf templates

🎓 Google Classroom Automation

Search courses and coursework directly from chat
Upload generated artifacts to Google Drive
Attach files and turn in assignments — all hands-free

📅 Smart Scheduling & Reminders

Natural language — "Remind me every day at 9am to study OS"
Cron-backed — Persistent schedules via node-cron
Messaging delivery — Reminders sent to your active chat channel

🧘 Student Life & Productivity

Domain	Capabilities
Finance	Log expenses, set monthly budgets, spending summaries, bill splitting
Personal	Todo lists, habit tracking with streaks
Emotional	Mood logging, journaling, guided breathing & grounding exercises
Career	Job application tracking and status management

🛡️ Safety: The emotional toolset includes crisis detection — if the user expresses distress, the agent immediately surfaces helpline numbers and crisis resources. Breathing/grounding exercises are deterministic (no LLM improvisation).

🔌 MCP (Model Context Protocol) Support

Extend StudentSync with external tools via MCP servers:

Google MCP — Gmail, Calendar, Google Chat integration
GitHub MCP — Issues, PRs, repository management
Add custom MCP servers through the onboarding wizard

🏗️ Architecture

┌──────────────────────────────────────────────────────────┐
│                     MESSAGING LAYER                       │
│  ┌─────────┐   ┌───────────┐   ┌───────┐                │
│  │Telegram │   │ WhatsApp  │   │ Slack │                 │
│  │(Telegraf)│   │ (Baileys) │   │ (WIP) │                │
│  └────┬────┘   └─────┬─────┘   └───┬───┘                │
│       └───────────────┼─────────────┘                    │
│                       ▼                                   │
│              ┌────────────────┐                           │
│              │   Bot Router   │  ← unified message handler│
│              └───────┬────────┘                           │
├──────────────────────┼───────────────────────────────────┤
│                      ▼           AGENT CORE               │
│         ┌─────────────────────┐                           │
│         │   Orchestrator      │                           │
│         │  ┌──────────────┐   │                           │
│         │  │  Classifier  │   │  Task type detection      │
│         │  └──────┬───────┘   │                           │
│         │  ┌──────▼───────┐   │                           │
│         │  │   Planner    │   │  Dynamic execution plan   │
│         │  └──────────────┘   │                           │
│         └─────────┬───────────┘                           │
│                   ▼                                       │
│         ┌─────────────────────┐                           │
│         │    Agent Loop       │  ← LLM + tool dispatch    │
│         │    (llm.ts)         │                           │
│         └─────────┬───────────┘                           │
│                   ▼                                       │
│  ┌────────────────────────────────────────┐               │
│  │          Tool Plugin Registry          │               │
│  │  ┌──────────┐  ┌───────────────────┐   │               │
│  │  │ Builtin  │  │  Domain Plugins   │   │               │
│  │  │ ┌──────┐ │  │ ┌───────────────┐ │   │               │
│  │  │ │ Code │ │  │ │   Finance     │ │   │               │
│  │  │ │ PDF  │ │  │ │   Personal    │ │   │               │
│  │  │ │ LaTeX│ │  │ │   Emotional   │ │   │               │
│  │  │ │ Shell│ │  │ │   Career      │ │   │               │
│  │  │ │ Web  │ │  │ └───────────────┘ │   │               │
│  │  │ │ File │ │  └───────────────────┘   │               │
│  │  │ └──────┘ │                          │               │
│  │  └──────────┘                          │               │
│  └────────────────────────────────────────┘               │
├──────────────────────────────────────────────────────────┤
│                    INTEGRATIONS                           │
│  ┌──────────┐  ┌──────────┐  ┌────────────┐              │
│  │ Google   │  │   MCP    │  │  Whisper   │              │
│  │Classroom │  │ Servers  │  │   (STT)    │              │
│  │ + Drive  │  │          │  │            │              │
│  └──────────┘  └──────────┘  └────────────┘              │
└──────────────────────────────────────────────────────────┘

Source Layout

src/
├── bot/                              # Messaging adapters
│   ├── TelegramBot.ts                #   Telegraf wrapper
│   ├── WhatsAppBot.ts                #   Baileys multi-device
│   ├── WhatsAppConnectionController  #   QR auth + reconnection
│   └── index.ts                      #   Unified bot router
├── cli/                              # Setup wizard + CLI commands
├── core/
│   ├── llm.ts                        # Agent loop (Gemini / OpenAI)
│   ├── plugins/                      # Tool plugin system
│   │   ├── ToolRegistry.ts           #   Registry + dispatch
│   │   ├── builtin/                  #   Academic + workspace tools
│   │   └── domains/                  #   Finance, personal, emotional, career
│   ├── orchestrator/                 # Task classifier + planner
│   ├── mcp/                          # MCP client manager + tool bridge
│   ├── scheduler/                    # Cron-backed reminders
│   ├── classroom.ts                  # Google Classroom + Drive
│   ├── formatter.ts                  # Output formatting engine
│   ├── parser.ts                     # PDF parsing
│   ├── store.ts                      # Local user/session persistence
│   ├── schema.ts                     # Config schema + defaults
│   ├── session-manager.ts            # Multi-user session management
│   ├── tools.ts                      # Artifact workspace + tool executor
│   └── transcriber.ts                # whisper.cpp voice transcription
├── api/                              # (Future) REST API surface
├── types/                            # TypeScript type definitions
├── templates/                        # LaTeX templates
├── resume-templates/                 # Bundled resume templates
├── skills/                           # Document/artifact generation skills
└── scripts/                          # Runtime provisioning

🔧 Configuration

Environment Variables

StudentSync uses a JSON config at ~/.studentsync/config.json (created by the onboarding wizard). Environment variables always take priority over the config file.

Core

Variable	Required	Description
`LLM_PROVIDER`	✅	`gemini`, `openai`, `openrouter`, or `ollama`
`LLM_API_KEY`	✅*	API key for your chosen provider (*not needed for Ollama)
`LLM_MODEL`	—	Model override (default: `gpt-4o` for OpenAI, `gemini-2.0-flash` for Gemini)
`TELEGRAM_BOT_TOKEN`	—	Token from @BotFather
`WHATSAPP_ENABLED`	—	Set to `true` to enable WhatsApp
`WHATSAPP_AUTH_DIR`	—	Path to store WhatsApp auth session

Google Classroom

Variable	Required	Description
`GOOGLE_CLIENT_ID`	—	OAuth 2.0 client ID
`GOOGLE_CLIENT_SECRET`	—	OAuth 2.0 client secret
`GOOGLE_REFRESH_TOKEN`	—	OAuth refresh token

Voice Transcription (whisper.cpp)

Variable	Default	Description
`WHISPER_CPP_MODEL`	(required)	Path to ggml model, e.g. `…/models/ggml-base.en.bin`
`WHISPER_CPP_BIN`	`whisper-cli`	Path to the whisper.cpp CLI binary
`FFMPEG_BIN`	`ffmpeg`	Override if ffmpeg isn't on `PATH`
`WHISPER_CPP_LANG`	(auto)	Language hint (e.g. `en`). Omit to auto-detect
`WHISPER_CPP_THREADS`	(default)	Thread count for transcription

Runtime

Variable	Default	Description
`STUDENTSYNC_HOME`	`~/.studentsync`	Base directory for all runtime data
`STUDENTSYNC_PYTHON`	Auto-provisioned	Path to Python binary
`PUPPETEER_EXECUTABLE_PATH`	Auto-provisioned	Path to Chrome/Chromium
`TECTONIC_BIN`	Auto-provisioned	Path to Tectonic LaTeX engine

If voice-note env vars aren't set, StudentSync simply replies with a friendly "voice notes aren't set up" message and continues working for text/PDF/image input.

Runtime Provisioning

The npm run setup command (also runs on postinstall) provisions into ~/.studentsync:

Component	What it does	Override
Python venv	Creates `venv/` with `reportlab`, `fpdf2`, `matplotlib`, `numpy`, `pandas`, `sympy`, etc.	`STUDENTSYNC_PYTHON`
Chrome	`puppeteer browsers install chrome` for HTML→PDF and screenshots	`PUPPETEER_EXECUTABLE_PATH`
Tectonic	Downloads the Tectonic LaTeX engine into `bin/`	`TECTONIC_BIN`

Every step is best-effort — if one toolchain can't be installed, the rest still proceed and the agent degrades gracefully.

📖 Commands Reference

NPM Scripts

Command	Description
`npm run build`	Compile TypeScript → `dist/`
`npm start`	Build + start messaging runtime
`npm start onboard`	Re-run the interactive setup wizard, then launch
`npm run setup`	Provision runtime toolchains (Python, Chrome, Tectonic)
`npm run start:daemon`	Start as a background service with PM2
`npm run stop:daemon`	Stop and remove the PM2 daemon

Chat Commands

Command	Description
`/start`	Initialize the bot conversation
`/new`	Start a fresh session
`/context`	Show current session info, workspace, and turn count
`/help`	List available commands
`/expense`	Log a new expense
`/todo`	Manage your todo list
`/mood`	Log how you're feeling
`/budget`	View budget summary
`/habits`	Track daily/weekly habits
`/apps`	View job application tracker

Pro tip: You don't need slash commands. Just type naturally — "add milk to my todo", "I spent ₹200 on lunch", "I'm feeling stressed". The agent understands.

🎙️ Voice Notes Setup

Voice transcription runs fully local — no API keys, no cloud dependency.

# 1. Install ffmpeg
sudo apt install ffmpeg            # Debian/Ubuntu
brew install ffmpeg                # macOS

# 2. Build whisper.cpp
git clone https://github.com/ggerganov/whisper.cpp && cd whisper.cpp
cmake -B build && cmake --build build --config Release
sh ./models/download-ggml-model.sh base.en

Then set the environment variables:

export WHISPER_CPP_MODEL="/path/to/whisper.cpp/models/ggml-base.en.bin"
export WHISPER_CPP_BIN="/path/to/whisper.cpp/build/bin/whisper-cli"

🧩 Extending StudentSync

Adding a New Tool Plugin

Creating a new capability is a one-file change:

// src/core/plugins/builtin/myTool.ts
import type { ToolPlugin } from '../ToolRegistry.js';

export const myToolPlugin: ToolPlugin = {
  name: 'my_tool',
  description: 'Does something awesome',
  parameters: {
    input: { type: 'string', description: 'The input to process' }
  },
  required: ['input'],
  async execute(args, context) {
    // Your logic here
    return { result: `Processed: ${args.input}` };
  }
};

Then export it from builtin/index.ts — done. The LLM can now call it. No prompt engineering, no router changes, no bot code modifications.

Adding a Life Domain

For persistent user data, extend StoredUserProfile in store.ts with a typed sub-store, initialize it in normalizeUserProfile(), and create domain plugins in src/core/plugins/domains/.

Adding an MCP Server

Add a preset to MCP_PRESETS in schema.ts, or configure custom servers through the onboarding wizard. The MCPClientManager handles lifecycle and the MCPToolBridge exposes external tools to the agent loop.

🛠️ Tech Stack

Layer	Technology
Runtime	Node.js 20+ · TypeScript
AI/LLM	Google GenAI SDK · OpenAI-compatible APIs · Ollama
Messaging	Telegraf (Telegram) · Baileys (WhatsApp)
Automation	Puppeteer · Google APIs (Classroom + Drive)
Documents	pdf-parse · Tesseract.js · md2latex · Tectonic
Scheduling	node-cron
CLI	Commander · @clack/prompts · picocolors
Frontend	Next.js 16 · Tailwind CSS · Radix UI · Framer Motion
Voice	whisper.cpp · ffmpeg
Process Mgmt	PM2

🤝 Contributing

Contributions are welcome! Here's how to get started:

Fork the repository
Create a feature branch: git checkout -b feat/my-feature
Make your changes and ensure the build passes: npm run build
Commit with a descriptive message: git commit -m "feat: add X capability"
Push to your fork: git push origin feat/my-feature
Open a Pull Request

Development Setup

git clone https://github.com/Ayush-Vish/studentsync.git
cd studentsync
npm install
npm run setup    # provision runtime toolchains
npm run build    # compile TypeScript
npm start onboard  # configure LLM + bot tokens

Project Structure

Directory	What lives there
`studentsync/`	Core agent runtime (TypeScript)
`frontend/`	Next.js marketing/docs site

📝 Design Philosophy

LLM-as-Router — No hardcoded workflows. The LLM decides which tools to call based on the user's intent.
Plugin-First — Every capability is a self-describing ToolPlugin. Adding features = adding files.
Graceful Degradation — Missing toolchains? The agent adapts. No Chrome? Skip screenshots. No Tectonic? Fall back to HTML PDFs.
Privacy-First Voice — Voice transcription runs entirely on-device with whisper.cpp. Zero cloud dependency.
Safety by Design — Crisis detection in emotional tools, no medical/diagnostic language, deterministic grounding exercises.

📄 License

This project is licensed under the MIT License — see the LICENSE file for details.

_{Built with ☕ and 🧠 by Ayush Vishwakarma}

Name		Name	Last commit message	Last commit date
Latest commit History 60 Commits
.github		.github
.vscode		.vscode
resume-templates		resume-templates
scripts		scripts
skills		skills
src		src
templates/latex		templates/latex
.codex		.codex
.gitignore		.gitignore
DESIGN_CHOICES.md		DESIGN_CHOICES.md
LICENSE		LICENSE
README.md		README.md
design_docs.md		design_docs.md
install.sh		install.sh
package-lock.json		package-lock.json
package.json		package.json
test-ddg-html.js		test-ddg-html.js
test-ddg.js		test-ddg.js
test-googlethis.js		test-googlethis.js
tsconfig.json		tsconfig.json

Folders and files

Latest commit

History

Repository files navigation