Friday AI Assistant - Open Source

Friday is an advanced, multimodal Voice AI Assistant designed to automate PC tasks, manage files, fetch real-time information, and integrate with custom MCP servers. It uses state-of-the-art LLMs combined with LiveKit's robust real-time communication infrastructure to deliver a seamless voice experience.

Features

• Real-Time Voice Interaction: Powered by Gemini Realtime API and LiveKit for sub-second latency conversational AI.
• System Automation: Control PC settings, manipulate files, launch applications, and manage windows using natural language.
• Tool Calling: Extendable tool framework (automation.py & tools.py) for custom functions (e.g., fetching weather, managing reminders, creating project structures).
• Persistent Memory: Uses mem0 for long-term personalized context across sessions.
• MCP Client: Connects to Model Context Protocol (MCP) servers to leverage external integrations.
• Modern Web UI: Comes with a Next.js frontend built on LiveKit Components for an immersive user interface.

Video Demo

Video.Project.4.mp4

Prerequisites

• Python 3.10+
• Node.js & pnpm
• LiveKit Cloud account (for API keys and WebRTC infrastructure)
• API Keys for your preferred LLMs (OpenRouter, Gemini, OpenAI)

Setup Guide

1. Backend Setup (Python)

1. Clone the repository (or download the files) and navigate to the project directory.
2. Create a virtual environment:

   python -m venv myenv
   myenv\Scripts\activate

3. Install Dependencies:

   pip install -r requirements.txt

4. Environment Variables: Copy the provided .env.example to .env and fill in your details:

   copy .env.example .env

5. Run the Backend Agent: Start the LiveKit agent in development mode (which supports hot-reloading):

   python agent.py dev

2. Frontend Setup (React/Next.js UI)

1. Navigate to the frontend directory:

   cd agent-starter-react-main

2. Install Dependencies:

   pnpm install

3. Configure Frontend Environment: Ensure your .env.local inside the frontend directory contains your LiveKit credentials (matching the backend).
4. Run the UI:

   pnpm dev

   Open http://localhost:3000 in your browser.

Customizing Tools

Friday's capabilities are split between:

• tools.py: General web queries, content generation, and APIs.
• automation.py: OS-level operations (file management, system controls).
• mob_automation.py: Mobile-specific or external device automations.

To add new capabilities, define a new async function using the @function_tool decorator in one of these files and append it to the exported registered tools list.

Licensing

This project is open-sourced under the MIT License. Infrastructure and real-time networking powered by LiveKit. Original project architecture by yours Truly- Debajeet. feel free to clone, test, use and contribute for any bug findings or feature suggestions.