Jarvis

A real-time AI assistant that provides contextual help during video calls, interviews, presentations, and meetings using screen capture and audio analysis.

Jarvis listens to the other side of the conversation, watches your screen, and streams concise, formatted answers into a transparent always-on-top overlay — so you always have the right thing to say or the right code on screen.

Note

Use the latest macOS and Windows versions; older versions have limited support.

Note

During testing it won't answer if you ask something — it answers the other speaker (e.g. the interviewer). Simulate the other side asking a question and Jarvis will respond.

Features

• Live AI assistance — real-time help powered by Google Gemini
• Screen & audio capture — analyzes what you see and hear for contextual responses
• Multiple profiles — Interview, Sales Call, Business Meeting, Presentation, Negotiation
• Transparent overlay — always-on-top window you can position anywhere
• Click-through mode — make the window transparent to clicks when needed
• Content protection — hidden from screen sharing/recording
• Cross-platform — macOS, Windows, and Linux (Linux is experimental)

Setup

1. Get a Gemini API key — visit Google AI Studio
2. Install dependencies — npm install
3. Run the app — npm start

Usage

1. Enter your Gemini API key in the main window
2. Choose your profile and language in settings
3. Click Start Session to begin
4. Position the window using keyboard shortcuts
5. Jarvis provides real-time assistance based on your screen and what the other side says

Keyboard Shortcuts

• Window movement — Ctrl/Cmd + Arrow Keys — move the window
• Click-through — Ctrl/Cmd + M — toggle mouse events
• Close/Back — Ctrl/Cmd + \ — close the window or go back
• Send message — Enter — send text to the AI

Audio Capture

• macOS: A Core Audio Process Tap helper (SystemAudioCapture) is used by default. It taps audio at the HAL layer and therefore captures all speaker output — including iPhone Continuity / FaceTime / CallKit phone calls that ScreenCaptureKit silently filters out. Requires macOS 14.2+ and "System Audio Recording" permission (System Settings → Privacy & Security → System Audio Recording). If the helper can't produce audio (e.g. permission denied on older macOS), the app falls back to the legacy ScreenCaptureKit-based SystemAudioDump so browser-based calls keep working.
• Windows: Loopback audio capture
• Linux: Microphone input

Rebuilding the macOS audio helper

SystemAudioCapture is a small Objective-C helper compiled from src/native/macos/SystemAudioCapture.m. A pre-built universal binary is checked in at src/assets/SystemAudioCapture. To rebuild it locally:

npm run build:macos-audio

This produces a universal (arm64 + x86_64 when both SDKs are available) binary at src/assets/SystemAudioCapture.

Requirements

• Electron-compatible OS (macOS, Windows, Linux)
• Gemini API key
• Screen recording permissions
• Microphone/audio permissions

License

GPL-3.0