Emo-Aware-LLM

An emotion-aware voice assistant that reads your facial expression before it answers.

Most voice assistants are blind to how you feel. Emo-Aware-LLM looks at you first. It grabs a webcam frame, uses DeepFace to pick your dominant emotion, then folds that emotion into the prompt it sends to an LLM. The response is spoken back to you. The whole loop is a few hundred lines of Python, runs on a laptop webcam or a Raspberry Pi, and is released as open source under the MIT license.

How a turn works

1. The webcam opens. DeepFace labels your dominant emotion live on the preview.
2. Press Space to capture a frame.
3. The assistant says "Wow, you look {emotion}!" and listens to your voice.
4. Your words and your emotion go to the LLM as one combined prompt.
5. The LLM's reply is spoken back to you.

What is in the box

• src/vision/ - webcam capture and DeepFace emotion recognition with OpenCV.
• src/audio/ - speech to text with SpeechRecognition and text to speech with gTTS.
• src/ai/ - OpenAI handler that blends emotion and user input into one prompt.
• src/utils/ - platform setup, including optional Raspberry Pi ALSA config.

Prerequisites

• Python 3.8+
• A webcam
• A microphone
• An OpenAI API key

Install

git clone https://github.com/siddvoh/Emo-Aware-LLM.git
cd Emo-Aware-LLM
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
# add your OpenAI API key to .env

Run

python src/main.py

Notes

• On Apple Silicon, the requirements pin tensorflow-macos and tensorflow for DeepFace. Set up a clean virtual environment so those pins do not clash with other projects.
• The .env.example includes optional ALSA_PCM_CARD and ALSA_PCM_DEVICE values for Raspberry Pi audio routing.

License

MIT. See LICENSE.