Voice Trigger: ESP32-S3 with TensorFlow Lite Micro

Real-time wake word detection running entirely on ESP32-S3 using TensorFlow Lite Micro. No cloud required - all AI inference happens on-device.

Project Origin

This project is based on the microWakeWord library from the ESPHome project, adapted for standalone ESP32-S3 deployment using ESP-IDF. The original example code was modified and optimized for the XIAO ESP32-S3 Sense hardware with PDM microphone support.

Key modifications:

• Adapted PDM microphone configuration for XIAO ESP32-S3 Sense
• Integrated pre-trained professional models (Alexa, Hey Jarvis, Hey Mycroft)
• Optimized audio parameters (16kHz, 16-bit) for better accuracy
• Multi-model support with easy switching

Overview

Detect wake words like "Alexa", "Hey Jarvis", or "Hey Mycroft" with <70ms latency on a $7 microcontroller. Perfect showcase of TinyML/Edge AI in action.

• ✅ On-device inference - <10ms per prediction, no cloud
• ✅ Production ready - >90% accuracy with professional models

Quick Start

Usage

Say "ALEXA"

✅ Ready! Say 'ALEXA' to trigger detection.
[Listening... loops=500]

*** WAKE WORD DETECTED! ***

Available Wake Word Models

Model	Wake Word	Size	Accuracy
alexa.h	"Alexa"	55KB	>90% (active)
hey_jarvis.h	"Hey Jarvis"	52KB	>90%
hey_mycroft.tflite	"Hey Mycroft"	56KB	>90%
okay_nabu.tflite	"Okay Nabu"	55KB	>90%

Training Your Own Wake Word

You have two options:

Option 1: microWakeWord (Recommended for this project)

How to train:

1. Visit microWakeWord releases
2. Use training scripts to create custom model
3. Export as .tflite and convert to .h file
4. Drop into your project

Option 2: Edge Impulse

How to train:

1. Create project at edgeimpulse.com
2. Record 100+ samples via phone
3. Train in web UI
4. Download C++ library and integrate

Configuration

Adjust Detection Sensitivity

Edit main/main.cpp line 60:

// Stricter (fewer false positives)
wakeWord.add_wake_word_model(model, 0.9f, 5, "Alexa", 22348);

// More sensitive (easier detection, more false positives)
wakeWord.add_wake_word_model(model, 0.5f, 5, "Alexa", 22348);

How It Works

PDM Mic (16kHz) → I2S Buffer → FFT → Mel Filterbank →
MFCC (13 coeffs) → Neural Network → Softmax → Threshold → DETECTED!

Performance:

• Inference: ~10ms
• Total latency: <70ms
• RAM usage: ~150KB
• Model size: 55KB
• Power: ~100mW

Project Structure

main/
├── main.cpp                 # Main application (Alexa model)
├── alexa.h                  # Alexa model (55KB)
└── hey_jarvis.h             # Hey Jarvis model (52KB)

models/
├── alexa.tflite            # Pre-trained models
├── hey_mycroft.tflite
└── okay_nabu.tflite

managed_components/
├── micro_wake_word/        # Wake word engine
├── esp-tflite-micro/       # TensorFlow Lite runtime
└── esp-nn/                 # Hardware acceleration