Real-time Voice Phishing Detection System
Submitted to the 2023 KISA AI+Security Idea Competition. Used FSS and AI Hub datasets for academic research.
Voice phishing targeting people in their 20s now accounts for 30% of total cases, but existing prevention methods only work after the damage is done. WatchCall detects suspicious patterns in real-time during phone calls to prevent fraud.
- Real-time analysis: Detects suspicious patterns during calls
- Text + Voice analysis: Analyzes both conversation content and voice tone
- Call summary: Summarizes long conversations
- Instant alerts: Warns immediately when risks are detected
Searches for phishing keywords like "urgent transfer", "legal action", "account suspension". Uses context analysis instead of simple keyword matching.
AI voices sound different from real human voices:
- Background noise (AI voices are usually clean)
- Voice intensity and frequency patterns
- Emotional changes (AI voices are unnatural)
Since scammers now use AI-generated voices, we added a model to distinguish them. Uses voice-to-text confidence scores and MFCC features for detection.
Datasets
- Financial Supervisory Service: Real voice phishing cases and transcripts
- AI Hub: Customer service call data
Model Architecture
- NER-based text analysis
- MFCC voice feature extraction
- Deepvoice detection model
- Ensemble for final prediction
Sends alerts when both text and voice models flag the call as suspicious.
After each call:
- Call summary (who and what)
- Risk level (red/yellow/green alerts)
- Full conversation text with highlighted suspicious parts
- Real-time fraud pattern updates via police database integration
- SMS phishing detection
- Bank FDS system integration
- Guardian alerts for elderly users
Feel free to reach out if you have questions.