A comprehensive curated list of papers, codes, datasets, and resources for Talking Head Generation research. This collection covers audio-driven, image-driven, text-driven methods, as well as NeRF, 3D Gaussian Splatting, and other cutting-edge techniques.
🔆 Last Updated: January 2026
📩 Contributions Welcome: Feel free to submit PRs for missing papers, new papers, or corrections!
- Surveys
- Datasets
- Audio-Driven Methods
- Image-Driven Methods
- Text-Driven Methods
- NeRF & 3D Gaussian Splatting
- Evaluation Metrics
- Tools & Software
- Related Research Groups
| Year | Title | Venue | Links |
|---|---|---|---|
| 2025 | Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functions | arXiv | Paper |
| 2024 | A Comparative Study of Perceptual Quality Metrics for Audio-driven Talking Head Videos | arXiv | Paper Code |
| 2024 | Deepfake Generation and Detection: A Benchmark and Survey | arXiv | Paper Code |
| 2024 | A Survey on 3D Human Avatar Modeling — From Reconstruction to Generation | arXiv | Paper |
| 2024 | Neural Radiance Fields: Past, Present, and Future | arXiv | Paper |
| 2024 | A Survey on 3D Gaussian Splatting | arXiv | Paper |
| 2023 | From Pixels to Portraits: A Comprehensive Survey of Talking Head Generation Techniques and Applications | arXiv | Paper |
| 2024 | A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing | arXiv | Paper |
| 2023 | Human-Computer Interaction System: A Survey of Talking-Head Generation | IEEE | Paper |
| 2022 | Deep Learning for Visual Speech Analysis: A Survey | arXiv | Paper |
| 2020 | What comprises a good talking-head video generation?: A Survey and Benchmark | arXiv | Paper |
| Dataset | Description | Download |
|---|---|---|
| VoxCeleb1 | Over 100,000 utterances from 1,251 celebrities extracted from YouTube videos | Link |
| VoxCeleb2 | Largest public audio-visual dataset, primarily for speaker recognition. Requires 300GB+ storage | Link |
| HDTF | High-definition Talking-Face Dataset with ~362 videos totaling 15.8 hours at 720P/1080P | Link |
| TalkingHead-1KH | 500k video clips, ~80k at 512x512+ resolution under permissive licenses | Link |
| CelebV-HQ | 35,666 high-quality clips (512x512+), 15,653 identities, 83 facial attributes | Link |
| MEAD | Emotional talking head dataset with 8 emotions at 3 intensity levels | Link |
| LRW | Lip Reading in the Wild - 1000+ speakers, 29-frame clips with diverse speaking styles | Link |
| AVSpeech | Large-scale audio-visual speech dataset | Link |
| CREMA-D | 7,442 clips from 91 actors expressing 6 emotions at 4 intensity levels | Link |
| Hallo3 | Training data for Hallo3 model | Link |
| Dataset | Description | Download |
|---|---|---|
| MultiTalk | 420+ hours across 20 languages, 293,812 clips at 512x512, 25fps | Link |
| VFHQ | 16,000+ high-fidelity clips for video face super-resolution | Link |
| Faceforensics++ | Deepfake detection benchmark dataset | Link |
| ObamaSet | Specialized dataset focused on Barack Obama's visual speech | Link |
| CN-CVS | Large-scale Mandarin Chinese visual-speech dataset | Link |
| DH-FaceVid-1K | 1,200 hours, 270,043 clips from 20,000+ individuals with multi-ethnic coverage | Link |
| Dataset | Description | Download |
|---|---|---|
| MMFace4D | 35,000+ sequences from 431 subjects for audio-driven 3D facial animation | Link |
| VOCA | 4D-face dataset with ~29 minutes of 4D face scans and synchronized audio | Link |
| Multiface | 65TB high-quality multi-view recordings from 13 people with various expressions | Link |
| BIWI | 3D Audiovisual Corpus of Affective Communication | Link |
| Name | Title | Venue | Links |
|---|---|---|---|
| OmniHuman-1 | Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models | ICCV 2025 | Paper Project |
| MoDA | Multi-modal Diffusion Architecture for Talking Head Generation | arXiv | Paper Project |
| DiTalker | A Unified DiT-based Framework for High-Quality and Speaking Styles Controllable Portrait Animation | arXiv | Paper Project |
| MVPortrait | Text-Guided Motion and Emotion Control for Multi-view Vivid Portrait Animation | arXiv | Paper |
| ACTalker | Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modelling | arXiv | Paper Project |
| OmniAvatar | Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animation | arXiv | Paper Code |
| EchoMimicV2 | Towards Striking, Simplified, and Semi-Body Human Animation | CVPR 2025 | Paper Code |
| Sonic | Shifting Focus to Global Audio Perception in Portrait Animation | CVPR 2025 | Paper |
| HunyuanPortrait | Implicit Condition Control for Enhanced Portrait Animation | CVPR 2025 | Paper Code |
| Teller | Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generation | CVPR 2025 | Paper |
| Loopy | Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency | ICLR 2025 (Oral) | Paper Project |
| FADA | Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation | CVPR 2025 | Paper |
| MobilePortrait | Real-Time One-Shot Neural Head Avatars on Mobile Devices | CVPR 2025 | Paper |
| KeyFace | Expressive Audio-Driven Facial Animation for Long Sequences via KeyFrame Interpolation | CVPR 2025 | Paper |
| MuseTalk | Real-Time High-Fidelity Video Dubbing via Spatio-Temporal Sampling | arXiv | Paper Code |
| LatentSync | Audio Conditioned Latent Diffusion Models for Lip Sync | arXiv | Paper Code |
| InsTaG | Learning Personalized 3D Talking Head from Few-Second Video | CVPR 2025 | Paper |
| Name | Title | Venue | Links |
|---|---|---|---|
| EMO | Emote Portrait Alive - Generating Expressive Portrait Videos with Audio2Video Diffusion Model | arXiv | Paper Project Code |
| VASA-1 | Lifelike Audio-Driven Talking Faces Generated in Real Time | NeurIPS 2024 (Oral) | Paper Project |
| Real3D-Portrait | One-shot Realistic 3D Talking Portrait Synthesis | ICLR 2024 | Paper Project Code |
| Hallo | Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation | arXiv | Paper Code |
| Hallo2 | Long-Duration and High-Resolution Audio-Driven Portrait Image Animation | ICLR 2025 | Paper |
| EchoMimic | Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions | AAAI 2025 | Paper Code |
| AniPortrait | Audio-Driven Synthesis of Photorealistic Portrait Animations | arXiv | Paper Code |
| EDTalk | Efficient Disentanglement for Emotional Talking Head Synthesis | ECCV 2024 (Oral) | Paper Code |
| V-Express | Conditional Dropout for Progressive Training of Portrait Video Generation | Tech Report | Paper |
| SadTalker | Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Talking Head Animation | CVPR 2023 | Paper Code |
| LivePortrait | Efficient Portrait Animation with Stitching and Retargeting Control | arXiv | Paper Code |
| MEMO | Memory-Guided Diffusion for Expressive Talking Video Generation | arXiv | Paper Code |
| FLOAT | Generative Motion Latent Flow Matching for Audio-driven Talking Portrait | ICCV 2025 | Paper Project |
| LetsTalk | Latent Diffusion Transformer for Talking Video Synthesis | arXiv | Paper Code |
| DAWN | Dynamic Frame Avatar with Non-autoregressive Diffusion Framework | arXiv | Paper Code |
| JoyVASA | Portrait and Animal Image Animation with Diffusion-Based Audio-Driven Generation | arXiv | Paper Code |
| VQTalker | Multilingual Talking Avatars through Facial Motion Tokenization | arXiv | Paper |
| PortraitTalk | Towards Customizable One-Shot Audio-to-Talking Face Generation | arXiv | Paper |
| IF-MDM | Implicit Face Motion Diffusion Model for High-Fidelity Realtime Talking Head Generation | arXiv | Paper |
| INFP | Audio-Driven Interactive Head Generation in Dyadic Conversations | arXiv | Paper Project |
| Hallo3 | Highly Dynamic and Realistic Portrait Image Animation with Diffusion Transformer Networks | arXiv | Paper |
| Ditto | Motion-Space Diffusion for Controllable Realtime Talking Head Synthesis | arXiv | Paper |
| EmotiveTalk | Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusion | arXiv | Paper |
| LES-Talker | Fine-Grained Emotion Editing for Talking Head Generation in Linear Emotion Space | arXiv | Paper Project |
| HelloMeme | Integrating Spatial Knitting Attentions to Embed High-Level and Fidelity-Rich Conditions in Diffusion Models | arXiv | Paper Code |
| Takin-ADA | Emotion Controllable Audio-Driven Animation with Canonical and Landmark Loss Optimization | arXiv | Paper |
| LaDTalk | Latent Denoising for Synthesizing Talking Head Videos with High Frequency Details | arXiv | Paper |
| TalkinNeRF | Animatable Neural Fields for Full-Body Talking Humans | ECCVW 2024 | Paper |
| JoyHallo | Digital Human Model for Mandarin | arXiv | Paper Code |
| JEAN | Joint Expression and Audio-guided NeRF-based Talking Face Generation | BMVC 2024 | Paper |
| StyleTalk++ | A Unified Framework for Controlling the Speaking Styles of Talking Heads | TPAMI 2024 | Paper |
| DiffTED | One-shot Audio-driven TED Talk Video Generation with Diffusion-based Co-speech Gestures | CVPRW 2024 | Paper |
| EMOdiffhead | Continuously Emotional Control in Talking Head Generation via Diffusion | arXiv | Paper |
| SVP | Style-Enhanced Vivid Portrait Talking Head Diffusion Model | arXiv | Paper |
| SegTalker | Segmentation-based Talking Face Generation with Mask-guided Local Editing | arXiv | Paper |
| PoseTalk | Text-and-Audio-based Pose Control and Motion Refinement for One-Shot Talking Head Generation | arXiv | Paper |
| CyberHost | Taming Audio-driven Avatar Diffusion Model with Region Codebook Attention | arXiv | Paper Project |
| TalkLoRA | Low-Rank Adaptation for Speech-Driven Animation | arXiv | Paper |
| S3D-NeRF | Single-Shot Speech-Driven Neural Radiance Field for High Fidelity Talking Head Synthesis | arXiv | Paper |
| FD2Talk | Towards Generalized Talking Head Generation with Facial Decoupled Diffusion Model | arXiv | Paper |
| LinguaLinker | Audio-Driven Portraits Animation with Implicit Facial Control Enhancement | arXiv | Paper Code |
| RealTalk | Real-time and Realistic Audio-driven Face Generation with 3D Facial Prior-guided Identity Alignment Network | arXiv | Paper |
| Let's Go Real Talk | Spoken Dialogue Model for Face-to-Face Conversation | arXiv | Paper Project |
| MyTalk | Make Your Actor Talk: Generalizable and High-Fidelity Lip Sync with Motion and Appearance Disentanglement | arXiv | Paper Project |
| Faces that Speak | Jointly Synthesising Talking Face and Speech from Text | arXiv | Paper Project |
| SwapTalk | Audio-Driven Talking Face Generation with One-Shot Customization in Latent Space | arXiv | Paper Project |
| AniTalker | Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding | arXiv | Paper Code |
| NeRFFaceSpeech | One-shot Audio-driven 3D Talking Head Synthesis via Generative Prior | CVPRW 2024 | Paper Code |
| THQA | A Perceptual Quality Assessment Database for Talking Heads | arXiv | Paper Code |
| Talk3D | High-Fidelity Talking Portrait Synthesis via Personalized 3D Generative Prior | arXiv | Paper Code |
| MoDiTalker | Motion-Disentangled Diffusion Model for High-Fidelity Talking Head Generation | arXiv | Paper Project |
| Style2Talker | High-Resolution Talking Head Generation with Emotion Style and Art Style | arXiv | Paper |
| FlowVQTalker | High-Quality Emotional Talking Face Generation through Normalizing Flow and Quantization | arXiv | Paper |
| FaceChain-ImagineID | Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled Audio | arXiv | Paper Code |
| DynTet | Learning Dynamic Tetrahedra for High-Quality Talking Head Synthesis | CVPR 2024 | Paper Code |
| G4G | A Generic Framework for High Fidelity Talking Face Generation with Fine-grained Intra-modal Alignment | arXiv | Paper |
| EmoSpeaker | One-shot Fine-grained Emotion-Controlled Talking Face Generation | arXiv | Paper Code |
| GPAvatar | Generalizable and Precise Head Avatar from Image(s) | ICLR 2024 | Paper Code |
| EmoTalker | Emotionally Editable Talking Face Generation via Diffusion Model | ICASSP 2024 | Paper |
| CVTHead | One-shot Controllable Head Avatar with Vertex-feature Transformer | WACV 2024 | Paper Code |
| Name | Title | Venue | Links |
|---|---|---|---|
| DiffTalk | Crafting Diffusion Models for Generalized Talking Head Synthesis | CVPR 2023 | Paper Code |
| Diffused Heads | Diffusion Models Beat GANs on Talking-Face Generation | arXiv | Paper Project |
| GeneFace++ | Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation | arXiv | Paper Code |
| VividTalk | One-Shot Audio-Driven Talking Head Generation Based on 3D Hybrid Prior | arXiv | Paper Code |
| StyleSync | High-Fidelity Generalized and Personalized Lip Sync in Style-based Generator | CVPR 2023 | Paper Code |
| TalkLip | Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert | CVPR 2023 | Paper Code |
| IP_LAP | Identity-Preserving Talking Face Generation with Landmark and Appearance Priors | CVPR 2023 | Paper Code |
| EAT | Efficient Emotional Adaptation for Audio-Driven Talking-Head Generation | ICCV 2023 | Paper Code |
| ER-NeRF | Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis | ICCV 2023 | Paper Code |
| VectorTalker | SVG Talking Face Generation with Progressive Vectorisation | arXiv | Paper |
| DreamTalk | When Expressive Talking Head Generation Meets Diffusion Probabilistic Models | arXiv | Paper Project |
| GMTalker | Gaussian Mixture based Emotional talking video Portraits | arXiv | Paper Project |
| DiT-Head | High-Resolution Talking Head Synthesis using Diffusion Transformers | arXiv | Paper |
| R2-Talker | Realistic Real-Time Talking Head Synthesis with Hash Grid Landmarks Encoding | arXiv | Paper |
| FT2TF | First-Person Statement Text-To-Talking Face Generation | arXiv | Paper |
| GAIA | Zero-shot Talking Avatar Generation | arXiv | Paper |
| MODA | Mapping-Once Audio-driven Portrait Animation with Dual Attentions | ICCV 2023 | Paper Project |
| ToonTalker | Cross-Domain Face Reenactment | ICCV 2023 | Paper |
| EMMN | Emotional Motion Memory Network for Audio-driven Emotional Talking Face Generation | ICCV 2023 | Paper |
| MetaPortrait | Identity-Preserving Talking Head Generation with Fast Personalized Adaptation | CVPR 2023 | Paper Code |
| LipFormer | High-fidelity and Generalizable Talking Face Generation with A Pre-learned Facial Codebook | CVPR 2023 | Paper |
| GANHead | Towards Generative Animatable Neural Head Avatars | CVPR 2023 | Paper Code |
| OTAvatar | One-shot Talking Face Avatar with Controllable Tri-plane Rendering | CVPR 2023 | Paper Code |
| HiDe-NeRF | One-Shot High-Fidelity Talking-Head Synthesis with Deformable Neural Radiance Field | CVPR 2023 | Paper Project |
| DAE-Talker | High Fidelity Speech-Driven Talking Face Generation with Diffusion Autoencoder | arXiv | Paper Project |
| DisCoHead | Audio-and-Video-Driven Talking Head Generation by Disentangled Control of Head Pose and Facial Expressions | ICASSP 2023 | Paper Code |
| StyleTalk | One-shot Talking Head Generation with Controllable Speaking Styles | AAAI 2023 | Paper Code |
| Name | Title | Venue | Links |
|---|---|---|---|
| VideoReTalking | Audio-based Lip Synchronization for Talking Head Video Editing In the Wild | SIGGRAPH 2022 | Paper |
| EAMM | One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model | SIGGRAPH 2022 | Paper |
| RAD-NeRF | Real-time Neural Talking Portrait Synthesis via Audio-spatial Decomposition | arXiv | Paper Code |
| SPACEx | Speech-driven Portrait Animation with Controllable Expression | arXiv | Paper Project |
| GC-AVT | Expressive Talking Head Generation with Granular Audio-Visual Control | CVPR 2022 | Paper |
| Name | Title | Venue | Links |
|---|---|---|---|
| PC-AVS | Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation | CVPR 2021 | Paper Code |
| AD-NeRF | Audio Driven Neural Radiance Fields for Talking Head Synthesis | ICCV 2021 | Paper Code |
| Wav2Lip | A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild | ACM MM 2020 | Paper Code |
| MakeItTalk | Speaker-Aware Talking-Head Animation | SIGGRAPH Asia 2020 | Paper Code |
| Neural Voice Puppetry | Audio-driven Facial Reenactment | ECCV 2020 | Paper Project |
| DAVS | Talking Face Generation by Adversarially Disentangled Audio-Visual Representation | AAAI 2019 | Paper Code |
| ATVGnet | Hierarchical Cross-modal Talking Face Generation with Dynamic Pixel-wise Loss | CVPR 2019 | Paper Code |
| Synthesizing Obama | Learning Lip Sync From Audio | SIGGRAPH 2017 | Paper Project |
| Name | Title | Venue | Links |
|---|---|---|---|
| HunyuanPortrait | Implicit Condition Control for Enhanced Portrait Animation | CVPR 2025 | Paper Code |
| Name | Title | Venue | Links |
|---|---|---|---|
| X-Portrait | Expressive Portrait Animation with Hierarchical Motion Attention | arXiv | Paper |
| LivePortrait | Efficient Portrait Animation with Stitching and Retargeting Control | arXiv | Paper Code |
| EMOPortraits | Emotion-enhanced Multimodal One-shot Head Avatars | CVPR 2024 | Paper Code |
| SMA | Synergizing Motion and Appearance: Multi-Scale Compensatory Codebooks | CVPR 2024 | Paper |
| Name | Title | Venue | Links |
|---|---|---|---|
| AVFR-GAN | Audio-Visual Face Reenactment | WACV 2023 | Paper Code |
| TS-Net | Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis | WACV 2023 | Paper Code |
| MCNET | Implicit Identity Representation Conditioned Memory Compensation Network | ICCV 2023 | Paper Code |
| Name | Title | Venue | Links |
|---|---|---|---|
| DaGAN | Depth-Aware Generative Adversarial Network for Talking Head Video Generation | CVPR 2022 | Paper Code |
| TPSM | Thin-Plate Spline Motion Model for Image Animation | CVPR 2022 | Paper Code |
| StyleHEAT | One-Shot High-Resolution Editable Talking Face Generation via Pretrained StyleGAN | ECCV 2022 | Paper Code |
| MegaPortraits | One-shot Megapixel Neural Head Avatars | ACM MM 2022 | Paper Project |
| LIA | Latent Image Animator: Learning to Animate Images via Latent Space Navigation | ICLR 2022 | Paper Code |
| Name | Title | Venue | Links |
|---|---|---|---|
| face-vid2vid | One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing | CVPR 2021 Oral | Paper Project |
| PIRenderer | Controllable Portrait Image Generation via Semantic Neural Rendering | ICCV 2021 | Paper Code |
| HeadGAN | One-shot Neural Head Synthesis and Editing | ICCV 2021 | Paper Project |
| FOMM | First Order Motion Model for Image Animation | NeurIPS 2019 | Paper Code |
| fs-vid2vid | Few-shot Video-to-Video Synthesis | NeurIPS 2019 | Paper Code |
| X2Face | A network for controlling face generation by using images, audio, and pose codes | ECCV 2018 | Paper Code |
| Face2Face | Real-time face capture and reenactment of RGB videos | CVPR 2016 | Paper |
| Year | Name | Title | Venue | Links |
|---|---|---|---|---|
| 2025 | When Words Smile | Generating Diverse Emotional Facial Expressions from Text | EMNLP 2025 | Paper Code |
| 2024 | HeadStudio | Text to Animatable Head Avatars with 3D Gaussian Splatting | ECCV 2024 | Paper Code |
| 2024 | T3M | Text Guided 3D Human Motion Synthesis from Speech | arXiv | Paper Code |
| 2024 | STAR | Skeleton-aware Text-based 4D Avatar Generation with In-Network Motion Retargeting | Tech Report | Paper Project |
| 2024 | InstructAvatar | Text-Guided Emotion and Motion Control for Avatar Generation | arXiv | Paper Project |
| 2023 | TalkCLIP | Talking Head Generation with Text-Guided Expressive Speaking Styles | arXiv | Paper |
| 2022 | Text2Video | Text-driven Talking-head Video Synthesis with Phonetic Dictionary | ICASSP 2022 | Paper Code |
| 2021 | Write-a-speaker | Text-based Emotional and Rhythmic Talking-head Generation | AAAI 2021 | Paper Code |
| Year | Name | Title | Venue | Links |
|---|---|---|---|---|
| 2025 | TaoAvatar | Real-Time Lifelike Full-Body Talking Avatars for Augmented Reality | CVPR 2025 (Highlight) | Paper Project |
| 2025 | LAM | Large Avatar Model for One-shot Animatable Gaussian Head | arXiv | Paper Code |
| 2025 | Avat3r | Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatars | arXiv | Paper Project |
| 2024 | GaussianAvatars | Photorealistic Head Avatars with Rigged 3D Gaussians | CVPR 2024 | Paper Code |
| 2024 | GaussianTalker | Real-Time High-Fidelity Talking Head Synthesis with Audio-Driven 3D Gaussian Splatting | ACM MM 2024 | Paper |
| 2024 | TalkingGaussian | Structure-Persistent 3D Talking Head Synthesis via Gaussian Splatting | ECCV 2024 | Paper |
| 2024 | GSTalker | Real-time Audio-Driven Talking Face Generation via Deformable Gaussian Splatting | arXiv | Paper |
| 2024 | HeadGaS | Real-Time Animatable Head Avatars via 3D Gaussian Splatting | arXiv | Paper |
| 2024 | 3DGS-Avatar | Animatable Avatars via Deformable 3D Gaussian Splatting | arXiv | Paper Code |
| 2024 | SplattingAvatar | Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting | CVPR 2024 | Paper Code |
| 2024 | MeGA | Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing | arXiv | Paper Code |
| Year | Name | Title | Venue | Links |
|---|---|---|---|---|
| 2024 | SyncTalk | The Devil is in the Synchronization for Talking Head Synthesis | CVPR 2024 | Paper Code |
| 2023 | ER-NeRF | Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis | ICCV 2023 | Paper Code |
| 2023 | GeneFace | Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis | ICLR 2023 | Paper Code |
| 2022 | RAD-NeRF | Real-time Neural Talking Portrait Synthesis via Audio-spatial Decomposition | arXiv | Paper Code |
| 2022 | DFRF | Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis | ECCV 2022 | Paper Code |
| 2022 | HeadNeRF | A Real-time NeRF-based Parametric Head Model | CVPR 2022 | Paper Code |
| 2021 | AD-NeRF | Audio Driven Neural Radiance Fields for Talking Head Synthesis | ICCV 2021 | Paper Code |
| 2021 | NerFACE | Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction | CVPR 2021 Oral | Paper Code |
| Year | Name | Title | Venue | Links |
|---|---|---|---|---|
| 2024 | FaceTalk | Audio-Driven Motion Diffusion for Neural Parametric Head Models | CVPR 2024 | Paper Code |
| 2024 | DiffSpeaker | Speech-Driven 3D Facial Animation with Diffusion Transformer | arXiv | Paper Code |
| 2023 | CodeTalker | Speech-Driven 3D Facial Animation with Discrete Motion Prior | CVPR 2023 | Paper Project |
| 2023 | EmoTalk | Speech-Driven Emotional Disentanglement for 3D Face Animation | ICCV 2023 | Paper Project |
| 2022 | FaceFormer | Speech-Driven 3D Facial Animation with Transformers | CVPR 2022 | Paper Code |
| 2021 | MeshTalk | 3D Face Animation from Speech using Cross-Modality Disentanglement | ICCV 2021 | Paper |
| 2024 | MimicTalk | Mimicking a Personalized and Expressive 3D Talking Face in Few Minutes | NeurIPS 2024 | Paper Code |
| 2024 | ScanTalk | 3D Talking Heads from Unregistered Scans | ECCV 2024 | Paper Code |
| 2024 | 3DFacePolicy | Speech-Driven 3D Facial Animation with Diffusion Policy | arXiv | Paper |
| 2024 | ProbTalk3D | Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAE | arXiv | Paper Code |
| 2024 | KMTalk | Speech-Driven 3D Facial Animation with Key Motion Embedding | ECCV 2024 | Paper Code |
| 2024 | EmoFace | Emotion-Content Disentangled Speech-Driven 3D Talking Face with Mesh Attention | arXiv | Paper |
| 2024 | DEEPTalk | Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation | arXiv | Paper |
| 2024 | JambaTalk | Speech-Driven 3D Talking Head Generation Based on Hybrid Transformer-Mamba Model | arXiv | Paper |
| 2024 | GLDiTalker | Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer | arXiv | Paper |
| 2024 | UniTalker | Scaling up Audio-Driven 3D Facial Animation through A Unified Model | arXiv | Paper |
| 2024 | EmoTalk3D | High-Fidelity Free-View Synthesis of Emotional 3D Talking Head | arXiv | Paper |
| 2024 | MultiTalk | Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Dataset | InterSpeech 2024 | Paper Project |
| 2024 | 3D Gaussian Blendshapes | 3D Gaussian Blendshapes for Head Avatar Animation | SIGGRAPH 2024 | Paper |
| 2024 | CSTalk | Correlation Supervised Speech-driven 3D Emotional Facial Animation Generation | arXiv | Paper |
| 2024 | Learn2Talk | 3D Talking Face Learns from 2D Talking Face | arXiv | Paper Project |
| 2024 | AnimateMe | 4D Facial Expressions via Diffusion Models | arXiv | Paper |
| 2024 | EmoVOCA | Speech-Driven Emotional 3D Talking Heads | arXiv | Paper |
| 2024 | AVI-Talking | Learning Audio-Visual Instructions for Expressive 3D Talking Face Generation | arXiv | Paper |
| 2024 | Media2Face | Co-speech Facial Animation Generation With Multi-Modality Guidance | arXiv | Paper Project |
| 2024 | One Shot One Talk | Whole-body Talking Avatar from a Single Image | arXiv | Paper Project |
| 2024 | Stereo-Talker | Audio-driven 3D Human Synthesis with Prior-Guided Mixture-of-Experts | arXiv | Paper |
| 2023 | EMOTE | Emotional Speech-Driven Animation with Content-Emotion Disentanglement | SIGGRAPH Asia 2023 | Paper Project |
| 2023 | PMMTalk | Speech-Driven 3D Facial Animation from Complementary Pseudo Multi-modal Features | arXiv | Paper |
| 2023 | 3DiFACE | Diffusion-based Speech-driven 3D Facial Animation and Editing | arXiv | Paper Code |
| 2023 | DiffusionTalker | Personalization and Acceleration for Speech-Driven 3D Face Diffuser | arXiv | Paper |
| 2023 | DiffPoseTalk | Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models | arXiv | Paper Code |
| 2023 | Imitator | Personalized Speech-driven 3D Facial Animation | ICCV 2023 | Paper Code |
| 2023 | Speech4Mesh | Speech-Assisted Monocular 3D Facial Reconstruction for Speech-Driven 3D Facial Animation | ICCV 2023 | Paper |
| 2023 | FaceXHuBERT | Text-less Speech-driven Expressive 3D Facial Animation Synthesis Using Self-Supervised Speech Representation Learning | arXiv | Paper Code |
| Metric | Description | Paper |
|---|---|---|
| PSNR | Peak Signal-to-Noise Ratio - measures reconstruction quality | - |
| SSIM | Structural Similarity Index Measure - evaluates structural similarity | Image quality assessment: from error visibility to structural similarity |
| LPIPS | Learned Perceptual Image Patch Similarity - perceptual similarity metric | The Unreasonable Effectiveness of Deep Features as a Perceptual Metric |
| FID | Fréchet Inception Distance - measures quality and diversity of generated images | - |
| LMD | Landmark Distance - measures facial landmark accuracy | - |
| Sync | Audio-Visual Sync Score - evaluates lip-sync quality | - |
| CPBD | Cumulative Probability of Blur Detection - measures image sharpness | A no-reference image blur metric based on the cumulative probability of blur detection |
| NIQE | Natural Image Quality Evaluator - blind image quality assessment | Making a 'Completely Blind' Image Quality Analyzer |
| AV Offset | Audio-Visual Offset - measures temporal alignment | - |
| FVD | Fréchet Video Distance - measures video generation quality | - |
| Tool | Description | Link |
|---|---|---|
| Wav2Lip | Accurate lip-sync from audio | GitHub |
| SadTalker | Stylized audio-driven talking head from single image | GitHub |
| LivePortrait | Efficient portrait animation | GitHub |
| MuseTalk | Real-time high-quality lip synchronization | GitHub |
| Hallo | Audio-driven visual synthesis | GitHub |
| EchoMimic | Audio-driven portrait animations with editable landmarks | GitHub |
| AniPortrait | Audio-driven photorealistic portrait animations | GitHub |
| Platform | Description |
|---|---|
| D-ID | AI video creation platform with talking avatar generation |
| HeyGen | AI-powered video generation with avatars |
| Synthesia | AI video generation platform |
| Colossyan | AI video creator with diverse AI avatars |
| Resource | Description | Link |
|---|---|---|
| Hugging Face Trending Papers | Search for latest talking head papers on HF | Link |
| arXiv cs.CV | Latest computer vision papers | Link |
| Group | Affiliation | Focus |
|---|---|---|
| MMLab@NTU | Nanyang Technological University | Computer Vision, Deep Learning |
| Alibaba TongYi XR | Alibaba | Human-AI Interaction, Avatar Generation |
| NVIDIA Research | NVIDIA | Neural Rendering, 3D Generation |
| Microsoft Research | Microsoft | VASA-1, AI Avatars |
| Meta AI Research | Meta | Codec Avatars, VR/AR |
- MoDA (Jul 2025) - Multi-modal Diffusion Architecture with flow matching for efficient talking head generation
- DiTalker (Jul 2025) - Unified DiT-based framework for speaking style controllable portrait animation
- MVPortrait (Mar 2025) - First text-guided multi-view portrait animation with FLAME representation
- ACTalker (Apr 2025) - Audio-visual controlled video diffusion with Mamba architecture
- VASA-1 (Microsoft, 2024) - Real-time lifelike talking faces with unprecedented quality
- EMO (Alibaba, 2024) - Emote Portrait Alive with weak condition diffusion
- OmniHuman-1 (ByteDance, 2025) - Scalable one-stage human animation
- LivePortrait (Kuaishou, 2024) - Efficient stitching and retargeting control
- FOMM (2019) - First Order Motion Model, seminal work in image animation
- Wav2Lip (2020) - Expert lip-sync that remains widely used
- AD-NeRF (2021) - Pioneer in NeRF-based talking heads
- SadTalker (2023) - Accessible 3D-aware audio-driven generation
- GaussianAvatars (2024) - Rigged 3D Gaussians for head avatars
- ER-NeRF (2023) - Efficient region-aware neural rendering
- TaoAvatar (2025) - Real-time AR avatars with 3DGS
- 🔥 Diffusion Models - EMO, VASA-1, Hallo series
- 🔥 3D Gaussian Splatting - Real-time rendering, animatable avatars
- 🔥 Long-form Generation - Extended video synthesis with temporal consistency
- 🔥 Emotional Control - Fine-grained emotion and expression control
- 🔥 Real-time Synthesis - Mobile deployment, VR/AR applications
- Multi-character interaction and conversation
- Full-body talking avatar generation
- Cross-lingual and multilingual synthesis
- Privacy-preserving avatar generation
- Hybrid 2D-3D representations
If you find this resource helpful, please consider citing:
@misc{awesome-talking-head,
title={Awesome Talking Head Generation},
author={Community Contributors},
year={2025},
howpublished={\url{https://github.com/awesome-talking-head}}
}This project is licensed under the MIT License.
⭐ If you find this repository useful, please give it a star!
🤝 Contributions are welcome! Feel free to submit PRs or open issues.