MagicTalk: Implicit and Explicit Correlation Learning for Diffusion-based Emotional Talking Face Generation

MagicTalk: Implicit and Explicit Correlation Learning for Diffusion-based

Emotional Talking Face Generation

Chenxu Zhang^1*,

Chao Wang^1*,

Jianfeng Zhang¹,

Hongyi Xu¹,

Guoxian Song¹,

You Xie¹,

Linjie Luo¹,

Yapeng Tian²,

Jiashi Feng¹,

Xiaohu Guo²,

¹Bytedance Inc. ²The University of Texas at Dallas

Video Results

▶ Qualitative Comparisons with MakeitTalk, SadTalker, EAMM and PD-FGC

▶ Qualitative Comparisons with EVP

Applications

▶ More Characters

Results with Real Human Faces and AIGC-generated Faces.

▶ Multiple Emotions

Results with Various Emotions, Such as Anger, Happy, and Surprise.

▶ A Conversation Across Time and Space

Leonardo predominantly expresses anger, while Mona Lisa exhibits happiness.

▶ Generation of Different Languages

Multiple Language Support for Emotional Talking Faces Generation, Including Chinese, Japanese, French, German, etc.

BibTeX

@inproceedings{magictalk2025,
    title = {MagicTalk: Implicit and Explicit Correlation Learning for Diffusion-based Emotional Talking Face Generation},
    authors = {Zhang, Chenxu and Wang, Chao and Zhang, Jianfeng and Xu, Hongyi and Song, Guoxian and Xie, You and Luo, Linjie and Tian, Yapeng and Feng, Jiashi and Guo, Xiaohu },
    year={2025}
}