3D Talking Face With Personalized Pose Dynamics
Recently, we have witnessed a boom in applications for 3D talking face generation. However, most existing 3D face generation methods can only generate 3D faces with a static head pose, which is inconsistent with how humans perceive faces. Only a few articles focus on head pose generation, but even t...
| Veröffentlicht in: | IEEE transactions on visualization and computer graphics. - 1996. - 29(2023), 2 vom: 04. Feb., Seite 1438-1449 |
|---|---|
| 1. Verfasser: | |
| Weitere Verfasser: | , , , , , |
| Format: | Online-Aufsatz |
| Sprache: | English |
| Veröffentlicht: |
2023
|
| Zugriff auf das übergeordnete Werk: | IEEE transactions on visualization and computer graphics |
| Schlagworte: | Journal Article Research Support, U.S. Gov't, Non-P.H.S. Research Support, Non-U.S. Gov't |
| Zusammenfassung: | Recently, we have witnessed a boom in applications for 3D talking face generation. However, most existing 3D face generation methods can only generate 3D faces with a static head pose, which is inconsistent with how humans perceive faces. Only a few articles focus on head pose generation, but even these ignore the attribute of personality. In this article, we propose a unified audio-driven approach to endow 3D talking faces with personalized pose dynamics. To achieve this goal, we establish an original person-specific dataset, providing corresponding head poses and face shapes for each video. Our framework is composed of two separate modules: PoseGAN and PGFace. Given an input audio, PoseGAN first produces a head pose sequence for the 3D head, and then, PGFace utilizes the audio and pose information to generate natural face models. With the combination of these two parts, a 3D talking head with dynamic head movement can be constructed. Experimental evidence indicates that our method can generate person-specific head pose sequences that are in sync with the input audio and that best match with the human experience of talking heads |
|---|---|
| Beschreibung: | Date Completed 06.04.2023 Date Revised 03.05.2023 published: Print-Electronic Citation Status MEDLINE |
| ISSN: | 1941-0506 |
| DOI: | 10.1109/TVCG.2021.3117484 |