Perceptually-Aligned Dynamic Facial Projection Mapping by High-Speed Face-Tracking Method and Lens-Shift Co-Axial Setup

Dynamic Facial Projection Mapping (DFPM) overlays computer-generated images onto human faces to create immersive experiences that have been used in the makeup and entertainment industries. In this study, we propose two concepts to reduce the misalignment artifacts between projected images and target...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on visualization and computer graphics. - 1996. - 31(2025), 10 vom: 17. Sept., Seite 6824-6838
1. Verfasser: Peng, Hao-Lun (VerfasserIn)
Weitere Verfasser: Sato, Kengo, Nakagawa, Soran, Watanabe, Yoshihiro
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2025
Zugriff auf das übergeordnete Werk:IEEE transactions on visualization and computer graphics
Schlagworte:Journal Article
LEADER 01000caa a22002652c 4500
001 NLM385075596
003 DE-627
005 20250906233455.0
007 cr uuu---uuuuu
008 250508s2025 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2025.3527203  |2 doi 
028 5 2 |a pubmed25n1558.xml 
035 |a (DE-627)NLM385075596 
035 |a (NLM)40031225 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Peng, Hao-Lun  |e verfasserin  |4 aut 
245 1 0 |a Perceptually-Aligned Dynamic Facial Projection Mapping by High-Speed Face-Tracking Method and Lens-Shift Co-Axial Setup 
264 1 |c 2025 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 05.09.2025 
500 |a published: Print 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Dynamic Facial Projection Mapping (DFPM) overlays computer-generated images onto human faces to create immersive experiences that have been used in the makeup and entertainment industries. In this study, we propose two concepts to reduce the misalignment artifacts between projected images and target faces, which is a persistent challenge for DFPM. Our first concept is a high-speed face-tracking method that exploits temporal information. We first introduce a cropped-area-limited inter/extrapolation-based face detection framework, which allows parallel execution with facial landmark detection. We then propose a novel hybrid facial landmark detection method that combines fast Ensemble of Regression Trees (ERT)-based detections and an auxiliary detection. ERT-based detections rapidly produce results in 0.107 ms using temporal information with the support of auxiliary detection to recover from detection errors. To train the facial landmark detection method, we propose an innovative method for simulating high-frame-rate video annotations to address the lack of publicly available high-frame-rate annotated datasets. Our second concept is a lens-shift co-axial projector-camera setup that maintains a high optical alignment with only a 1.274-pixel error between 1 m and 2 m depth. This setup reduces misalignment by applying the same optical designs to the projector and camera without causing large misalignment as in conventional methods. Based on these concepts, we developed a novel high-speed DFPM system that achieves nearly perfect alignment with human visual perception 
650 4 |a Journal Article 
700 1 |a Sato, Kengo  |e verfasserin  |4 aut 
700 1 |a Nakagawa, Soran  |e verfasserin  |4 aut 
700 1 |a Watanabe, Yoshihiro  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g 31(2025), 10 vom: 17. Sept., Seite 6824-6838  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnas 
773 1 8 |g volume:31  |g year:2025  |g number:10  |g day:17  |g month:09  |g pages:6824-6838 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2025.3527203  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 31  |j 2025  |e 10  |b 17  |c 09  |h 6824-6838