HFM-GS : half-face mapping 3DGS avatar based real-time HMD removal
In extended reality (XR) applications, enhancing user perception often necessitates head-mounted display (HMD) removal. However, existing methods suffer from low time performance and suboptimal reconstruction quality. In this paper, we propose a half face mapping 3D Gaussian splatting avatar based H...
| Publié dans: | IEEE transactions on visualization and computer graphics. - 1996. - PP(2025) vom: 02. Okt. |
|---|---|
| Auteur principal: | |
| Autres auteurs: | , , , , |
| Format: | Article en ligne |
| Langue: | English |
| Publié: |
2025
|
| Accès à la collection: | IEEE transactions on visualization and computer graphics |
| Sujets: | Journal Article |
| Résumé: | In extended reality (XR) applications, enhancing user perception often necessitates head-mounted display (HMD) removal. However, existing methods suffer from low time performance and suboptimal reconstruction quality. In this paper, we propose a half face mapping 3D Gaussian splatting avatar based HMD removal method (HFM-GS), which can perform real-time and high-fidelity online restoration of the complete face in HMD-occluded videos for XR applications after a short un-occluded face registration. We establish a mapping field between the upper and lower face Gaussians to enhance the adaptability to deformation. Then, we introduce correlation weight-based sampling to improve time performance and handle variations in the number of Gaussians. At last, we ensure model robustness through Gaussian Segregation Strategy. Compared to two state-of-the-art methods, our method achieves better quality and time performance. The results of the user study show that fidelity is significantly improved with our method |
|---|---|
| Description: | Date Revised 02.10.2025 published: Print-Electronic Citation Status Publisher |
| ISSN: | 1941-0506 |
| DOI: | 10.1109/TVCG.2025.3616801 |