Relightable Detailed Human Reconstruction from Sparse Flashlight Images

We present a lightweight system for reconstructing human geometry and appearance from sparse flashlight images. Our system produces detailed geometry including garment wrinkles and surface reflectance, which are exportable for direct rendering and relighting in traditional graphics pipelines. By cap...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on visualization and computer graphics. - 1996. - PP(2024) vom: 09. Sept.
1. Verfasser: Lu, Jiawei (VerfasserIn)
Weitere Verfasser: Shao, Tianjia, Wang, He, Yang, Yong-Liang, Yang, Yin, Zhou, Kun
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2024
Zugriff auf das übergeordnete Werk:IEEE transactions on visualization and computer graphics
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM377374822
003 DE-627
005 20240910233826.0
007 cr uuu---uuuuu
008 240910s2024 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2024.3450591  |2 doi 
028 5 2 |a pubmed24n1529.xml 
035 |a (DE-627)NLM377374822 
035 |a (NLM)39250391 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Lu, Jiawei  |e verfasserin  |4 aut 
245 1 0 |a Relightable Detailed Human Reconstruction from Sparse Flashlight Images 
264 1 |c 2024 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 10.09.2024 
500 |a published: Print-Electronic 
500 |a Citation Status Publisher 
520 |a We present a lightweight system for reconstructing human geometry and appearance from sparse flashlight images. Our system produces detailed geometry including garment wrinkles and surface reflectance, which are exportable for direct rendering and relighting in traditional graphics pipelines. By capturing multi-view flashlight images using a consumer camera equipped with an co-located LED (e.g., a cell phone), we obtain view-specific shading cues that aid in the determination of surface orientation and help disambiguate between shading and material. To enable the reconstruction of geometry and appearance from sparse-view flashlight images, we integrate a pre-trained model into a differentiable physics-based rendering framework. As the learned image features from synthetic data cannot accurately reflect the shading features on real images, which is crucial for the high-quality reconstruction of geometry details and appearance, we propose to jointly optimize the image feature extractor with two MLPs for SDF and BRDF prediction using the differentiable physics-based rendering. Compared with existing methods for relightable human reconstruction, our system is able to produce high-fidelity 3D human models with more accurate geometry and appearance under the same condition. Our code and data are available at http://github.com/Jarvisss/Relightable_human_recon 
650 4 |a Journal Article 
700 1 |a Shao, Tianjia  |e verfasserin  |4 aut 
700 1 |a Wang, He  |e verfasserin  |4 aut 
700 1 |a Yang, Yong-Liang  |e verfasserin  |4 aut 
700 1 |a Yang, Yin  |e verfasserin  |4 aut 
700 1 |a Zhou, Kun  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g PP(2024) vom: 09. Sept.  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnns 
773 1 8 |g volume:PP  |g year:2024  |g day:09  |g month:09 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2024.3450591  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d PP  |j 2024  |b 09  |c 09