HDhuman : High-Quality Human Novel-View Rendering From Sparse Views

In this paper, we aim to address the challenge of novel view rendering of human performers that wear clothes with complex texture patterns using a sparse set of camera views. Although some recent works have achieved remarkable rendering quality on humans with relatively uniform textures using sparse...

Description complète

Détails bibliographiques
Publié dans:IEEE transactions on visualization and computer graphics. - 1996. - 30(2024), 8 vom: 12. Aug., Seite 5328-5338
Auteur principal: Zhou, Tiansong (Auteur)
Autres auteurs: Huang, Jing, Yu, Tao, Shao, Ruizhi, Li, Kun
Format: Article en ligne
Langue:English
Publié: 2024
Accès à la collection:IEEE transactions on visualization and computer graphics
Sujets:Journal Article Research Support, Non-U.S. Gov't
LEADER 01000caa a22002652c 4500
001 NLM358857597
003 DE-627
005 20250304233742.0
007 cr uuu---uuuuu
008 231226s2024 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2023.3290543  |2 doi 
028 5 2 |a pubmed25n1195.xml 
035 |a (DE-627)NLM358857597 
035 |a (NLM)37384477 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Zhou, Tiansong  |e verfasserin  |4 aut 
245 1 0 |a HDhuman  |b High-Quality Human Novel-View Rendering From Sparse Views 
264 1 |c 2024 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Completed 01.07.2024 
500 |a Date Revised 06.01.2025 
500 |a published: Print-Electronic 
500 |a Citation Status MEDLINE 
520 |a In this paper, we aim to address the challenge of novel view rendering of human performers that wear clothes with complex texture patterns using a sparse set of camera views. Although some recent works have achieved remarkable rendering quality on humans with relatively uniform textures using sparse views, the rendering quality remains limited when dealing with complex texture patterns as they are unable to recover the high-frequency geometry details that are observed in the input views. To this end, we propose HDhuman, which uses a human reconstruction network with a pixel-aligned spatial transformer and a rendering network with geometry-guided pixel-wise feature integration to achieve high-quality human reconstruction and rendering. The designed pixel-aligned spatial transformer calculates the correlations between the input views and generates human reconstruction results with high-frequency details. Based on the surface reconstruction results, the geometry-guided pixel-wise visibility reasoning provides guidance for multi-view feature integration, enabling the rendering network to render high-quality images at 2k resolution on novel views. Unlike previous neural rendering works that always need to train or fine-tune an independent network for a different scene, our method is a general framework that is able to generalize to novel subjects. Experiments show that our approach outperforms all the prior generic or specific methods on both synthetic data and real-world data. Source code and test data will be made publicly available for research purposes at http://cic.tju.edu.cn/faculty/likun/projects/HDhuman/index.html 
650 4 |a Journal Article 
650 4 |a Research Support, Non-U.S. Gov't 
700 1 |a Huang, Jing  |e verfasserin  |4 aut 
700 1 |a Yu, Tao  |e verfasserin  |4 aut 
700 1 |a Shao, Ruizhi  |e verfasserin  |4 aut 
700 1 |a Li, Kun  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g 30(2024), 8 vom: 12. Aug., Seite 5328-5338  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnas 
773 1 8 |g volume:30  |g year:2024  |g number:8  |g day:12  |g month:08  |g pages:5328-5338 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2023.3290543  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 30  |j 2024  |e 8  |b 12  |c 08  |h 5328-5338