VPRF : Visual Perceptual Radiance Fields for Foveated Image Synthesis

Neural radiance fields (NeRF) has achieved revolutionary breakthrough in the novel view synthesis task for complex 3D scenes. However, this new paradigm struggles to meet the requirements for real-time rendering and high perceptual quality in virtual reality. In this paper, we propose VPRF, a novel...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on visualization and computer graphics. - 1996. - 30(2024), 11 vom: 08. Okt., Seite 7183-7192
1. Verfasser: Wang, Zijun (VerfasserIn)
Weitere Verfasser: Wu, Jian, Fan, Runze, Ke, Wei, Wang, Lili
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2024
Zugriff auf das übergeordnete Werk:IEEE transactions on visualization and computer graphics
Schlagworte:Journal Article
LEADER 01000caa a22002652 4500
001 NLM377466794
003 DE-627
005 20241011232355.0
007 cr uuu---uuuuu
008 240912s2024 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2024.3456184  |2 doi 
028 5 2 |a pubmed24n1564.xml 
035 |a (DE-627)NLM377466794 
035 |a (NLM)39259633 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Wang, Zijun  |e verfasserin  |4 aut 
245 1 0 |a VPRF  |b Visual Perceptual Radiance Fields for Foveated Image Synthesis 
264 1 |c 2024 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 10.10.2024 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Neural radiance fields (NeRF) has achieved revolutionary breakthrough in the novel view synthesis task for complex 3D scenes. However, this new paradigm struggles to meet the requirements for real-time rendering and high perceptual quality in virtual reality. In this paper, we propose VPRF, a novel visual perceptual based radiance fields representation method, which for the first time integrates the visual acuity and contrast sensitivity models of human visual system (HVS) into the radiance field rendering framework. Initially, we encode both the appearance and visual sensitivity information of the scene into our radiance field representation. Then, we propose a visual perceptual sampling strategy, allocating computational resources according to the HVS sensitivity of different regions. Finally, we propose a sampling weight-constrained training scheme to ensure the effectiveness of our sampling strategy and improve the representation of the radiance field based on the scene content. Experimental results demonstrate that our method renders more efficiently, with higher PSNR and SSIM in the foveal and salient regions compared to the state-of-the-art FoV-NeRF. The results of the user study confirm that our rendering results exhibit high-fidelity visual perception 
650 4 |a Journal Article 
700 1 |a Wu, Jian  |e verfasserin  |4 aut 
700 1 |a Fan, Runze  |e verfasserin  |4 aut 
700 1 |a Ke, Wei  |e verfasserin  |4 aut 
700 1 |a Wang, Lili  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g 30(2024), 11 vom: 08. Okt., Seite 7183-7192  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnns 
773 1 8 |g volume:30  |g year:2024  |g number:11  |g day:08  |g month:10  |g pages:7183-7192 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2024.3456184  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 30  |j 2024  |e 11  |b 08  |c 10  |h 7183-7192