Scene-aware Foveated Neural Radiance Fields

Foveated rendering provides an idea for improving the image synthesis performance of neural radiance fields (NeRF) methods. In this paper, we propose a scene-aware foveated neural radiance fields method to synthesize high-quality foveated images in complex VR scenes at high frame rates. Firstly, we...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on visualization and computer graphics. - 1996. - PP(2024) vom: 16. Juli
1. Verfasser: Shi, Xuehuai (VerfasserIn)
Weitere Verfasser: Wang, Lili, Liu, Xinda, Wu, Jian, Shao, Zhiwen
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2024
Zugriff auf das übergeordnete Werk:IEEE transactions on visualization and computer graphics
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM375004629
003 DE-627
005 20240717233359.0
007 cr uuu---uuuuu
008 240717s2024 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2024.3429416  |2 doi 
028 5 2 |a pubmed24n1473.xml 
035 |a (DE-627)NLM375004629 
035 |a (NLM)39012750 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Shi, Xuehuai  |e verfasserin  |4 aut 
245 1 0 |a Scene-aware Foveated Neural Radiance Fields 
264 1 |c 2024 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 17.07.2024 
500 |a published: Print-Electronic 
500 |a Citation Status Publisher 
520 |a Foveated rendering provides an idea for improving the image synthesis performance of neural radiance fields (NeRF) methods. In this paper, we propose a scene-aware foveated neural radiance fields method to synthesize high-quality foveated images in complex VR scenes at high frame rates. Firstly, we construct a multi-ellipsoidal neural representation to enhance the neural radiance field's representation capability in salient regions of complex VR scenes based on the scene content. Then, we introduce a uniform sampling based foveated neural radiance field framework to improve the foveated image synthesis performance with one-pass color inference, and improve the synthesis quality by leveraging the foveated scene-aware objective function. Our method synthesizes high-quality binocular foveated images at the average frame rate of 66 frames per second (FPS) in complex scenes with high occlusion, intricate textures, and sophisticated geometries. Compared with the state-of-the-art foveated NeRF method, our method achieves significantly higher synthesis quality in both the foveal and peripheral regions with 1.41-1.46× speedup. We also conduct a user study to prove that the perceived quality of our method has a high visual similarity with the ground truth 
650 4 |a Journal Article 
700 1 |a Wang, Lili  |e verfasserin  |4 aut 
700 1 |a Liu, Xinda  |e verfasserin  |4 aut 
700 1 |a Wu, Jian  |e verfasserin  |4 aut 
700 1 |a Shao, Zhiwen  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g PP(2024) vom: 16. Juli  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnns 
773 1 8 |g volume:PP  |g year:2024  |g day:16  |g month:07 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2024.3429416  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d PP  |j 2024  |b 16  |c 07