Light Field Neural Rendering

Classical light field rendering for novel view synthesis can accurately reproduce view-dependent effects such as reflection, refraction, and translucency, but requires a dense view sampling of the scene. Methods based on geometric reconstruction need only sparse views, but cannot accurately model no...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - PP(2023) vom: 27. Sept.
1. Verfasser: Suhail, Mohammed (VerfasserIn)
Weitere Verfasser: Esteves, Carlos, Sigal, Leonid, Makadia, Ameesh
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2023
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article
LEADER 01000caa a22002652 4500
001 NLM362524815
003 DE-627
005 20240212231925.0
007 cr uuu---uuuuu
008 231226s2023 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2023.3316992  |2 doi 
028 5 2 |a pubmed24n1289.xml 
035 |a (DE-627)NLM362524815 
035 |a (NLM)37756168 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Suhail, Mohammed  |e verfasserin  |4 aut 
245 1 0 |a Light Field Neural Rendering 
264 1 |c 2023 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 12.02.2024 
500 |a published: Print-Electronic 
500 |a Citation Status Publisher 
520 |a Classical light field rendering for novel view synthesis can accurately reproduce view-dependent effects such as reflection, refraction, and translucency, but requires a dense view sampling of the scene. Methods based on geometric reconstruction need only sparse views, but cannot accurately model non-Lambertian effects. We introduce a model that combines the strengths and mitigates the limitations of these two directions. By operating on a four-dimensional representation of the light field, our model learns to represent view-dependent effects accurately. By enforcing geometric constraints during training and inference, the scene geometry is implicitly learned from a sparse set of views. Concretely, we introduce a two-stage transformer-based model that first aggregates features along epipolar lines, then aggregates features along reference views to produce the color of a target ray. Additionally, we propose modifications that allow the model to generalize to scenes without any fine-tuning. Our model outperforms the state-of-the-art on multiple forward-facing and 360 ° datasets, with larger margins on scenes with severe view-dependent variations. Code and results can be found at https://light-field-neural-rendering.github.io/ 
650 4 |a Journal Article 
700 1 |a Esteves, Carlos  |e verfasserin  |4 aut 
700 1 |a Sigal, Leonid  |e verfasserin  |4 aut 
700 1 |a Makadia, Ameesh  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g PP(2023) vom: 27. Sept.  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnns 
773 1 8 |g volume:PP  |g year:2023  |g day:27  |g month:09 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2023.3316992  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d PP  |j 2023  |b 27  |c 09