SLS4D : Sparse Latent Space for 4D Novel View Synthesis

Neural radiance fields (NeRF) have achieved great success in novel view synthesis and 3D representation for static scenarios. Existing dynamic NeRFs usually exploit a locally dense grid to fit the deformation fields; however, they fail to capture the global dynamics and concomitantly yield models of...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on visualization and computer graphics. - 1996. - PP(2024) vom: 16. Juli
1. Verfasser: Feng, Qi-Yuan (VerfasserIn)
Weitere Verfasser: Chen, Hao-Xiang, Xu, Qun-Ce, Mu, Tai-Jiang
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2024
Zugriff auf das übergeordnete Werk:IEEE transactions on visualization and computer graphics
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM375004653
003 DE-627
005 20240717233359.0
007 cr uuu---uuuuu
008 240717s2024 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2024.3429421  |2 doi 
028 5 2 |a pubmed24n1473.xml 
035 |a (DE-627)NLM375004653 
035 |a (NLM)39012751 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Feng, Qi-Yuan  |e verfasserin  |4 aut 
245 1 0 |a SLS4D  |b Sparse Latent Space for 4D Novel View Synthesis 
264 1 |c 2024 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 17.07.2024 
500 |a published: Print-Electronic 
500 |a Citation Status Publisher 
520 |a Neural radiance fields (NeRF) have achieved great success in novel view synthesis and 3D representation for static scenarios. Existing dynamic NeRFs usually exploit a locally dense grid to fit the deformation fields; however, they fail to capture the global dynamics and concomitantly yield models of heavy parameters. We observe that the 4D space is inherently sparse. Firstly, the deformation fields are sparse in spatial but dense in temporal due to the continuity of motion. Secondly, the radiance fields are only valid on the surface of the underlying scene, usually occupying a small fraction of the whole space. We thus represent the 4D scene using a learnable sparse latent space, a.k.a. SLS4D. Specifically, SLS4D first uses dense learnable time slot features to depict the temporal space, from which the deformation fields are fitted with linear multi-layer perceptions (MLP) to predict the displacement of a 3D position at any time. It then learns the spatial features of a 3D position using another sparse latent space. This is achieved by learning the adaptive weights of each latent feature with the attention mechanism. Extensive experiments demonstrate the effectiveness of our SLS4D: It achieves the best 4D novel view synthesis using only about 6% parameters of the most recent work 
650 4 |a Journal Article 
700 1 |a Chen, Hao-Xiang  |e verfasserin  |4 aut 
700 1 |a Xu, Qun-Ce  |e verfasserin  |4 aut 
700 1 |a Mu, Tai-Jiang  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g PP(2024) vom: 16. Juli  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnns 
773 1 8 |g volume:PP  |g year:2024  |g day:16  |g month:07 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2024.3429421  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d PP  |j 2024  |b 16  |c 07