EV-LFV : Synthesizing Light Field Event Streams from an Event Camera and Multiple RGB Cameras

Light field videos captured in RGB frames (RGB-LFV) can provide users with a 6 degree-of-freedom immersive video experience by capturing dense multi-subview video. Despite its potential benefits, the processing of dense multi-subview video is extremely resource-intensive, which currently limits the...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on visualization and computer graphics. - 1996. - 29(2023), 11 vom: 03. Nov., Seite 4546-4555
1. Verfasser: Lu, Zhicheng (VerfasserIn)
Weitere Verfasser: Chen, Xiaoming, Chung, Vera Yuk Ying, Cai, Weidong, Shen, Yiran
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2023
Zugriff auf das übergeordnete Werk:IEEE transactions on visualization and computer graphics
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM362819866
003 DE-627
005 20231226092034.0
007 cr uuu---uuuuu
008 231226s2023 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2023.3320271  |2 doi 
028 5 2 |a pubmed24n1209.xml 
035 |a (DE-627)NLM362819866 
035 |a (NLM)37788211 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Lu, Zhicheng  |e verfasserin  |4 aut 
245 1 0 |a EV-LFV  |b Synthesizing Light Field Event Streams from an Event Camera and Multiple RGB Cameras 
264 1 |c 2023 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 06.11.2023 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Light field videos captured in RGB frames (RGB-LFV) can provide users with a 6 degree-of-freedom immersive video experience by capturing dense multi-subview video. Despite its potential benefits, the processing of dense multi-subview video is extremely resource-intensive, which currently limits the frame rate of RGB-LFV (i.e., lower than 30 fps) and results in blurred frames when capturing fast motion. To address this issue, we propose leveraging event cameras, which provide high temporal resolution for capturing fast motion. However, the cost of current event camera models makes it prohibitive to use multiple event cameras for RGB-LFV platforms. Therefore, we propose EV-LFV, an event synthesis framework that generates full multi-subview event-based RGB-LFV with only one event camera and multiple traditional RGB cameras. EV-LFV utilizes spatial-angular convolution, ConvLSTM, and Transformer to model RGB-LFV's angular features, temporal features, and long-range dependency, respectively, to effectively synthesize event streams for RGB-LFV. To train EV-LFV, we construct the first event-to-LFV dataset consisting of 200 RGB-LFV sequences with ground-truth event streams. Experimental results demonstrate that EV-LFV outperforms state-of-the-art event synthesis methods for generating event-based RGB-LFV, effectively alleviating motion blur in the reconstructed RGB-LFV 
650 4 |a Journal Article 
700 1 |a Chen, Xiaoming  |e verfasserin  |4 aut 
700 1 |a Chung, Vera Yuk Ying  |e verfasserin  |4 aut 
700 1 |a Cai, Weidong  |e verfasserin  |4 aut 
700 1 |a Shen, Yiran  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g 29(2023), 11 vom: 03. Nov., Seite 4546-4555  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnns 
773 1 8 |g volume:29  |g year:2023  |g number:11  |g day:03  |g month:11  |g pages:4546-4555 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2023.3320271  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 29  |j 2023  |e 11  |b 03  |c 11  |h 4546-4555