Content-Aware Warping for View Synthesis

Existing image-based rendering methods usually adopt depth-based image warping operation to synthesize novel views. In this paper, we reason the essential limitations of the traditional warping operation to be the limited neighborhood and only distance-based interpolation weights. To this end, we pr...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - 45(2023), 8 vom: 06. Aug., Seite 9486-9503
1. Verfasser: Guo, Mantang (VerfasserIn)
Weitere Verfasser: Hou, Junhui, Jin, Jing, Liu, Hui, Zeng, Huanqiang, Lu, Jiwen
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2023
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM355271176
003 DE-627
005 20231226064010.0
007 cr uuu---uuuuu
008 231226s2023 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2023.3242709  |2 doi 
028 5 2 |a pubmed24n1184.xml 
035 |a (DE-627)NLM355271176 
035 |a (NLM)37022422 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Guo, Mantang  |e verfasserin  |4 aut 
245 1 0 |a Content-Aware Warping for View Synthesis 
264 1 |c 2023 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Completed 03.07.2023 
500 |a Date Revised 03.07.2023 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Existing image-based rendering methods usually adopt depth-based image warping operation to synthesize novel views. In this paper, we reason the essential limitations of the traditional warping operation to be the limited neighborhood and only distance-based interpolation weights. To this end, we propose content-aware warping, which adaptively learns the interpolation weights for pixels of a relatively large neighborhood from their contextual information via a lightweight neural network. Based on this learnable warping module, we propose a new end-to-end learning-based framework for novel view synthesis from a set of input source views, in which two additional modules, namely confidence-based blending and feature-assistant spatial refinement, are naturally proposed to handle the occlusion issue and capture the spatial correlation among pixels of the synthesized view, respectively. Besides, we also propose a weight-smoothness loss term to regularize the network. Experimental results on light field datasets with wide baselines and multi-view datasets show that the proposed method significantly outperforms state-of-the-art methods both quantitatively and visually. The source code is publicly available at https://github.com/MantangGuo/CW4VS 
650 4 |a Journal Article 
700 1 |a Hou, Junhui  |e verfasserin  |4 aut 
700 1 |a Jin, Jing  |e verfasserin  |4 aut 
700 1 |a Liu, Hui  |e verfasserin  |4 aut 
700 1 |a Zeng, Huanqiang  |e verfasserin  |4 aut 
700 1 |a Lu, Jiwen  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g 45(2023), 8 vom: 06. Aug., Seite 9486-9503  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnns 
773 1 8 |g volume:45  |g year:2023  |g number:8  |g day:06  |g month:08  |g pages:9486-9503 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2023.3242709  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 45  |j 2023  |e 8  |b 06  |c 08  |h 9486-9503