Local-to-Global Panorama Inpainting for Locale-Aware Indoor Lighting Prediction

Predicting panoramic indoor lighting from a single perspective image is a fundamental but highly ill-posed problem in computer vision and graphics. To achieve locale-aware and robust prediction, this problem can be decomposed into three sub-tasks: depth-based image warping, panorama inpainting and h...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on visualization and computer graphics. - 1996. - 29(2023), 11 vom: 02. Nov., Seite 4405-4416
1. Verfasser: Bai, Jiayang (VerfasserIn)
Weitere Verfasser: He, Zhen, Yang, Shan, Guo, Jie, Chen, Zhenyu, Zhang, Yan, Guo, Yanwen
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2023
Zugriff auf das übergeordnete Werk:IEEE transactions on visualization and computer graphics
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM362784345
003 DE-627
005 20231226091949.0
007 cr uuu---uuuuu
008 231226s2023 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2023.3320233  |2 doi 
028 5 2 |a pubmed24n1209.xml 
035 |a (DE-627)NLM362784345 
035 |a (NLM)37782598 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Bai, Jiayang  |e verfasserin  |4 aut 
245 1 0 |a Local-to-Global Panorama Inpainting for Locale-Aware Indoor Lighting Prediction 
264 1 |c 2023 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 03.11.2023 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Predicting panoramic indoor lighting from a single perspective image is a fundamental but highly ill-posed problem in computer vision and graphics. To achieve locale-aware and robust prediction, this problem can be decomposed into three sub-tasks: depth-based image warping, panorama inpainting and high-dynamic-range (HDR) reconstruction, among which the success of panorama inpainting plays a key role. Recent methods mostly rely on convolutional neural networks (CNNs) to fill the missing contents in the warped panorama. However, they usually achieve suboptimal performance since the missing contents occupy a very large portion in the panoramic space while CNNs are plagued by limited receptive fields. The spatially-varying distortion in the spherical signals further increases the difficulty for conventional CNNs. To address these issues, we propose a local-to-global strategy for large-scale panorama inpainting. In our method, a depth-guided local inpainting is first applied on the warped panorama to fill small but dense holes. Then, a transformer-based network, dubbed PanoTransformer, is designed to hallucinate reasonable global structures in the large holes. To avoid distortion, we further employ cubemap projection in our design of PanoTransformer. The high-quality panorama recovered at any locale helps us to capture spatially-varying indoor illumination with physically-plausible global structures and fine details 
650 4 |a Journal Article 
700 1 |a He, Zhen  |e verfasserin  |4 aut 
700 1 |a Yang, Shan  |e verfasserin  |4 aut 
700 1 |a Guo, Jie  |e verfasserin  |4 aut 
700 1 |a Chen, Zhenyu  |e verfasserin  |4 aut 
700 1 |a Zhang, Yan  |e verfasserin  |4 aut 
700 1 |a Guo, Yanwen  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g 29(2023), 11 vom: 02. Nov., Seite 4405-4416  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnns 
773 1 8 |g volume:29  |g year:2023  |g number:11  |g day:02  |g month:11  |g pages:4405-4416 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2023.3320233  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 29  |j 2023  |e 11  |b 02  |c 11  |h 4405-4416