Structure-Guided Image Completion With Image-Level and Object-Level Semantic Discriminators

Structure-guided image completion aims to inpaint a local region of an image according to an input guidance map from users. While such a task enables many practical applications for interactive editing, existing methods often struggle to hallucinate realistic object instances in complex natural scen...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - 46(2024), 12 vom: 27. Nov., Seite 7669-7681
1. Verfasser: Zheng, Haitian (VerfasserIn)
Weitere Verfasser: Lin, Zhe, Lu, Jingwan, Cohen, Scott, Shechtman, Eli, Barnes, Connelly, Zhang, Jianming, Liu, Qing, Amirghodsi, Sohrab, Zhou, Yuqian, Luo, Jiebo
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2024
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article
LEADER 01000caa a22002652 4500
001 NLM371580188
003 DE-627
005 20241108232140.0
007 cr uuu---uuuuu
008 240427s2024 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2024.3393898  |2 doi 
028 5 2 |a pubmed24n1594.xml 
035 |a (DE-627)NLM371580188 
035 |a (NLM)38669165 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Zheng, Haitian  |e verfasserin  |4 aut 
245 1 0 |a Structure-Guided Image Completion With Image-Level and Object-Level Semantic Discriminators 
264 1 |c 2024 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 08.11.2024 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a Structure-guided image completion aims to inpaint a local region of an image according to an input guidance map from users. While such a task enables many practical applications for interactive editing, existing methods often struggle to hallucinate realistic object instances in complex natural scenes. Such a limitation is partially due to the lack of semantic-level constraints inside the hole region as well as the lack of a mechanism to enforce realistic object generation. In this work, we propose a learning paradigm that consists of semantic discriminators and object-level discriminators for improving the generation of complex semantics and objects. Specifically, the semantic discriminators leverage pretrained visual features to improve the realism of the generated visual concepts. Moreover, the object-level discriminators take aligned instances as inputs to enforce the realism of individual objects. Our proposed scheme significantly improves the generation quality and achieves state-of-the-art results on various tasks, including segmentation-guided completion, edge-guided manipulation and panoptically-guided manipulation on Places2 datasets. Furthermore, our trained model is flexible and can support multiple editing use cases, such as object insertion, replacement, removal and standard inpainting. In particular, our trained model combined with a novel automatic image completion pipeline achieves state-of-the-art results on the standard inpainting task 
650 4 |a Journal Article 
700 1 |a Lin, Zhe  |e verfasserin  |4 aut 
700 1 |a Lu, Jingwan  |e verfasserin  |4 aut 
700 1 |a Cohen, Scott  |e verfasserin  |4 aut 
700 1 |a Shechtman, Eli  |e verfasserin  |4 aut 
700 1 |a Barnes, Connelly  |e verfasserin  |4 aut 
700 1 |a Zhang, Jianming  |e verfasserin  |4 aut 
700 1 |a Liu, Qing  |e verfasserin  |4 aut 
700 1 |a Amirghodsi, Sohrab  |e verfasserin  |4 aut 
700 1 |a Zhou, Yuqian  |e verfasserin  |4 aut 
700 1 |a Luo, Jiebo  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g 46(2024), 12 vom: 27. Nov., Seite 7669-7681  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnns 
773 1 8 |g volume:46  |g year:2024  |g number:12  |g day:27  |g month:11  |g pages:7669-7681 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2024.3393898  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 46  |j 2024  |e 12  |b 27  |c 11  |h 7669-7681