SAC-GAN : Structure-Aware Image Composition

We introduce an end-to-end learning framework for image-to-image composition, aiming to plausibly compose an object represented as a cropped patch from an object image into a background scene image. As our approach emphasizes more on semantic and structural coherence of the composed images, rather t...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on visualization and computer graphics. - 1996. - 30(2024), 7 vom: 01. Juni, Seite 3151-3165
1. Verfasser: Zhou, Hang (VerfasserIn)
Weitere Verfasser: Ma, Rui, Zhang, Ling-Xiao, Gao, Lin, Mahdavi-Amiri, Ali, Zhang, Hao
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2024
Zugriff auf das übergeordnete Werk:IEEE transactions on visualization and computer graphics
Schlagworte:Journal Article
LEADER 01000caa a22002652 4500
001 NLM355202328
003 DE-627
005 20240628231857.0
007 cr uuu---uuuuu
008 231226s2024 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2022.3226689  |2 doi 
028 5 2 |a pubmed24n1454.xml 
035 |a (DE-627)NLM355202328 
035 |a (NLM)37015486 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Zhou, Hang  |e verfasserin  |4 aut 
245 1 0 |a SAC-GAN  |b Structure-Aware Image Composition 
264 1 |c 2024 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 28.06.2024 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a We introduce an end-to-end learning framework for image-to-image composition, aiming to plausibly compose an object represented as a cropped patch from an object image into a background scene image. As our approach emphasizes more on semantic and structural coherence of the composed images, rather than their pixel-level RGB accuracies, we tailor the input and output of our network with structure-aware features and design our network losses accordingly, with ground truth established in a self-supervised setting through the object cropping. Specifically, our network takes the semantic layout features from the input scene image, features encoded from the edges and silhouette in the input object patch, as well as a latent code as inputs, and generates a 2D spatial affine transform defining the translation and scaling of the object patch. The learned parameters are further fed into a differentiable spatial transformer network to transform the object patch into the target image, where our model is trained adversarially using an affine transform discriminator and a layout discriminator. We evaluate our network, coined SAC-GAN, for various image composition scenarios in terms of quality, composability, and generalizability of the composite images. Comparisons are made to state-of-the-art alternatives, including Instance Insertion, ST-GAN, CompGAN and PlaceNet, confirming superiority of our method 
650 4 |a Journal Article 
700 1 |a Ma, Rui  |e verfasserin  |4 aut 
700 1 |a Zhang, Ling-Xiao  |e verfasserin  |4 aut 
700 1 |a Gao, Lin  |e verfasserin  |4 aut 
700 1 |a Mahdavi-Amiri, Ali  |e verfasserin  |4 aut 
700 1 |a Zhang, Hao  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g 30(2024), 7 vom: 01. Juni, Seite 3151-3165  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnns 
773 1 8 |g volume:30  |g year:2024  |g number:7  |g day:01  |g month:06  |g pages:3151-3165 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2022.3226689  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 30  |j 2024  |e 7  |b 01  |c 06  |h 3151-3165