Multimodal Image Synthesis and Editing : The Generative AI Era

As information exists in various modalities in real world, effective interaction and fusion among multimodal information plays a key role for the creation and perception of multimodal data in computer vision and deep learning research. With superb power in modeling the interaction among multimodal i...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence. - 1979. - 45(2023), 12 vom: 25. Dez., Seite 15098-15119
1. Verfasser: Zhan, Fangneng (VerfasserIn)
Weitere Verfasser: Yu, Yingchen, Wu, Rongliang, Zhang, Jiahui, Lu, Shijian, Liu, Lingjie, Kortylewski, Adam, Theobalt, Christian, Xing, Eric
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2023
Zugriff auf das übergeordnete Werk:IEEE transactions on pattern analysis and machine intelligence
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM361229488
003 DE-627
005 20231226084712.0
007 cr uuu---uuuuu
008 231226s2023 xx |||||o 00| ||eng c
024 7 |a 10.1109/TPAMI.2023.3305243  |2 doi 
028 5 2 |a pubmed24n1204.xml 
035 |a (DE-627)NLM361229488 
035 |a (NLM)37624713 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Zhan, Fangneng  |e verfasserin  |4 aut 
245 1 0 |a Multimodal Image Synthesis and Editing  |b The Generative AI Era 
264 1 |c 2023 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 07.11.2023 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a As information exists in various modalities in real world, effective interaction and fusion among multimodal information plays a key role for the creation and perception of multimodal data in computer vision and deep learning research. With superb power in modeling the interaction among multimodal information, multimodal image synthesis and editing has become a hot research topic in recent years. Instead of providing explicit guidance for network training, multimodal guidance offers intuitive and flexible means for image synthesis and editing. On the other hand, this field is also facing several challenges in alignment of multimodal features, synthesis of high-resolution images, faithful evaluation metrics, etc. In this survey, we comprehensively contextualize the advance of the recent multimodal image synthesis and editing and formulate taxonomies according to data modalities and model types. We start with an introduction to different guidance modalities in image synthesis and editing, and then describe multimodal image synthesis and editing approaches extensively according to their model types. After that, we describe benchmark datasets and evaluation metrics as well as corresponding experimental results. Finally, we provide insights about the current research challenges and possible directions for future research 
650 4 |a Journal Article 
700 1 |a Yu, Yingchen  |e verfasserin  |4 aut 
700 1 |a Wu, Rongliang  |e verfasserin  |4 aut 
700 1 |a Zhang, Jiahui  |e verfasserin  |4 aut 
700 1 |a Lu, Shijian  |e verfasserin  |4 aut 
700 1 |a Liu, Lingjie  |e verfasserin  |4 aut 
700 1 |a Kortylewski, Adam  |e verfasserin  |4 aut 
700 1 |a Theobalt, Christian  |e verfasserin  |4 aut 
700 1 |a Xing, Eric  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on pattern analysis and machine intelligence  |d 1979  |g 45(2023), 12 vom: 25. Dez., Seite 15098-15119  |w (DE-627)NLM098212257  |x 1939-3539  |7 nnns 
773 1 8 |g volume:45  |g year:2023  |g number:12  |g day:25  |g month:12  |g pages:15098-15119 
856 4 0 |u http://dx.doi.org/10.1109/TPAMI.2023.3305243  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 45  |j 2023  |e 12  |b 25  |c 12  |h 15098-15119