A Unified Framework for Generalizable Style Transfer : Style and Content Separation

Image style transfer has drawn broad attention recently. However, most existing methods aim to explicitly model the transformation between different styles, and the learned model is often not generalizable to new styles. Based on the idea of style and content separation, we here propose a unified st...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on image processing : a publication of the IEEE Signal Processing Society. - 1992. - (2020) vom: 31. Jan.
1. Verfasser: Zhang, Yexun (VerfasserIn)
Weitere Verfasser: Zhang, Ya, Cai, Wenbin
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2020
Zugriff auf das übergeordnete Werk:IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
Schlagworte:Journal Article
LEADER 01000caa a22002652 4500
001 NLM30607611X
003 DE-627
005 20240229162519.0
007 cr uuu---uuuuu
008 231225s2020 xx |||||o 00| ||eng c
024 7 |a 10.1109/TIP.2020.2969081  |2 doi 
028 5 2 |a pubmed24n1308.xml 
035 |a (DE-627)NLM30607611X 
035 |a (NLM)32012013 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Zhang, Yexun  |e verfasserin  |4 aut 
245 1 2 |a A Unified Framework for Generalizable Style Transfer  |b Style and Content Separation 
264 1 |c 2020 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 27.02.2024 
500 |a published: Print-Electronic 
500 |a Citation Status Publisher 
520 |a Image style transfer has drawn broad attention recently. However, most existing methods aim to explicitly model the transformation between different styles, and the learned model is often not generalizable to new styles. Based on the idea of style and content separation, we here propose a unified style transfer framework that consists of style encoder, content encoder, mixer and decoder. The style encoder and the content encoder are used to extract the style and content representations from the corresponding reference images. The two representations are integrated by the mixer and fed to the decoder, which generates images with the target style and content. Assuming the same encoder could be shared among different styles/contents, the style/content encoder explores a generalizable way to represent style/content information, i.e. the encoders are expected to capture the underlying representation for different styles/contents and generalize to new styles/contents. Training simultaneously with a number of styles and contents, the framework enables building one single transfer network for multiple styles and further leads to a key merit of the framework, i.e. its generalizability to new styles and contents. To evaluate the proposed framework, we apply it to both supervised and unsupervised style transfer, using character typeface transfer and neural style transfer as respective examples. For character typeface transfer, to separate the style features and content features, we leverage the conditional dependence of styles and contents given an image. For neural style transfer, we leverage the statistical information of feature maps in certain layers to represent style. Extensive experimental results have demonstrated the effectiveness and robustness of the proposed methods. Furthermore, models learned under the proposed framework are shown to be better generalizable to new styles and contents 
650 4 |a Journal Article 
700 1 |a Zhang, Ya  |e verfasserin  |4 aut 
700 1 |a Cai, Wenbin  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on image processing : a publication of the IEEE Signal Processing Society  |d 1992  |g (2020) vom: 31. Jan.  |w (DE-627)NLM09821456X  |x 1941-0042  |7 nnns 
773 1 8 |g year:2020  |g day:31  |g month:01 
856 4 0 |u http://dx.doi.org/10.1109/TIP.2020.2969081  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |j 2020  |b 31  |c 01