A Domain-Guided Noise-Optimization-Based Inversion Method for Facial Image Manipulation

A style-based architecture (StyleGAN2) yields outstanding results in data-driven unconditional generative image modeling. This work proposes a Domain-guided Noise-optimization-based Inversion (DNI) method to perform facial image manipulation. It works based on an inverse code that includes: 1) a nov...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on image processing : a publication of the IEEE Signal Processing Society. - 1992. - 30(2021) vom: 22., Seite 6198-6211
1. Verfasser: Yang, Nan (VerfasserIn)
Weitere Verfasser: Zheng, Zeyu, Zhou, Mengchu, Guo, Xiwang, Qi, Liang, Wang, Tianran
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2021
Zugriff auf das übergeordnete Werk:IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
Schlagworte:Journal Article
LEADER 01000naa a22002652 4500
001 NLM327042923
003 DE-627
005 20231225195913.0
007 cr uuu---uuuuu
008 231225s2021 xx |||||o 00| ||eng c
024 7 |a 10.1109/TIP.2021.3089905  |2 doi 
028 5 2 |a pubmed24n1090.xml 
035 |a (DE-627)NLM327042923 
035 |a (NLM)34156940 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Yang, Nan  |e verfasserin  |4 aut 
245 1 2 |a A Domain-Guided Noise-Optimization-Based Inversion Method for Facial Image Manipulation 
264 1 |c 2021 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Revised 12.07.2021 
500 |a published: Print-Electronic 
500 |a Citation Status PubMed-not-MEDLINE 
520 |a A style-based architecture (StyleGAN2) yields outstanding results in data-driven unconditional generative image modeling. This work proposes a Domain-guided Noise-optimization-based Inversion (DNI) method to perform facial image manipulation. It works based on an inverse code that includes: 1) a novel domain-guided encoder called Image2latent to project the image to StyleGAN2 latent space, which can reconstruct an input image with high-quality and maintain its semantic meaning well; 2) a noise optimization mechanism in which a set of noise vectors are used to capture the high-frequency details such as image edges, further improving image reconstruction quality; and 3) a mask for seamless image fusion and local style migration. We further propose a novel semantic alignment evaluation pipeline. It evaluates the semantic alignment with an inverse code by using different attribute boundaries. Extensive qualitative and quantitative comparisons show that DNI can capture rich semantic information and achieve a satisfactory image reconstruction. It can realize a variety of facial image manipulation tasks and outperform state of the art 
650 4 |a Journal Article 
700 1 |a Zheng, Zeyu  |e verfasserin  |4 aut 
700 1 |a Zhou, Mengchu  |e verfasserin  |4 aut 
700 1 |a Guo, Xiwang  |e verfasserin  |4 aut 
700 1 |a Qi, Liang  |e verfasserin  |4 aut 
700 1 |a Wang, Tianran  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on image processing : a publication of the IEEE Signal Processing Society  |d 1992  |g 30(2021) vom: 22., Seite 6198-6211  |w (DE-627)NLM09821456X  |x 1941-0042  |7 nnns 
773 1 8 |g volume:30  |g year:2021  |g day:22  |g pages:6198-6211 
856 4 0 |u http://dx.doi.org/10.1109/TIP.2021.3089905  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 30  |j 2021  |b 22  |h 6198-6211