Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network

Jialu HUANG, Jing LIAO, Sam KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review

30 Citations (Scopus)

Abstract

Image-to-Image (I2I) translation is an emerging topic in academia, and it also has been applied in real-world industry for tasks like image synthesis, super-resolution, and colorization. Traditional I2I translation methods usually train data in two or more domains together. This requires lots of computation resources. The results are of lower quality, and contain more artifacts. The training process could be unstable when the data in different domains are not balanced, and modal collapse is more likely to happen. In this paper, we propose a new I2I translation method that generates a new model in the target domain via a series of model transformations on a pre-trained StyleGAN2 model in the source domain. After that, we develop an inversion method to achieve the conversion between an image and its latent vector. By feeding the latent vector into the generated model, we can perform I2I translation between the source domain and target domain. Both qualitative and quantitative evaluations were conducted to verify that the proposed method can achieve better performance in terms of image quality, diversity and semantic similarity to the input and reference images compared to state-of-the-art works.
Original languageEnglish
Pages (from-to)1435-1448
Number of pages15
JournalIEEE Transactions on Multimedia
Volume24
Early online date17 Mar 2021
DOIs
Publication statusPublished - 17 Mar 2022
Externally publishedYes

Keywords

  • GAN
  • machine learning
  • neural network

Fingerprint

Dive into the research topics of 'Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network'. Together they form a unique fingerprint.

Cite this