InvGAN: Invertible GANs

2022

Conference Paper

al

ps

Generation of photo-realistic images, semantic editing and representation learning are only a few of many applications of high-resolution generative models. Recent progress in GANs have established them as an excellent choice for such tasks. However, since they do not provide an inference model, downstream tasks such as classification cannot be easily applied on real images using the GAN latent space. Despite numerous efforts to train an inference model or design an iterative method to invert a pre-trained generator, previous methods are dataset (e.g. human face images) and architecture (e.g. StyleGAN) specific. These methods are nontrivial to extend to novel datasets or architectures. We propose a general framework that is agnostic to architecture and datasets. Our key insight is that, by training the inference and the generative model together, we allow them to adapt to each other and to converge to a better quality model. Our InvGAN, short for Invertible GAN, successfully embeds real images in the latent space of a high quality generative model. This allows us to perform image inpainting, merging, interpolation and online data augmentation. We demonstrate this with extensive qualitative and quantitative experiments.

@inproceedings{InvGAN:GCPR:2022,
  title = {{InvGAN}: Invertible {GANs}},
  author = {Ghosh, Partha and Zietlow, Dominik and Black, Michael J. and Davis, Larry S. and Hu, Xiaochen},
  booktitle = {Pattern Recognition},
  pages = {3--19},
  series = {Lecture Notes in Computer Science, 13485},
  editors = {Andres, Björn and Bernard, Florian and Cremers, Daniel and Frintrop, Simone and Goldlücke, Bastian and Ihrke, Ivo},
  publisher = {Springer},
  address = {Cham},
  month = sep,
  year = {2022},
  doi = {10.1007/978-3-031-16788-1_1},
  month_numeric = {9}
}