Timezone: »

 
Poster
Low-Rank Subspaces in GANs
Jiapeng Zhu · Ruili Feng · Yujun Shen · Deli Zhao · Zheng-Jun Zha · Jingren Zhou · Qifeng Chen

Thu Dec 09 12:30 AM -- 02:00 AM (PST) @
in Poster Session 5 »

The latent space of a Generative Adversarial Network (GAN) has been shown to encode rich semantics within some subspaces. To identify these subspaces, researchers typically analyze the statistical information from a collection of synthesized data, and the identified subspaces tend to control image attributes globally (i.e., manipulating an attribute causes the change of an entire image). By contrast, this work introduces low-rank subspaces that enable more precise control of GAN generation. Concretely, given an arbitrary image and a region of interest (e.g., eyes of face images), we manage to relate the latent space to the image region with the Jacobian matrix and then use low-rank factorization to discover steerable latent subspaces. There are three distinguishable strengths of our approach that can be aptly called LowRankGAN. First, compared to analytic algorithms in prior work, our low-rank factorization of Jacobians is able to find the low-dimensional representation of attribute manifold, making image editing more precise and controllable. Second, low-rank factorization naturally yields a null space of attributes such that moving the latent code within it only affects the outer region of interest. Therefore, local image editing can be simply achieved by projecting an attribute vector into the null space without relying on a spatial mask as existing methods do. Third, our method can robustly work with a local region from one image for analysis yet well generalize to other images, making it much easy to use in practice. Extensive experiments on state-of-the-art GAN models (including StyleGAN2 and BigGAN) trained on various datasets demonstrate the effectiveness of our LowRankGAN.

Keywords: Generative Model

Author Information

Jiapeng Zhu (Hong Kong University of Science and Technology)
Ruili Feng (University of Science and Technology of China)
Yujun Shen (ByteDance Inc.)
Deli Zhao (Xiaomi AI Lab)
Zheng-Jun Zha (University of Science and Technology of China)
Jingren Zhou (Alibaba Group)
Qifeng Chen (HKUST)

More from the Same Authors