1. PixelGAN Autoencoders
Alireza Makhzani, Brendan Frey
University of Toronto
Liu ze
Dec 30th, 2017
中国科学技术大学
University of Science and Technology of China

2. Outline 1. Background 2. PixelGAN Autoencoders 3. Experiments
PixelCNNs
Variational Autoencoders
Adversarial Autoencoders
2. PixelGAN Autoencoders
Limitations of VAE/AAE
Structure and Training
Benefits of PixelGAN Autoencoders
3. Experiments
4. Conclusion

3. Outline 1. Background 2. PixelGAN Autoencoders 3. Experiments
PixelCNNs
Variational Autoencoders
Adversarial Autoencoders
2. PixelGAN Autoencoders
Limitations of VAE/AAE
Structure and Training
Benefits of PixelGAN Autoencoders
3. Experiments
4. Conclusion

4. PixelCNNs

5. Contional PixelCNNs h h

6. Conditional PixelCNNsh h
Learn the image statistics directly at the pixel level.
Good at modelling low-level pixel statistics.
Conditional PixelCNNs can learn conditional densities.
Samples lack global structure.
Lacking latent representation.

7. Variational Autoencoders
Good at capturing the global structure, but samples are blurry.

8. Adversarial Autoencoders
Code Space of MNIST:
Gaussian Prior
Mixture of Gaussians

9. Outline 1. Background 2. PixelGAN Autoencoders 3. Experiments
PixelCNNs
Variational Autoencoders
Adversarial Autoencoders
2. PixelGAN Autoencoders
Limitations of VAE/AAE
Structure and Training
Benefits of PixelGAN Autoencoders
3. Experiments
4. Conclusion

10. Limitations of VAE/AAE
✦ All the image statistics are captured by the single latent vector.
VAE
label, style global and local
p(z)
Latent Variable
Deterministic (factorized Gaussians)
p(x|z)
None

11. Structure and Training
Cost function of PixelGAN = Reconstruction + Adversarial Cost

12. Benefits of PixelGAN Autoencoders
✦ The image statistics are captured jointly by the latent vector and the autoregressive decoder.
p(z)
Latent Variable
p(x|z)
PixelCNN

13. Benefits of PixelGAN Autoencoders
✦ The image statistics are captured jointly by the latent vector and the autoregressive decoder.
PixelGAN (Gaussian)
PixelGAN (Categorical)
Discrete (label)
Global (low-frequency)
p(z)
Latent Variable
Local (high-frequency)
Continuous (Style)
p(x|z)
PixelCNN

14. PixelGAN (Categorical)
Benefits of PixelGAN Autoencoders
✦ The image statistics are captured jointly by the latent vector and the autoregressive decoder.
PixelGAN (Gaussian)
PixelGAN (Categorical)
Discrete (label)
Global (low-frequency)
p(z)
Latent Variable
Local (high-frequency)
Continuous (Style)
p(x|z)
PixelCNN
Semi-supervised Learning

15. Outline 1. Background 2. PixelGAN Autoencoders 3. Experiments
PixelCNNs
Variational Autoencoders
Adversarial Autoencoders
2. PixelGAN Autoencoders
Limitations of VAE/AAE
Structure and Training
Benefits of PixelGAN Autoencoders
3. Experiments
4. Conclusion

16. Global vs. Local Decomposition

17. Code Space
Code Space of MNIST:

18. PixelGAN Autoencoders with Categorical Priors

19. Discrete vs. Continuous Decomposition (Clustering)

20. Discrete vs. Continuous Decomposition (Clustering)

21. Unsupervised Clustering
5% Error rate

22. Semi-supervised Learning

23. Semi-supervised Classification

24. Outline 1. Background 2. PixelGAN Autoencoders 3. Experiments
PixelCNNs
Variational Autoencoders
Adversarial Autoencoders
2. PixelGAN Autoencoders
Limitations of VAE/AAE
Structure and Training
Benefits of PixelGAN Autoencoders
3. Experiments
4. Conclusion

25. Unsupervised ClusteringA
Proposed the PixelGAN autoencoder, which is a generative autoencoder that combines a generative PixelCNN with a GAN inference network that can impose arbitrary priors on the latent code. B
Showed that imposing different distributions as the prior enables us to learn a latent representation that captures the type of statistics that we care about, while the remaining structure of the image is captured by the PixelCNN decoder. C
Demonstrate the application of PixelGAN autoencoders in downstream tasks such as semi-supervised learning; Discussed how these architectures have other potentials such as learning cross-domain relations between two different domains

26. Thank you!