1. David Healey BYU Capstone Course 15 Nov 2018
Lecture 4: VAEs, GANs
David Healey
BYU Capstone Course
15 Nov 2018

2. Administrative things

3. Homework: Kaggle stock exchange prediction
Our homework will be a group assignment to do a Kaggle competition involving stock prediction from the new york stock exchange

4. About Recursion Pharmaceuticals
To get a data science job at Recursion Pharmaceuticals:
Pass a resume screen
Practical experience
Advanced degree helps
Looking for depth
Pass a work sample
2 hours to complete several questions testing general ability around data analysis
”What kind of analysis would constitute an answer to this question?”
Evaluation of the method you used
Pass 4 or 5 onsite interviews
Light coding, quantitative reasoning, machine learning
Kinds of jobs at Recursion:
Data scientist
Data engineer
Software Engineer

5. How to contact me
LinkedIn

6. Generative models: Variational Autoencoders and GANs

8. Unsupervised generative models: VAEs and GANs
The goal is to create a decoder or a generator: a neural network that maps an arbitrary low- dimensional vector into a very high-dimensional and complicated output that matches a very complicated data distribution like images or language
A model that can create something new that looks like its from the data “understands” the distribution of the data very well.
Example: can you create the face of a person who doesn’t exist yet z
Low dimensional
“latent space”
High dimensional

9. Variational autoencoders (VAE)
Normal autoencoder
Variational autoencoder
Encode to a distribution instead of a single point
Decoder first samples from the distribution.
Try to force the space to be continuous and meaningful so that arbitrary vectors will map to something real-seeming
Encode data to a vector whose dimension is less than the

10. VAE loss 2 parts to the loss
Reconstruction loss (mse): did you get the same data out that you put in?
This makes sure the model learns the distribution of the data, like a regular autoencoder
K-L loss (KL divergence from unit normal): Did you encode the point to a distribution that somewhat resembles a unit normal? (this makes sure the latent space is more or less continuous)

11. Variational Autoencoders: Generating Data!
Data manifold for 2-d z
Dim 1
Labeled Faces in the Wild
Figures copyright (L) Dirk Kingma et al. 2016; (R) Anders Larsen et al Reproduced with permission.
Dimension 2
Remember: these are new numbers, not part of the training set
Kingma and Welling, “Auto-Encoding Variational Bayes”, ICLR 2014
Used with permission from Justin Johnson

12. Generative Adversarial Networks
Ian Goodfellow et al., “Generative
Adversarial Nets”, NIPS 2014
Problem: Want to sample from complex, high-dimensional training distribution. No direct
way to do this!
Solution: Sample from a simple distribution, e.g. random noise. Learn transformation to
training distribution.
Output: Sample from
training distribution
Q: What can we use to
represent this complex
transformation?
A: A neural network!
Transform
Input: Random noise
sample z 10 3
Lecture 12 -
May 15, 2018
Used with permission from Justin Johnson

13. Training GANs: Two-player game
Ian Goodfellow et al., “Generative
Adversarial Nets”, NIPS 2014
Generator network : try to fool the discriminator by generating real-looking images
Discriminator network : try to distinguish between real and fake images
Real or Fake
Discriminator Network
(CNN)
Fake Images
(from generator)
Real Images
(from training set)
Generator Network
(upsampling convolutional network)
Random noise z
Fake and real images copyright Emily Denton et al Reproduced with permission. 10 5
Lecture 12 -
May 15, 2018
Used with permission from Justin Johnson

14. Training GANs
Alternate between training the generator and discriminator (you can train for one step each or multiple steps each)
Discriminator:
Generate some fake examples with the generator
Get some real examples
Discriminator tries to score the fake examples low and the real ones high
So GAN loss punishes discriminator for high scores on fake examples and low scores on real examples
Generator:
Generate some fake images and give them to the discriminator to score
Generator tries to get high scores from the discriminator
So GAN loss punishes generator for low scores from the discriminator
Warning: notoriously hard to train and unstable. See for tips:

15. Generative Adversarial Nets: Convolutional Architectures
Samples
from the
model look
much
better!
Radford et al,
ICLR 2016 12
Lecture 12 -
May 15, 2018
Used with permission from Justin Johnson

16. Generative Adversarial Nets: Interpretable Vector Math
Glasses man No glasses man No glasses woman
Radford et al, ICLR 2016
Radford et al,
ICLR 2016
Lecture 12 -
May 15, 2018
Used with permission from Justin Johnson

17. Generative Adversarial Nets: Interpretable Vector Math
Glasses man
No glasses man
No glasses woman
Radford et al, ICLR 2016
Woman with glasses 12 6
Lecture 12 -
May 15, 2018
Used with permission from Justin Johnson

18. See how far we’ve come since then

19. Resources to look for for help
When you don’t know what approach to take:
Talk to someone
Examples to work from, when you know what you need to build but don’t know how to code and train it:
Keras blog has a lot of examples (our translation example was adapted from one)
Google search usually brings up at least one github repo if you search something like Keras GAN
Concepts and Courses for learning and clarifying concepts:
Deeplearning.ai Andrew Ng (pretty friendly)
Cs231n Stanford course (heavy on equations). A lot of the slides here were adapted w/permission from that course.
Geoff Hinton’s DL Coursera (downside: OCTAVE not Python)

20. Some things to say about data science and artificial intelligence

21. The demand for machine learning is crazy high right now
Data scientist is the highest rated job in the country 3 years running
Machine learning scientists have higher compensation than comparably-credentialed peers
Utah has a particular shortage of data scientists which is about to get worse

22. The future of Artificial Intelligence

23. What happens when machines displace human labor?

24. Real earnings relative to 1963 (men)

25. “And now behold, my beloved brethren, … do not suppose that this is all; for after ye have done all these things, if ye turn away the needy, and the naked, and visit not the sick and afflicted, and impart of your substance, if ye have, to those who stand in need—I say unto you, if ye do not any of these things, behold, your prayer is vain, and availeth you nothing, and ye are as hypocrites who do deny the faith.” (Alma 34:28.)