1. Neural Networks and Deep Learning

3. How does the
brain do it?

6. If features are complex enough, anything can be classified?

7. Single neurons are not able to solve complex tasks (linear
decision boundaries).
More layers of linear units are not enough (still linear).

13. How can we learn the weights?
Theoretical result [Cybenko, 1989]: 2-layer net with linear
output can approximate any continuous function over compact
domain to arbitrary accuracy (given enough hidden units!)

14. Why use Deep Multi Layered Models?
Is there a theoretical justification? No

18. Neural Network examples
Standard NN
Convolutional NN
Recurrent NN

29. No closed form solution for the Maximum Likelihood for this
model!

31. Feed forward Networks

66. Books and Resources
We will mostly follow Deep Learning by Ian Goodfellow,Yoshua Bengio and Aaron Courville (MIT Press, 2016)
Stanford CS 231n: by Li, Karpathy & Johnson
Neural Networks and Deep Learning by Michael Nielsen
Bishop - Pattern Recognition And Machine Learning - Springer 2006
Uncertainty in Deep Learning Yarin Gal Department of Engineering University of Cambridge

67. Books and Resources
Probabilistic machine learning and artificial intelligence Zoubin Ghahramani1