1. Deep Learning: What is it good for? R. Burgmann
ITC571 Emerging technologies and Innovation

2. Introduction A brief history.
What are neural networks?
What are artificial neural networks (ANN)?
Early problem, solutions, stagnation.
Where does deep learning (DL) fit in the toolbox?
Relationship between ANN and DL
When is it the right choice?
How close are we to human level artificial intelligence (are we there yet)?
Questions and Answers.

3. What are neural networks?
Image courtesy of Hagan, Demuth & Beal. Neural Network Design PWS Publishing. Boston. Page 1-8

4. What are artificial neural networsk?
Image courtesy of Hagan, Demuth & Beal. Neural Network Design PWS Publishing. Boston. Page 2-11

5. Early problems, solutions and stagnation
Problem 1. Single layer networks could only solve linear problems.
Problem 2. At first no algorithm known to train multi-layer networks until backpropagation algorithm.
Problem 3. Feature detectors needed to be hand coded or in some other fashion made available to the network.
Problem 4. Not enough training data for serious problems.
Problem 5. If you had a serious problem with lots of training data then you needed a super computer cluster.
Problem 6. If you could solve it with an ANN back in the 1980s and 1990s then there where easier ways.

6. Where does deep learning fit in the toolbox?
Relationship between ANN and DL
Rather than 3 to 5 layers, deep learning uses neural networks hundreds of layers deep.
How is this achieved?
Problem 3. Feature detectors needed to be hand coded or in some other fashion made available to the network.
Convolution
Pooling
Problem 4. Not enough training data for serious problems.
Unlimited training data (thank you internet)
Problem 5. If you had a serious problem with lots of training data then you needed a super computer cluster.
Unlimited processing capacity (thank you computer games industry and graphical processing chips)

7. Where does deep learning fit in the toolbox
Where does deep learning fit in the toolbox? Other machine learning techniques
Is deep learning the only game in town?
Well… no.
Bayesian Networks
Classification Rules
Linear Models
Clustering
Ensemble Learning

8. When is it the right choice?
Tens to hundreds of millions of training examples
Unknown features
Examples
Baldi et al, searching for exotic particles in high energy particle physics.
Druzhkov, et al, image classification and object detection.
Noda, et al, Audio-visual speech recognition.
Sheehan, et al, Population genetics
Suk, et al, Brain imaging of Alzheimers patients

9. Are we there yet? How close are we to human level ai?
From deep learning to AIXI, “an optimal rational reinforcement learning agent” Hutter (2005)

10. Questions and Answers
Thank you for your time.