Deep Learning: What is it good for? R. Burgmann ITC571 Emerging technologies and Innovation

Introduction A brief history. What are neural networks? What are artificial neural networks (ANN)? Early problem, solutions, stagnation. Where does deep learning (DL) fit in the toolbox? Relationship between ANN and DL When is it the right choice? How close are we to human level artificial intelligence (are we there yet)? Questions and Answers.

What are neural networks? Image courtesy of Hagan, Demuth & Beal. Neural Network Design PWS Publishing. Boston. Page 1-8

What are artificial neural networsk? Image courtesy of Hagan, Demuth & Beal. Neural Network Design PWS Publishing. Boston. Page 2-11

Early problems, solutions and stagnation Problem 1. Single layer networks could only solve linear problems. Problem 2. At first no algorithm known to train multi-layer networks until backpropagation algorithm. Problem 3. Feature detectors needed to be hand coded or in some other fashion made available to the network. Problem 4. Not enough training data for serious problems. Problem 5. If you had a serious problem with lots of training data then you needed a super computer cluster. Problem 6. If you could solve it with an ANN back in the 1980s and 1990s then there where easier ways.

Where does deep learning fit in the toolbox? Relationship between ANN and DL Rather than 3 to 5 layers, deep learning uses neural networks hundreds of layers deep. How is this achieved? Problem 3. Feature detectors needed to be hand coded or in some other fashion made available to the network. Convolution Pooling Problem 4. Not enough training data for serious problems. Unlimited training data (thank you internet) Problem 5. If you had a serious problem with lots of training data then you needed a super computer cluster. Unlimited processing capacity (thank you computer games industry and graphical processing chips)

Where does deep learning fit in the toolbox Where does deep learning fit in the toolbox? Other machine learning techniques Is deep learning the only game in town? Well… no. Bayesian Networks Classification Rules Linear Models Clustering Ensemble Learning

When is it the right choice? Tens to hundreds of millions of training examples Unknown features Examples Baldi et al, searching for exotic particles in high energy particle physics. Druzhkov, et al, image classification and object detection. Noda, et al, Audio-visual speech recognition. Sheehan, et al, Population genetics Suk, et al, Brain imaging of Alzheimers patients

Are we there yet? How close are we to human level ai? From deep learning to AIXI, “an optimal rational reinforcement learning agent” Hutter (2005)

Questions and Answers Thank you for your time.