Introduction to Deep Learning with Keras

Machine Learning on Images Given examples, can we train a computer to do: Source - http://cs231n.github.io/

Machine Learning on Images Source - http://cs231n.github.io/

Artificial Neural Networks Signal goes in, via input layer Weighted links transfer input values to neurons in hidden layers Signals are summed at hidden neurons and passed through transfer/activation function Processed signal arrives at output layer Decisions made using output signal(s)

What’s in an (Artificial) Neuron? Source - http://cs231n.github.io/

More layers can encapsulate more knowledge Why Deep Learning? More layers can encapsulate more knowledge More weights to train – need more data, need more computation

Weights encapsulate the knowledge of a network Where’s the Knowledge? Weights encapsulate the knowledge of a network Network learns using an algorithm that optimize weights given examples Back propagation is commonly used – learn weights from examples using some linear algebra

Classification Example – MNIST Which digit is it?

DEEP Different Classifiers on the MNIST Database Why Deep Learning? Type Classifier Distortion Preprocessing Error rate (%) Linear classifier Pairwise linear classifier None Deskewing 7.6[9] K-Nearest Neighbors K-NN with non-linear deformation (P2DHMDM) Shiftable edges 0.52[18] Boosted Stumps Product of stumps on Haar features Haar features 0.87[19] Non-linear classifier 40 PCA + quadratic classifier 3.3[9] Support vector machine Virtual SVM, deg-9 poly, 2-pixel jittered 0.56[20] Neural network 2-layer 784-800-10 1.6[21] elastic distortions 0.7[21] Deep neural network 6-layer 784-2500-2000-1500-1000-500-10 0.35[22] Convolutional neural network 6-layer 784-40-80-500-1000-2000-10 Expansion of the training data 0.31[15] 6-layer 784-50-100-500-1000-10-10 0.27[16] Committee of 35 CNNs, 1-20-P-40-P-150-10 Width normalizations 0.23[8] Committee of 5 CNNs, 6-layer 784-50-100-500-1000-10-10 0.21[17] DEEP https://en.wikipedia.org/wiki/MNIST_database

Convolutional Neural Networks Complex architectures, many layers – really good for image recognition tasks Lots of computing power needed to do the training mathematics!

GPU cards are exceptionally well suited to Neural Network Mathematics GPUs to the Rescue! GPU cards are exceptionally well suited to Neural Network Mathematics Orders of magnitude faster than CPU-based training

https://keras.io High-level, open-source Python API “Being able to go from idea to result with the least possible delay is key to doing good research” Interface for TensorFlow, Microsoft Cognitive Toolkit, and Theano

$ module load python/3.6.4-anaconda Installing a Conda Environment for Keras and TensorFlow with Jupyter Support $ module load python/3.6.4-anaconda $ conda create --name py3.6-keras python=3.6 ipykernel keras tensorflow-gpu pillow matplotlib $ ipython kernel install --user --name py3.6-tfgpu --display-name=“Keras (GPU)"