Convolutional Neural Network
Feature Representation Domain Knowledge Specific for each task Learning Representations Computer Vision, Speech Recognition, Natural Language Processing, Transfer Learning Unsupervised Supervised
Feature representation 5/14/2018 Feature Learning Unlabeled images Learning algorithm Feature representation
Basic Concept of CNN Convolutional neural networks Signal, image, video
Architecture of LeNet Convolutional layers Sub-sampling layers Fully-connected layers
Convolutional layers Biologically-inspired animal visual cortex: the most powerful visual processing system emulate its behavior
Convolution Generating feature maps Different kernels Input ... Input Feature Map
Convolutional kernels Different scales and orientations ... Input Feature Map
Edge Detector Randomly initialized filter uniform distribution [-1/fan-in, 1/fan-in] fan-in: the number of inputs to a hidden unit
Sub-sampling layers Reduce the spatial resolution of each feature map A certain degree of shift and distortion invariance is achieved.
Fully connected layers Correspond to a traditional MLP hidden layer + logistic regression
Softmax Probability: Prediction:
Loss function Likelihood: Negative log-likelihood:
Stochastic gradient descent Average loss: L2 regularization:
SGD Current update: Updating the parameter:
LeNet Example C1 S2 C3 S4 C5
LeNet Example Layer-1, 3, 5 Input
Our architecture
Reference LeCun, Yann. "LeNet-5, convolutional neural networks." Internet: http://yann. lecun. com/exdb/lenet (2013). http://www.cs.nyu.edu/~fergus/tutorials/deep_learning_cvpr12 /fergus_dl_tutorial_final.pptx https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=4&cad=rja&u act=8&ved=0ahUKEwjyxJyuwcbKAhVE4D4KHeITCEAQFggvMAM&url=http%3A%2F%2F ce.sharif.edu%2Fcourses%2F85-86%2F2%2Fce667%2Fresources%2Froot%2F15%2520- %2520Convolutional%2520N.%2520N.%2Fnn.ppt&usg=AFQjCNH9- tf2cj6doUAKQwDMctuUG7TBdQ&sig2=zHQlMhh4IDy87TB8YgWTawUnsupervised