1. Convolutional Neural Network

2. Feature Representation
Domain Knowledge
Specific for each task
Learning Representations
Computer Vision, Speech Recognition, Natural Language Processing, Transfer Learning
Unsupervised
Supervised

3. Feature representation
5/14/2018
Feature Learning
Unlabeled images
Learning
algorithm
Feature representation

4. Basic Concept of CNN Convolutional neural networks
Signal, image, video

5. Architecture of LeNet Convolutional layers Sub-sampling layers
Fully-connected layers

6. Convolutional layers Biologically-inspired
animal visual cortex: the most powerful visual processing system
emulate its behavior

7. Convolution Generating feature maps Different kernels Input
...
Input
Feature Map

8. Convolutional kernels
Different scales and orientations
...
Input
Feature Map

9. Edge Detector Randomly initialized filter
uniform distribution [-1/fan-in, 1/fan-in]
fan-in: the number of inputs to a hidden unit

10. Sub-sampling layers Reduce the spatial resolution of each feature map
A certain degree of shift and distortion invariance is achieved.

11. Fully connected layers
Correspond to a traditional MLP
hidden layer + logistic regression

12. Softmax
Probability:
Prediction:

13. Loss function
Likelihood:
Negative log-likelihood:

14. Stochastic gradient descent
Average loss:
L2 regularization:

15. SGD
Current update:
Updating the parameter:

16. LeNet Example
C1 S2 C3 S4 C5

17. LeNet Example
Layer-1, 3, 5
Input

18. Our architecture

19. Reference
LeCun, Yann. "LeNet-5, convolutional neural networks." Internet: lecun. com/exdb/lenet (2013).
/fergus_dl_tutorial_final.pptx
act=8&ved=0ahUKEwjyxJyuwcbKAhVE4D4KHeITCEAQFggvMAM&url=http%3A%2F%2F ce.sharif.edu%2Fcourses%2F85-86%2F2%2Fce667%2Fresources%2Froot%2F15%2520- %2520Convolutional%2520N.%2520N.%2Fnn.ppt&usg=AFQjCNH9- tf2cj6doUAKQwDMctuUG7TBdQ&sig2=zHQlMhh4IDy87TB8YgWTawUnsupervised