Convolutional Neural Network

Slides:



Advertisements
Similar presentations
Neural Networks and Kernel Methods
Advertisements

Face Recognition: A Convolutional Neural Network Approach
Advanced topics.
Tiled Convolutional Neural Networks TICA Speedup Results on the CIFAR-10 dataset Motivation Pretraining with Topographic ICA References [1] Y. LeCun, L.
Aula 5 Alguns Exemplos PMR5406 Redes Neurais e Lógica Fuzzy.
Spatial Pyramid Pooling in Deep Convolutional
Overview of Back Propagation Algorithm
Face Recognition Using Neural Networks Presented By: Hadis Mohseni Leila Taghavi Atefeh Mirsafian.
What is the Best Multi-Stage Architecture for Object Recognition Kevin Jarrett, Koray Kavukcuoglu, Marc’ Aurelio Ranzato and Yann LeCun Presented by Lingbo.
Convolutional Neural Networks for Image Processing with Applications in Mobile Robotics By, Sruthi Moola.
Hurieh Khalajzadeh Mohammad Mansouri Mohammad Teshnehlab
Building high-level features using large-scale unsupervised learning Anh Nguyen, Bay-yuan Hsu CS290D – Data Mining (Spring 2014) University of California,
Dr. Z. R. Ghassabi Spring 2015 Deep learning for Human action Recognition 1.
LeCun, Bengio, And Hinton doi: /nature14539
ECE 6504: Deep Learning for Perception Dhruv Batra Virginia Tech Topics: –(Finish) Backprop –Convolutional Neural Nets.
Deep Convolutional Nets
Neural Networks Presented by M. Abbasi Course lecturer: Dr.Tohidkhah.
CSC321 Lecture 5 Applying backpropagation to shape recognition Geoffrey Hinton.
CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 6: Applying backpropagation to shape recognition Geoffrey Hinton.
Object Recognizing. Deep Learning Success in 2012 DeepNet and speech processing.
Deep Learning and Deep Reinforcement Learning. Topics 1.Deep learning with convolutional neural networks 2.Learning to play Atari video games with Deep.
1 Convolutional neural networks Abin - Roozgard. 2  Introduction  Drawbacks of previous neural networks  Convolutional neural networks  LeNet 5 
Deep Learning Overview Sources: workshop-tutorial-final.pdf
Xintao Wu University of Arkansas Introduction to Deep Learning 1.
Yann LeCun Other Methods and Applications of Deep Learning Yann Le Cun The Courant Institute of Mathematical Sciences New York University
Big data classification using neural network
Deep Learning Amin Sobhani.
Randomness in Neural Networks
Data Mining, Neural Network and Genetic Programming
Data Mining, Neural Network and Genetic Programming
Jure Zbontar, Yann LeCun
Learning Mid-Level Features For Recognition
COMP24111: Machine Learning and Optimisation
Restricted Boltzmann Machines for Classification
CSCI 5922 Neural Networks and Deep Learning: Convolutional Nets For Image And Speech Processing Mike Mozer Department of Computer Science and Institute.
LECTURE ??: DEEP LEARNING
Multimodal Learning with Deep Boltzmann Machines
Intelligent Information System Lab
Neural networks (3) Regularization Autoencoder
Supervised Training of Deep Networks
Convolution Neural Networks
Deep Learning Qing LU, Siyuan CAO.
Deep Learning Workshop
Adversarially Tuned Scene Generation
Goodfellow: Chap 6 Deep Feedforward Networks
Image Classification.
Convolutional Neural Networks for Visual Tracking
Deep learning Introduction Classes of Deep Learning Networks
Smart Robots, Drones, IoT
Convolutional neural networks Abin - Roozgard.
CSC 578 Neural Networks and Deep Learning
LECTURE 35: Introduction to EEG Processing
Neural Networks Geoff Hulten.
On Convolutional Neural Network
Lecture: Deep Convolutional Neural Networks
LECTURE 33: Alternative OPTIMIZERS
Classification Boundaries
Convolutional Neural Networks
Softmax Classifier.
实习生汇报 ——北邮 张安迪.
Neural networks (3) Regularization Autoencoder
CSCI 5922 Neural Networks and Deep Learning: Convolutional Nets For Image And Speech Processing Mike Mozer Department of Computer Science and Institute.
Deep Learning Authors: Yann LeCun, Yoshua Bengio, Geoffrey Hinton
by Khaled Nasr, Pooja Viswanathan, and Andreas Nieder
Introduction to Neural Networks
Image recognition.
CSC 578 Neural Networks and Deep Learning
Week 7 Presentation Ngoc Ta Aidean Sharghi
Sign Language Recognition With Unsupervised Feature Learning
Presentation transcript:

Convolutional Neural Network

Feature Representation Domain Knowledge Specific for each task Learning Representations Computer Vision, Speech Recognition, Natural Language Processing, Transfer Learning Unsupervised Supervised

Feature representation 5/14/2018 Feature Learning Unlabeled images Learning algorithm Feature representation

Basic Concept of CNN Convolutional neural networks Signal, image, video

Architecture of LeNet Convolutional layers Sub-sampling layers Fully-connected layers

Convolutional layers Biologically-inspired animal visual cortex: the most powerful visual processing system emulate its behavior

Convolution Generating feature maps Different kernels Input ... Input Feature Map

Convolutional kernels Different scales and orientations ... Input Feature Map

Edge Detector Randomly initialized filter uniform distribution [-1/fan-in, 1/fan-in] fan-in: the number of inputs to a hidden unit

Sub-sampling layers Reduce the spatial resolution of each feature map A certain degree of shift and distortion invariance is achieved.

Fully connected layers Correspond to a traditional MLP hidden layer + logistic regression

Softmax Probability: Prediction:

Loss function Likelihood: Negative log-likelihood:

Stochastic gradient descent Average loss: L2 regularization:

SGD Current update: Updating the parameter:

LeNet Example C1 S2 C3 S4 C5

LeNet Example Layer-1, 3, 5 Input

Our architecture

Reference LeCun, Yann. "LeNet-5, convolutional neural networks." Internet: http://yann. lecun. com/exdb/lenet (2013). http://www.cs.nyu.edu/~fergus/tutorials/deep_learning_cvpr12 /fergus_dl_tutorial_final.pptx https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=4&cad=rja&u act=8&ved=0ahUKEwjyxJyuwcbKAhVE4D4KHeITCEAQFggvMAM&url=http%3A%2F%2F ce.sharif.edu%2Fcourses%2F85-86%2F2%2Fce667%2Fresources%2Froot%2F15%2520- %2520Convolutional%2520N.%2520N.%2Fnn.ppt&usg=AFQjCNH9- tf2cj6doUAKQwDMctuUG7TBdQ&sig2=zHQlMhh4IDy87TB8YgWTawUnsupervised