Download presentation
Presentation is loading. Please wait.
1
Handwritten Digits Recognition
MNIST Based Handwritten Digits Recognition Li Linjun ECE 539 Introduction to Artificial Neural Network and Fuzzy Systems
2
Study Result TABLE OF CONTENTS Introduction Approaches Implementation
Problems Conclusion Next to do Reference
3
INTRODUCTION Widely Studied LeCun’s Benchmark Application Problem Description 60,000 images Fit in 28x28 0~9 Recognition
4
APPROACHES Principal: Studying the Data Trade off: Complexity and Accuracy Devils in the Details: Software Engineering KNN SLP&MLP CNN LeNet-5 VGG-5 VGG-16 Committee/Voting
5
IMPLEMENTATION KNN L1 L2 L3 Norm,5Neighbors Accuracy: 97.0% CNN LeNet % VGG % VGG % Committee 99.80% SLP & MLP Softmax & Cross Entropy Accuracy 91.75% & 94.12%
6
IMPLEMENTATION Change the size of kernel is not useful More convolution layers are helpful
7
PROBLEMS Software Rounding problem Different Frameworks Cross Validation How to -> sklearn Value k For Same Model: Accuracy: Caffe2 and MxNet Easy to use: Tensorflow and Pytorch
8
CONCLUSION Voting/Committee : engineers’ dirty way 99.80% is even better than committee of 35 conv. net with 0.23% error rate on LeCun’s website. The more complex the better might not be true. Devils are in the details, talk is cheap Understanding your data is the most important thing.
9
NEXT TO DO Dive into the deep models Extracted feature for KNN or SVM? Something fun: what about RCNN on MNIST?
10
REFERENCES [1] http://yann.lecun.com/exdb/mnist/
[2] Y. LeCun, L. Bottou, Y. Bengio and P. Haffner: Gradient-Based Learning Applied to Document Recognition, Proceedings of the IEEE, 86(11): , November 1998 [3] [4] Dan Cireşan, Ueli Meier, Juergen Schmidhuber: Multi-column Deep Neural Networks for Image Classification,CVPR,2012 [5] Liu,Qing,Tang Xianlun, Zhang,Na: Structure Optimized Convolutional Neural Network Based on Unsupervised Pre-training, Gongcheng Kexue Yu Jishu/Advanced Engineering Science , 2017, Vol.49, p [6] Quoc V. Le, Navdeep Jaitly, Geoffrey E. Hinton:A Simple Way to Initialize Recurrent Networks of Rectified Linear Units,arxiv: v2 [cs.NE] 7 Apr 2015 [7] Li Deng, Dong Yu, Deep Convex Network: A Scalable Architecture for Speech Pattern Classification
![REFERENCES [1]](http://slideplayer.com./slide/14537400/90/images/10/REFERENCES+%5B1%5D.jpg "[2] Y. LeCun, L. Bottou, Y. Bengio and P. Haffner: Gradient-Based Learning Applied to Document Recognition, Proceedings of the IEEE, 86(11): , November [3] [4] Dan Cireşan, Ueli Meier, Juergen Schmidhuber: Multi-column Deep Neural Networks for Image Classification,CVPR,2012. [5] Liu,Qing,Tang Xianlun, Zhang,Na: Structure Optimized Convolutional Neural Network Based on Unsupervised Pre-training, Gongcheng Kexue Yu Jishu/Advanced Engineering Science , 2017, Vol.49, p [6] Quoc V. Le, Navdeep Jaitly, Geoffrey E. Hinton:A Simple Way to Initialize Recurrent Networks of Rectified Linear Units,arxiv: v2 [cs.NE] 7 Apr [7] Li Deng, Dong Yu, Deep Convex Network: A Scalable Architecture for Speech Pattern Classification.")
Similar presentations
![Tiled Convolutional Neural Networks TICA Speedup Results on the CIFAR-10 dataset Motivation Pretraining with Topographic ICA References [1] Y. LeCun, L.](/14/4420869/big_thumb.jpg "Tiled Convolutional Neural Networks TICA Speedup Results on the CIFAR-10 dataset Motivation Pretraining with Topographic ICA References [1] Y. LeCun, L.>")
![AN ANALYSIS OF SINGLE- LAYER NETWORKS IN UNSUPERVISED FEATURE LEARNING [1] Yani Chen 10/14/2014 1.](/17/5275044/big_thumb.jpg "AN ANALYSIS OF SINGLE- LAYER NETWORKS IN UNSUPERVISED FEATURE LEARNING [1] Yani Chen 10/14/2014 1.>")
