Analysis of Classification Algorithms In Handwritten Digit Recognition Logan Helms Jon Daniele.

Slides:

Advertisements

Similar presentations

Patient information extraction in digitized X-ray imagery Hsien-Huang P. Wu Department of Electrical Engineering, National Yunlin University of Science.

Advertisements

Face Recognition: A Convolutional Neural Network Approach

Page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401)

Neural Networks: Backpropagation algorithm Data Mining and Semantic Web University of Belgrade School of Electrical Engineering Chair of Computer Engineering.

Machine Learning Lecture 4 Multilayer Perceptrons G53MLE | Machine Learning | Dr Guoping Qiu1.

Analysis of Classification Algorithms in Handwriting Pattern Recognition Logan Helms Jon Daniele.

Tiled Convolutional Neural Networks TICA Speedup Results on the CIFAR-10 dataset Motivation Pretraining with Topographic ICA References [1] Y. LeCun, L.

Thesis title: “Studies in Pattern Classification – Biological Modeling, Uncertainty Reasoning, and Statistical Learning” 3 parts: (1)Handwritten Digit.

CS771 Machine Learning : Tools, Techniques & Application Gaurav Krishna Y Harshit Maheshwari Pulkit Jain Sayantan Marik

Handwritten Character Recognition Using Artificial Neural Networks Shimie Atkins & Daniel Marco Supervisor: Johanan Erez Technion - Israel Institute of.

Aula 5 Alguns Exemplos PMR5406 Redes Neurais e Lógica Fuzzy.

CS 590M Fall 2001: Security Issues in Data Mining Lecture 3: Classification.

Rutgers CS440, Fall 2003 Neural networks Reading: Ch. 20, Sec. 5, AIMA 2 nd Ed.

Document Classification Comparison Evangel Sarwar, Josh Woolever, Rebecca Zimmerman.

Classifiers for Recognition Reading: Chapter 22 (skip 22.3) Slide credits for this chapter: Frank Dellaert, Forsyth & Ponce, Paul Viola, Christopher Rasmussen.

Text Classification: An Implementation Project Prerak Sanghvi Computer Science and Engineering Department State University of New York at Buffalo.

Presenting: Itai Avron Supervisor: Chen Koren Characterization Presentation Spring 2005 Implementation of Artificial Intelligence System on FPGA.

How does computer know what is spam and what is ham?

Convolutional Neural Networks for Image Processing with Applications in Mobile Robotics By, Sruthi Moola.

West Virginia University

Presented by: Kamakhaya Argulewar Guided by: Prof. Shweta V. Jain

A Genetic Algorithms Approach to Feature Subset Selection Problem by Hasan Doğu TAŞKIRAN CS 550 – Machine Learning Workshop Department of Computer Engineering.

Hurieh Khalajzadeh Mohammad Mansouri Mohammad Teshnehlab

Human Gesture Recognition Using Kinect Camera Presented by Carolina Vettorazzo and Diego Santo Orasa Patsadu, Chakarida Nukoolkit and Bunthit Watanapa.

Neural Networks Chapter 6 Joost N. Kok Universiteit Leiden.

Explorations in Neural Networks Tianhui Cai Period 3.

COMPARISON OF IMAGE ANALYSIS FOR THAI HANDWRITTEN CHARACTER RECOGNITION Olarik Surinta, chatklaw Jareanpon Department of Management Information System.

Yang, Luyu.  Postal service for sorting mails by the postal code written on the envelop  Bank system for processing checks by reading the amount of.

Dünaamiliste süsteemide modelleerimine Identification for control in a non- linear system world Eduard Petlenkov, Automaatikainstituut, 2013.

Handwritten Recognition with Neural Network Chatklaw Jareanpon, Olarik Surinta Mahasarakham University.

1 Pattern Recognition Pattern recognition is: 1. A research area in which patterns in data are found, recognized, discovered, …whatever. 2. A catchall.

Handwritten Hindi Numerals Recognition Kritika Singh Akarshan Sarkar Mentor- Prof. Amitabha Mukerjee.

Logan Lebanoff Mentor: Haroon Idrees. Two-layer method  Trying a method that will have two layers of neural networks.

EE459 Neural Networks Examples of using Neural Networks Kasin Prakobwaitayakit Department of Electrical Engineering Chiangmai University.

School of Engineering and Computer Science Victoria University of Wellington Copyright: Peter Andreae, VUW Image Recognition COMP # 18.

CSC321: Introduction to Neural Networks and Machine Learning Lecture 19: Learning Restricted Boltzmann Machines Geoffrey Hinton.

Neural Networks Vladimir Pleskonjić 3188/ /20 Vladimir Pleskonjić General Feedforward neural networks Inputs are numeric features Outputs are in.

Hand-written character recognition

Each neuron has a threshold value Each neuron has weighted inputs from other neurons The input signals form a weighted sum If the activation level exceeds.

Convolutional Restricted Boltzmann Machines for Feature Learning Mohammad Norouzi Advisor: Dr. Greg Mori Simon Fraser University 27 Nov

Convolutional Neural Network

1 Restricted Boltzmann Machines and Applications Pattern Recognition (IC6304) [Presentation Date: ] [ Ph.D Candidate,

1 Convolutional neural networks Abin - Roozgard. 2  Introduction  Drawbacks of previous neural networks  Convolutional neural networks  LeNet 5 

Feasibility of Using Machine Learning Algorithms to Determine Future Price Points of Stocks By: Alexander Dumont.

1 A Statistical Matching Method in Wavelet Domain for Handwritten Character Recognition Presented by Te-Wei Chiang July, 2005.

Machine Learning Artificial Neural Networks MPλ ∀ Stergiou Theodoros 1.

A Document-Level Sentiment Analysis Approach Using Artificial Neural Network and Sentiment Lexicons Yan Zhu.

Handwritten Digit Recognition Using Stacked Autoencoders

Deep Learning Amin Sobhani.

Data Mining, Neural Network and Genetic Programming

DeepCount Mark Lenson.

Applications of Deep Learning and how to get started with implementation of deep learning Presentation By : Manaswi Advisor : Dr.Chinmay.

Classification with Perceptrons Reading:

Self organizing networks

CAMCOS Report Day December 9th, 2015 San Jose State University

State-of-the-art face recognition systems

Handwritten Digits Recognition

Outline Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no.

Non-linear hypotheses

شبکه عصبی تنظیم: بهروز نصرالهی-فریده امدادی استاد محترم: سرکار خانم کریمی دانشگاه آزاد اسلامی واحد شهرری.

Introduction to Deep Learning with Keras

network of simple neuron-like computing elements

Convolutional neural networks Abin - Roozgard.

[Figure taken from googleblog

On Convolutional Neural Network

Prepared by: Mahmoud Rafeek Al-Farra

CSC321 Winter 2007 Lecture 21: Some Demonstrations of Restricted Boltzmann Machines Geoffrey Hinton.

Automatic Handwriting Generation

Presentation transcript:

Analysis of Classification Algorithms In Handwritten Digit Recognition Logan Helms Jon Daniele

Classification Algorithms Template Matching Naïve Bayes Classifier Neural Network

Benchmarks 1.Gradient-Based Learning Applied to Document Recognition by LeCun, Y., Bottou, L., Bengio, Y., Haffner, P. 2.Comparison of Machine Learning Classifiers for Recognition of Online and Offline Handwritten Digits by Omidiora, E., Adeyanju, I., Fenwa, O.

MNIST Training Set:60,000 samples Test Set:10,000 samples Accuracy: Number of correctly guessed test samples/ 10,000

NAÏVE BAYES CLASSIFIER

Naïve Bayes Classifier Each pixel value (on/off) is independent of any other pixel value Each pixel has a probability associated with being on or off in any given digit class The probability of each pixel is used to determine the probability of an unknown digit being classified in one of the known classes

Naïve Bayes Classifier Training set: digits Test set: digits Success rate: Abysmal: 08.13% correct classification rate Benchmark: WEKA: Multimodal Naive Bayes: 83.65% 08.13% <<<<<<<<< 83.65%

Naïve Bayes Classifier Challenges: Pixel probabilities change according to the shape of the digit Are pixels the best feature set by which to compare different digits? Input size: 28x28 digit image results 786 pixels Requirements for matrix manipulation

Naïve Bayes Classifier Improvements Discarding extraneous pixel data Pixel values are mainly contained in a 20x20 matrix Using a binary pixel value vs a range of pixel values (0-255) Edge detection Incorporate feature extractor(s) and evaluate images based on those features

NEURAL NETWORK

Neural Network Type: Feed Forward Training: Back-propagation algorithm Response Function: Architectures: NameInput LayerHidden LayerOutput Layer NN NN

Training

NN300 Training time:~17 hours (~52 mins/epoch) Learning rate: EpochRate 1, , 4, , 7, , 10, 11, , 14, 15, 16, 17, 18, 19,

NN1000 Training time:~2.5 days (~3 hrs/epoch) Learning rate: EpochRate 1, , 4, , 7, , 10, 11, , 14, 15, 16, 17, 18, 19,

Results After 20 epochs NetworkAccuracy Benchmark % NN % Benchmark % NetworkAccuracy Benchmark % NN1000-

Benchmark %On MNIST test set as is. 96.4%Generated more training data by using artificial distortions 98.4%When using deslanted images

Future Work Further training of NN300 with the MNIST test set has increased accuracy to 84.01% Experiment with hidden neuron count and multiple hidden layers Research other types of neural networks