MATLAB Implementation of the Optimal Brain Surgeon (OBS) Algorithm

Slides:

Advertisements

Similar presentations

A Brief Overview of Neural Networks By Rohit Dua, Samuel A. Mulder, Steve E. Watkins, and Donald C. Wunsch.

Advertisements

CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.

Brain Damage: Algorithms for Network Pruning Andrew Yip HMC Fall 2003.

Artificial Neural Networks

Artificial Neural Networks - Introduction -

Machine Learning: Connectionist McCulloch-Pitts Neuron Perceptrons Multilayer Networks Support Vector Machines Feedback Networks Hopfield Networks.

Machine Learning Neural Networks

Neural Nets Using Backpropagation Chris Marriott Ryan Shirley CJ Baker Thomas Tannahill.

Supervised learning 1.Early learning algorithms 2.First order gradient methods 3.Second order gradient methods.

Prénom Nom Document Analysis: Artificial Neural Networks Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.

Slide 1 EE3J2 Data Mining EE3J2 Data Mining Lecture 15: Introduction to Artificial Neural Networks Martin Russell.

2806 Neural Computation Multilayer neural networks Lecture Ari Visa.

Presenting: Itai Avron Supervisor: Chen Koren Characterization Presentation Spring 2005 Implementation of Artificial Intelligence System on FPGA.

Improved BP algorithms ( first order gradient method) 1.BP with momentum 2.Delta- bar- delta 3.Decoupled momentum 4.RProp 5.Adaptive BP 6.Trinary BP 7.BP.

Artificial Neural Networks

Introduction to Artificial Neural Network and Fuzzy Systems

Last lecture summary.

Implementation of Nonlinear Conjugate Gradient Method for MLP Matt Peterson ECE 539 December 10, 2001.

Biointelligence Laboratory, Seoul National University

© N. Kasabov Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering, MIT Press, 1996 INFO331 Machine learning. Neural networks. Supervised.

1 Artificial Neural Networks Sanun Srisuk EECP0720 Expert Systems – Artificial Neural Networks.

Waqas Haider Khan Bangyal. Multi-Layer Perceptron (MLP)

11 CSE 4705 Artificial Intelligence Jinbo Bi Department of Computer Science & Engineering

Artificial Neural Network Yalong Li Some slides are from _24_2011_ann.pdf.

Appendix B: An Example of Back-propagation algorithm

Michigan REU Final Presentations, August 10, 2006Matt Jachowski 1 Multivariate Analysis, TMVA, and Artificial Neural Networks Matt Jachowski

From Machine Learning to Deep Learning. Topics that I will Cover (subject to some minor adjustment) Week 2: Introduction to Deep Learning Week 3: Logistic.

A note about gradient descent: Consider the function f(x)=(x-x 0 ) 2 Its derivative is: By gradient descent. x0x0 + -

Last lecture summary. biologically motivated synapses Neuron accumulates (Σ) positive/negative stimuli from other neurons. Then Σ is processed further.

Multi-Layer Perceptron

Non-Bayes classifiers. Linear discriminants, neural networks.

11 1 Backpropagation Multilayer Perceptron R – S 1 – S 2 – S 3 Network.

Intro. ANN & Fuzzy Systems Lecture 14. MLP (VI): Model Selection.

ADALINE (ADAptive LInear NEuron) Network and

An Artificial Neural Network Approach to Surface Waviness Prediction in Surface Finishing Process by Chi Ngo ECE/ME 539 Class Project.

CS621 : Artificial Intelligence

Variations on Backpropagation.

Neural Networks Vladimir Pleskonjić 3188/ /20 Vladimir Pleskonjić General Feedforward neural networks Inputs are numeric features Outputs are in.

Hand-written character recognition

Automatic Screening of Sonar Imagery Using Artificial Intelligence Techniques John Tran.

Intro. ANN & Fuzzy Systems Lecture 28 Modeling (3): Expert System and Reinforcement Learning.

Intro. ANN & Fuzzy Systems Lecture 11. MLP (III): Back-Propagation.

Learning: Neural Networks Artificial Intelligence CMSC February 3, 2005.

Evolutionary Computation Evolving Neural Network Topologies.

Speech Recognition through Neural Networks By Mohammad Usman Afzal Mohammad Waseem.

1 Neural Networks MUMT 611 Philippe Zaborowski April 2005.

ANN-based program for Tablet PC character recognition

Lecture 12. MLP (IV): Programming & Implementation

CSE 473 Introduction to Artificial Intelligence Neural Networks

ECE 539 Final Project Mark Slosarek

Java Implementation of Optimal Brain Surgeon

Going Backwards In The Procedure and Recapitulation of System Identification By Ali Pekcan 65570B.

Classification / Regression Neural Networks 2

Generalization ..

CSC 578 Neural Networks and Deep Learning

Optimal Brain Surgeon Algorithm

Lecture 12. MLP (IV): Programming & Implementation

Lecture 11. MLP (III): Back-Propagation

System Level Diesel Engine Emission Modeling Using Neural Networks

Training a Neural Network

Capabilities of Threshold Neurons

Lecture Notes for Chapter 4 Artificial Neural Networks

Backpropagation.

COSC 4335: Part2: Other Classification Techniques

Neural Network Training

Backpropagation.

CS621: Artificial Intelligence Lecture 22-23: Sigmoid neuron, Backpropagation (Lecture 20 and 21 taken by Anup on Graphical Models) Pushpak Bhattacharyya.

CS621: Artificial Intelligence Lecture 18: Feedforward network contd

Artificial Neural Networks / Spring 2002

CS621: Artificial Intelligence Lecture 17: Feedforward network (lecture 16 was on Adaptive Hypermedia: Debraj, Kekin and Raunak) Pushpak Bhattacharyya.

Presentation transcript:

MATLAB Implementation of the Optimal Brain Surgeon (OBS) Algorithm by Mengu Sukan December 10th, 2001 ECE 539: Intro to Artificial Neural Networks Prof. Yu Hen Hu

Optimal Brain Surgeon (OBS) Algorithm A network pruning technique Minimizes network size while maintaining good performance Advantages of smaller network Calculation efficiency Less affected by training noise, thus better generalization to new data

OBS in words Train multilayer perceptron to minimum square error Calculate 2nd order-derivative of error surface – the inverse Hessian (matrix form) Calculate saliency of each weight (saliency = the increase in error when deleting weight) If saliency small enough, eliminate weight

OBS History Introduced by Hassibi in 1992 NETtalk perceptron (1987) Text-to-speech engine 203 sensory, 80 hidden, 26 output 18,629 weights Pruned to 1560 weights

MATLAB’s Suitability for OBS Matrix based calculations Familiarity with software Availability of network training algorithm