Optimal Brain Surgeon Algorithm

Slides:

Advertisements

Similar presentations

Neural Network I Week 7 1. Team Homework Assignment #9 Read pp. 327 – 334 and the Week 7 slide. Design a neural network for XOR (Exclusive OR) Explore.

Advertisements

Artificial Neural Networks - Introduction -

Neural Networks Dr. Peter Phillips. Neural Networks What are Neural Networks Where can neural networks be used Examples Recognition systems (Voice, Signature,

1 Automated Feature Abstraction of the fMRI Signal using Neural Network Clustering Techniques Stefan Niculescu and Tom Mitchell Siemens Medical Solutions,

An Introduction To The Backpropagation Algorithm Who gets the credit?

Last lecture summary.

Traffic Sign Recognition Using Artificial Neural Network Radi Bekker

Artificial Neural Networks

Artificial Neural Networks An Overview and Analysis.

Appendix B: An Example of Back-propagation algorithm

Back-Propagation MLP Neural Network Optimizer ECE 539 Andrew Beckwith.

Artificial Neural Network Supervised Learning دكترمحسن كاهاني

NEURAL NETWORKS FOR DATA MINING

LINEAR CLASSIFICATION. Biological inspirations  Some numbers…  The human brain contains about 10 billion nerve cells ( neurons )  Each neuron is connected.

Radial Basis Function Networks:

Applying Neural Networks Michael J. Watts

Neural and Evolutionary Computing - Lecture 9 1 Evolutionary Neural Networks Design  Motivation  Evolutionary training  Evolutionary design of the architecture.

CS344: Introduction to Artificial Intelligence (associated lab: CS386) Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 31: Feedforward N/W; sigmoid.

Last lecture summary. biologically motivated synapses Neuron accumulates (Σ) positive/negative stimuli from other neurons. Then Σ is processed further.

Soft Computing Lecture 8 Using of perceptron for image recognition and forecasting.

Solving Polynomial Functions involving Complex Numbers.

Implementing Local Relative Sensitivity Pruning Paul Victorey.

Intro. ANN & Fuzzy Systems Lecture 11. MLP (III): Back-Propagation.

An Introduction To The Backpropagation Algorithm.

Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 6: Artificial Neural Networks for Data Mining.

Neural Networks - Berrin Yanıkoğlu1 MLP & Backpropagation Issues.

Fundamental ARTIFICIAL NEURAL NETWORK Session 1st

Neural Networks.

Applying Neural Networks

Other Classification Models: Neural Network

MATLAB Implementation of the Optimal Brain Surgeon (OBS) Algorithm

Artificial Intelligence (CS 370D)

Artificial neural networks:

Other Classification Models: Neural Network

Lecture 12. MLP (IV): Programming & Implementation

ECE 539 Final Project Mark Slosarek

with Daniel L. Silver, Ph.D. Christian Frey, BBA April 11-12, 2017

MLP Based Feedback System for Gas Valve Control in a Madison Symmetric Torus Andrew Seltzman Dec 14, 2010.

Java Implementation of Optimal Brain Surgeon

Going Backwards In The Procedure and Recapitulation of System Identification By Ali Pekcan 65570B.

CS621: Artificial Intelligence

Hebb and Perceptron.

BACKPROPOGATION OF NETWORKS

Lecture 12. MLP (IV): Programming & Implementation

Lecture 11. MLP (III): Back-Propagation

Training a Neural Network

Brute Force Approaches

OVERVIEW OF BIOLOGICAL NEURONS

BACKPROPAGATION Multlayer Network.

XOR problem Input 2 Input 1

An Introduction To The Backpropagation Algorithm

The use of Neural Networks to schedule flow-shop with dynamic job arrival ‘A Multi-Neural Network Learning for lot Sizing and Sequencing on a Flow-Shop’

Neural Networks Geoff Hulten.

Artificial Intelligence Lecture No. 28

Capabilities of Threshold Neurons

Neural Networks II Chen Gao Virginia Tech ECE-5424G / CS-5824

Artificial neurons Nisheeth 10th January 2019.

Artificial Intelligence 12. Two Layer ANNs

2. Matrix-Vector Formulation of Backpropagation Learning

Artificial Neural Networks

Structure of a typical back-propagated multilayered perceptron used in this study. Structure of a typical back-propagated multilayered perceptron used.

Computer Vision Lecture 19: Object Recognition III

Neural Network Training

ARTIFICIAL NEURAL networks.

Neural Networks II Chen Gao Virginia Tech ECE-5424G / CS-5824

Introduction to Neural Network

CS621: Artificial Intelligence Lecture 22-23: Sigmoid neuron, Backpropagation (Lecture 20 and 21 taken by Anup on Graphical Models) Pushpak Bhattacharyya.

Sanguthevar Rajasekaran University of Connecticut

CS621: Artificial Intelligence Lecture 18: Feedforward network contd

CSC 578 Neural Networks and Deep Learning

Presentation transcript:

Optimal Brain Surgeon Algorithm Payne Y. Chang

What is OBS(Optical Brain Surgeon) ? To solve real-world problems with ANN : Usually Need Highly connected Larger Not alwayse good OBS algorithm An approach to prune the ANN Goal : 1. Set some synaptic weights to zero 2. Modify the remaining weights 3. Maintain good performance

How does OBS work ? 1. Train the MLP 2. Calculate the saliency ( Si ) of each synaptic weight ( wi ) Si : The increase in the cost function from the deletion of wi 3. If Si is smaller than a threshold, delete wi 4. Go to step 2 5. Stop when no more weights can be deleted

Program : OBS.exe 1. C++ 2. FLTK Input Output Hidden N Bias (1) Deleted w

Results and Summary 1. Efficient ( Time & Storage Saving ) Learning Rate = 0.01 Momentum = 0.8 Activation function : Hyperbolic tangen 1. Efficient ( Time & Storage Saving ) 2. Better Generalization