Radial Basis Networks: An Implementation of Adaptive Centers Nivas Durairaj ECE539 Final Project.

Slides:

Advertisements

Similar presentations

Computational Intelligence Winter Term 2009/10 Prof. Dr. Günter Rudolph Lehrstuhl für Algorithm Engineering (LS 11) Fakultät für Informatik TU Dortmund.

Advertisements

Computational Intelligence Winter Term 2011/12 Prof. Dr. Günter Rudolph Lehrstuhl für Algorithm Engineering (LS 11) Fakultät für Informatik TU Dortmund.

Computational Intelligence Winter Term 2014/15 Prof. Dr. Günter Rudolph Lehrstuhl für Algorithm Engineering (LS 11) Fakultät für Informatik TU Dortmund.

A Brief Overview of Neural Networks By Rohit Dua, Samuel A. Mulder, Steve E. Watkins, and Donald C. Wunsch.

Computer Vision Lecture 18: Object Recognition II

Support Vector Machines

Adaptive Resonance Theory (ART) networks perform completely unsupervised learning. Their competitive learning algorithm is similar to the first (unsupervised)

Ch. 4: Radial Basis Functions Stephen Marsland, Machine Learning: An Algorithmic Perspective. CRC 2009 based on slides from many Internet sources Longin.

Soft computing Lecture 6 Introduction to neural networks.

RBF Neural Networks x x1 Examples inside circles 1 and 2 are of class +, examples outside both circles are of class – What NN does.

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2010 Shreekanth Mandayam ECE Department Rowan University.

Radial Basis-Function Networks. Back-Propagation Stochastic Back-Propagation Algorithm Step by Step Example Radial Basis-Function Networks Gaussian response.

Radial Basis Functions

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks ECE /ECE Fall 2008 Shreekanth Mandayam ECE Department Rowan University.

I welcome you all to this presentation On: Neural Network Applications Systems Engineering Dept. KFUPM Imran Nadeem & Naveed R. Butt &

Radial Basis Function Networks 표현아 Computer Science, KAIST.

November 2, 2010Neural Networks Lecture 14: Radial Basis Functions 1 Cascade Correlation Weights to each new hidden node are trained to maximize the covariance.

Prediction Networks Prediction –Predict f(t) based on values of f(t – 1), f(t – 2),… –Two NN models: feedforward and recurrent A simple example (section.

Hazırlayan NEURAL NETWORKS Radial Basis Function Networks I PROF. DR. YUSUF OYSAL.

October 28, 2010Neural Networks Lecture 13: Adaptive Networks 1 Adaptive Networks As you know, there is no equation that would tell you the ideal number.

CS Instance Based Learning1 Instance Based Learning.

Aula 4 Radial Basis Function Networks

Radial Basis Function (RBF) Networks

Radial Basis Function G.Anuradha.

Radial-Basis Function Networks

Hazırlayan NEURAL NETWORKS Radial Basis Function Networks II PROF. DR. YUSUF OYSAL.

Radial Basis Function Networks

Face Recognition Using Neural Networks Presented By: Hadis Mohseni Leila Taghavi Atefeh Mirsafian.

Radial Basis Function Networks

Radial Basis Function Networks

Soft Computing Colloquium 2 Selection of neural network, Hybrid neural networks.

Chapter 4 Supervised learning: Multilayer Networks II.

Artificial Neural Networks Shreekanth Mandayam Robi Polikar …… …... … net k   

MML Inference of RBFs Enes Makalic Lloyd Allison Andrew Paplinski.

Explorations in Neural Networks Tianhui Cai Period 3.

Radial Basis Function Networks:

COMPARISON OF IMAGE ANALYSIS FOR THAI HANDWRITTEN CHARACTER RECOGNITION Olarik Surinta, chatklaw Jareanpon Department of Management Information System.

George F Luger ARTIFICIAL INTELLIGENCE 6th edition Structures and Strategies for Complex Problem Solving Machine Learning: Connectionist Luger: Artificial.

Well Log Data Inversion Using Radial Basis Function Network Kou-Yuan Huang, Li-Sheng Weng Department of Computer Science National Chiao Tung University.

So Far……  Clustering basics, necessity for clustering, Usage in various fields : engineering and industrial fields  Properties : hierarchical, flat,

Handwritten Recognition with Neural Network Chatklaw Jareanpon, Olarik Surinta Mahasarakham University.

1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 24 Nov 2, 2005 Nanjing University of Science & Technology.

EE459 Neural Networks Examples of using Neural Networks Kasin Prakobwaitayakit Department of Electrical Engineering Chiangmai University.

ADALINE (ADAptive LInear NEuron) Network and

381 Self Organization Map Learning without Examples.

Radial Basis Function ANN, an alternative to back propagation, uses clustering of examples in the training set.

Semiconductors, BP&A Planning, DREAM PLAN IDEA IMPLEMENTATION.

Computational Intelligence Winter Term 2015/16 Prof. Dr. Günter Rudolph Lehrstuhl für Algorithm Engineering (LS 11) Fakultät für Informatik TU Dortmund.

Computational Intelligence: Methods and Applications Lecture 29 Approximation theory, RBF and SFN networks Włodzisław Duch Dept. of Informatics, UMK Google:

COMP53311 Other Classification Models: Neural Network Prepared by Raymond Wong Some of the notes about Neural Network are borrowed from LW Chan’s notes.

Perceptrons Michael J. Watts

Intelligent Numerical Computation1 Center:Width:.

Deep Belief Network Training Same greedy layer-wise approach First train lowest RBM (h 0 – h 1 ) using RBM update algorithm (note h 0 is x) Freeze weights.

Big data classification using neural network

Chapter 4 Supervised learning: Multilayer Networks II

RADIAL BASIS FUNCTION NEURAL NETWORK DESIGN

Neural Networks Winter-Spring 2014

Adavanced Numerical Computation 2008, AM NDHU

CSC321: Neural Networks Lecture 22 Learning features one layer at a time Geoffrey Hinton.

Other Classification Models: Neural Network

Radial Basis Function G.Anuradha.

Chapter 4 Supervised learning: Multilayer Networks II

LINEAR AND NON-LINEAR CLASSIFICATION USING SVM and KERNELS

Lecture 25 Radial Basis Network (II)

Neuro-Computing Lecture 4 Radial Basis Function Network

A RBF-GA Project By: No name given.

Introduction to Radial Basis Function Networks

Prediction Networks Prediction A simple example (section 3.7.3)

Using Clustering to Make Prediction Intervals For Neural Networks

Computational Intelligence

Presentation transcript:

Radial Basis Networks: An Implementation of Adaptive Centers Nivas Durairaj ECE539 Final Project

Brief Description of RBF Networks Consists of 3 layers (input, hidden, output) Input layer made up of nodes that connect network to environment At input of each neuron (hidden layer), distance between neuron center & input vector is calculated Apply RBF (Gaussian bell function) to form output of the neurons. Output layer is linear and supplies response of network to activation function.

Project Overview Purpose: Develop a Radial Basis Network with a supervised selection of centers Question: Are there any disadvantages or advantages between a fixed center RBF network and an adaptive RBF network? A RBF network with multiple outputs

Adaptation Formulas RBF with supervised selection of centers require the following formulas: 1.Linear Weights (output layer) 2. Positions of centers (hidden layer) 3. Spreads of centers (hidden layer) W: 1x1 T: 1xm vector : mxm matrix M is the feature dimension

Programming Used Matlab to implement RBF Network with Adaptive Centers Sample code for calculation of linear weights given below: %Calculation of linear weights weightdiff=0; for j=1:n g=exp(-0.5((x(j,:)-t(i,:)))*covinv(:,:,i)*((x(j,:)-t(i,:))')); weightdiff = weightdiff + e(j)*g; end w(i)=w(i) - (eta1*weightdiff);

Testing & Comparison Tested Adaptive Center RBF against Fixed Center RBF. Used data for three functions, namely sinusoidal, piecewise-linear, and polynomial functions. Made use of the cost function given below analyze differences between two networks Cost Function where

Sinusoidal Function Testing For fewer radial basis functions, adaptive center RBF network seems to perform a bit better. However, after number of RBFs increase, results in cost function are negligible.

Piecewise Linear Function Testing Adaptive center RBF network performed better till the number of radial basis functions reached 6. I found that at higher numbers of radial basis functions (9 and above), both RBF networks were providing similar approximations of piecewise-linear function.

Polynomial Function Testing The adaptive center RBF network was clearly the winner in the approximation of the polynomial function. Differences in cost function for higher numbers of RBFs were too small for Excel to plot.

Conclusion Results show RBF network with adaptive centers performs slightly better than fixed-center RBF. Advantage of Adaptive RBF: Performs better with fewer RBFs Disadvantage of Adaptive RBF: Takes longer to run. Unless situation is known, one cannot say with certainty that one model is better than other.