Introduction to Neural Networks John Paxton Montana State University Summer 2003.

Slides:



Advertisements
Similar presentations
Bioinspired Computing Lecture 16
Advertisements

Pattern Association.
NEURAL NETWORKS Backpropagation Algorithm
Marković Miljan 3139/2011
Data mining in wireless sensor networks based on artificial neural-networks algorithms Authors: Andrea Kulakov and Danco Davcev Presentation by: Niyati.
Neural Networks Chapter 9 Joost N. Kok Universiteit Leiden.
-Artificial Neural Network- Counter Propagation Network
Adaptive Resonance Theory (ART) networks perform completely unsupervised learning. Their competitive learning algorithm is similar to the first (unsupervised)
G5BAIM Artificial Intelligence Methods Graham Kendall Neural Networks.
Financial Informatics –XVI: Supervised Backpropagation Learning
ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.
INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.
Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.
The back-propagation training algorithm
Slides are based on Negnevitsky, Pearson Education, Lecture 8 Artificial neural networks: Unsupervised learning n Introduction n Hebbian learning.
Introduction to Neural Networks John Paxton Montana State University Summer 2003.
Neural Networks Part 4 Dan Simon Cleveland State University 1.
Introduction to Neural Networks John Paxton Montana State University Summer 2003.
-Artificial Neural Network- Adaptive Resonance Theory(ART) 朝陽科技大學 資訊管理系 李麗華 教授.
Chapter 6: Multilayer Neural Networks
Introduction to Neural Networks John Paxton Montana State University Summer 2003.
Before we start ADALINE
Introduction to Neural Networks John Paxton Montana State University Summer 2003.
Data Mining with Neural Networks (HK: Chapter 7.5)
Introduction to Neural Networks John Paxton Montana State University Summer 2003.
ART (Adaptive Resonance Theory)
Neural Networks An Introduction.
Introduction to Neural Networks John Paxton Montana State University Summer 2003.
Neural Networks Lecture 17: Self-Organizing Maps
-Artificial Neural Network- Chapter 3 Perceptron 朝陽科技大學 資訊管理系 李麗華教授.
CS623: Introduction to Computing with Neural Nets (lecture-10) Pushpak Bhattacharyya Computer Science and Engineering Department IIT Bombay.
Artificial Neural Networks Dr. Abdul Basit Siddiqui Assistant Professor FURC.
1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 23 Nov 2, 2005 Nanjing University of Science & Technology.
Chapter 3 Neural Network Xiu-jun GONG (Ph. D) School of Computer Science and Technology, Tianjin University
Introduction to Artificial Neural Network Models Angshuman Saha Image Source: ww.physiol.ucl.ac.uk/fedwards/ ca1%20neuron.jpg.
Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.
Adaptive Resonance Theory
Artificial Intelligence Methods Neural Networks Lecture 4 Rakesh K. Bissoondeeal Rakesh K. Bissoondeeal.
1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 21 Oct 28, 2005 Nanjing University of Science & Technology.
IE 585 Associative Network. 2 Associative Memory NN Single-layer net in which the weights are determined in such a way that the net can store a set of.
Chapter 5. Adaptive Resonance Theory (ART) ART1: for binary patterns; ART2: for continuous patterns Motivations: Previous methods have the following problem:
Soft Computing Lecture 14 Clustering and model ART.
Ming-Feng Yeh1 CHAPTER 16 AdaptiveResonanceTheory.
Activations, attractors, and associators Jaap Murre Universiteit van Amsterdam
Linear Discrimination Reading: Chapter 2 of textbook.
UNSUPERVISED LEARNING NETWORKS
Non-Bayes classifiers. Linear discriminants, neural networks.
1 Adaptive Resonance Theory. 2 INTRODUCTION Adaptive resonance theory (ART) was developed by Carpenter and Grossberg[1987a] ART refers to the class of.
381 Self Organization Map Learning without Examples.
Introduction to Neural Networks Introduction to Neural Networks Applied to OCR and Speech Recognition An actual neuron A crude model of a neuron Computational.
What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship.
Unsupervised Learning Networks 主講人 : 虞台文. Content Introduction Important Unsupervised Learning NNs – Hamming Networks – Kohonen’s Self-Organizing Feature.
Neural Networks Teacher: Elena Marchiori R4.47 Assistant: Kees Jong S2.22
Image Source: ww.physiol.ucl.ac.uk/fedwards/ ca1%20neuron.jpg
Intelligent Database Systems Lab Advisor : Dr. Hsu Graduate : Yu Cheng Chen Author : Yongqiang Cao Jianhong Wu 國立雲林科技大學 National Yunlin University of Science.
Artificial Intelligence Methods Neural Networks Lecture 3 Rakesh K. Bissoondeeal Rakesh K. Bissoondeeal.
Pattern Recognition Lecture 20: Neural Networks 3 Dr. Richard Spillman Pacific Lutheran University.
J. Kubalík, Gerstner Laboratory for Intelligent Decision Making and Control Artificial Neural Networks II - Outline Cascade Nets and Cascade-Correlation.
CSE343/543 Machine Learning Mayank Vatsa Lecture slides are prepared using several teaching resources and no authorship is claimed for any slides.
Machine Learning 12. Local Models.
Chapter 5 Unsupervised learning
Adaptive Resonance Theory (ART)
Counter propagation network (CPN) (§ 5.3)
Dr. Unnikrishnan P.C. Professor, EEE
Adaptive Resonance Theory
Adaptive Resonance Theory
Dr. Unnikrishnan P.C. Professor, EEE
Artificial Neural Networks
Pattern Recognition: Statistical and Neural
Presentation transcript:

Introduction to Neural Networks John Paxton Montana State University Summer 2003

Chapter 5: Adaptive Resonance Theory 1987, Carpenter and Grossberg ART1: clusters binary vectors ART2: clusters continuous vectors

General Weights on a cluster unit can be considered to be a prototype pattern Relative similarity is used instead of an absolute difference. Thus, a difference of 1 in a vector with only a few non-zero components becomes more significant.

General Training examples may be presented several times. Training examples may be presented in any order. An example might change clusters. Nets are stable (patterns don’t oscillate). Nets are plastic (examples can be added).

Architecture Input layer (x i ) Output layer or cluster layer – competitive (y i ) Units in the output layer can be active, inactive, or inhibited.

Sample Network t (top down weight), b (bottom up weight) xnxn x1x1 ymym y1y1 b nm t 11

Nomenclature b ij : bottom up weight t ij : top down weight s: input vector x: activation vector n: number of components in input vector m: maximum number of clusters || x ||:  x i p: vigilance parameter

Training Algorithm 1.L > 1, 0 < p <= 1 t ji (0) = 1 0 < b ij (0) < L / (L – 1 + n) 2.while stopping criterion is false do steps 3 – 12 3.for each training example do steps

Training Algorithm 4.y i = 0 5.compute || s || 6.x i = s i 7.if y j (do for each j) is not inhibited then y j =  b ij x i 8.find largest y j that is not inhibited 9.x i = s i * t ji

Training Algorithm 10.compute || x || 11.if || x || / || s || < p then y j = -1, go to step 8 12.b ij = L x i / ( L – 1 + || x || ) t ji = x i

Possible Stopping Criterion No weight changes. Maximum number of epochs reached.

What Happens If All Units Are Inhibited? Lower p. Add a cluster unit. Throw out the current input as an outlier.

Example n = 4 m = 3 p = 0.4 (low vigilance) L = 2 b ij (0) = 1/(1 + n) = 0.2 t ji (0) = 1 x2x2 x3x3 x4x4 x1x1 y3y3 y2y2 y1y1

Example 3.input vector ( ) 4.y i = 0 5.|| s || = 2 6.x = ( ) 7.y 1 =.2(1) +.2(1) +.2(0) +.2(0) = 0.4 y 2 = y 3 = y 4 = 0.4

Example 8.j = 1 (use lowest index to break ties) 9.x 1 = s 1 * t 11 = 1 * 1 = 1 x 2 = s 2 * t 12 = 1 * 1 = 1 x 3 = s 3 * t 13 = 0 * 1 = 0 x 4 = s 4 * t 14 = 0 * 1 = 0 10.|| x || = 2 11.|| x || / || s || = 1 >= 0.4

Example 12.b 11 = 2 * x i / ( || x ||) = 2 * 1 / (1 + 2) =.667 b 21 =.667 b 31 = b 41 = 0 t 11 = x 1 = 1 t 12 = 1 t 13 = t 14 = 0

Exercise Show the network after the training example ( ) is processed.

Observations Typically, stable weight matrices are obtained quickly. The cluster units are all topologically independent of one another. We have just looked at the fast learning version of ART1. There is also a slow learning version that updates just one weight per training example.