Download presentation
Presentation is loading. Please wait.
1
S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Neural Networks 0909.560.01/0909.454.01 Fall 2004 Shreekanth Mandayam ECE Department Rowan University http://engineering.rowan.edu/~shreek/fall04/ann/ Lecture 7 October 25, 2004
2
S. Mandayam/ ANN/ECE Dept./Rowan UniversityPlan RBF Design Issues K-means clustering algorithm Adaptive techniques ANN Design Issues Input data processing Selection of training and test data - cross-validation Pre-processing: Feature Extraction Approximation Theory Universal approximation Lab Project 3
3
S. Mandayam/ ANN/ECE Dept./Rowan University RBF Network RBF Network 1 1 1 x1x1 x2x2 x3x3 y1y1 y2y2 1 w ij Input Layer Hidden Layer Output Layer Inputs Outputs -55 0 0.5 1 (t) t
4
S. Mandayam/ ANN/ECE Dept./Rowan University RBF - Center Selection x1x1 x2x2 Data points Centers
5
S. Mandayam/ ANN/ECE Dept./Rowan University K-means Clustering Algorithm N data points, x i ; i = 1, 2, …, N At time-index, n, define K clusters with cluster centers c j (n) ; j = 1, 2, …, K Initialization: At n=0, let c j (n) = x j ; j = 1, 2, …, K (i.e. choose the first K data points as cluster centers) Compute the Euclidean distance of each data point from the cluster center, d(x j, c j (n) ) = d ij Assign x j to cluster c j (n) if d ij = min i,j {d ij }; i = 1, 2, …, N, j = 1, 2, …, K For each cluster j = 1, 2, …, K, update the cluster center c j (n+1) = mean {x j c j (n) } Repeat until ||c j (n+1) - c j (n) || <
6
S. Mandayam/ ANN/ECE Dept./Rowan University Selection of Training and Test Data: Method of Cross-Validation Train Test Train TestTrain TestTrain TestTrain Trial 1 Trial 2 Trial 3 Trial 4 Vary network parameters until total mean squared error is minimum for all trials Find network with the least mean squared output error
7
S. Mandayam/ ANN/ECE Dept./Rowan University Feature Extraction Objective: Increase information content Decrease vector length Parametric invariance Invariance by structure Invariance by training Invariance by transformation
8
S. Mandayam/ ANN/ECE Dept./Rowan University Approximation Theory: Distance Measures Supremum Norm Infimum Norm Mean Squared Norm
9
S. Mandayam/ ANN/ECE Dept./Rowan University Recall: Metric Space Reflexivity Positivity Symmetry Triangle Inequality
10
S. Mandayam/ ANN/ECE Dept./Rowan University Approximation Theory: Terminology Compactness Closure K F
11
S. Mandayam/ ANN/ECE Dept./Rowan University Approximation Theory: Terminology Best Approximation Existence Set E M u0u0 f min E M u0u0 ALL f min
12
S. Mandayam/ ANN/ECE Dept./Rowan University Approximation Theory: Terminology Denseness F f g
13
S. Mandayam/ ANN/ECE Dept./Rowan University Fundamental Problem E M ? g min Theorem 1: Every compact set is an existence set (Cheney) Theorem 2: Every existence set is a closed set (Braess)
14
S. Mandayam/ ANN/ECE Dept./Rowan University Stone-Weierstrass Theorem Identity Separability Algebraic Closure F f g x 1 x1x1 f(x 1 ) x2x2 f(x 2 ) F af+bg
15
S. Mandayam/ ANN/ECE Dept./Rowan University Lab Project 3: Radial Basis Function Neural Networks http://engineering.rowan.edu/~shreek/fall04/ann/lab3.html
16
S. Mandayam/ ANN/ECE Dept./Rowan UniversitySummary
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.