1. S. Mandayam/ ANN/ECE Dept./Rowan University Shreekanth Mandayam ECE Department Rowan University http://engineering.rowan.edu/~shreek/fall08/ann/ Lecture 11 November 17, 2008 Artificial Neural Networks ECE.09.454/ECE.09.560 Fall 2008

2. S. Mandayam/ ANN/ECE Dept./Rowan UniversityPlan ANN Pre-processing Feature Extraction Approximation Theory Universal approximation Final Project Discussion

3. S. Mandayam/ ANN/ECE Dept./Rowan University Feature Extraction Objective: Increase information content Decrease vector length Parametric invariance Invariance by structure Invariance by training Invariance by transformation

4. S. Mandayam/ ANN/ECE Dept./Rowan University Approximation Theory: Distance Measures Supremum Norm Infimum Norm Mean Squared Norm

5. S. Mandayam/ ANN/ECE Dept./Rowan University Recall: Metric Space Reflexivity Positivity Symmetry Triangle Inequality

6. S. Mandayam/ ANN/ECE Dept./Rowan University Approximation Theory: Terminology Compactness Closure K F

7. S. Mandayam/ ANN/ECE Dept./Rowan University Approximation Theory: Terminology Best Approximation Existence Set E M u0u0 f min E M u0u0 ALL f min

8. S. Mandayam/ ANN/ECE Dept./Rowan University Approximation Theory: Terminology Denseness F f g 

9. S. Mandayam/ ANN/ECE Dept./Rowan University Fundamental Problem E M ? g min Theorem 1: Every compact set is an existence set (Cheney) Theorem 2: Every existence set is a closed set (Braess)

10. S. Mandayam/ ANN/ECE Dept./Rowan University Stone-Weierstrass Theorem Identity Separability Algebraic Closure F f g  x 1 x1x1 f(x 1 ) x2x2 f(x 2 ) F af+bg

11. S. Mandayam/ ANN/ECE Dept./Rowan University Final Project Discussion