Intro. ANN & Fuzzy Systems Lecture 24 Radial Basis Network (I)

Intro. ANN & Fuzzy Systems (C) by Yu Hen Hu 2 Outline Interpolation Problem Formulation Radial Basis Network Type 1

Intro. ANN & Fuzzy Systems (C) by Yu Hen Hu 3 What is Radial Basis Function? RBF is a kernel function that is symmetric w. r. t. origin. Hence its variable is r that is the norm-distance from origin. Examples of RBF

Intro. ANN & Fuzzy Systems (C) by Yu Hen Hu 4 Interpolation Problem Formulation Radial Basis function for interpolation: Given {x i ; 1  i  K} and {d i ; 1  i  K }, find a function F(x) that satisfies the interpolation condition: F(x i ) = d i 1  i  K (1) One possible choice of F(x) is a radial basis function of the following form: (2) where {x i ; 1  i  K } are the centers of the radial basis functions.

Intro. ANN & Fuzzy Systems (C) by Yu Hen Hu 5 Solving Radial Basis Coefficients Substitute (1) into (2), we obtain a set of linear system of equations M w = d(3) where M = [M(i,j), 1  i, j,  K] is the interpolation matrix, M(i,j) =  (||x i – x j ||), w = [w 1, w 2,, w K ] t, and d = [d 1, d 2,, d K ] t. Given M and d, assuming the N centers are distinct, w can be solved as: w = M  1 d if M is non-singular. If the  (r) = (r 2 + c 2 ) –1/2, or  (r) = exp(–r 2 /(2s 2 )), it can further be shown that M is also positive definite.

Intro. ANN & Fuzzy Systems (C) by Yu Hen Hu 6 An Example Let F(–1) = 0.2, F(–0.5) = 0.5, and F(1) = –0.5. Use a triangular radial basis function  (r) = (1–r)[u(r) –u(r –1)] u(r) = 1 if r  0 and = 0 if r < 0. rbfexample1.m

Intro. ANN & Fuzzy Systems (C) by Yu Hen Hu 7 Example continued Use Gaussian rbfs: Parzen window: No weighting, and no target values of F(x) needed.,

Intro. ANN & Fuzzy Systems (C) by Yu Hen Hu 8 Example (Comparison)

Intro. ANN & Fuzzy Systems (C) by Yu Hen Hu 9 Regularization Problem Formulation When there are too many data points, the M matrix may become singular. This is because by impose a rbf to each data point, we have an over-determined system. Regularization is the mathematical tool that addresses this problem. By regularization, we add an additional term to the cost function that represents additional constraints on the solution: Regularization term (e.g.):

Intro. ANN & Fuzzy Systems (C) by Yu Hen Hu 10 Solution to Regularization Problem The solution to this regularization problem is G(x; x i ) is the Green's function corresponding to the self- adjoin differential operator P * P such that P * P G(x; x i ) =  (x – x i ) A solution to the Green function that is of special interests to us is a multi-variate Gaussian function Hence With individual training data substituted into G(x, x i ), a matrix equation (G + I) w = d can be solved for w.

Intro. ANN & Fuzzy Systems (C) by Yu Hen Hu 11 Implementation Consideration However, other radial basis function other than the multi-variate Gaussian rbf can also be used. The regularized F(x) may no longer match data points exactly, but it will be more smooth. The value of is usually determined empirically although generalized cross-validation (GCV) may be applied here.