A Fast Fixed-Point Algorithm for Independent Component Analysis Neural Computation, 9:1483-1492, 1997 A. Hyvarinen, E. Oja Summarized by Seong-woo Chung 2001.6.8

(C) 2001, SNU CSE Biointelligence Lab Introduction Independent Component Analysis (ICA) is to express a set of random variables as linear combinations of statistically independent component variables Two applications of ICA are blind source separation and feature extraction (C) 2001, SNU CSE Biointelligence Lab

(C) 2001, SNU CSE Biointelligence Lab Introduction(2) In the simplest form of ICA Observable m scalar variables v1, v2, …, vm n unknown independent components s1, s2, …, sn n < m v (vector) is linear combinations of s (vector) with an unknown m×n matrix (called the mixing matrix) Can only estimate non-Gaussian independent components (except if just one of the independent components is Gaussian) Defines that the independent components si have unit variance (C) 2001, SNU CSE Biointelligence Lab

(C) 2001, SNU CSE Biointelligence Lab Introduction(3) The problem of estimating the matrix A can be simplified by performing sphering or prewhitening of the data v v is linearly transformed to a vector x = Mv such that its elements xi are mutually uncorrelated and all have unit variance Thus the correlation matrix of x equals unity: B=MA is an orthogonal matrix due to assumptions on the components si (C) 2001, SNU CSE Biointelligence Lab

ICA by Kurtosis Minimization and Maximization ICA use the fourth-order cumulant or kurtosis of the signals, defined for a zero-mean random variable v as For a Gaussian, kurtosis is zero, for densities peaked at zero, positive, and for flatter densities, negative To find w satisfing , following object function have to be minimized or maximized (C) 2001, SNU CSE Biointelligence Lab

ICA by Kurtosis Minimization and Maximization(2) Using gradient rule, The advantage is fast adaptation in a non-stationary environment A resulting trade-off is that the convergence is slow, and depends on a good choice of the learning rate sequence μ(t) (C) 2001, SNU CSE Biointelligence Lab

Fixed-Point Algorithm Using the above derivation, we get the following fixed-point algorithm for ICA Take a random initial vector w(0) of norm 1. Let k=1 Let Divide w(k) by its norm If |w(k)w(k-1)| is not close to 1, let k=k+1 and go back to step 2, Otherwise, output the vector w(k) (C) 2001, SNU CSE Biointelligence Lab

(C) 2001, SNU CSE Biointelligence Lab Application Blind source separation Feature extraction (C) 2001, SNU CSE Biointelligence Lab

Blind source separation <Eight independent components of the EEG data> (C) 2001, SNU CSE Biointelligence Lab

(C) 2001, SNU CSE Biointelligence Lab Feature extraction <Some ICA basis vectors of natural image data> (C) 2001, SNU CSE Biointelligence Lab

Discussion(about the Fast Fixed-Point Analysis) The convergence of the algorithm is very fast There is no learning rate or other adustable parameters Finds the independent components one at a time Both components of negative and positive kurtosis can be found (C) 2001, SNU CSE Biointelligence Lab