Independent Component Analysis From PCA to ICA Bell Sejnowski algorithm Kurtosis method Demonstrations

Bell and Sejnowski 1995 Consider y=g(x)+noise with f depending on w I(y;x)=H(y)- H(y|x) H(y|x)=E_x E_y|x [-log P(y|x)]

ICA based on Kurtosis Oja and Hyvarinen

Independent Component Analysis ? Perform “blind separation” of signals recorded at multiple sensors ? Use minimal assumptions about the characteristics of the signal sources. An overview of applications of ICA to biological data and general data mining, Computational Neurobiology Laboratory Salk Institute, La Jolla CA (April, 1999). Enter [Enter] to advance, [up-arrow] to rewind.

Principle: Maximize Information Q:Q: How to extract maximum information from multiple visual channels ? Set of 144 ICA filters AA: ICA does this -- it maximizes joint entropy & minimizes mutual information between output channels (Bell & Sejnowski, 1995). ICA produces brain-like visual filters for natural images.

ICA versus PCA Independent Component Analysis (ICA) finds directions of maximal independence in non- Gaussian data (higher-order statistics). Principal Component Analysis (PCA) finds directions of maximal variance in Gaussian data (second-order statistics).

Example: Audio decomposition Play MixturesPlay Components Perform ICA Mic 1 Mic 2 Mic 3 Mic 4 Terry Scott Te-WonTzyy-Ping

Electroencephalography (EEG) ICA separates brain signals from artifacts. Artifacts Brain signals Allows monitoring of multiple brain processes. Permits study of brain activity in noisy conditions.

Functional Brain Imaging Functional magnetic resonance imaging (fMRI) data are noisy and complex. ICA identifies concurrent hemodynamic processes. Does not require a priori knowledge of time courses or spatial distributions.

Data Mining ICA was applied to Armed Forces Vocation Aptitude Battery (ASVAB) test scores and Navy Fire Control School grades. ICA may suggest more efficient and balanced selection criteria. Two ICA components contributed to final school grade.

This presentation by Scott Makeig, Naval Health Research Center, San Diego Tzyy-Ping Jung, Institute for Neural Computation, UCSD, La Jolla CA Te-Won Lee, Salk Institute, La Jolla CA Sigurd Enghoff, Salk Institute Terrence J. Sejnowski, Salk Institute & UCSD

From Barak Pearlmutter  Contextual ICA  The first demo applies the Contextual ICA blind source separation algorithm. Lucas Parra and I digitally extracted ten five-second clips from ten audio CDs. These were digitally mixed, without time delays or echos, and with random gains, to form the output of a synthetic microphone. Ten such microphone outputs were synthesized. These synthetic microphone outputs formed the input to the Bell-Sejnowski Independent Components Analysis algorithm. The sources are somewhat separated in the output of the Bell-Sejnowski ICA algorithm, but not fully.Lucas ParraI ten five-second clips from ten audio CDs.microphone outputs the output of the Bell-Sejnowski ICA algorithm,  The same synthetic microphone outputs were then used as input to our new cICA algorithm (see my publications page for technical details). The sources are almost fully separated in the output of cICA.publications pagethe output of cICA.