1. Independent Component Analysis (ICA)
Adopted from:
Independent Component Analysis: A Tutorial
Aapo Hyvärinen and Erkki Oja Helsinki University of Technology

2. Motivation
Example: Cocktail-Party-Problem

3. Motivation
2 speakers, speaking simultaneously.

4. Motivation
2 microphones in different locations

5. Motivation
aij ... depends on the distances of the microphones from the speakers

6. Problem Definition
Get the original signals out of the recorded ones.

7. Noise-free ICA model Use statistical „latent variables“ system
Random variable sk instead of time signal
xj = aj1s1 + aj2s ajnsn, for all j
x = As
x = Sum(aisi)
ai ... basis functions
si ... independent components (IC‘s)

8. Generative Model IC‘s s are latent variables => unknown
Mixing matrix A is also unknown
Task: estimate A and s using only the observeable random vector x

9. Restrictions si are statistically independent
p(y1,y2) = p(y1)p(y2)
Non-gaussian distributions
Note: if only one IC is gaussian, the estimation is still possible

10. Solving the ICA model Additional assumptions:
# of IC‘s = # of observable mixtures
=> A is square and invertible
A is identifiable => estimate A
Compute W = A-1
Obtain IC‘s from:
s = Wx

11. Ambiguities (I) Can‘t determine the variances (energies) of the IC‘s
x = Sum[(1/Ci)aisiCi]
Fix magnitudes of IC‘s assuming unit variance: E{si2} = 1
Only ambiguity of sign remains

12. Ambiguities (II) Can‘t determine the order of the IC‘s
Terms can be freely interchanged, because both s and A are unknown
x = AP-1Ps
P ... permutation matrix

13. Centering the variables
Simplifying the algorithm:
Assume that both x and s have zero mean
Preprocessing:
x = x‘ – E{x‘}
IC‘s are also zero mean because of:
E{s} = A-1E{x}
After ICA: add A-1E{x‘} to zero mean IC‘s

14. Noisy ICA model x = As + n A ... mxn mixing matrix
s ... n-dimensional vector of IC‘s
n ... m-dimensional random noise vector
Same assumptions as for noise-free model

15. General ICA model Find a linear transformation: s = Wx
si as independent as possible
Maximize F(s) : Measure of independence
No assumptions on data
Problem:
definition for measure of independence
Strict independence is in general impossible

16. Illustration (I) 2 IC‘s with distribution: Joint distribution of IC‘s:
zero mean and variance equal to 1
Joint distribution of IC‘s:

17. Illustration (II) Mixing matrix:
Joint distribution of observed mixtures:

18. Other Problems Blind Source/Signal Separation (BSS) Feature extraction
Cocktail Party Problem (another definition)
Electroencephalogram
Radar
Mobile Communication
Feature extraction
Image, Audio, Video, ...representation

19. Principles of ICA Estimation
“Nongaussian is independent”: central limit theorem
Measure of nonguassianity
Kurtosis: (Kurtosis=0 for a gaussian distribution)
Negentropy: a gaussian variable has the largest entropy among all random variables of equal variance:

20. Approximations of Negentropy (1)

21. Approximations of Negentropy (2)

22. The FastICA Algorithm

23. 4 Signal BSS demo (original)

24. 4 Signal BSS demo (Mixtures)

25. 4 Signal BSS demo (ICA)

26. FastICA demo (mixtures)

27. FastICA demo (whitened)

28. FastICA demo (step 1)

29. FastICA demo (step 2)

30. FastICA demo (step 3)

31. FastICA demo (step 4)

32. FastICA demo (step 5 - end)

33. Other Algorithms for BSS
Temporal Predictability
TP of mixture < TP of any source signal
Maximize TP to seperate signals
Works also on signals with Gaussian PDF
CoBliSS
Works in frequency domain
Only using the covariance matrix of the observation
JADE

34. Links 1 Feature extraction (Images, Video) Aapo Hyvarinen: ICA (1999)
Aapo Hyvarinen: ICA (1999)
ICA demo step-by-step
Lots of links

35. Links 2 object-based audio capture demos
Demo for BBS with „CoBliSS“ (wav-files)
Tomas Zeman‘s page on BSS research
Virtual Laboratories in Probability and Statistics

36. Links 3 An efficient batch algorithm: JADE
Dr JV Stone: ICA and Temporal Predictability
BBS with Degenerate Unmixing Estimation Technique (papers)

37. Links 4
detailed information for scientists, engineers and industrials about ICA
FastICA package for matlab
Aapo Hyvärinen
Erkki Oja