1. Slide 1 EE3J2 Data Mining EE3J2 Data Mining Lecture 9(b) Principal Components Analysis Martin Russell

2. Slide 2 EE3J2 Data Mining Objectives  To illustrate PCA through an example application  3D dance motion modelling

3. Slide 3 EE3J2 Data Mining Data  Analysis of dance sequence data  Body position represented as 90 dimensional vector  Dance sequence represented as a sequence of these vectors  MEng FYP 2004/5, Wan Ni Chong

4. Slide 4 EE3J2 Data Mining Data Capture (1)

5. Slide 5 EE3J2 Data Mining Data Capture (2)

6. Slide 6 EE3J2 Data Mining Data Capture (3)

7. Slide 7 EE3J2 Data Mining Calculating PCA  Step 1: Arrange data as a matrix –Rows correspond to individual data points –Number of columns = dimension of data (= 90) –Number of rows = number of data points = N

8. Slide 8 EE3J2 Data Mining Calculating PCA (step 2)  Compute the covariance matrix of the data  In MATLAB >>C = cov(X)  Alternatively (as in slides from last lecture): – calculate the mean vector m, –subtract m from each row of X to give Y –Then

9. Slide 9 EE3J2 Data Mining Calculating PCA (step 3)  Do an eigenvector decomposition of C, so that: C = UDU T  Where –U is a unitary (rotation) matrix –D is a diagonal matrix (in fact all elements of D will be real and non-negative)  In MATLAB type >>[U,D] = eig(C)

10. Slide 10 EE3J2 Data Mining Calculating PCA (step 4)  Each column of U is a principal vector  The corresponding eigenvalue indictates the variance of the data along that dimension  Large eigenvalues indicate significant components of the data  Small eigenvalues indicate that the variation along the corresponding eigenvectors may be noise

11. Slide 11 EE3J2 Data Mining Eigenvalues Insignificant Component s More Significant Components PCsData 1 st 2 nd 3 rd 90 th 1 st 90 th Eigenvalues

12. Slide 12 EE3J2 Data Mining Calculating PCA (step 6)  It may be advantageous to ignore dimensions which correspond to small eigenvalues and only consider the projection of the data onto the most significant eigenvectors  In this way the dimension of the data can be reduced

13. Slide 13 EE3J2 Data Mining Visualising PCA Original pattern (blue) U Eigenspace Set coordinates 11 – 90 to zero U -1 Reduced pattern (red)

14. Slide 14 EE3J2 Data Mining PCA Example  Original 90 dimensional data reduced to just 1 dimension

15. Slide 15 EE3J2 Data Mining PCA Example  Original 90 dimensional data reduced to 10 dimensions

16. Slide 16 EE3J2 Data Mining Summary  Example of PCA  Analysis of 90 dimensional 3D dance data  Analysis shows that PCA can reduce 90 dimensional representation to just 10 dimensions with minimal loss of accuracy