1. Presented by: Muhammad Wasif Laeeq (BSIT07-1) Muhammad Aatif Aneeq (BSIT07-15) Shah Rukh (BSIT07-22) Mudasir Abbas (BSIT07-34) Ahmad Mushtaq (BSIT07-45) PRINCIPLE COMPONENT ANALYSIS

2. BZUPAGES.COM Study Hours G.P.A. Student A64 Student B53.2 Student C42.75 Student D22 Student E0.251.2 5 x 2 Matrix

3. BZUPAGES.COM

4. BZUPAGES.COM Var1Var9000 Student A Student B Student C Student D Student E 5 x 9000 Matrix

5. BZUPAGES.COM

6. BZUPAGES.COM

7. BZUPAGES.COM

8. BZUPAGES.COM

9. BZUPAGES.COM

10. BZUPAGES.COM

11. BZUPAGES.COM

12. BZUPAGES.COM THE DEFINITION Principal component analysis (PCA) is a mathematical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of uncorrelated variables called principal components.

13. BZUPAGES.COM Principal Components are always perpendicular to each other The number of principal components is less than or equal to the number of original variables First Principal Component has highest Variance

14. BZUPAGES.COM

15. BZUPAGES.COM

16. BZUPAGES.COM WHERE TO USE Used to uncover unknown trends Used to summarize Data Visualization of high dimension Data Dimensionality Reduction

17. BZUPAGES.COM BACKGROUND MATHEMATICS BY SHAHRUKH

18. Mean Standard Deviation Variance Covariance The covariance Matrix STATISTICS

19. Eigenvectors Eigenvalues MATHEMATICS Matrix Algebra

20. STATISTICS Mean

21. STATISTICS Standard deviation The average distance from the mean of the data set to a point

22. STATISTICS Cont.. To compute the squares of the distance from each data point to the mean of the set, add them all up, divide by n-1 and take the positive square root

23. STATISTICS Variance Variance is another measure of the spread of data in a data set. In fact it is almost identical to the standard deviation. The formula is this:

24. STATISTICS Covariance When there are than two dimensions of data then we use this, The formula is :

25. STATISTICS The covariance Matrix When there are more than two dimensions of data then we use this. The formula is : Suppose we have three dimensions of data:

26. Eigenvectors Eigenvalues MATHEMATICS Matrix Algebra

27. Eigenvectors:- The eigenvectors of a square matrix are the non- zero vectors which, after being multiplied by the matrix, remain proportional to the original vector. MATHEMATICS Cont…

28. Eigenvalues:- For each eigenvector, the corresponding eigenvalue is the factor by which the eigenvector changes when multiplied by the matrix. MATHEMATICS Cont…

29. MATHEMATICS Eigenvectors, Eigenvalues

30. PCA IN

31. BZUPAGES.COM

32. BZUPAGES.COM

33. BZUPAGES.COM

34. BZUPAGES.COM

35. BZUPAGES.COM

36. BZUPAGES.COM

37. BZUPAGES.COM

38. BZUPAGES.COM

39. BZUPAGES.COM So how do we detect a face? Any face can be identified by multiplying the eigen-faces to the name minus average. If the image we “project” from the face space is close enough to the actual image detected then we found what we are looking for…

40. PCA IN ACTION!

41. BZUPAGES.COM Suppose we have a 2 dimensional data:

42. BZUPAGES.COM The first step of PCA is to obtain covariance matrix Variance of x1 Variance of x2 Covariance of x1-x2 Variance(1) Cov(1,2) Cov(1,2)Variance(2)

43. BZUPAGES.COM Formula for variance: Formula for Covariance

44. BZUPAGES.COM Step 2: is to obtain Eigen Values by solving function determinant Solving a the above equation gives two values of And these two values are Eigen Values

45. BZUPAGES.COM Step 3: is to obtain Eigen Vector by solving for matrix X in such a way that, Cov Matrix

46. BZUPAGES.COM Step 4: is to obtain coordinates of data point in the direction of Eigen Vectors We obtain this by multiplying centered data matrix to the Eigen vector matrix.

47. BZUPAGES.COM Lets have a look at an Excell Workbook

48. BZUPAGES.COM Our covariance matrix is: 6.4228 7.9876 7.9876 9.9528

49. BZUPAGES.COM Lets find out the Eigen Values: By solving function determinant: - 16.3756* +0.12214 Solving =16.36809984, 0.007462657

50. BZUPAGES.COM Now we will find out Eigen Vectors: By solving the following matrix: Substract the 1 st Eigen value from variance terms of co-variance matrix of Step 1, we obtain -9.9453 7.9876 7.9876 -6.4153

51. BZUPAGES.COM Finding Eigen Vectors: Now for 1 st Eigen Vector: -9.9453 7.9876 7.9876 -6.4153 X= abab 0000 We get a=0.6262, b=0.7797 Similarly for 2 nd Eigen value 0.007462657, We get a=0.7797 and b=-0.6262

52. BZUPAGES.COM Now obtain cordinates of data point in the direction of Eigen Vectors

53. BZUPAGES.COM