1. Lecture 19 Quadratic Shapes and Symmetric Positive Definite Matrices Shang-Hua Teng

2. Singular Value Decomposition Proof

3. All Singular Values are non Negative Positive

4. Row and Column Space Projection Suppose A is an m by n matrix that has rank r and r << n, and r << m. –Then A has r non-zero singular values –Let A = U  V T be the SVD of A where S is an r by r diagonal matrix –Examine:

5. The Singular Value Projection · A U VTVT m x n m x r r x r r x n =  0 0

6. Therefore Rows of U  are r dimensional projections of rows of A Columns of  V T are r dimensional projections of columns of A So we can compute their distances or dot products in a lower dimensional space

7. Eigenvalues and Determinants Product law: Summation Law: Both can be proved by examining the characteristic polynomial

8. Eigenvalues and Pivots If A is symmetric the number of positive (negative) eigenvalues equals to the number of positive (negative) pivots A = LDL T Topological Proof: scale down the off-diagonal entries of L continuously to 0, i.e., moving L continuously to I. Any change sign in eigenvalue must cross 0

9. Next Lecture Dimensional reduction for Latent Semantic Analysis Eigenvalue Problems in Web Analysis