1. MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 6. Eigenvalue problems.

2. Eigenvalue problems Eigenvalue problems occur in many areas of science and engineering, such as structural analysis Eigenvalues are also important in analyzing numerical methods Theory and algorithms apply to complex matrices as well as real matrices With complex matrices, we use conjugate transpose, A H, instead of usual transpose, A T

3. Formulation Matrix expands or shrinks any vector lying in direction of eigenvector by scalar factor Expansion or contraction factor is given by corresponding eigenvalue Eigenvalues and eigenvectors decompose complicated behavior of general linear transformation into simpler actions

4. Examples

5. Characteristic polynomial

6. Example

7. Companion matrix

8. Characteristic polynomial Computing eigenvalues using characteristic polynomial is not recommended because of work in computing coefficients of characteristic polynomial sensitivity of coefficients of characteristic polynomial work in solving for roots of characteristic polynomial Characteristic polynomial is powerful theoretical tool but usually not useful computationally

9. Example

10. Diagonalizability

11. Eigenspaces

12. Some relevant definitions

13. Examples

15. Properties of eigenvalue problems Properties of eigenvalue problem affecting choice of algorithm and software Are all eigenvalues needed, or only a few? Are only eigenvalues needed, or are corresponding eigenvectors also needed? Is matrix real or complex? Is matrix relatively small and dense, or large and sparse? Does matrix have any special properties, such as symmetry, or is it general matrix? Condition of eigenvalue problem is sensitivity of eigenvalues and eigenvectors to changes in matrix Conditioning of eigenvalue problem is not same as conditioning of solution to linear system for same matrix Different eigenvalues and eigenvectors are not necessarily equally sensitive to perturbations in matrix

16. Conditioning of eigenvalues

18. Problem transformations

19. Similarity transformation

21. Diagonal form Eigenvalues of diagonal matrix are diagonal entries, and eigenvectors are columns of identity matrix Diagonal form is desirable in simplifying eigenvalue problems for general matrices by similarity transformations But not all matrices are diagonalizable by similarity transformation Closest one can get, in general, is Jordan form, which is nearly diagonal but may have some nonzero entries on first superdiagonal, corresponding to one or more multiple eigenvalues

22. Triangular form

23. Block triangular form

24. Forms attainable by similarity

25. Power iteration

26. Convergence of Power iteration

27. Example

28. Limitations

29. Normalized Power iteration

30. Example

31. Geometric interpretation

32. Power Iteration with Shift In earlier example, for instance, if we pick shift of σ = 1, (which is equal to other eigenvalue) then ratio becomes zero and method converges in one iteration In general, we would not be able to make such fortuitous choice, but shifts can still be extremely useful in some contexts, as we will see later

33. Inverse iteration Inverse iteration converges to eigenvector corresponding to smallest eigenvalue of A. Eigenvalue obtained is dominant eigenvalue of A −1, and hence its reciprocal is smallest eigenvalue of A in modulus

34. Example

35. Shifted inverse iteration

36. Rayleigh Quotient

37. Example

38. Rayleigh Quotient iteration

40. Deflation

43. Simultaneous Iteration

44. Orthogonal iteration

45. QR iteration

47. Example

48. QR iteration with shifts

49. Example

50. Preliminary reduction

53. Cost of QR iteration

54. Krylov subspaces methods

56. Arnoldi iteration

59. Lanczos Iteration

60. Lanczos iteration

62. Krylov subspace methods cont.

63. Example of Lanczos iteration

64. Jacobi method

66. Plane rotation

67. Jacobi method cont.

69. Jacobi method example

71. Process continues until off-diagonal entries reduced to as small as desired Result is diagonal matrix orthogonally similar to original matrix, with the orthogonal similarity transformation given by product of plane rotations

72. Other methods (spectrum-slicing)

73. Sturm sequence

74. Divide-and-conquer algorithm

75. Relatively robust representation

76. Generalized eigenvalue problems

77. QZ algorithm

78. Computing SVD