1. Orthogonal Matrices & Symmetric Matrices
Hung-yi Lee

2. Announcement 如果三個作業都滿分請忽略以下訊息 …… We have a bonus homework
三個作業都滿分就是 300
Bonus homework 全對可以加 50
最多可以加到 300
助教第二堂課會來講解

3. Orthogonal Matrices Symmetric Matrices Outline Reference: Chapter 7.5

4. Norm-preserving A linear operator is norm-preserving if 𝑇 𝑢 = 𝑢
𝑇 𝑢 = 𝑢
For all u
Example: linear operator T on R2 that rotates a vector by .
 Is T norm-preserving?
Anything in Common?
Example: linear operator T is refection
 Is T norm-preserving?
𝐴= −1

5. Norm-preserving A linear operator is norm-preserving if 𝑇 𝑢 = 𝑢
𝑇 𝑢 = 𝑢
For all u
Example: linear operator T is projection
 Is T norm-preserving? 𝐴=
Example: linear operator U on Rn that has an eigenvalue   ±1.
 U is not norm-preserving, since for the corresponding
eigenvector v, U(v) = v = ·v  v.

6. Orthogonal Matrix
An nxn matrix Q is called an orthogonal matrix (or simply orthogonal) if the columns of Q form an orthonormal basis for Rn
Orthogonal operator: standard matrix is an orthogonal matrix.
unit
unit
is an orthogonal matrix.
orthogonal

7. Norm-preserving Necessary conditions: Norm-preserving
Orthogonal Matrix ?
???
Linear operator Q is norm-preserving
Why there is no northonormal matrix??????????????????????????????????????????????????????
qj = 1
qj = Qej = ej
qi and qj are orthogonal
畢式定理
qi + qj2 = Qei + Qej2 = Q(ei + ej)2 = ei + ej2 = 2 = qi2 + qj2

8. Orthogonal Matrix Q is an orthogonal matrix 𝑄𝑄 𝑇 = 𝐼 𝑛
Those properties are used to check orthogonal matrix.
Q is an orthogonal matrix
𝑄𝑄 𝑇 = 𝐼 𝑛
𝑄 is invertible, and 𝑄 −1 = 𝑄 𝑇
𝑄𝑢∙𝑄𝑣=𝑢∙𝑣 for any u and v
𝑄𝑢 = 𝑢 for any u
Simple inverse
Q preserves dot projects
Q preserves norms
Proof (b)  (c) By definition of invertible matrices
(a)  (b) with Q = [ q1 q2  qn ], qi  qi = 1 = [QTQ]ii i,
and qi  qj = 0 = [QTQ]ij i  j.
 QTQ = In
(c)  (d) u, v  Rn, Qu  Qv = u  QTQv = u  Q1Qv = u  v.
(d)  (e) u  Rn, Qu = (Qu  Qu)1/2 = (u  u)1/2 = u.
(e)  (a) The above necessary conditions.
Norm-preserving
Orthogonal Matrix

9. Orthogonal Matrix Rows and columns
Q is orthogonal if and only if 𝑄 𝑇 is orthogonal.
Let P and Q be n x n orthogonal matrices
𝑑𝑒𝑡𝑄=±1
𝑃𝑄 is an orthogonal matrix
𝑄 −1 is an orthogonal matrix
𝑄 𝑇 is an orthogonal matrix
Proof
Check by 𝑄 −1 = 𝑄 𝑇
Check by 𝑃𝑄 −1 = 𝑃𝑄 𝑇
Proof (a) QQT = In  1 = det(In) = det(QQT) = det(Q)det(QT)
= det(Q)2  det(Q) = ±1.
(b) (PQ)T = QTPT = Q1P1 = (PQ)1.

10. Orthogonal Operator
Applying the properties of orthogonal matrices on orthogonal operators
T is an orthogonal operator
𝑇 𝑢 ∙𝑇 𝑣 =𝑢∙𝑣 for all 𝑢 and 𝑣
𝑇 𝑢 = 𝑢 for all 𝑢
T and U are orthogonal operators, then 𝑇𝑈 and 𝑇 −1 are orthogonal operators.
Preserves dot product
Preserves norms

11. Example: Find an orthogonal operator T on R3 such that
𝑇 =
Norm-preserving 𝑣=
Find 𝐴 −1 first
𝐴𝑣= 𝑒 2
𝑣= 𝐴 −1 𝑒 2
Because 𝐴 −1 = 𝐴 𝑇
𝐴 −1 = ∗ ∗ ∗ 0 ∗ ∗ ∗
Also orthogonal
𝐴 −1 = −
−1 2
0 1 0
𝐴= 𝐴 −1 𝑇 = −

12. Conclusion Orthogonal Matrix (Operator)
Columns and rows are orthogonal unit vectors
Preserving norms, dot products
Its inverse is equal its transpose

13. Orthogonal Matrices Symmetric Matrices Outline Reference: Chapter 7.5

14. Eigenvalues are real
The eigenvalues for symmetric matrices are always real.
How about more general cases?
Consider 2 x 2 symmetric matrices
實係數多項式虛根共軛
The symmetric matrices always have real eigenvalues.

15. Orthogonal Eigenvectors
A is symmetric
𝑑𝑒𝑡 𝐴−𝑡 𝐼 𝑛
Factorization
= 𝑡− 𝜆 1 𝑚 1 𝑡− 𝜆 2 𝑚 2 … 𝑡− 𝜆 𝑘 𝑚 𝑘 …… ……
Eigenvalue:
𝜆 1
𝜆 2
𝜆 𝑘 ……
Eigenspace:
𝑑 1
≤ 𝑚 1
𝑑 2
≤ 𝑚 2
𝑑 𝑘
≤ 𝑚 𝑘
(dimension)
Independent
orthogonal

16. Orthogonal Eigenvectors
A is symmetric.
If 𝑢 and 𝑣 are eigenvectors corresponding to eigenvalues 𝜆 and 𝜇 (𝜆≠𝜇)
𝑢 and 𝑣 are orthogonal.

17. P is an orthogonal matrix
Diagonalization
A=𝑃D P 𝑇
P560 ?
A is symmetric
P 𝑇 A𝑃=D
P is an orthogonal matrix :
simple
D is a diagonal matrix
P 𝑇 A𝑃=D
P −1 A𝑃=D
A=𝑃D P −1
Diagonalization
A=𝑃D P 𝑇
P consists of eigenvectors
, D are eigenvalues

18. Diagonalization Example A=𝑃D P 𝑇 A=𝑃D P −1 P 𝑇 A𝑃=D
A has eigenvalues 1 = 6 and 2 = 1,
with corresponding eigenspaces E1 = Span{[ 1 2 ]T} and
E2 = Span{[ 2 1 ]T}
orthogonal
 B1 = {[ 1 2 ]T/5} and B2 = {[ 2 1 ]T/5}

19. Example of Diagonalization of Symmetric Matrix
A=𝑃D P −1
A=𝑃D P 𝑇
P is an orthogonal
matrix
Intendent
Gram-
Schmidt
1 = 2
𝑆𝑝𝑎𝑛 − , −2 6
Eigenspace: 𝑆𝑝𝑎𝑛 − , −1 0 1
normalization
Not orthogonal
2 = 8
𝑆𝑝𝑎𝑛
Eigenspace: 𝑆𝑝𝑎𝑛
normalization
𝑃= − − 𝐷=

20. P is an orthogonal matrix
Diagonalization
P is an orthogonal matrix
A is symmetric
P 𝑇 A𝑃=D
A=𝑃D P 𝑇
P consists of eigenvectors
, D are eigenvalues
Finding an orthonormal basis consisting of eigenvectors of A (1) Compute all distinct eigenvalues 1, 2, , k of A.
(2) Determine the corresponding eigenspaces E1, E2, , Ek.
(3) Get an orthonormal basis B i for each Ei.
(4) B = B 1 B 2 B k is an orthonormal basis for A.

21. Diagonalization of Symmetric Matrix
𝑢= 𝑐 1 𝑣 1 + 𝑐 2 𝑣 2 +⋯+ 𝑐 𝑘 𝑣 𝑘
𝑢∙ 𝑣 1
𝑢∙ 𝑣 2
𝑢∙ 𝑣 𝑘
Orthonormal basis 𝐷
𝑇 B
𝑣 B
𝑇 𝑣 B
simple
Eigenvectors form the good system
𝑃 −1 𝑃
𝐵 −1 𝐵
Properly selected
Properly selected
𝐴=𝑃𝐷 𝑃 −1 𝑣
𝑇 𝑣
A is symmetric

22. Spectral Decomposition
Orthonormal basis
A = PDPT
Let P = [ u1 u2  un ] and D = diag[ 1 2  n ].
= P[ 1e1 2e2  nen ]PT
= [ 1Pe1 2Pe2  nPen ]PT = [ 1u1 2u2  nun ]PT
𝑃 1
𝑃 2
𝑃 𝑛
= 𝜆 1 P 1 + 𝜆 2 P 2 +⋯+ 𝜆 𝑛 P 𝑛
𝑃 𝑖 are symmetric

23. Spectral Decomposition
Orthonormal basis
A = PDPT
Let P = [ u1 u2  un ] and D = diag[ 1 2  n ].
= 𝜆 1 P 1 + 𝜆 2 P 2 +⋯+ 𝜆 𝑛 P 𝑛

24. Spectral Decomposition
Example
𝐴= 3 −4 −4 −3
Find spectrum decomposition.
𝑃 1 = 𝑢 1 𝑢 1 𝑇 = −2 5 −
Eigenvalues 1 = 5 and 2 = 5.
An orthonormal basis consisting of eigenvectors of A is
𝐵= − ,
𝑃 2 = 𝑢 2 𝑢 2 𝑇 =
𝑢 1
𝑢 2
𝐴= 𝜆 1 𝑃 1 + 𝜆 2 𝑃 2

25. P is an orthogonal matrix
Conclusion
Any symmetric matrix
has only real eigenvalues
has orthogonal eigenvectors.
is always diagonalizable
A is symmetric
P is an orthogonal matrix
P 𝑇 A𝑃=D
A=𝑃D P 𝑇

26. Appendix

27. Diagonalization By induction on n. n = 1 is obvious.
Assume it holds for n  1, and consider A  R(n+1)(n+1).
A has an eigenvector b1  Rn+1 corresponding to a real eigenvalue , so  an orthonormal basis B = {b1, b2, , bn+1}
by the Extension Theorem and Gram- Schmidt Process.

28. S = ST  Rnn   an orthogonal C  Rnn and a diagonal L  Rnn
such that CTSC = L by the induction hypothesis.
Gram-Schmidt

29. Example: reflection operator T about a line L passing the origin.
Question: Is T an orthogonal operator?
(An easier) Question:
Is T orthogonal if L is the x-axis?
b1 is a unit vector along L .
b2 is a unit vector perpendicular to L .
P = [ b1 b2 ] is an orthogonal matrix.
B = {b1, b2} is an orthonormal basis of R2.
[T]B = diag[1 1] is an orthogonal matrix.
Let the standard matrix of T be Q. Then [T]B = P1QP, or Q =
P[T]B P1  Q is an orthogonal matrix.  T is an orthogonal operator.