1. unité #5
Analyse
numérique
matricielle
Giansalvo EXIN Cirrincione

2. Décomposition en valeurs singulières (SVD)
am1  
amn
a11
a1n U
 n  1 V =
Full SVD
valeurs singulières

3. Décomposition en valeurs singulières (SVD)
am1  
amn
a11
a1n U
 n  1 V =
am1  
amn
a11
a1n U
 n  1 V = ^
Reduced SVD

4. Décomposition en valeurs singulières (SVD)
Full SVD
Reduced SVD

5. Approximation au sens des moindres carrées
Example: polynomial data fitting

6. Approximation au sens des moindres carrées
0.2
0.4
0.6
0.8 1
1.2
1.4
1.6
1.8 -4 -3 -2 -1 2
f(x) xi yi

7. Approximation au sens des moindres carrées
discrete square wave
interpolation
m = n = 11
least squares
m = 11 , n = 8

8. Approximation au sens des moindres carrées
Posons le problème matriciellement

9. Approximation au sens des moindres carrées
système linéaire de n équations et n inconnues
erreur
d’approximation
Matrice de Vandermonde
( )

10. Approximation au sens des moindres carrées
forme quadratique
Équations normales

12. The system is nonsingular iff A has full rank.b
r = b - A x
range(A)
y = A x
= Pb
The system is nonsingular iff A has full rank.

13. The system is nonsingular iff A has full rank.

14. Solution par les équations normales
factorisation de Cholesky
AHA est une matrice n x n hermitienne strictement définie positive
1. Form the matrix AHA and the vector AH b
2. Compute the Cholesky factorization AHA = RHR
3. Solve the lower-triangular system RH w = AH b for w
4. Solve the upper-triangular system R x = w for x

15. Solution par la factorisation QR (Householder)
reduced
QR factorization
1. Compute the reduced QR factorization
2. Compute the vector
3. Solve the upper-triangular system for x

16. Solution par la SVD 1. Compute the reduced SVD 2. Compute the vector
3. Solve the diagonal system for w
4. Set

17. Comparison of algorithms
speed : normal equations
standard : QR factorization
A close to singular : SVD
Drawbacks
normal equations : not always stable in the presence of rounding errors
QR factoriz.: less-than-ideal stability properties if A is close to singular
SVD : expensive for m  n

18. Conditionnement et précision

19. Conditionnement du problème des moindres carrées
Données : A , b Solutions : x , y
range(A)
y = A x
r = b - A x b
= Pb 
closeness of the fit

20. Conditionnement du problème des moindres carrées
Données : A , b Solutions : x , y
2-norm relative condition numbers
exact for certain  b
upper bounds

21.  highly ill-conditioned basis
Stabilité des méthodes des moindres carrées
exemple
Least squares fitting of the function exp(sin(4)) on the interval [0,1] by a polynomial of degree 14
 x15 = 1
 highly ill-conditioned basis
 very close fit

22. Stabilité des méthodes des moindres carrées
exemple
reduced
factorisation QR (Householder)
The rounding errors have been amplified by a factor of order This inaccuracy is explained by ill-conditioning, not instability.

23. Stabilité des méthodes des moindres carrées
exemple
factorisation QR (Householder)
implicit calculation of the product QH b

24. Stabilité des méthodes des moindres carrées
exemple
factorisation QR (Householder)
implicit calculation of the product QH b

25. Stabilité des méthodes des moindres carrées
exemple
factorisation QR (Householder)
backward stable

26. Stabilité des méthodes des moindres carrées
exemple
SVD
It beats Householder triangularization with column pivoting ( MATLAB's \ ) by a factor of about 3
backward stable

27. Stabilité des méthodes des moindres carrées
exemple
équations normales
factorisation de Cholesky
not even a single digit of accuracy
unstable

28. Stabilité des méthodes des moindres carrées
BS least squares algorithm
The condition number of the LS problem may lie anywhere in the range  to 2 .

29. Stabilité des méthodes des moindres carrées
BS least squares algorithm
Cholesky factorization (BS)
The normal equations are typically unstable for ill-conditioned problems involving close fits.

30. Stabilité des méthodes des moindres carrées
The solution of the full-rank least squares problem via the normal equations is unstable. Stability can be achieved, however, by restriction to a class of problems in which (A) is uniformly bounded above or (tan)/ is uniformly bounded below.
The normal equations are typically unstable for ill-conditioned problems involving close fits.

31. FINE