1. Linear System expensive p=[0,0.2,0.4,0.45,0.5,0.55,0.6,0.8,1]; t=[1:8; 2:9]; e=[1,9]; n = length(p); % number of nodes m = size(t,2); % number of elements uexact = inline('x*(1-x)','x'); [M] = createM(p,e,t) [K] = createK(p,e,t) [L] = createL(p,e,t) A = K + M; for i=1:length(e) ib = e(i); A(ib,:)=sparse(1,n);A(:,ib)=sparse(n,1); A(ib,ib)=1;L(ib)=0; end U = A\L; ezplot('x*(1-x)',[0,1]); hold on; plot(p,U,'r-o'); grid on; hold off

2. tic; any matlab line; toc p=[0,0.2,0.4,0.45,0.5,0.55,0.6,0.8,1]; t=[1:8; 2:9]; e=[1,9]; n = length(p); % number of nodes m = size(t,2); % number of elements uexact = inline('x*(1-x)','x'); [M] = createM(p,e,t) [K] = createK(p,e,t) [L] = createL(p,e,t) A = K + M; for i=1:length(e) ib = e(i); A(ib,:)=sparse(1,n);A(:,ib)=sparse(n,1); A(ib,ib)=1;L(ib)=0; End tic U = A\L; toc ezplot('x*(1-x)',[0,1]); hold on; plot(p,U,'r-o'); grid on; hold off The most expensive line in the code is solving the linear system of equations

3. Linear System We want to solve the following linear system These methods generate a sequence of approximate solutions Gaussian Elimination LU, Choleski 2 classes of methods Direct Methods Iterative Methods

4. Linear System (Numerical Linear Algebra, Trefethen) Gaussian Elimination LU, Choleski Direct Methods We want to solve the following linear system We apply a sequence of row operations to convert the matrix A into an upper triangular matrix

5. Linear System We want to solve the following linear system 2 classes of methods Direct Methods Iterative Methods (Trefethen book) The following table gives a thumbnail of matrix computations over the year: 1950n=20 1965n=200 1980n=2000 1995n=20000 These numbers represent a rough approximation to what dimensions might have been considered “very large” for a dense, direct matrix computations at the indicated dates

6. observations made by Horst Simon [173]. He predicted that by now we will have to solve routinely linear problems with some 5 × 10^9 unknowns. From extrapolation of the CPU times observed for a characteristic model problem, he estimated the CPU time for the most efficient direct method as 520 040 years, provided that the computation can be carried out at a speed of 1 TFLOPS. On the other hand, the extrapolated guess for the CPU time with preconditioned conjugate gradients, still assuming a processing speed of 1 TFLOPS, is 575 seconds. ‘flops’ floating point operations. Each addition, subtraction, multiplication, division, or square root counts as one flop.

7. Linear System Iterative Methods Krylov subspace splitting others Jacobi, GS, SOR,..PCG, MINRES, GMRES, …

8. Linear System Remark: (1) has a unique solution We want to solve the following linear system A is invertable Remark: A is invertable det(A)=0 Remark: A is invertable Example: Solve: 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8

9. Linear System We want to solve the following linear system Remark: Definition: Example:

10. Iterative Method These methods generate a sequence of approximate solutions Remark:

11. DEF: with eigenvalues Splittings and Convergence DEF: spectral radius of A is defined to be Splitting A large family of iteration

12. Splittings and Convergence A large family of iteration DiagonalLowerUpper Jacobi:Gauss-Seidel:

13. Jacobi Method Consider 4x4 case 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8 Example

14. K=6K=7K=8K=9K=10 x1 x2 x3 X4 K=1K=2K=3K=4K=5 x1 x2 x3 X4 Jacobi Method 0.6000 1.0473 0.9326 1.0152 0.9890 2.2727 1.7159 2.0533 1.9537 2.0114 -1.1000 -0.8052 -1.0493 -0.9681 -1.0103 1.8750 0.8852 1.1309 0.9738 1.0214 1.0032 0.9981 1.0006 0.9997 1.0001 1.9922 2.0023 1.9987 2.0004 1.9998 -0.9945 -1.0020 -0.9990 -1.0004 -0.9998 0.9944 1.0036 0.9989 1.0006 0.9998 11.3537 4.9910 2.0299 0.8911 0.3686 0.1605 0.0671 0.0290 0.0122 0.0053

15. K=1K=2K=3K=4K=5 x1 x2 x3 X4 Gauss Seidel Method 0.6000 1.0302 1.0066 1.0009 1.0001 2.3273 2.0369 2.0036 2.0003 2.0000 -0.9873 -1.0145 -1.0025 -1.0003 -1.0000 0.8789 0.9843 0.9984 0.9998 1.0000 5.6930 0.4300 0.0662 0.0082 0.0009 Note that in the Jacobi iteration one does not use the most recently available information.

16. Gauss-Seidel iteration for general n: Gauss Seidel Method Jacobi iteration for general n:

17. Splittings and Convergence THM: A large family of iteration

18. Splittings and Convergence Example: 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8 10 11 10 8 2 -1 0 3 -1 -1 2 0 -1 3 Jacobi: U=triu(A,1) L=tril(A,-1) D=diag(diag(A)) eig(inv(M)*N) -0.4264 -0.1040 0.1860 0.3445 GS: 0 0 0.0839 - 0.0322i 0.0839 + 0.0322i

19. Splittings and Convergence THM: A large family of iteration Proof: (Golub p511) Remarks:

20. Splittings and Convergence THM: Proof: (Golub p512) show that all eigenvalues are less than one. Positive Definite Symmetric Stiffness Matrix Mass Matrix

21. Example: Splittings and Convergence DEF: IF THM:

22. Successive over Relaxation The Gauss-Seidel iteration is very attractive because of its simplicity. Unfortunately, if the spectral radius is close to one, then convergence is vey slow. One solution for this Successive over Relaxation GS:

23. Successive over Relaxation Example: 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8

24. K=1K=2K=3 x1 x2 x3 X4 Successive over Relaxation Example: 10 -1 2 0 -1 11 -1 3 2 -1 10 -1 0 3 -1 8 0.6180 1.0553 1.0055 2.3988 2.0273 2.0004 -1.0132 -1.0211 -1.0014 0.8743 0.9905 1.0000

25. MATLAB CODE Jacobi iteration for general n: Ex: Write a Matlab function for Jacobi function [sol,X]=jacobi(A,b,x0) n=length(b); maxiter=10; x=x0; for k=1:maxiter for i=1:n sum1=0; for j=1:i-1 sum1=sum1+A(i,j)*x(j); end sum2=0; for j=i+1:n sum2=sum2+A(i,j)*x(j); end xnew(i)=(b(i)-sum1-sum2)/A(i,i) end X(1:n,k)=xnew; x=xnew; end sol=xnew;

26. MATLAB CODE GS iteration for general n: Ex: Write a Matlab function for GS function [sol,X]=gs(A,b,x0) n=length(b); maxiter=10; x=x0; for k=1:maxiter for i=1:n sum1=0; for j=1:i-1 sum1=sum1+A(i,j)*x(j); end sum2=0; for j=i+1:n sum2=sum2+A(i,j)*x(j); end x(i)=(b(i)-sum1-sum2)/A(i,i) end X(1:n,k)=x; end sol=x;