Linear System expensive p=[0,0.2,0.4,0.45,0.5,0.55,0.6,0.8,1]; t=[1:8; 2:9]; e=[1,9]; n = length(p); % number of nodes m = size(t,2); % number of elements uexact = inline('x*(1-x)','x'); [M] = createM(p,e,t) [K] = createK(p,e,t) [L] = createL(p,e,t) A = K + M; for i=1:length(e) ib = e(i); A(ib,:)=sparse(1,n);A(:,ib)=sparse(n,1); A(ib,ib)=1;L(ib)=0; end U = A\L; ezplot('x*(1-x)',[0,1]); hold on; plot(p,U,'r-o'); grid on; hold off

tic; any matlab line; toc p=[0,0.2,0.4,0.45,0.5,0.55,0.6,0.8,1]; t=[1:8; 2:9]; e=[1,9]; n = length(p); % number of nodes m = size(t,2); % number of elements uexact = inline('x*(1-x)','x'); [M] = createM(p,e,t) [K] = createK(p,e,t) [L] = createL(p,e,t) A = K + M; for i=1:length(e) ib = e(i); A(ib,:)=sparse(1,n);A(:,ib)=sparse(n,1); A(ib,ib)=1;L(ib)=0; End tic U = A\L; toc ezplot('x*(1-x)',[0,1]); hold on; plot(p,U,'r-o'); grid on; hold off The most expensive line in the code is solving the linear system of equations

Linear System We want to solve the following linear system These methods generate a sequence of approximate solutions Gaussian Elimination LU, Choleski 2 classes of methods Direct Methods Iterative Methods

Linear System (Numerical Linear Algebra, Trefethen) Gaussian Elimination LU, Choleski Direct Methods We want to solve the following linear system We apply a sequence of row operations to convert the matrix A into an upper triangular matrix

Linear System We want to solve the following linear system 2 classes of methods Direct Methods Iterative Methods (Trefethen book) The following table gives a thumbnail of matrix computations over the year: 1950n= n= n= n=20000 These numbers represent a rough approximation to what dimensions might have been considered “very large” for a dense, direct matrix computations at the indicated dates

observations made by Horst Simon [173]. He predicted that by now we will have to solve routinely linear problems with some 5 × 10^9 unknowns. From extrapolation of the CPU times observed for a characteristic model problem, he estimated the CPU time for the most efficient direct method as years, provided that the computation can be carried out at a speed of 1 TFLOPS. On the other hand, the extrapolated guess for the CPU time with preconditioned conjugate gradients, still assuming a processing speed of 1 TFLOPS, is 575 seconds. ‘flops’ floating point operations. Each addition, subtraction, multiplication, division, or square root counts as one flop.

Linear System Iterative Methods Krylov subspace splitting others Jacobi, GS, SOR,..PCG, MINRES, GMRES, …

Linear System Remark: (1) has a unique solution We want to solve the following linear system A is invertable Remark: A is invertable det(A)=0 Remark: A is invertable Example: Solve:

Linear System We want to solve the following linear system Remark: Definition: Example:

Iterative Method These methods generate a sequence of approximate solutions Remark:

DEF: with eigenvalues Splittings and Convergence DEF: spectral radius of A is defined to be Splitting A large family of iteration

Splittings and Convergence A large family of iteration DiagonalLowerUpper Jacobi:Gauss-Seidel:

Jacobi Method Consider 4x4 case Example

K=6K=7K=8K=9K=10 x1 x2 x3 X4 K=1K=2K=3K=4K=5 x1 x2 x3 X4 Jacobi Method

K=1K=2K=3K=4K=5 x1 x2 x3 X4 Gauss Seidel Method Note that in the Jacobi iteration one does not use the most recently available information.

Gauss-Seidel iteration for general n: Gauss Seidel Method Jacobi iteration for general n:

Splittings and Convergence THM: A large family of iteration

Splittings and Convergence Example: Jacobi: U=triu(A,1) L=tril(A,-1) D=diag(diag(A)) eig(inv(M)*N) GS: i i

Splittings and Convergence THM: A large family of iteration Proof: (Golub p511) Remarks:

Splittings and Convergence THM: Proof: (Golub p512) show that all eigenvalues are less than one. Positive Definite Symmetric Stiffness Matrix Mass Matrix

Example: Splittings and Convergence DEF: IF THM:

Successive over Relaxation The Gauss-Seidel iteration is very attractive because of its simplicity. Unfortunately, if the spectral radius is close to one, then convergence is vey slow. One solution for this Successive over Relaxation GS:

Successive over Relaxation Example:

K=1K=2K=3 x1 x2 x3 X4 Successive over Relaxation Example:

MATLAB CODE Jacobi iteration for general n: Ex: Write a Matlab function for Jacobi function [sol,X]=jacobi(A,b,x0) n=length(b); maxiter=10; x=x0; for k=1:maxiter for i=1:n sum1=0; for j=1:i-1 sum1=sum1+A(i,j)*x(j); end sum2=0; for j=i+1:n sum2=sum2+A(i,j)*x(j); end xnew(i)=(b(i)-sum1-sum2)/A(i,i) end X(1:n,k)=xnew; x=xnew; end sol=xnew;

MATLAB CODE GS iteration for general n: Ex: Write a Matlab function for GS function [sol,X]=gs(A,b,x0) n=length(b); maxiter=10; x=x0; for k=1:maxiter for i=1:n sum1=0; for j=1:i-1 sum1=sum1+A(i,j)*x(j); end sum2=0; for j=i+1:n sum2=sum2+A(i,j)*x(j); end x(i)=(b(i)-sum1-sum2)/A(i,i) end X(1:n,k)=x; end sol=x;