Review Taylor Series and Error Analysis Roots of Equations Linear Algebraic Equations Optimization Numerical Differentiation and Integration Ordinary Differential Equations Partial Differential Equations Curve Fitting Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Taylor Series Lagrange remainder Numerical Methods Prof. Jinbo Bi Through mean-value theorem, we can derive the Lagrange remainder for Taylor Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Roots of Equations Bracketing Methods Open Methods Bisection Method False Position Method Open Methods Fixed Point Iteration Newton-Raphson Method Secant Method Roots of Polynomials Müller’s Method Bairstow’s Method Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Bisection Method Example: Use range of [202:204] Root is in upper subinterval Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Bisection Method Use range of [203:204] Root is in lower subinterval Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Fixed Point Iteration Example Special attention Read Chap 6.1, 6.6 Fixed Point Iteration Example Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Newton-Raphson Method Use tangent to guide you to the root Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Linear Algebraic Systems Gaussian Elimination Forward Elimination Back Substitution LU Decomposition Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Gaussian Elimination Forward elimination Eliminate x1 from row 2 Multiply row 1 by a21/a11 Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Gaussian Elimination Eliminate x1 from row 2 Subtract row 1 from row 2 Eliminate x1 from all other rows in the same way Then eliminate x2 from rows 3-n and so on Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Gaussian Elimination Forward elimination Back substitute to solve for x Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
LU Decomposition Substitute the factorization into the linear system We have transformed the problem into two steps Factorize A into L and U Solve the two sub-problems LD = B UX = D Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
LU Decomposition Example Factorize A using forward elimination Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
LU Decomposition Example Numerical Methods Prof. Jinbo Bi Lecture 22 CSE, UConn
LU Decomposition Example Numerical Methods Prof. Jinbo Bi Lecture 22 CSE, UConn
LU Decomposition Example Numerical Methods Prof. Jinbo Bi Lecture 22 CSE, UConn
Optimization Methods One-dimensional unconstrained optimization Golden-Section Quadratic Interpolation Newton’s Method Multidimensional unconstrained optimization Direct Methods Gradient Methods Constrained Optimization Linear Programming Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Golden-section search Algorithm Pick two interior points in the interval using the golden ratio Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Golden-section search Two possibilities Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Golden-section search Example Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Golden-section search Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Golden-section search Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Newton’s Method Newton-Raphson could be used to find the root of an function When finding a function optimum, use the fact that we want to find the root of the derivative and apply Newton-Raphson Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Newton’s Method Example Numerical Methods Prof. Jinbo Bi Lecture 22 CSE, UConn
Newton’s Method Example Numerical Methods Prof. Jinbo Bi Lecture 22 CSE, UConn
Quadratic interpolation Special attention Quadratic interpolation Use a second order polynomial as an approximation of the function near the optimum Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Special attention Gradient Methods Given a starting point, use the gradient to tell you which direction to proceed The gradient gives you the largest slope out from the current position Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Numerical Integration Newton-Cotes Trapezoidal Rule Simpson’s Rules (Special attention for unevenly distributed points) Romberg Integration Gauss Quadrature Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Newton-Cotes Formulas Special attention Read Chap 21.2-3 Newton-Cotes Formulas Trapezoidal Rule Simpson’s 1/3 Rule Simpson’s 3/8 Rule Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Integration of Equations Romberg Integration Use two estimates of integration and then extrapolate to get a better estimate Special case where you always halve the interval - i.e. h2=h1/2 Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Romberg Integration Numerical Methods Prof. Jinbo Bi Lecture 22 CSE, UConn
Ordinary Differential Equations Runge-Kutta Methods Euler’s Method Heun’s Method RK4 Multistep Methods Boundary Value Problems Eigenvalue Problems Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Euler’s Method Example: True: h=0.5 Numerical Methods Prof. Jinbo Bi Lecture 22 Prof. Jinbo Bi CSE, UConn
Heun’s Method Local truncation error is O(h3) and global truncation error is O(h2) Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Heun’s Method Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Classic 4th-order R-K method Special attention to ODE equation system Not only one equation Classic 4th-order R-K method Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Curve Fitting Least Squares Regression Interpolation Fourier Approximation Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Polynomial Regression Special attention Lecture note 19 Chap 17.1 Polynomial Regression An mth order polynomial will require that you solve a system of m+1 linear equations Standard error Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Newton (divided difference) Interpolation polynomials Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Newton (divided difference) Interpolation polynomials Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Interpolation General Scheme for Divided Difference Coefficients Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Interpolation General Scheme for Divided Difference Coefficients Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Interpolation Example: Estimate ln 2 with data points at (1,0), (4,1.386294) Linear interpolation Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Interpolation Example: Estimate ln 2 with data points at (1,0), (4,1.386294), (5,1.609438) Quadratic interpolation Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Interpolation Example: Estimate ln 2 with data points at (1,0), (4,1.386294), (5,1.609438), (6,1.791759) Cubic interpolation Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Spline Interpolation Spline interpolation applies low-order polynomial to connect two neighboring points and uses it to interpolate between them. Typical Spline functions Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Cubic Spline Functions This gives us n-1 equation with n-1 unknowns – the second derivatives Once we solve for the second derivatives, we can plug it into the Lagrange interpolating polynomial for second derivative to solve for the splines Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Cubic Spline Functions Example: (3,2.5), (4.5,1), (7,2.5), (9,0.5) At x=x1=4 Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Cubic Spline Functions At x=x2=7 Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Cubic Spline Functions Solve the system of equations to find the second derivatives Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Cubic Spline Equations Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Cubic Spline Equations Substituting for other intervals Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn
Final Exam December 13 Friday, 10:30 AM~12:30 PM, ITE 119 Closed book, three cheat sheets (8.5x11in) allowed Office hours: December 12, 1-3pm, or by appointment TA December 10, 11am-12noon or by appointment Numerical Methods Lecture 22 Prof. Jinbo Bi CSE, UConn