1. CSE 245: Computer Aided Circuit Simulation and Verification
Fall 2004, Nov
Lecture 8:
Numerical Integration

2. Outline One-step Method for ODE (IVP) Linear MultiStep Method
Forward Euler
Backward Euler
Trapezoidal Rule
Equivalent Circuit Model
Linear MultiStep Method
Convergence Analysis
Consistence
Stability
Time Step Control (next lecture)
Stability Region
Stiff System
Dynamic Time Step Control
Over-Relaxation Method & ADI
November 27, 2018

3. Circuit Equation Formulation
For dynamical circuits the equations can be written compactly:
For sake of simplicity, we shall discuss first order ODEs in the form:
November 27, 2018
courtesy Alessandra Nardi UCB

4. Ordinary Differential Equations
Typically analytic solutions are not available
 solve it numerically
November 27, 2018
courtesy Alessandra Nardi UCB

5. Ordinary Differential Equations
Assumptions and Simplifications
Not necessarily a solution exists and is unique for:
It turns out that, under rather mild conditions on the continuity and differentiability of F, it can be proven that there exists a unique solution.
Also, for sake of simplicity only consider
linear case:
We shall assume that has a unique solution
November 27, 2018
courtesy Alessandra Nardi UCB

6. Finite Difference Methods
Basic Concepts
First - Discretize Time
Second - Represent x(t) using values at ti
Approx.
sol’n
Exact
sol’n
Third - Approximate using the discrete
November 27, 2018
courtesy Alessandra Nardi UCB

7. Forward Euler Approximation
November 27, 2018
courtesy Alessandra Nardi UCB

8. Forward Euler Approximation
November 27, 2018
courtesy Alessandra Nardi UCB

9. Backward Euler Approximation
November 27, 2018
courtesy Alessandra Nardi UCB

10. Backward Euler Approximation
Solve with Gaussian Elimination
November 27, 2018
courtesy Alessandra Nardi UCB

11. Trapezoidal Rule Approximation
November 27, 2018
courtesy Alessandra Nardi UCB

12. courtesy Alessandra Nardi UCB
Trapezoidal Rule Approximation
Solve with Gaussian Elimination
November 27, 2018
courtesy Alessandra Nardi UCB

13. Numerical Integration View
Trap BE FE
November 27, 2018
courtesy Alessandra Nardi UCB

14. Equivalent Circuit Model-BE
Capacitor + + + C - - -
November 27, 2018

15. Equivalent Circuit Model-BE
Inductor + + - + L - -
November 27, 2018

16. Equivalent Circuit Model-TR
Capacitor + + + C - - -
November 27, 2018

17. Equivalent Circuit Model-TR
Inductor + + - + L - -
November 27, 2018

18. Summary of Basic Concepts
Trap Rule, Forward-Euler, Backward-Euler
Are all one-step methods
Forward-Euler is simplest
No equation solution explicit method.
Boxcar approximation to integral
Backward-Euler is more expensive
Equation solution each step implicit method
Trapezoidal Rule might be more accurate
Trapezoidal approximation to integral
November 27, 2018
courtesy Alessandra Nardi UCB

19. courtesy Alessandra Nardi UCB
Outline
One-step Method for ODE (IVP)
Forward Euler
Backward Euler
Trapezoidal Rule
Linear MultiStep Method
Convergence Analysis
Consistence
Stability
Stiff System and Time Step Control (next lecture)
Stiff System
Dynamic Time Step Control
November 27, 2018
courtesy Alessandra Nardi UCB

20. Linear Multistep Method (LMS)
Basic Equations
Nonlinear Differential Equation:
k-Step Multistep Approach:
Multistep coefficients
Solution at discrete points
Time discretization
November 27, 2018
courtesy Alessandra Nardi UCB

21. LMS: Common Algorithm TR, BE, FE are one-step methods
Multistep Equation:
Forward-Euler Approximation:
FE Discrete Equation:
Multistep Coefficients:
BE Discrete Equation:
Multistep Coefficients:
Trap Discrete Equation:
Multistep Coefficients:
November 27, 2018
courtesy Alessandra Nardi UCB

22. Adams-Bashforth formula
0 =0
The first order Adams-Bashforth formula (forward Euler)
The second order Adams-Bashforth formula
November 27, 2018

23. Adams-Moulton formula
0 0
The first order Adams-Moulton formula (backward Euler)
The second order Adams-Moulton formula (trapezoidal)
November 27, 2018

24. Convergence Analysis Convergence for one-step methods
Consistency for FE
Stability for FE
Convergence for multistep methods
Consistency (Exactness Constraints)
Selecting coefficients
Stability
Region of Absolute Stability
Dahlquist’s Stability Barriers
November 27, 2018

25. LMS: Convergence Analysis
Definition: A finite-difference method for solving initial value problems on [0,T] is said to be convergent if given any A and any initial condition
November 27, 2018
courtesy Alessandra Nardi UCB

26. LMS: Convergence Analysis
Order-p Convergence
Definition: A multi-step method for solving initial value problems on [0,T] is said to be order p convergent if given any A and any initial condition
Forward- and Backward-Euler are order 1 convergent
Trapezoidal Rule is order 2 convergent
November 27, 2018
courtesy Alessandra Nardi UCB

27. Convergence Analysis (1)
November 27, 2018
courtesy Alessandra Nardi UCB

28. Convergence Analysis (2)
For convergence we need to look at max error over the whole time interval [0,T]
We look at GTE
Not enough to look at LTE, in fact:
As I take smaller and smaller timesteps Dt, I would like my solution to approach exact solution better and better over the whole time interval, even though I have to add up LTE from more timesteps.
November 27, 2018
courtesy Alessandra Nardi UCB

29. Convergence Analysis (3)
1) Local Condition: One step errors are small (consistency)
Typically verified using Taylor Series
Exactness Constraints up to p0 (p0 must be > 0)
2) Global Condition: The single step errors do not grow too quickly (stability)
All one-step methods are stable in this sense.
November 27, 2018
courtesy Alessandra Nardi UCB

30. courtesy Alessandra Nardi UCB
Consistency
Definition: A one-step method for solving initial value problems on an interval [0,T] is said to be consistent if for any A and any initial condition
November 27, 2018
courtesy Alessandra Nardi UCB

31. One-step Methods – Convergence Analysis
Consistency for Forward Euler
Forward-Euler definition
Expanding in t about zero yields
Proves the theorem if derivatives of x are bounded
November 27, 2018
courtesy Alessandra Nardi UCB

32. One-step Methods – Convergence Analysis
Convergence Analysis for Forward Euler
Forward-Euler definition
November 27, 2018
courtesy Alessandra Nardi UCB

33. One-step Methods – Convergence Analysis
Convergence Analysis for Forward Euler
Subtracting the previous slide equations
Define the "Global" error
Taking norms and using the bound on l e
November 27, 2018
courtesy Alessandra Nardi UCB

34. One-step Methods – Convergence Analysis
A helpful bound on difference equations
A lemma bounding difference equation solutions
To prove, first write as a power series and sum u l
November 27, 2018
courtesy Alessandra Nardi UCB

35. One-step Methods – Convergence Analysis
A helpful bound on difference equations
Mapping the global error equation to the lemma
November 27, 2018
courtesy Alessandra Nardi UCB

36. One-step Methods – Convergence Analysis
Back to Convergence Analysis for Forward Euler
Applying the lemma and canceling terms
November 27, 2018
courtesy Alessandra Nardi UCB

37. One-step Methods – Convergence Analysis
Observations about Convergence Analysis for FE
Forward-Euler is order 1 convergent
The bound grows exponentially with time interval
C is related to the solution second derivative
The bound grows exponentially fast with norm(A).
November 27, 2018
courtesy Alessandra Nardi UCB

38. courtesy Alessandra Nardi UCB
Convergence
Convergence for one-step methods
Consistency for FE
Stability for FE
Convergence for multistep methods
Consistency (Exactness Constraints)
Selecting coefficients
Stability
Region of Absolute Stability
Dahlquist’s Stability Barriers
November 27, 2018
courtesy Alessandra Nardi UCB

39. Linear Multistep Methods (LMS)
Definition and Observations
Multistep Equation:
How does one pick good coefficients?
Want the highest accuracy
November 27, 2018
courtesy Alessandra Nardi UCB

40. Linear Multistep Methods (LMS)
Simplified Problem for Analysis
Scalar ODE:
Why such a simple Test Problem?
Nonlinear Analysis has many unrevealing subtleties
Scalar equivalent to vector for multistep methods.
multistep
discretization
Decoupled Equations
November 27, 2018
courtesy Alessandra Nardi UCB

41. Linear Multistep Methods
Simplified Problem for Analysis
Scalar ODE:
Scalar Multistep formula:
Decaying
Solutions
Osci l
lations
Growing
Solutions
November 27, 2018
courtesy Alessandra Nardi UCB

42. Multistep Methods – Convergence Analysis
Global Error Equation
Multistep formula:
Exact solution Almost
satisfies Multistep Formula:
Local Truncation Error (LTE)
Global Error:
Subtracting yields difference equation for global error
November 27, 2018
courtesy Alessandra Nardi UCB

43. Multistep Methods – Making LTE small
Exactness Constraints
Multistep methods are designed so that they are exact for a polynomial of order p. These methods are said to be of order p.
November 27, 2018
courtesy Alessandra Nardi UCB

44. Multistep Methods – Making LTE small
Exactness Constraints If
As any smooth v(t) has a locally accurate Taylor series in t: if
Then
November 27, 2018
courtesy Alessandra Nardi UCB

45. Multistep Methods – Making LTE small
Exactness Constraints – k=2 Example
For k=2, yields a 5x6 system of equations for Coefficients
p=0
p=1
p=2
p=3
p=4
November 27, 2018
courtesy Alessandra Nardi UCB

46. Multistep Methods – Making LTE small
Exactness Constraints – k=2 Example
Exactness Constraints for k=2
November 27, 2018
courtesy Alessandra Nardi UCB

47. Multistep Methods – Making LTE small
Exactness Constraints k=2 Example, generating Methods
Solve for the 2-step method with lowest LTE
Solve for the 2-step explicit method with lowest LTE
November 27, 2018
courtesy Alessandra Nardi UCB

48. Multistep Methods – Making LTE small10 -4 -3 -2 -1
-15
-10 -5 FE
LTE
Trap
Best Explicit Method has highest one-step
accurate
Beste
Timestep
November 27, 2018
courtesy Alessandra Nardi UCB

49. Multistep Methods – Making LTE small
Max Error FE
Where’s BESTE?
Trap
Timestep
November 27, 2018
courtesy Alessandra Nardi UCB

50. Multistep Methods – Making LTE small
worrysome 10 -4 -3 -2 -1
-100
100
200
Max Error
Best Explicit Method has lowest one-step error but global errror increases as timestep decreases
Beste
Trap FE
Timestep
November 27, 2018
courtesy Alessandra Nardi UCB

51. LMS: Stability
Why did the “best” 2-step explicit method fail to Converge?
Multistep Method Difference Equation
LTE
Global Error
We made the LTE so small, how come the Global error is so large?
November 27, 2018
courtesy Alessandra Nardi UCB

52. courtesy Alessandra Nardi UCB
Convergence
Convergence for one-step methods
Consistency for FE
Stability for FE
Convergence for multistep methods
Consistency (Exactness Constraints)
Selecting coefficients
Stability
Region of Absolute Stability
Dahlquist’s Stability Barriers
November 27, 2018
courtesy Alessandra Nardi UCB

53. An Aside on Solving Difference Equations
Consider a general kth order difference equation
Which must have k initial conditions
As is clear when the equation is in update form
Most important difference equation result
November 27, 2018
courtesy Alessandra Nardi UCB

54. An Aside on Solving Difference Equations
To understand how h is derived, first a simple case
November 27, 2018
courtesy Alessandra Nardi UCB

55. An Aside on Solving Difference Equations
Three important observations
November 27, 2018
courtesy Alessandra Nardi UCB

56. LMS: Convergence Analysis
Conditions for convergence – Consistency & Stability
1) Local Condition: One step errors are small (consistency)
Exactness Constraints up to p0 (p0 must be > 0)
2) Global Condition: One step errors grow slowly (stability)
Convergence Result:
November 27, 2018
courtesy Alessandra Nardi UCB

57. Multistep Methods – Stability
Difference Equation
Multistep Method Difference Equation
Definition: A multistep method is stable if and only if
Theorem: A multistep method is stable if and only if
Less than one in magnitude or equal to one and distinct
November 27, 2018
courtesy Alessandra Nardi UCB

58. Multistep Methods – Stability
Stability Theorem Proof
Given the Multistep Method Difference Equation
are either
If the roots of
less than one in magnitude
equal to one in magnitude but distinct
Then from the aside on difference equations
From which stability easily follows.
November 27, 2018
courtesy Alessandra Nardi UCB

59. Multistep Methods – Stability
Stability Theorem Proof Im Re -1 1
November 27, 2018
courtesy Alessandra Nardi UCB

60. Multistep Methods – Stability
A more formal approach
Def: A method is stable if all the solutions of the associated difference equation obtained from (1) setting q=0 remain bounded if l
The region of absolute stability of a method is the set of q such that all the solutions of (1) remain bounded if l
Note that a method is stable if its region of absolute stability contains the origin (q=0)
November 27, 2018
courtesy Alessandra Nardi UCB

61. courtesy Alessandra Nardi UCB
LMS: A-Stable
Def: A method is A-stable if the region of absolute stability contains the entire left hand plane (in the  space)
Re(z)
Im(z) -1 1
Im()
Re() -1 1
November 27, 2018
courtesy Alessandra Nardi UCB

62. courtesy Alessandra Nardi UCB
LMS: Stability
Each method is associated with two polynomials a and b:
a : associated with function past values
b: associated with derivative past values
Stability: roots of a must stay in |z|1 and be simple on |z|=1
Absolute stability: roots of (a-bq) must stay in |z|1 and be simple on |z|=1 when Re(q)<0.
November 27, 2018
courtesy Alessandra Nardi UCB

63. courtesy Alessandra Nardi UCB
LMS: Stability
Dahlquist’s First Stability Barrier
For a stable, explicit k-step multistep method, the maximum number of exactness constraints that can be satisfied is less than or equal to k (note there are 2k coefficients). For implicit methods, the number of constraints that can be satisfied is either k+2 if k is even or k+1 if k is odd.
November 27, 2018
courtesy Alessandra Nardi UCB

64. Stabilities
Froward Euler
November 27, 2018

65. FE region of absolute stability
Forward Euler
ODE stability region
Im(z)
Difference Eqn
Stability region
Region of
Absolute Stability
Re(z) -1 1
November 27, 2018
courtesy Alessandra Nardi UCB

66. Stabilities
Backward Euler
November 27, 2018

67. courtesy Alessandra Nardi UCB
BE region of absolute stability
Backward Euler
Im(z)
Difference Eqn
Stability region
Re(z) -1 1
Region of
Absolute Stability
November 27, 2018
courtesy Alessandra Nardi UCB

68. Stabilities
Trapezoidal
November 27, 2018

69. courtesy Alessandra Nardi UCB
Summary
Convergence for one-step methods
Consistency for FE
Stability for FE
Convergence for multistep methods
Consistency (Exactness Constraints)
Selecting coefficients
Stability
Region of Absolute Stability
Dahlquist’s Stability Barriers
November 27, 2018
courtesy Alessandra Nardi UCB