Ordinary Differential Equations (ODEs) 1Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers Daniel Baur ETH Zurich, Institut für Chemie- und Bioingenieurwissenschaften ETH Hönggerberg / HCI F128 – Zürich

Runge-Kutta methods  If we approximate the slope field more accurately between two iteration steps, we can generate methods higher local and global accuracy orders  Approximating the slope field in m stages: 2Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Butcher tableau  A convenient way of of writing down explicit RK methods is the Butcher tableau 3Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers Example: Heun, order 2

Definition of an implicit algorithm  In an implicit algorithm, the function value at the next step appears on the right hand side of the step equation:  A simple example is the Backward Euler method:  A system of equations has to be solved at every iteration step; Why even use implicit algorithms? 4Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Example  Consider a batch reactor where two reactions take place  This can be written in matrix form: 5Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Stability of numerical ODE solvers  Linear constant coefficient ODE systems such as this are used as «test-equations» for solvers  Example: Forward Euler  This iteration will only converge to a value if the spectral radius of the iteration matrix is smaller than 1: 6Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Stability region of the forward Euler algorithm  The forward Euler algorithm is stable if  Since eigenvalues are complex in general, we need to use the definition of the complex absolute value  Therefore the algorithm is stable if 7Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Visualizing the stability region  We can plot the value of our stability equation as a function of ha and hb 8Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Backward Euler Method  As mentioned before, the simplest implicit method is the Backward Euler method  If we substitute our problem  We obtain  In case of linear ODEs, this is a linear system of the form Ax = b that has to be solved at every iteration step  Note that in general this can be a non-linear system 9Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Stability of the Backward Euler Method  If we solve for the next step, we get  Again the spectral radius of the iteration matrix determines convergence:  Therefore, the algorithm is stable if 10Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Visualizing the stability region a = Re(λ) max < 0  We see that if a = Re(λ) max < 0, the stability is independent on the step size  This property is called A-stability 11Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

A-Stability of other algorithms  Explicit Runge-Kutta methods, as well as explicit multi-step methods can never be A-stable  Note that the Forward Euler method can be seen as both an explicit multi-step method or RK method of order 1  Implicit multi-step methods can only be A-stable if they are of order 2 or lower (second Dahlquist barrier)  However, implicit RK methods of higher order can be A- stable 12Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Stiff problems  A problem is considered stiff if e.g. the following apply:  A linear constant coefficient system is stiff if all of its eigenvalues have a negative real part and the stiffness ratio is large  Stability requirements, rather than accuracy considerations, constrain the step length  Some components of the solution show much faster dynamics than others  Explicit solvers require an impracticably small h to insure stability, therefore implicit solvers are preferred 13Daniel Baur / Numerical Methods for Chemical Engineers / Explicit ODE Solvers

Implicit algorithms and stiff problems  Since implicit algorithms are generally more stable than explicit algorithms (some are even A-stable!), they fare much better with stiff problems, where the step size is often restricted by stability issues  For non-A-stable implicit algorithms, the main goal is usually to get the largest possible stability region, since this is the main advantage of implicit algorithms  However, this stability comes at the price of larger computational demand per step, which is needed to solve the arising algebraic equation systems  There are however highly specialized algorithms to solve systems arising from implicit solvers, which can take into account special features of the problem like sparseness or bandedness 14Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Sparse and banded matrices  It is computationally very advatageous if sparse or in the best case even banded matrices can be used: 15Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers Sparse: Storing and operating on only 510 non-zero elements (times two for their position) instead of 10’000 Banded: All non-zero elements are grouped in a band around the diagonal, which further simplifies computations

Variable step size and order algorithms  Since the step size h is of such importance for stability and accuracy, sophisticated algorithms adjust it during runtime  This requires error estimation, which is usually done by comparing the result to what the same algorithm produces with a higher order  Some algorithms even adjust their order p dynamically 16Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Assignment 1.Consider the batch reactor given on slide 6. What is the maximum step size h that would give a stable forward Euler solution? det(A – λI) = 0 λ  Remember that the eigenvalues of a matrix can be calculated by solving det(A – λI) = 0 for λ  The determinant of a 2x2 matrix is 2.What is the maximum step size h that would give a stable backward Euler solution? 17Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers

Assignment (continued) 3.From the Butcher tableaus, implement the 2nd order Heun method and «the» 4th order RK method.  Solve the batch reactor and Plot the solutions.  The Butcher tableaus for the methods are: k 1 = 1; k 2 = 2; E 0 = 1; I 0 = 0; tSpan = [0,10];  Use k 1 = 1; k 2 = 2; E 0 = 1; I 0 = 0; tSpan = [0,10];  The analytical solution reads  What step size h is needed to get a maximum error of 0.1% at all time points, compared to the analytical solutions? 18Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers Heun 2 RK 4

Assignment (continued) 4.Change k 2 to ode45  Do the explicit algorithms still work? Try to solve the system with ode45. Does it perform well? What could be the problem? 5.Implement the backward Euler algorithm to solve the new system with k 2 = and plot the solutions. fsolve  Note that you cannot just put the backward Euler formula into Matlab as is! Use fsolve to solve for y n+1. ones(size(y0)) fsolve  Use ones(size(y0)) as initial guess for fsolve options = optimset('display','off'); y(i+1,:) = fsolve( , options);  You might want to use these to avoid spamming you command line: options = optimset('display','off'); y(i+1,:) = fsolve( , options);  What can you say about the behavior of the chemical reaction when you plot it? 19Daniel Baur / Numerical Methods for Chemical Engineers / Implicit ODE Solvers