1. Convergence in Numerical Science
12/11/2018
Kihwan Lee

2. Classification of Errors in Numerical Science
- 1 ρ 𝜕𝑝 𝜕𝑛 = 𝑢2 𝑅 , where R = radius of curvature
𝜎max=𝜎 𝑎 𝑅 , where 2*a = crack length, R = radius of curvature
Physical approximation error
Physical modeling error: is the governing equation representative of the actual physics?
Modeling approximation is always present.
Good example: inviscid approximation, turbulence models
Geometry modeling error: is the geometry good enough to capture the actual physics?
Round and corners
Is far-field boundary far enough? ~ 50 to 100 times of characteristic length
Is boundary layer resolved enough?
Is a round round-enough?
Round corners introduces additional acceleration to the fluid (centrifugal).
Is a sharp corner too sharp?
Is the stress concentration too much?
Computer round-off error
Subtraction error
Known problem in the computational science.
Active research in the effort of transforming the governing equation to remove “subtraction” operation in the formulation
Precision
32 bit computer: good up to 7 digit
64 bit computer: good up to 15 digit
Considering round-off errors and subtraction errors, about 3 ~ 4 digits in 32-bit computer and about 5 ~ 6 digits in the 64-bit computer is trusted
Iterative convergence error
Solver error
Direct vs. iterative solver
Multi-grid vs. single grid solver
Matrix behavior
Condition number
Importance of proper scaling – preconditioning matrix used often
Discretization error
Spatial discretization error
How close is the derivative to the definition of “derivative”?
Temporal discretization error
Are all required time scale resolved properly to capture the propagation of physical information?
Typical error estimate
Error ~ O(Δ2), where 2nd order is accepted as standard.
Programming error
Higher level programming error – Fortran, C, C++
Lower level programming error – compiler, chip instruction
System configuration error
Library version mismatch
User error
Often biggest source of error
Misinterpretation
Lack of understanding of the governing physics

3. What is Convergence?
Achievement of a limiting behavior in the solution of the governing equations and is typically represented by the diminishing residuals of the numerical solutions.
It satisfies the governing equation used in the formulation
Finite difference method
Discretization on the differential form equation. Good for capturing differential quantities.
Finite volume method
Discretization on the integral form of equation. Good for captures conservative quantities.
Finite element method
Discretization on the weighted residual form of the integral equation. Introduces additional layer of approximation. Good for meshing. Generally non-conservative.
It satisfies the applied discretization
1st order, 2nd order
Upwind (using information from the upstream of the characteristic lines) and central difference
Does it follow physical flow of information?
Shock capturing has become successful only after taking this effect into consideration.
Numerical damping
Limiter: artificially introduced for the purpose of damping spurious solutions.
Additional terms to satisfy convergence criteria.
Precision
How much precision to trust?
About 5 ~ 6 digits from 64 bit computer, based on Navier Stokes solution.
Mesh
Boundary?
How good is a good mesh?
Often the given PDE is solved in the mesh generation process in order to minimize errors introduced by meshing alone.

4. Nature of Solution Structural Dynamics Typical solvers: Ansys, Abacus
PDE: elliptic, hyperbolic
Quantity of interest:
Displacement: solver output
Stress: derivative of the output
Fluid
Typical solvers: Ansys, Fluent, FloTherm
PDE: hyperbolic, elliptic
Quantity of interest: velocity, thermodynamic state, stress
Velocity: solver output
Thermodynamic state: solver output
Electricity & Magnetism
Typical solvers: HFSS
E-fields, M-fields: derivative of the output
Obtaining correct order of convergence on the derivative quantity often requires finer mesh compared to the obtaining convergence on the direct solver output.
PDE
Linear vs. Non-linear
Coefficient independent/dependent of the solution.
Elliptic
Diffusion. Smooth solution.
Diagonally dominant matrix. Better matrix convergence property.
Parabolic
Time-dependent diffusion. Similar to the elliptic case, but in transient.
Hyperbolic
Wave equation, convection equation.
Sensitive to stability criteria.
Requires finer resolution of time step to minimize numerical dissipation.
Strong influence from the off-diagonal element. Poor matrix convergence property.
Mixed direction of physical information flow at the boundaries.
Heat Transfer
Typical solvers: Sinda, Fluent, FloTherm
PDE: parabolic, elliptic
Quantity of interest:
Temperature: solver output
Heat flux: derivative of the output

5. Error Analysis Taylor series expansion
Gradient x0
x0 + Δx
x0 - Δx
Gradient with 1st order approximation
Taylor series expansion
Expansion of a differentiable function
1st order form
2nd order formulation
Accuracy of y(x0) depends on the relative size of Δ𝑥, y’’(x0), and y’’’(x0)
Where y’’(x0) or y’’’(x0) is non-negligible, reduce Δ𝑥
Where y’’(x0) or y’’’(x0) is near zero, moderate size of Δ𝑥 is acceptable
How large is acceptable?
Gradient with 2nd order approximation
𝑦 𝑥0+Δ𝑥 =𝑦 𝑥0 + Δ𝑥 ∗ 𝑦 ′ 𝑥0 + Δ𝑥 2 2 𝑦 ′′ 𝑥0 + remainders
𝑦 ′ 𝑥0 = 𝑦 𝑥0+Δ𝑥 −𝑦 𝑥0 Δ𝑥 − Δ𝑥 2 𝑦 ′′ 𝑥0
𝑦 ′ 𝑥0 = 𝑦 𝑥0+Δ𝑥 −𝑦 𝑥0−Δ𝑥 2Δ𝑥 − Δ𝑥 2 6 𝑦 ′′′ 𝑥0

6. Convergence Criteria Analysis
𝜕𝑓 𝜕𝑡 +𝑢 𝜕𝑓 𝜕𝑥 −𝜇 𝜕2𝑓 𝜕𝑥2 =0
Convection-Diffusion Equation:
Length scale to resolve both convection and diffusion terms is of the order of
Convection term dominates in the region with larger cell length.
Diffusion term dominates in the region with smaller cell length.
Each convection and diffusion term require different max time scale.
Boundary layer resolution dominates time step
Combined effect of diffusion and convection for accuracy with 𝜇 = 10-5 and u = 1m/s,
Presence of different time scale and length scales makes the problem hard to converge
Multi-grid approach is often used for faster convergence – the difference is dramatic.
Matrix pre-conditioning is often used for faster convergence.
𝐿= 𝜇 𝑢
𝜏𝑐𝑜𝑛𝑣≤ ∆𝑥 𝑢
,𝜏𝑑𝑖𝑓𝑓≤ ∆𝑥2 𝜇
∆𝑥≈ 10 −5

7. Number of Equations Stencil in RANS Solver Boundary Condition
Number of equations to solve
Heat Transfer
1 equation
Diffusion
Average solution
Structural Dynamics
3 equations
Fluid
5 equation mean flow + 1 ~ 7 equations for turbulence
6 ~ 12 times longer on the 1st cut assessment
Each cell utilizes information from 32 adjacent cells
Matrix size increase by O(N2) ~ O(N*log(N))
Requires more memory space
N*log(N) operation count increase
More time to swap and access memories in the cache.
Shorter time step requirement.
Takes loner to converge – slower propagation speed.
Stronger non-linearity
Convection + diffusion
Diffusion: averaged quantity
Convection:
Dependent on the previous value
The scheme becomes extremely dissipative without proper resolution of the current state
Requires fine resolution to stay within acceptable accuracy
Typical error analysis
Error ~ O(Δ2), where 2nd order is a standard error acceptance
Resolution of turbulence requires exponentially larger number of cells.
Boundary Condition

8. Useful Concepts in CFD

9. Residual 𝜕𝑓(𝑥) 𝜕𝑡 +𝑢(𝑥) 𝜕𝑓(𝑥) 𝜕𝑥 −𝜇 𝜕2𝑓(𝑥) 𝜕𝑥2 =0
For a given PDE,
𝜕𝑓(𝑥) 𝜕𝑡 +𝑢(𝑥) 𝜕𝑓(𝑥) 𝜕𝑥 −𝜇 𝜕2𝑓(𝑥) 𝜕𝑥2 =0
Focus on the steady state form of the PDE,
𝑢(𝑥) 𝜕𝑓(𝑥) 𝜕𝑥 −𝜇 𝜕2𝑓(𝑥) 𝜕𝑥2 =0
Take any x0 value and plug it into the steady state part of the PDE. If the x0 is not a solution, the value in the left hand side of the equation will not be equal to zero.
𝑢(𝑥0) 𝜕𝑓(𝑥0) 𝜕𝑥 −𝜇 𝜕2𝑓(𝑥0) 𝜕𝑥2 ≠0
Then, the amount of deviation is called residual.
𝑅 𝑥0 = 𝑢(𝑥0) 𝜕𝑓(𝑥0) 𝜕𝑥 −𝜇 𝜕2𝑓(𝑥0) 𝜕𝑥2 ≠0

10. CFL Number For a given PDE, After applying discretization,
𝜕𝑓(𝑥) 𝜕𝑡 +𝑢(𝑥) 𝜕𝑓(𝑥) 𝜕𝑥 =0
After applying discretization,
𝑓(𝑛+1,𝑖) −𝑓(𝑛,𝑖) Δ𝑡 +𝑢(𝑥) 𝑓(𝑛,𝑖+1) −𝑓(𝑛,𝑖) Δ𝑥 =0
After rearrangement,
𝑓 𝑛+1,𝑖 =𝑓 𝑛,𝑖 −𝑢(𝑥) Δ𝑡 Δ𝑥 (𝑓 𝑛,𝑖+1 −𝑓 𝑛,𝑖 )=0
The time to spatial discretization ratio is called Courant-Friedrichs-Lewy (CFL) number. CFL number larger than 1 corrupts the obtained solution – it calculates physical impossible solution.
The full numerical domain of dependence must contain the physical domain of dependence.
CFL=𝑢(𝑥) Δ𝑡 Δ𝑥 <1

11. Relaxation 𝐴𝑥=𝑏 𝐴=𝐷+𝑈+𝐿 𝐷+ω𝐿 𝑥 𝑛+1 =ω𝑏− ω𝑈+ ω−1 𝐷 𝑥 𝑛
For a matrix form of a discretized PDE,
𝐴𝑥=𝑏
The matrix A is decomposed into a diagonal, upper triangular, and lower triangular matrices
𝐴=𝐷+𝑈+𝐿
Then, the updated value can be obtained by the following formula with a relaxation factor, ω
𝐷+ω𝐿 𝑥 𝑛+1 =ω𝑏− ω𝑈+ ω−1 𝐷 𝑥 𝑛
In simple terms, it is a measure of how much correction to apply for the current update. In other perspective, it is a measure of mixing old and the new solution .
If 0<ω<1, it is called under-relaxation.
If 1<ω<2, it is called over-relaxation.

12. Multigrid
Most advanced convergence acceleration technique along with preconditioner.
Achieves small scale and large scale convergence at the same time.
V-cycle and W-cycle typically used

13. Back-Up

14. Physical Approximation Error 1/2
Viscous Governing Equation
Reality
Inviscid Governing Equation
Turbulence Governing Equation

15. Physical Approximation Error 2/2
Forward difference
y 𝑥0+Δ𝑥 =𝑦 𝑥0 + Δ𝑥 ∗ 𝑦 ′ 𝑥0 + Δ𝑥 2 2 𝑦 ′′ 𝑥0 + remainders, where x is near x0
y 𝑥0+Δ𝑥 −𝑦 𝑥0 = Δ𝑥 ∗ 𝑦 ′ 𝑥0 + Δ𝑥 2 2 𝑦 ′′ 𝑥0
𝑦 ′ 𝑥0 = 𝑦 𝑥0+Δ𝑥 −𝑦 𝑥0 Δ𝑥 − Δ𝑥 2 𝑦 ′′ 𝑥0
Slope approximation + O(Δ𝑥)
Using information from one additional node
1 stencil
Central difference
y 𝑥0+Δ𝑥 −𝑦 𝑥0 = Δ𝑥 ∗ 𝑦 ′ 𝑥0 + Δ𝑥 2 2 𝑦 ′′ 𝑥 Δ𝑥 3 6 𝑦 ′′′ 𝑥0
y 𝑥0−Δ𝑥 −𝑦 𝑥0 = −Δ𝑥 ∗ 𝑦 ′ 𝑥0 + −Δ𝑥 2 2 𝑦 ′′ 𝑥 −Δ𝑥 3 6 𝑦 ′′′ 𝑥0
𝑦 ′ 𝑥0 = 𝑦 𝑥0+Δ𝑥 −𝑦 𝑥0−Δ𝑥 2Δ𝑥 − Δ𝑥 2 6 𝑦 ′′′ 𝑥0
Slope approximation + O(Δ𝑥2)
Using information from two additional node
2 stencils
More stencils
Higher order scheme possible
Computationally costly
Boundary condition not well defined
How to convert from continuous formula to discrete formula?

16. Finite Volume Formulation 1/2

17. Finite Volume Formulation 2/2

18. Finite Difference

19. Finite Element 1/2

20. Finite Element 2/2

21. Time Scale Time scale Physical length Flow of information Start-up
Too small or too large
Flow of information
Diffusion: Average. Less sensitive to spatial resolution.
Convection: Directional flow. Finer spatial resolution required to satisfy conservation.
Coarse grid introduces excessive loss.
Start-up
Reaching the steady state may take a long time, depending on the start-up condition
Numerical instability
CFL number
CSV (C/Sum(G))
Reduce time step
Lack of damping
Increase relaxation factor
Slower update
Artificial damping
Modification of the governing equation to achieve better convergence condition, but with same order of accuracy

22. Condition Number
Condition number in general means a measure of how much the output changes for a small change in the input.
For a linear system, Ax = b
Condition number for the matrix A in Ax = b gives a bound on how inaccurate the solution will be after approximation.
The rate at which the solution x will change with respect to the change in b.
It is defined as the maximum ratio of the relative error in x to the relative error in b.
Pre-conditioner generally reduces the condition number of the matrix
Faster convergence.
Solution with less error.