“Solve the equations right” Mathematical problem Validation : Definitions Verification : “Solve the equations right” Mathematical problem Validation : “Solve the right equations” Science/engineering activity Roache, 1998 1

Verification includes two different activities: - Code Verification Definitions Verification includes two different activities: - Code Verification Guarantee that the computer code does not contain errors (bugs). Error evaluation that requires the knowledge of the exact solution - Solution/calculations Verification Estimate the numerical uncertainty for a numerical solution that has an unkown exact solution (Error estimation) 2

- Assessing the modeling uncertainty of Definitions Validation - Assessing the modeling uncertainty of (a selected aspect of) a mathematical model Needed: the definition of the quantities of interest the selection of a metric to quantify the modeling uncertainty (on-going topic of research) 3

An error is defined as the difference between Definitions − Error vs Uncertainty An error is defined as the difference between a given solution and its “true/exact” value. It has a sign and it requires the knowledge of the “truth” An uncertainty defines an interval that should contain the “true/exact” value with a certain degree of confidence. It is defined with a ∓ 4

- Discretization error Three components of the numerical error behave - Round-off error - Iterative error - Discretization error Three components of the numerical error behave differently with the increase of the number of degrees of freedom (grid refinement) 5

- Due to the finite precision of computers Numerical Error Round-off error: - Due to the finite precision of computers - Decreased by the use of double precision - May be dominant in ill-conditioned problems - It grows with the increase of the number of degrees of freedom (grid refinement) 6

− Round-off error, example: Polynomial interpolation in 2-D (or 3-D) Numerical Error − Round-off error, example: Polynomial interpolation in 2-D (or 3-D) (n+1)2 coefficients determined from (n+1)2 points where fi(xi,yi) is known 7

− Round-off error, example: Polynomial interpolation in 2-D (or 3-D) Numerical Error − Round-off error, example: Polynomial interpolation in 2-D (or 3-D) Coefficients of the polynomial obtained from the solution of a linear systems of algebraic equations First term of the main diagonal : 1 Last term of the main diagonal : xnyn 8

- Non-linear equations (convection in momentum equations) Numerical Error Iterative error: - Non-linear equations (convection in momentum equations) - Decoupling of the equations (turbulence model solved for a given mean velocity field and momentum equations solved with a “known” eddy-viscosity) 9

- Discretization schemes with explicit corrections Numerical Error Iterative error: - Discretization schemes with explicit corrections for the high-order terms - Solution of the systems of algebraic equations with iterative methods (Jacobi, Gauss-Seidel, Conjugate Gradients, GMRES, ...) 10

- In principle, it can be reduced to machine Numerical Error Iterative error: - In principle, it can be reduced to machine accuracy (if there are no problems with the round-off error) - The reduction of the iterative error becomes hardest with the increase of the number of degrees of freedom (grid refinement). Multigrid techniques may avoid problems with the size of the system of algebraic equations to solve 11

- It is important to define (know) the meaning of one iteration Numerical Error Iterative error: - It is important to define (know) the meaning of one iteration - Iterative error estimates based on differences or normalized residuals of the last iteration performed are not reliable - For iterative error estimates, the L∞ norm is a more appropriate choice than the L1and L2 norms 12

- RANS (with eddy-viscosity model) calculation Numerical Error Iterative error: - RANS (with eddy-viscosity model) calculation of the turbulent flow in a channel - Initial velocity field obtained from the specified profiles at the inlet - Solution converged to machine precision (10-14) 13

- Convergence criteria based on the maximum Numerical Error Iterative error: - Convergence criteria based on the maximum difference between consecutive iterations, et - Iterative error obtained from the difference to the solution converged to machine accuracy - Example of iterative errors for the horizontal velocity component, U1 14

Iterative error: Numerical Error Largest iterative error is 2 orders of magnitude larger than et! 15

Discretization error: Numerical Error Discretization error: - Consequence of the transformation of the continuum equation(s) into a system of algebraic equations - It may have a geometric component, which may even be the dominant contribution in domains bounded by surfaces with high curvature 16

Discretization error: - Usually, it is the main contribution to the Numerical Error Discretization error: - Usually, it is the main contribution to the numerical error - Can only be determined with the knowledge of the exact solution - Tends to diminish with the increase of the number of degrees of freedom (grid refinement) - Estimate of the discretization error may be obtained from grid refinement studies 17

Its goal is: check that the code is bug-free Code Verification Its goal is: check that the code is bug-free Does the numerical error tend to zero when the grid refinement / time step tends to zero? Is the observed order of convergence in agreement with the “theoretical order” of the code? M - If not, find out why?!

Error described by a Power Series Expansions Code Verification Error described by a Power Series Expansions fi → Local or functional flow quantity for grid i fexact → Exact solution of local or functional flow quantity eo → Error for cell size zero (supposed to be “0”) a → Constant hi → Typical cell size p → Observed order of grid convergence If

Geometrical Similarity - Typical cell size definition, hi Code Verification Geometrical Similarity - Typical cell size definition, hi

Geometrical Similarity (a 1-D example) Code Verification Geometrical Similarity (a 1-D example) - Definition of a typical cell size (single parameter) requires sets of geometrically similar grids - Consider a 1-D problem for - Geometrically similar grids have a single definition of the dimensionless distance (s) as a function of the “dimensionless” node counter (x)

Geometrical Similarity (a 1-D example) Code Verification Geometrical Similarity (a 1-D example) Sets of 31 grids with 5 ≤ Ncells ≤320

Geometrical Similarity (a 1-D example) Code Verification Geometrical Similarity (a 1-D example) Similar grids

Geometrical Similarity (a 1-D example) Code Verification Geometrical Similarity (a 1-D example) Unsimilar grids

Geometrical Similarity (a 1-D example) Code Verification Geometrical Similarity (a 1-D example) Similar grids

Geometrical Similarity (a 1-D example) Code Verification Geometrical Similarity (a 1-D example) Unsimilar grids

Geometrical Similarity (a 1-D example) - Error norms: Code Verification Geometrical Similarity (a 1-D example) - Error norms: - eo(f), p and a obtained in the least squares sense from the data of the six finest grids

Geometrical Similarity (a 1-D example) Code Verification Geometrical Similarity (a 1-D example) Similar grids

Geometrical Similarity (a 1-D example) Code Verification Geometrical Similarity (a 1-D example) Unsimilar grids

Geometrical Similarity (a 1-D example) Code Verification Geometrical Similarity (a 1-D example) Unsimilar grids

Geometrical Similarity (a 1-D example) Code Verification Geometrical Similarity (a 1-D example) Unsimilar grids

- In some practical applications, there are no Code Verification Code Verification - In some practical applications, there are no exact solutions available. For example, the Reynolds-averaged Navier-Stokes equations supplemented by a turbulence model do not have any known exact solutions − The Method of Manufactured Solutions (MMS) is a viable alternative to perform Code Verification when analytical solutions are not available. 32

Method of Manufactured Solutions: Choose the calculation domain Code Verification Method of Manufactured Solutions: Choose the calculation domain Choose the analytical solution for the dependent variables of the problem Substitute the chosen solution in the differential equations to determine source terms that guarantee that the selected exact solution satisfies the “new” system of differential equations 33

Solution Verification Solution/Calculations Verification − Exact solution is not available − Numerical error estimation usually assumes that the discretization error is dominant (this requires an iterative error at least two orders of magnitude smaller) − Methods based on grid refinement studies are one of the alternatives for the estimation of the discretization error/uncertainty − Mathematical problem 34

Solution Verification − Estimate the uncertainty, U, of a numerical solution of a flow quantity f for which the exact solution is not known Objective: with a confidence level of 95% Safety factor Error estimate 35

Solution Verification Error described by a Power Series Expansions fi → Local or functional flow quantity for grid i fo → Estimate of the exact solution of local or functional flow quantity dRE → Estimate of the error a → Constant hi → Typical cell size p → Observed order of grid convergence 36

Solution Verification At least 3 grids are required to determine fo, a, p 37

Solution Verification Apparent convergence or divergence for a grid triplet with h2/h1=h3/h2! Convergence ratio : Previous equation can only be used for Monotonic Convergence 0 < R <1  Monotonic Convergence -1 < R <0  Oscillatory Convergence R > 1  Monotonic Divergence R <-1  Oscillatory Divergence 38

Solution Verification Example I Sets of 31 grids with 5 ≤ Ncells ≤320 Geometrically similar grids require L A simple 1-D example

Solution Verification Example I - Determination of solution at x=0.5 - Three grid sets of 21 grids with 5 ≤ Ncells ≤80 a) Equally-spaced geometrically similar grids, Equal b) Stretched geometrically similar grids, Stret c) Stretched non-geometrically similar grids, StretN L Different grid node distributions requiring interpolation (Stret) and with no interpolation (Equal)

Solution Verification Example I Determination of solution at x=0.5 L Disturbance to grid similarity

Solution Verification Example I Determination of solution at x=0.5 L The effect of grid node distribution. The “asymptotic range”

Solution Verification Example I Determination of solution at x=0.5 L The effect of grid node distribution. The asymptotic range

Solution Verification Example I Determination of solution at x=0.5 L Disturbances from lack of geometric similarity

Solution Verification Example I Determination of solution at x=0.5 L

Solution Verification Example I Determination of solution at x=0.5 M Disturbances from interpolation

Solution Verification Example I Determination of solution at x=0.5 M Disturbances from interpolation

Solution Verification Example II - Flow around the KVLCC2 tanker at model and full scale Reynolds number (double-body) - PARNASSOS with SST k-w model - Sets of 6 nearly geometrically similar grids with direct application of no slip condition with y2+<0.6 - Blending between third (b=1) and first-order (b=0) upwind L A “real test case” - Selected flow quantities are resistance coefficients and nominal wake fraction

Solution Verification Example II L High quality grid sets (3 Workshops with the same test case!)

Solution Verification Example II L

Solution Verification Example II Rn=4.6×106 L Numerics, numerics and numerics... Explain the figure KVLCC2 tanker SST k-w model

Solution Verification Example II Rn=2.03×109 L Numerics, numerics and numerics... KVLCC2 tanker SST k-w model

Solution Verification Example II Rn=4.6×106 L Numerics, numerics and numerics... KVLCC2 tanker SST k-w model

Solution Verification Example II Rn=2.03×109 L Numerics, numerics and numerics... KVLCC2 tanker SST k-w model

Solution Verification Example II Rn=4.6×106 L Numerics, numerics and numerics... KVLCC2 tanker SST k-w model

Solution Verification Example II Rn=2.03×109 L Numerics, numerics and numerics... KVLCC2 tanker SST k-w model

Solution Verification Example II M An example of what careless CFD can do... 40 minutes KVLCC2 tanker SST k-w model

Solution Verification Example II Rn=4.6×106 M An example of what careless CFD can do... KVLCC2 tanker SST k-w model

Solution Verification Example II Rn=4.6×106 M An example of what careless CFD can do... KVLCC2 tanker SST k-w model

Goal: Estimate an interval that contains the modeling error, Validation (ASME V&V 20) ASME V&V 20 procedure Goal: Estimate an interval that contains the modeling error, with 95% confidence Needed: a numerical simulation result with uncertainty estimate a corresponding experimental data set with uncertainty estimate L the parameter uncertainty

Goal: Estimate an interval that contains the modeling error, Validation (ASME V&V 20) ASME V&V 20 procedure Goal: Estimate an interval that contains the modeling error, with 95% confidence Comparison error, Validation uncertainty, L

Analysis of the outcome: Validation (ASME V&V 20) ASME V&V 20 procedure Analysis of the outcome: If too big: “Houston, we have a problem...” L If is acceptable: The model is good enough, for this “noise level”

Flow at the propeller plane of a tanker at model scale Validation (ASME V&V 20) Flow at the propeller plane of a tanker at model scale - Double body (no gravity waves) Comparison of axial velocity at the propeller plane RANS equations with an eddy-viscosity model Parameter uncertainty equal to zero 63

Validation (ASME V&V 20) Experimental Numerical Numérico 64

Validation (ASME V&V 20) Usual Comparison 65

Validation (ASME V&V 20) Introduction of experimental uncertainty 66

Validation (ASME V&V 20) Introduction of numerical uncertainty 67

Comparasion of validation error with validation uncertainty Validation (ASME V&V 20) Comparasion of validation error with validation uncertainty 68

Validation (ASME V&V 20) What we want to avoid! 69