Verification & Validation QIP Course on CFD June 2-6, 2003 K Sudhakar SKSane Amitay Isaacs Department of Aerospace Engineering Indian Institute of technology, Mumbai

Verification & Validation What? Why? When? How? Who?

Verification & Validation What? Verification & Validation No computer code can be proved to have no errors. But Can be proved wrong if evidence to this effect is collected. Process of increasing our confidence in results of a computation Practiced with rigour in critical applications. IV&V, ADA, Bangalore ISRO, DRDL, . . CFVDS - Centre for Formal Verification & Design of Software, IIT Bombay

Verification & Validation What? Verification & Validation Validation simple systems m c k Reality Validation Model Qualification Code Verification

What? Validation Validation - Is the correct model being solved? k c If resistance due to air is significant the model will be incorrect. How to confirm if model is good? Analytically integrate? Conduct experiments. Compare Possible only for simple Models. Verification - Is the model being solved correctly

Verification & Validation What? Verification & Validation Reality Model Code Verification Validation Qualification

Verification & Validation Why? Verification & Validation Affects critical performance Loss of life? Loss of prestige? Financial/Social loss Affects important performance Loss of revenue How frequently etc. Incorrect results Integrity Level scale of 0 to 4 0  No IV&V required …………… 4  rigorous IV&V required

Verification & Validation When? Verification & Validation Most effective if conducted in parallel with s/w development Concept V&V Requirements V&V Design V&V Implementation V&V Test V&V Installation & Checkout V&V Operation V&V Maintenance V&V

Verification & Validation When? Verification & Validation V&V are defined as the process of determining; ie. They are on-going activities No end point 0% 100% Confidence Cost Value Time

Verification & Validation How? Verification & Validation Are the results given by a computer program correct? How to investigate this? Any standard processes to perform V&V? IEEE Std 1012-1998, etc. Common sense, logic, discipline, . .

How? Verification Verification - Is the model being solved correctly? Solution Verification. Numerical techniques Sources of error - spatial & temporal discretisation, iterative convergence, round off ODE’s Time step adequate for all situations? Adaptive? PDEs round off, iterative convergence are better understood and can be addressed. Discretisation errors in PDE are less understood

How? Verification Verification - Is the model being solved correctly? Code Verification Coding language, Coding practices, . . . (S/W Engineering) Model to code translation Static & dynamic analysis. Statically unreachable code Dynamically unreachable code FTNCHK, FORCHEK etc.

How? Verification = Model Solution Code Answers from highly accurate solutions Analytical Benchmark Solution =

How? Verification Checks must cover all paths through the code? if (0<m=<0.9) then gama = 1.4 call abc (….) elseif (0.9<m<1.2) then gama =14. call def (. . . .) elseif (1.2=<m<2.0) call ghi ( ….) endif Verification checks m = 0.6, 0.8, 0.9, 1.3, 1.5, 2. Does not check one path. m = 0.6, 0.8, 0.9, 1.0, 1.3, 1.5, 2. Checks all paths

How? Validation Is the model correct? Conceptual Model Code Reality Results Experience Validation domain Simple systems Verification

Validation for Complex Systems How? Validation for Complex Systems Code Validation Experiments Outcome of Experiment Computational Predictions Model Experimental Prediction

Validation for Complex Systems How? Validation for Complex Systems Computational Predictions Code Computational Results of Exp outcome Model Difference Validation Experiments Outcome of Experiment

How? Validation Prediction Confidence? Predictive domain Validation domain Can validation domain cover the predictive requirements?

How? Validation x - Validation experiments x Predictive domain x Where to conduct experiments? How to assess predictive confidence?

How? Validation Computation Region 3 Region 1 Region 2 Validation Experiment Computation Region 1 Region 2 Region 3

How? Validation Design & Analysis of Computer Experiments Where to conduct computational & physical experiments for validation? How to build predictive models? How to assess predictive confidence? Computational results corrected using fixed no of physical experiments x Estimates of Predictive error

How? Validation = Real World Model Solution Answers from validation experiments Benchmark solutions Specially conducted Code Solution = Model

Verification & Validation Who? Verification & Validation Independent Verification & Validation (IV&V) Technical Independence. V&V done by people who are not involved in design & development Managerial Independence. An organization separate from that did design & development. Financial Independence. Budget for V&V is not in anyway controlled by those responsible to design & development.

Verification & Validation in CFD Less valued in CFD than in FEM (for historical reasons) AIAA Aerosp. Sciences Meeting & Exhibit 2002 DLR-F4 Wing Body 35 different CFD Calculations (Advanced research codes & commercial codes) After weeding out outliers this gave a standard deviation in CD = 0.0021 (for wind tunnel testing this figure is put at 0.0004)

Thank you

Coding Practice - Not Done! How? Coding Practice - Not Done! if (x) 100, 200, 300 200 if (y) 400, 500, 600 z= z +1 goto 750 500 xsq = x**2. goto 850 300 xcu = x**3 ….. t1 = a1*2. + b1 t2 = sqrt(t1)+c1 t3 = t2**1.3+b1 . . . real a(100) integer b(50) complex c(10) equivalence (a(1),b(1)),