1. Chapter 2.11 Program Validation

2. Reliable System = Reliable Hardware AND Reliable Software AND Compatible Hardware and Software

3. Reliable Hardware Good Design Diagnostic Circuits Build-in Redundancy Diagnostic Programs

4. Reliable Software should be ADEQUATE (accomplishes the tasks it is used for) and ROBUST (survives all abnormal inputs)

5. Software Validation Before being used a program should be VALIDATED for ADEQUACY and ROBUSTNESS

6. Software Validation WARNING ! Validation can never improve the quality of software, it can only prevent the use of erroneous programs

7. Program Errors Syntax errors –Detected by the compiler Run-time errors –The program stops with an error message –Special recovery software is activated Erroneous results –Consequences can be catastrophic !!! Software engineering tries to push errors upwards in the list

8. Mandatory Declarations VAR LOAD, Force : REAL; BEGIN... L0AD := 1000. Undeclared:L0AD... Force := LOAD *... BEGIN... L0AD := 1000.... Force:= LOAD *...... Cross reference:... L0AD : 15 LOAD : 20,27,39,44 Normal execution Erroneous value of Force No execution Compilation error

9. Static vs. Dynamic Validation Static validation: –based on the analysis of the program –source code required –could guarantee correctness Dynamic validation (testing): –based on the execution of the program –only the object code is required –can only prove the presence of errors, never their absence

10. Bottom-up Validation Start with small building blocks Finish with the entire system Natural and intuitively attractive Major incompatibilities between building blocks discovered last For dynamic validation: –Test programs required for each building block

11. Top-down Validation Start with the entire system, ignoring details. Finish with the small building blocks Somewhat counter-intuitive Major incompatibilities between building blocks discovered first For dynamic validation: –Dummy substitutes for all building blocks have to be made for high-level tests.

12. Static Validation Techniques Syntax checking by the compiler Manual code checking –Desk checking –Walk through –Inspection Data flow analysis Correctness proofs

13. Static Validation Techniques Syntax checking by the compiler Manual code checking –Desk checking –Walk through –Inspection Data flow analysis Correctness proofs

14. Static Validation Techniques Syntax checking by the compiler Manual code checking –Desk checking –Walk through –Inspection Data flow analysis Correctness proofs

15. Manual code checking Desk checking –Informal code reading –preferably by somebody else Walk through –Formal meeting –Simulated execution of code on blackboard Inspection –Formal meeting –Contradictory discussion of code and programming style

16. Static Validation Techniques Syntax checking by the compiler Manual code checking –Desk checking –Walk through –Inspection Data flow analysis Correctness proofs

17. Example : Second order equation d > 0 ? Read a,b,c Write x 1, x 2 No d:=b 2 - 4ac d = 0 ? x 1 := … x 2 := … x := … Write x Write "no roots" No

18. Dataflow of a : d > 0 ? Read a,b,c Write x 1, x 2 No d:=b 2 - 4ac d = 0 ? x 1 := … x 2 := … x := … Write x Write "no roots" No

19. Dataflow of x 1 : d > 0 ? Read a,b,c Write x 1, x 2 No d:=b 2 - 4ac d = 0 ? x 1 := … x 2 := … x := … Write x Write "no roots" No

20. Uninitialized variable ? x :=... := X

21. Useless statement Write x x := exp1 x := exp2 Why this statement ???

22. Static Validation Techniques Syntax checking by the compiler Manual code checking –Desk checking –Walk through –Inspection Data flow analysis Correctness proofs

23. Correctness Proof InputsOutputs Specifications Program EQUIVALENCE

24. Correctness Proof Formal proofs can be long and complex –Impossible for large systems –Very error prone –Could be automated Doesn't help with specification errors… Useful for well specified, short, but complex, algorithms.

25. Dynamic Validation Techniques Testing –No guarantee !!! –Black box vs. white box testing. –Test data Debugging –The program state –Control flow breakpoints –Data flow breakpoints –Real time considerations

26. Dynamic Validation Techniques Testing –No guarantee !!! –Black box vs. white box testing. –Test data Debugging –The program state –Control flow breakpoints –Data flow breakpoints –Real time considerations

27. TESTING "Testing can proof the presence of errors but not their absence" E.W.Dijkstra

28. Dynamic Validation Techniques Testing –No guarantee !!! –Black box vs. white box testing. –Test data Debugging –The program state –Control flow breakpoints –Data flow breakpoints –Real time considerations

29. Dynamic Validation Techniques Testing –No guarantee !!! –Black box vs. white box testing. –Test data Debugging –The program state –Control flow breakpoints –Data flow breakpoints –Real time considerations

30. Test data d > 0 ? Read a,b,c Write x 1, x 2 No d:=b 2 - 4ac d = 0 ? x 1 := … x 2 := … x := … Write x Write "no roots" No Tests should exercise all paths through a program abcd 1321 1210 111 -3

31. d > 0 ? Read a,b,c Write x 1, x 2 No d:=b 2 - 4ac d = 0 ? x 1 := … x 2 := … x := … Write x Write "no roots" No Test data Tests should exercise exceptional values abcd 0111 101 -4 1101

32. d > 0 ? Read a,b,c Write x 1, x 2 No d:=b 2 - 4ac d = 0 ? x 1 := … x 2 := … x := … Write x Write "no roots" No Test data Tests should exercise very large and very small values a b c 1 10000000001 10000000000

33. Dynamic Validation Techniques Testing –No guarantee !!! –Black box vs. white box testing. –Test data Debugging –The program state –Control flow breakpoints –Data flow breakpoints –Real time considerations

34. Dynamic Validation Techniques Testing –No guarantee !!! –Black box vs. white box testing. –Test data Debugging –The program state –Control flow breakpoints –Data flow breakpoints –Real time considerations

35. Dynamic Validation Techniques Testing –No guarantee !!! –Black box vs. white box testing. –Test data Debugging –The program state –Control flow breakpoints –Data flow breakpoints –Real time considerations