1. Software Testing Copyright, 1999 © Jerzy R. Nawrocki Jerzy.Nawrocki@put.poznan.plwww.cs.put.poznan.pl/jnawrocki/mse/psp/ Personal Software Process Lecture 14

2. J. Nawrocki, PSP, Lecture 14 { n+=NF; } { n+=NF; } END { print n; } From the previous lecture.. InspectionInspection (very formal) (very formal) Walk-throughWalk-through (a presentation) (a presentation) Personal reviewPersonal review (DIY) (DIY)

3. J. Nawrocki, PSP, Lecture 14 From the previous lecture.. DesignDesignCodeCodeUnittestUnittest I1I1 I2I2 I3I3 Net savings (hours/KLOC): I 1 : 94 I 2 : 51 I 3 : -20 Fagan inspections

4. J. Nawrocki, PSP, Lecture 14 From the previous lecture.. 1. Overview (whole team) 2. Preparation (individual) 3. Inspection (whole team) 4. Rework 5. Follow-up Designer Implem. Moderator Tester Review session Fagan inspections

5. J. Nawrocki, PSP, Lecture 14 From the previous lecture.. Are all constants defined? If a queue is being manipulated, can the execution be interrupted; If so, is queue protected by a locking structure? Are registers being restored on exits? Are all increment counts properly initialised (0 or 1)? Are absolutes shown where there should be symbolics? Are all blocks shown in design necessary? Checklist for design inspection Ex Wr Missing

6. J. Nawrocki, PSP, Lecture 14 From the previous lecture.. PR/M/Min L3: the prologue in the REMARKS section needs expansion. section needs expansion. LO/W/Maj L172: NAME-CHECK is performed one time too few. time too few. DE/W/Min L175: the design should allow for the occurrence of a period in a last occurrence of a period in a last name. name. Error list

7. J. Nawrocki, PSP, Lecture 14 From the previous lecture.. CC: Code comments CU: CB usage DE: Design error IC: Interconnect calls LO: Logic MN: Maintainability OT: Other PE: Performance PR: Prolog Major Minor Major Minor M W E M W E M W E M W E Date.............. Code inspection report Mod/Mac:....................... Total

8. J. Nawrocki, PSP, Lecture 14 From the previous lecture.. Questions posed by the author of the design - to encourage a thorough reviewQuestions posed by the author of the design - to encourage a thorough review Several brief reviews focusing on a part of a work product (part of a design document)Several brief reviews focusing on a part of a work product (part of a design document) A sequence of small inspectionsA sequence of small inspections One-person reviews and multi-person inspectionsOne-person reviews and multi-person inspections Indigenous and seeded defectsIndigenous and seeded defects Active design reviews Phased inspections

9. J. Nawrocki, PSP, Lecture 14 Introduction Software testing is a critical element of SQA and represents the ultimate review of specification, design, and coding. -- Roger Pressman97

10. J. Nawrocki, PSP, Lecture 14 Testing objectives Testing = executing a program to find an error Quality of a test case: probability of finding an as-yet undiscovered error. A successful test uncovers an as-yet undiscovered error. -- Glen Myers79

11. J. Nawrocki, PSP, Lecture 14 Dijkstras saying Testing cannot show the absence of defects, it can only show that software errors are present. -- E.W. Dijkstra

12. J. Nawrocki, PSP, Lecture 14 Testing effort Testing: ~ 30% - 40% of total effort Testing mission-critical systems: 70% - 80% of the total effort (!) -- Roger Pressman97

13. J. Nawrocki, PSP, Lecture 14 Testing principles All tests should be traceable to customer requirementsAll tests should be traceable to customer requirements Tests should be planned long before testing beginsTests should be planned long before testing begins The Pareto principle applies to software testingThe Pareto principle applies to software testing Testing should begin in-the-small and progress toward in-the-largeTesting should begin in-the-small and progress toward in-the-large Exhaustive testing is not possibleExhaustive testing is not possible Some testing should be conducted by an independent third partySome testing should be conducted by an independent third party

14. J. Nawrocki, PSP, Lecture 14 A good test.... has a high probability of finding an error.. is not redundant.. should be best of breed.. should be neither too simple nor too complex

15. J. Nawrocki, PSP, Lecture 14 White-box testing Flow chartFlow chart The number of independent paths (cyclomatic complexity - McCabe76):The number of independent paths (cyclomatic complexity - McCabe76): V(G) = P + 1 V(G) = P + 1 P: the number of predicate nodes P: the number of predicate nodes Construct a basis set of independent pathsConstruct a basis set of independent paths Prepare test cases: each path in the basis set must be takenPrepare test cases: each path in the basis set must be taken

16. J. Nawrocki, PSP, Lecture 14 White-box testing void PrintGraph(int n){ int r, c; int r, c; for(r=0; r<n; r++){ for(r=0; r<n; r++){ printf("\n%d:", r); printf("\n%d:", r); for(c=0; c<n; c++) for(c=0; c<n; c++) if(Connected(r, c)) if(Connected(r, c)) printf(" %d", c); } printf("\n"); printf("\n"); } 0: 1 2 3 1: 0 2: 0 3: 0 0 123

17. J. Nawrocki, PSP, Lecture 14 White-box testing void PrintGraph(int n){ int r, c; int r, c; for(r=0; r<n; r++){ for(r=0; r<n; r++){ printf("\n%d:", r); printf("\n%d:", r); for(c=0; c<n; c++) for(c=0; c<n; c++) if(Connected(r, c)) if(Connected(r, c)) printf(" %d", c); printf(" %d", c); } printf("\n"); printf("\n"); } r=0; if r>=n goto.. printf(, r); c=0; if c>=n goto.. if not Connected goto.. printf(, c) c++; r++; printf(\n);

18. J. Nawrocki, PSP, Lecture 14 White-box testing void PrintGraph(int n){ int r, c; int r, c; for(r=0; r<n; r++){ for(r=0; r<n; r++){ printf("\n%d:", r); printf("\n%d:", r); for(c=0; c<n; c++) for(c=0; c<n; c++) if(Connected(r, c)) if(Connected(r, c)) printf(" %d", c); printf(" %d", c); } printf("\n"); printf("\n"); } r=0; if r>=n goto.. printf(, r); c=0; if c>=n goto.. if not Connected goto.. printf(, c) c++; r++; printf(\n); V(G) = 3 + 1 = 4

19. J. Nawrocki, PSP, Lecture 14 White-box testing r=0; if r>=n goto.. printf(, r); c=0; if c>=n goto.. if !Connected goto.. printf(, c) c++; r++; printf(\n); A A G G B B C C D D E E F F A - G

20. J. Nawrocki, PSP, Lecture 14 White-box testing r=0; if r>=n goto.. printf(, r); c=0; if c>=n goto.. if !Connected goto.. printf(, c) c++; r++; printf(\n); A A G G B B C C D D E E F F A - G A - B - F - A - G

21. J. Nawrocki, PSP, Lecture 14 White-box testing r=0; if r>=n goto.. printf(, r); c=0; if c>=n goto.. if !Connected goto.. printf(, c) c++; r++; printf(\n); A A G G B B C C D D E E F F A - G A - B - F - A - G A - B - C - E - B - F - A - G

22. J. Nawrocki, PSP, Lecture 14 White-box testing r=0; if r>=n goto.. printf(, r); c=0; if c>=n goto.. if !Connected goto.. printf(, c) c++; r++; printf(\n); A A G G B B C C D D E E F F A - G A - B - F - A - G A - B - C - E - B - F - A - G A - B - C - D - E - B - F - A - G

23. J. Nawrocki, PSP, Lecture 14 White-box testing r=0; if r>=n goto.. printf(, r); c=0; if c>=n goto.. if !Connected goto.. printf(, c) c++; r++; printf(\n); A A G G B B C C D D E E F F A - G n = 0

24. J. Nawrocki, PSP, Lecture 14 White-box testing r=0; if r>=n goto.. printf(, r); c=0; if c>=n goto.. if !Connected goto.. printf(, c) c++; r++; printf(\n); A A G G B B C C D D E E F F A - B - F - A - G 0 = n ???

25. J. Nawrocki, PSP, Lecture 14 White-box testing r=0; if r>=n goto.. printf(, r); c=0; if c>=n goto.. if !Connected goto.. printf(, c) c++; r++; printf(\n); A A G G B B C C D D E E F F A - B - F - A - G 0 1 n = 2 A-B-C-E-B-C-D-E-B-F-A-..-A-G A test case for:

26. J. Nawrocki, PSP, Lecture 14 Black-box testing Targets: incorrect or missing functionsincorrect or missing functions interface errorsinterface errors errors in data structureserrors in data structures performance errorsperformance errors initialisation and termination errorsinitialisation and termination errors

27. J. Nawrocki, PSP, Lecture 14 Black-box testing Requirements-based: use scenariosuse scenarios Based on formal specification: boundary value analysisboundary value analysis

28. J. Nawrocki, PSP, Lecture 14 Black-box testing 01 n = 2 n = 0 n = 1 0 01 n = 2 n = 3 012 012 012 0 1 2 void PrintGraph( int n )

29. J. Nawrocki, PSP, Lecture 14 Black-box testing r=0; if r>=n goto.. printf(, r); c=0; if c>=n goto.. if !Connected goto.. printf(, c) c++; r++; printf(\n); A A G G B B C C D D E E F F 01 n = 2 n = 0 n = 1 0 01 n = 2 n = 3 012 012 012 0 1 2 McCabes completeness

30. J. Nawrocki, PSP, Lecture 14 Test report template Tester:........................ Date:........... Program:............... Test symbol:..................... Test objective:.............................................................................................................................................. Test conditions:............................................................................................................................................. Input Expected output Actual output

31. J. Nawrocki, PSP, Lecture 14 Test report template Tester: J. Nawrocki Date: 18.01.00 Program: GraphCol Test symbol: N2Full Test objective: To check the program for a graph consisting of 2 connected nodes. Test conditions: Use the program N2Full. Input Expected output 0: 1 1: 0 Actual output 0: 1 1: 0

32. J. Nawrocki, PSP, Lecture 14 Comparison testing Multiple implementationsMultiple implementations Sometimes interface filters can be helpful (two modules are implementing not the same but similar systems)Sometimes interface filters can be helpful (two modules are implementing not the same but similar systems)

33. J. Nawrocki, PSP, Lecture 14 Software testing strategies Verification vs. ValidationVerification vs. Validation The V modelThe V model Predicting the failure intensity (logarithmic Poisson execution time model):Predicting the failure intensity (logarithmic Poisson execution time model): f(t) = (1/p)ln(l 0 pt + 1) f(t) = (1/p)ln(l 0 pt + 1) f(t) - cumulative number of failures that are expected to occur after time t f(t) - cumulative number of failures that are expected to occur after time t

34. J. Nawrocki, PSP, Lecture 14 Summary Testing is a destructive activity. Testing cannot show the absence of defects. Testing > 30% of total effort. All tests should be traceable to customer requirements. White-box testing. Black-box testing.

35. J. Nawrocki, PSP, Lecture 14 Further readings W.S. Humphrey, A Discipline for Software Engineering, Addison-Wesley, Reading, Engineering, Addison-Wesley, Reading, 1995. 1995. R.S. Pressman, Software Engineering. A Practitioners Approach, McGraw-Hill, A Practitioners Approach, McGraw-Hill, New York, 1997. New York, 1997. R.A. DeMillo et al., Software Testing and Evaluation, Benjamin/Cummings, Evaluation, Benjamin/Cummings, Menlo Park, 1987. Menlo Park, 1987.

36. J. Nawrocki, PSP, Lecture 14 Quality assessment 1. What is your general impression ? (1 - 6) 2. Was it too slow or too fast ? 3. Did you learn something important to you ? 4. What to improve and how ?