Contents Introduction Requirements Engineering Project Management Software Design Detailed Design and Coding Quality Assurance Maintenance PVK-HT06

Quality Assurance Introduction Testing Testing Phases and Approaches Black-box Testing White-box Testing System Testing PVK-HT06

Test Methods Structural testing (white-box, glass-box) Uses code/detailed design to develop test cases Typically used in unit testing Approaches: Coverage-based testing Symbolic execution Data flow analysis ... Functional testing (black-box) Uses function specifications to develop test cases Typically used in system testing Equivalence partitioning Border case analysis develop black-box test cases develop white-box test cases perform white-box testing perform black-box testing time PVK-HT06

Test Preparation Exhaustive testing is prohibited, because of the combinatorial explosion of test cases Choose representative test data i paths to test #tests 1 X, Y 2 2 XX, XY, YX, YY 4 3 XXX, XXY, ... 8 ... 100 2100 for i := 1 to 100 do if a = b then X else Y; 2  2100 - 2 > 2,5  1030 With 106 tests/sec this would take 81016 years Choose test data (test cases) PVK-HT06

How to Choose Test Data Example 1 Example 2 Both paths must be tested! How can I know there is a “path”? if ((x + y + z)/3 = x) then writeln( “x, y, z are equal”) else writeln( “x, y, z are unequal”); Test case 1: x=1, y=2, z=3 Test case 2: x=y=z=2 if (d = 0) then writeln( “division by zero”) else x = y/n; (*-----------------------------*) PVK-HT06

Test Case Development Problems: Test case  test data Systematic way to develop test cases Find a satisfying set of test cases Test case  test data Test data: Inputs devised to test the system Test case: Situation to test Inputs to test this situation Expected outputs Test are reproducible Equivalence partitioning Coverage-based testing PVK-HT06

Equivalence Partitioning Input data Input- and output data can be grouped into classes where all members in the class behave in a comparable way. Define the classes Choose representatives Typical element Borderline cases Inputs causing anomalous behaviour System Outputs which reveal the presence of faults Output data class 1: x < 25 class 2: x >= 25 and x <= 100 class 3: x > 100 x  [25 .. 100] PVK-HT06

Equivalence Partitioning Example Grocery Store Consider a software module that is intended to accept the name of a grocery item and a list of the different sizes the item comes in, specified in ounces. The specifications state that the item name is to be alphabetic characters 2 to 15 characters in length. Each size may be a value in the range of 1 to 48, whole numbers only. The sizes are to be entered in ascending order (smaller sizes first). A maximum of five sizes may be entered for each item. The item name is to be entered first, followed by a comma, then followed by a list of sizes. A comma will be used to separate each size. Spaces (blanks) are to be ignored anywhere in the input. PVK-HT06

Derived Equivalence Classes Item name is alphabetic (valid) Item name is not alphabetic (invalid) Item name is less than 2 characters in length (invalid) Item name is 2 to 15 characters in length (valid) Item name is greater than 15 characters in length (invalid) Size value is less than 1 (invalid) Size value is in the range 1 to 48 (valid) Size value is greater than 48 (invalid) Size value is a whole number (valid) Size value is a decimal (invalid) Size value includes nonnumeric characters (invalid) Size values entered in ascending order (valid) Size values entered in non ascending order (invalid) No size values entered (invalid) One to five size values entered (valid) More than five sizes entered (invalid) Item name is first (valid) Item name is not first (invalid) A single comma separates each entry in list (valid) A comma does not separate two or more entries in the list (invalid) The entry contains no blanks (valid) The entry contains blanks (valid) PVK-HT06

Black Box Test Cases… …for the Grocery Item Example based on the Equivalence Classes Above Test case number Test Data Expected outcome Classes Covered TC1 xy,1 T 1,4,7,9,11,13,16,18,20,22 TC2 a2x,1 F 2 TC3 .... PVK-HT06

Statement Coverage Every statement is at least executed once in some test 4 2 7 6 1 8 3 5 2 test cases: 12467; 13567 PVK-HT06

Branch Coverage For every decision point in the graph, each branch is at least chosen once 4 2 7 6 1 8 3 5 2 test cases: 12467; 1358 PVK-HT06

Path Coverage Assure that every distinct paths in the control-flow graph is executed at least once in some test PVK-HT06

Path Coverage Assure that every distinct paths in the control-flow graph is executed at least once in some test 4 2 7 6 1 8 3 5 PVK-HT06

Path Coverage Assure that every distinct paths in the control-flow graph is executed at least once in some test 4 2 7 6 1 8 3 5 4 test cases: 12467; 1358; 1248; 13567 PVK-HT06

Test Coverage Statement Coverage: 5/10 = 50% Branch Coverage: 2/6 = 33% Path Coverage: 1/4 = 25% PVK-HT06

Data-flow testing Def-use analysis: match variable definitions (assignments) and uses. Example: x = 5; … if (x > 0) ... Does assignment get to the use? PVK-HT06

Data Flow Coverage All-uses coverage x :=1 x :=3 z :=x+y y :=2 r :=4 x :=2 z := 2*r z := 2*x-y Red path covers the defs y :=2; r :=4; x :=1 Blue path covers y :=2; x :=3. Does not cover x :=2 PVK-HT06

Coverage-based Testing Advantages Systematic way to develop test cases Measurable results (the coverage) Extensive tool support Flow graph generators Test data generators Bookkeeping Documentation support Disadvantages Code must be available Does not (yet) work well for data-driven programs PVK-HT06