1. Contents Introduction Requirements Engineering Project Management
Software Design
Detailed Design and Coding
Quality Assurance
Maintenance
PVK-HT06

2. Quality Assurance Introduction Testing Testing Phases and Approaches
Black-box Testing
White-box Testing
System Testing
PVK-HT06

3. Test Methods Structural testing (white-box, glass-box)
Uses code/detailed design to develop test cases
Typically used in unit testing
Approaches:
Coverage-based testing
Symbolic execution
Data flow analysis
...
Functional testing (black-box)
Uses function specifications to develop test cases
Typically used in system testing
Equivalence partitioning
Border case analysis
develop
black-box
test cases
develop
white-box
test cases
perform
white-box
testing
perform
black-box
testing
time
PVK-HT06

4. Test Preparation
Exhaustive testing is prohibited, because of the combinatorial explosion of test cases
Choose representative test data
i paths to test #tests
1 X, Y 2
2 XX, XY, YX, YY 4
3 XXX, XXY, ... 8
...
for i := 1 to 100 do
if a = b then X
else Y;
2  > 2,5  1030
With 106 tests/sec this would take 81016 years
Choose test data (test cases)
PVK-HT06

5. How to Choose Test Data Example 1 Example 2 Both paths must be tested!
How can I know there is a “path”?
if ((x + y + z)/3 = x) then
writeln( “x, y, z are equal”)
else
writeln( “x, y, z are unequal”);
Test case 1: x=1, y=2, z=3
Test case 2: x=y=z=2
if (d = 0) then
writeln( “division by zero”)
else
x = y/n;
(* *)
PVK-HT06

6. Test Case Development Problems: Test case  test data
Systematic way to develop test cases
Find a satisfying set of test cases
Test case  test data
Test data: Inputs devised to test the system
Test case:
Situation to test
Inputs to test this situation
Expected outputs
Test are reproducible
Equivalence partitioning
Coverage-based testing
PVK-HT06

7. Equivalence Partitioning
Input data
Input- and output data can be grouped into classes where all members in the class behave in a comparable way.
Define the classes
Choose representatives
Typical element
Borderline cases
Inputs causing
anomalous behaviour
System
Outputs which reveal
the presence of faults
Output data
class 1: x < 25
class 2: x >= 25 and x <= 100
class 3: x > 100
x  [ ]
PVK-HT06

8. Equivalence Partitioning Example
Grocery Store
Consider a software module that is intended to accept the name of a grocery item and a list of the different sizes the item comes in, specified in ounces. The specifications state that the item name is to be alphabetic characters 2 to 15 characters in length. Each size may be a value in the range of 1 to 48, whole numbers only. The sizes are to be entered in ascending order (smaller sizes first). A maximum of five sizes may be entered for each item. The item name is to be entered first, followed by a comma, then followed by a list of sizes. A comma will be used to separate each size. Spaces (blanks) are to be ignored anywhere in the input.
PVK-HT06

9. Derived Equivalence Classes
Item name is alphabetic (valid)
Item name is not alphabetic (invalid)
Item name is less than 2 characters in length (invalid)
Item name is 2 to 15 characters in length (valid)
Item name is greater than 15 characters in length (invalid)
Size value is less than 1 (invalid)
Size value is in the range 1 to 48 (valid)
Size value is greater than 48 (invalid)
Size value is a whole number (valid)
Size value is a decimal (invalid)
Size value includes nonnumeric characters (invalid)
Size values entered in ascending order (valid)
Size values entered in non ascending order (invalid)
No size values entered (invalid)
One to five size values entered (valid)
More than five sizes entered (invalid)
Item name is first (valid)
Item name is not first (invalid)
A single comma separates each entry in list (valid)
A comma does not separate two or more entries in the list (invalid)
The entry contains no blanks (valid)
The entry contains blanks (valid)
PVK-HT06

10. Black Box Test Cases…
…for the Grocery Item Example based on the Equivalence Classes Above
Test case number
Test Data
Expected outcome
Classes Covered
TC1
xy,1 T
1,4,7,9,11,13,16,18,20,22
TC2
a2x,1 F 2
TC3
....
PVK-HT06

11. Statement Coverage
Every statement is at least executed once in some test 4 2 7 6 1 8 3 5
2 test cases: 12467; 13567
PVK-HT06

12. Branch Coverage
For every decision point in the graph, each branch is at least chosen once 4 2 7 6 1 8 3 5
2 test cases: 12467; 1358
PVK-HT06

13. Path Coverage
Assure that every distinct paths in the control-flow graph is executed at least once in some test
PVK-HT06

14. Path Coverage
Assure that every distinct paths in the control-flow graph is executed at least once in some test 4 2 7 6 1 8 3 5
PVK-HT06

15. Path Coverage
Assure that every distinct paths in the control-flow graph is executed at least once in some test 4 2 7 6 1 8 3 5
4 test cases: 12467; 1358; 1248; 13567
PVK-HT06

16. Test Coverage Statement Coverage: 5/10 = 50%
Branch Coverage: 2/6 = 33%
Path Coverage: 1/4 = 25%
PVK-HT06

17. Data-flow testing
Def-use analysis: match variable definitions (assignments) and uses.
Example:
x = 5; …
if (x > 0) ...
Does assignment get to the use?
PVK-HT06

18. Data Flow Coverage All-uses coverage
x :=1
x :=3
z :=x+y
y :=2
r :=4
x :=2
z := 2*r
z := 2*x-y
Red path covers the defs y :=2; r :=4; x :=1
Blue path covers y :=2; x :=3. Does not cover x :=2
PVK-HT06

19. Coverage-based Testing
Advantages
Systematic way to develop test cases
Measurable results (the coverage)
Extensive tool support
Flow graph generators
Test data generators
Bookkeeping
Documentation support
Disadvantages
Code must be available
Does not (yet) work well for data-driven programs
PVK-HT06