1. CS 425/625 Software Engineering Software Testing
Based on Chapter 20 of the textbook [Somm00] Ian Sommerville,
Software Engineering, 6th Ed., Addison-Wesley, 2000 and on the
Ch20 PowerPoint presentation available at the book’s web-site:
November 12, 2002

2. Outline Introduction Defect Testing Integration Testing
Black Box Testing
Equivalence Partitioning
Structural (White Box) Testing
Path Testing
Integration Testing
Top-Down
Bottom-Up
Interface Testing
Stress Testing
Object Oriented Testing

3. Introduction..
A very high level view of testing phases [Fig. 20.1, Somm00]

4. .Introduction.
Component testing is concerned with checking the functionality of methods, classes, and modules
Integration testing focuses on sub-systems and their interaction as well as on the operation of the system as a whole
Component testing is individual programmers’ task while integration testing is assigned to an integration team
For critical systems independent testing teams are used during all testing
Integration testing is based on written requirements specifications

5. ..Introduction
In object-oriented testing the boundaries between component and integration testing are blurred because
Object-oriented systems do not have the clear distinction between program units and program modules characteristic to structured (function-oriented) systems
The hierarchy of objects is not necessarily “nested”

6. Defect Testing Defect testing is aimed at discovering errors in code
A successful defect test causes the system to perform incorrectly
During the testing process the following are created:
Test cases
Test data
Test reports

7. Defect Testing
Comprehensive testing is not possible. Subsets of tests cases are designed based on criteria such as:
All code statements should be executed at least once
All functions accessed through menus should be tested
All combinations of functions accessed through the same menu should be checked
All user input situations should be checked with both correct and incorrect data

8. Defect Testing: The Process
The defect testing process [Fig. 20.2, Somm00]

9. Defect Testing: Black Box Testing.
In black box testing (or functional testing) the system is viewed from “outside,” only in terms of its functionality
The software is tested against its specification
Internal (implementation) details are ignored
The behavior of the system is evaluated in terms of inputs received, outputs expected, and outputs actually produced
A defect is discovered if the actual system output differs from the expected output
Applicable to both function-oriented and object-oriented systems

10. Defect Testing: .Black Box Testing
The black-box testing model [Fig. 20.3, Somm00]

11. Defect Testing: Equivalence Partitioning….
Equivalence partitioning makes use of classes of input test data (e.g., positive numbers, negative numbers, strings, etc.)
Generally, systems tend to behave in a comparable way for all members of a class
Classes of input test data are called equivalence partitions or domains
Equivalence partitions are determined based on program specification and user documentation;
Test cases are derived for each equivalence partition
Partition mid-point and boundary values provide useful test data

12. .Defect Testing: .Equivalence Partitioning…
Equivalence partitioning [Fig. 20.4, Somm00]

13. ..Defect Testing: Equivalence Partitioning..
Equivalence partitions [Fig. 20.5, Somm00]

14. …Defect Testing: Equivalence Partitioning.
Example: A search routine [Fig. 20.6, Somm00]
procedure Search (Key : ELEM ; T: ELEM_ARRAY; Found : in out BOOLEAN; L: in out ELEM_INDEX) ; Pre-condition -- the array has at least one element T’FIRST <= T’LAST Post-condition -- the element is found and is referenced by L ( Found and T (L) = Key) or -- the element is not in the array ( not Found and not (exists i, T’FIRST <= i <= T’LAST, T (i) = Key ))

15. ….Defect Testing: Equivalence Partitioning
Equivalence partitions for search routine [Fig. 20.7, Somm00]

16. Defect Testing: Structural Testing….
In contrast to black box testing structural testing (or white box, clear box, glass box testing) is based on the knowledge the testers have about the structure and the implementation of the software
Applied generally to smaller program units
The tester analyzes the code to determine test cases
Types of structural testing:
Branch coverage: every branch of the unit is tested at least once
Statement coverage: every statement of the unit is executed at least once
Path coverage: every path through the unit is executed at least once

17. .Defect Testing: Structural Testing…
The structural testing process [Fig. 20.8, Somm00]

18. Structural Testing Example
Java
implementation
[Fig. 20.9, Somm00]

19. Defect Testing: …Structural Testing.
Equivalence classes for binary search [Fig , Somm00]

20. ….Defect Testing: Structural Testing
Test cases for binary search [Fig , Somm00]

21. Defect Testing: Path Testing…
Path testing is a form of structural testing aimed at exercising each individual execution path of a program unit. It ensures that
Each statement of the unit is executed at least once
Each branch condition is tested with both true and false conditions
The program flow graph is used for path testing. In a flow graph decisions are shown as nodes and the flow of control is represented by edges

22. Defect Testing: .Path Testing..
The cyclomatic complexity CC of a connected graph G is given by the formula:
CC(G) = Number of edges - Number of nodes + 2
If there are no go to statements and no compound decisions (involving more than one test) then
CC(G) = Number of predicate nodes + 1
(predicate nodes contain conditions that determine branching of the execution flow)
The cyclomatic complexity of a graph gives the minimum number of test cases needed to cover all paths

23. Defect Testing: ..Path Testing.
Flow graph for binary search [Fig , Somm00]

24. Defect Testing: …Path Testing
Independent paths in previous case
1, 2, 3, 8, 9
1, 2, 3, 4, 6, 7, 2
1, 2, 3, 4, 5, 7, 2
1, 2, 3, 4, 6, 7, 2, 8, 9
In programs with complex branching structure the number of paths is high and it is often difficult to predict the program’s behaviour
A dynamic program analyzer can be used to identify all paths

25. Integration Testing.…..
Integration involves building sub-systems from program units and the system from subsystems
Integration testing uses software specifications as basis for checking the system
Errors are more difficult to locate than in unit testing
The incremental approach is recommended for system integration and testing
Initially, a minimal system configuration can be used
Tests need be repeated after each addition to the system

26. .Integration Testing…..
Incremental integration testing [Fig , Somm00]

27. ..Integration Testing…. Strategies for system integration and testing:
Top-Down
Starts with high level components and continues at lower levels
Integral part of the top-down development process
Stubs are needed to test the system

28. …Integration Testing…
Strategies for system integration and testing [cont’d]:
Bottom-Up
Starts with lower level components and builds up the system by adding & testing higher level components
Can be used when components from previously developed systems are reused or when the system’s functionality relies on critical lower level modules
Driver modules are used to test the system
“Sandwich”
Combines the above two strategies by using the Top-Down approach for logic modules (higher level) and the Bottom-Up approach for operational (lower level) modules

29. ….Integration Testing..
Top-down integration testing [Fig , Somm00]

30. …..Integration Testing.
Bottom-up integration testing [Fig , Somm00]

31. …...Integration Testing Top-Down versus Bottom-Up:
Architectural validation
Top-Down allows earlier discovery of high-level design errors
System demonstration
Top-Down supports better an early demo; Bottom-Up can support such demo if many reusable components are utilized
Test implementation
Strict Top-Down testing is more difficult to implement given that stubs are needed. It is easier to use drivers in a strict Bottom-Up testing
Test observation
Equally challenging, since both stubs and drivers may not allow complete observations

32. Interface Testing…
Interface testing has the goal of detecting errors related to interfacing modules and sub-systems
Particularly important for object-oriented testing given objects’ collaboration relies on their interfaces
Types of interfaces:
Parameter interfaces
Shared memory interfaces
Procedural interfaces
Message passing interfaces
Classes of interface errors:
Interface misuse
Interface misunderstanding
Timing errors

33. .Interface Testing..
Interface testing [Fig , Somm00]: tests directed at the sub-system as a whole, not at particular components

34. ..Interface Testing. Guidelines for interface testing
Test all external calls with parameters at extreme ranges
When pointers are used, test interfaces with null pointers
For procedural interfaces try to design tests that would cause the components to fail
Use stress testing for message passing interfaces
Change activation order of components that share memory

35. …Interface Testing
Stress testing exercise the systems with overloading (e.g., number of transactions per minute, number of users in a distributed applications)
The main purposes of stress testing:
Make sure the system does not lose or corrupt data when overloaded
Reveal defects that otherwise would pass unnoticed

36. Object-Oriented Testing..
Differences from function-oriented systems:
Objects are usually larger units, which encompass several methods
Objects are typically loosely coupled and there may not be an obvious high top of the class hierarchy
Testers may not have access to the code of reused objects

37. .Object-Oriented Testing.
Levels of testing in object-oriented testing:
Testing operations of object classes (sequences of operations may be needed)
Testing object classes (operations, attributes, and states)
Testing clusters of objects (use case or scenarios based testing; thread testing, etc.); each operation in each class should be checked at least once
Testing the object-oriented system

38. ..Object-Oriented Testing
Compared with testing function-oriented systems, testing of object-oriented system is facilitated by inheritance + reuse (previously tested objects) but specific challenges exist because:
Information hiding is a complicating factor
Writing additional methods (invoked for testing purposes only) are often necessary
New and overwritten methods must be tested
Inherited methods must still be re-tested if they interact with newly written methods