1. Chapter 14 Testing Tactics

2. 14.1 Software Testing Fundamentals
What is a “Good” test?
A good test has a high probability of finding an error
A good test is not redundant
A good test should be “best of breed”
A good test should be neither too simple nor too complex
Test Case Design
OBJECTIVE: to uncover errors
CRITERIA: in a complete manner
CONSTRAINT: with a minimum of effort and time
1/21

3. 14.1 Software Testing Fundamentals
Testability
Operability - the better it works the more efficiently it can be tested
Observability - what you see is what you test
Controllability - the better software can be controlled the more testing can be automated and optimized
Decomposability - by controlling the scope of testing, the more quickly problems can be isolated and retested intelligently
Simplicity - the less there is to test, the more quickly we can test
Stability - the fewer the changes, the fewer the disruptions to testing
Understandability - the more information known, the smarter the testing
2/21

4. 14.2 Black-box an White-box Testing
Why not try exhaustive testing?
loop = 20 X
There are 5201014 possible paths! If we execute one test per millisecond, it would take 3,170 years to test this program!!
White-Box Testing
Note: Testing cannot show the absence of errors and defects.
Black-Box Testing
3/21

5. Definition: White-box or Glass-box Testing
14.3 White-box Testing
Definition: White-box or Glass-box Testing
Knowing the internal workings of a product, tests are performed to check the workings of all possible logic paths -- testing in the small.
Questions
Can you guarantee that all independent paths within a module will be executed at least once?
Can you exercise all logical decisions on their true and false branches?
Will all loops execute at their boundaries and within their operational bounds?
Can you exercise internal data structures to ensure their validity?
4/21

6. Goal: execute every statement at least once
14.4 Basis Path Testing
Goal: execute every statement at least once
Defines the number of
independent paths in the basis
set of a program and provides us with
an upper bound for the number of tests
that must be conducted. a b c d e f g h i j k
a（begin）
b（in） c d e f g h i
j（out）
k（stop）
TOTAL=TOTAL+A
K++
Input A
Output K, L, TOTAL
Stop T
Begin
K=0
L=0
TOTAL=0
Do While
TOTAL<1000 and
A0 F
A>0
L++
Compound conditions
flow graph
Predicate node d’ R1 R2 R3 R4
Cyclomatic Complexity =
# of regions
E  N + 2
P + 1
5/21

7. 14.4 Basis Path Testing
Prepare test cases that will force execution of each path in the basis set – only to execute every statement at least once is NOT enough
Example: in
A > 1
AND B=0 T
A=2
OR X > 1
X = X / A
X = X + 1
out F
CC = ? 5
Test case :
A=3, B=0, X=4.
Note: If AND is written as OR, the error will not be detected by the above test case.
6/21

8. 14.5 Control Structure Testing
Condition Testing: test every condition at least once
Examples: Test ( E1 > E2 ) AND ( E3 = E4 )
B1 AND B2
{ ( t, t ), ( f, t ), ( t, f ) }
B1 AND ( E3 = E4 )
{ ( t, = ), ( f, = ), ( t, < ), ( t, > ) }
( E1 > E2 ) AND ( E3 = E4 )
{ ( >, = ), ( =, = ), ( <, = ), ( >, < ), ( >, > ) }
7/21

9. 14.5 Control Structure Testing
Data Flow Testing: selects test paths according to the locations of variable definitions and uses in the program
DEF(S) = { X| S contains a definition of X };
USE(S) = { X| S contains a use of X }.
DU chain of X = [X, S, S’] where XDEF(S) and XUSE(S’), and the definition of X in S is live at S’ ---- the flow of X.
Every DU chain is covered once.
8/21

10. 14.5 Control Structure Testing
Loop Testing
Unstructured
Loops
Concatenated
Loops
0,1
Nested
Loops
Simple
loop 1 2
2<m<n
n-1 n
n+1
9/21

11. Definition: Black-box or Behavioral Testing
14.6 Black-box Testing
Definition: Black-box or Behavioral Testing
Knowing the specified function a product is to perform and demonstrating correct operation based solely on its specification without regard for its internal logic – testing in the large.
Questions
How is functional validity tested?
How is system behavior and performance tested?
What classes of input will make good test cases?
Is the system particularly sensitive to certain input values?
How are the boundaries of a data class isolated?
What data rates and data volume can the system tolerate?
What effect will specific combinations of data have on system operation?
10/21

12. Equivalence Partitioning Boundary Value Analysis
14.6 Black-box Testing
Techniques
Graph-Based Testing
Equivalence Partitioning
Boundary Value Analysis
Orthogonal Array Testing
11/21

13. 14.6 Black-box Testing Guidelines
Equivalence Partitioning
An equivalence class represents a set of valid or invalid states for input conditions, so that there is no particular reason to choose one element over another as a class representative.
Guidelines
If an input condition specifies a range, one valid and two invalid equivalence classes are defined.
If an input condition requires a specific value, one valid and two invalid equivalence classes are defined.
If an input condition specifies a member of a set, one valid and one invalid equivalence classes are defined.
If an input condition is Boolean, one valid and one invalid equivalence classes are defined.
12/21

14. 14.6 Black-box Testing Guidelines
Boundary Value Analysis (BVA)
"Bugs lurk in corners
and congregate at
boundaries ..."
Boris Beizer
Guidelines
If input condition specifies a range bounded by values a and b, test cases should include a and b, values just above and just below a and b.
If an input condition specifies and number of values, test cases should be exercise the minimum and maximum numbers, as well as values just above and just below the minimum and maximum values.
Apply guidelines 1 and 2 to output conditions, test cases should be designed to produce the minimum and maxim output reports.
If internal program data structures have boundaries (e.g., size limitations), be certain to test the boundaries.
13/21

15. 14.7 Object-Oriented Testing Methods
Test Case Design
1. Each test case should be uniquely identified and should be explicitly associated with the class to be tested,
2. The purpose of the test should be stated,
3. A list of testing steps should be developed for each test and should contain [BER94]:
a. a list of specified states for the object that is to be tested
b. a list of messages and operations that will be exercised as a consequence of the test
c. a list of exceptions that may occur as the object is tested
d. a list of external conditions
e. supplementary information that will aid in understanding or implementing the test.
Unlike conventional test case design, which is driven by an I-P-O view of software or the algorithmic detail of individual modules, OO testing focuses on designing appropriate sequences of operations to exercise the states of a class.
14/21

16. 14.7 Object-Oriented Testing Methods
Fault-Based Testing
The tester looks for plausible faults (i.e., aspects of the implementation of the system that may result in defects). To determine whether these faults exist, test cases are designed to exercise the design or code.
Class Testing and the Class Hierarchy
Inheritance does not obviate the need for thorough testing of all derived classes. In fact, it can actually complicate the testing process.
Scenario-Based Test Design
Scenario-based testing concentrates on what the user does, not what the product does. This means capturing the tasks (via use-cases) that the user has to perform, then applying them and their variants as tests.
15/21

17. 14.8 Class Level Testing Methods
Random testing
identify operations applicable to a class
define constraints on their use
identify a minimum test sequence
an operation sequence that defines the minimum life history of the class (object)
generate a variety of random (but valid) test sequences
exercise other (more complex) class instance life histories
 Requires large number of data permutations and combinations and can be inefficient.
16/21

18. 14.8 Class Level Testing Methods
Partition Testing
reduces the number of test cases required to test a class in much the same way as equivalence partitioning for conventional software
state-based partitioning
categorize and test operations based on their ability to change the state of a class
attribute-based partitioning
categorize and test operations based on the attributes that they use
category-based partitioning
categorize and test operations based on the generic function each performs
17/21

19. 14.9 Inter-class Test Case Design
Multiple Class Testing (random)
For each client class, use the list of class operators to generate a series of random test sequences. The operators will send messages to other server classes.
For each message that is generated, determine the collaborator class and the corresponding operator in the server object.
For each operator in the server object (that has been invoked by messages sent from the client object), determine the messages that it transmits.
For each of the messages, determine the next level of operators that are invoked and incorporate these into the test sequence.
18/21

20. 14.9 Inter-class Test Case Design
Behavior Testing
– The tests to be designed should achieve all state coverage [KIR94]. e m p t y a c o n s u A d i ( l ) w r k g h f b I F 1
4.12 S [ K R 9 4 ]
19/21

21. 14.10 Specialized Testing 14.11 Testing Patterns
Graphical User Interfaces
Client/Server Architectures
Documentation and Help Facilities
Real-Time Systems
Testing Patterns
Define the patterns and REUSE
20/21

22. Final Words About Testing
Testing never ends, it just gets transferred from you [ the software engineer ] to your customer. Every time your customer uses the program, a test is being conducted.
21/21