1. Control Structure Testing
Some Alternative Techniques
This should follow Testing_Flow
Copyright © Curt Hill

2. Introduction The purpose of the Cyclomatic Complexity metric was to:
Put an upper bound on basis path needed tests
Give a measure of module complexity
There are other methods for determining tests
These all revolve around control flow
This presentation considers some of them
Copyright © Curt Hill

3. Motivation
Basis path testing is satisfied that every statement executed
This may not be sufficient
Control flow is one of the places bugs hide
By examining closely all of these we should find a host of other bugs
Some of the alternatives to basis path testing should cover those test cases and provide others that are useful
Copyright © Curt Hill

4. Condition Testing The focus of this approach is to examine predicates
We look specifically at relational operations (comparisons) and Boolean operators
Thus, in C we only examine the parentheses following an if, while or switch
These are the units that drive all of the control flow of the module
Copyright © Curt Hill

5. Boolean/Relational Errors
The Boolean result that drives our control flow may have several possible errors
These may be in a:
Comparison
Boolean operator
Arithmetic
We then generate test cases looking for the possible errors
Copyright © Curt Hill

6. Error Types Arithmetic expression error Relational operator error
Usually an equal left out or put in, but sometimes a reversal of less/greater
Boolean operator
Wrong, extra or missing operator
Wrong Boolean variable
Precedence error
Wrong or missing parenthesis
Copyright © Curt Hill

7. Test Case Generation We set up the test data considering these errors
When done systematically we generally get more complete coverage than basis path testing
The computation of Cyclomatic Complexity or the construction of a flow graph is not needed
Copyright © Curt Hill

8. Branch Testing A form of condition testing
For every Boolean affecting flow we must generate tests that force the Boolean to be true and false
Consider: if(a==b && c<d) then four tests are required – each comparison to be true or false in every combination
An if or while with N conditions needs 2N tests
Copyright © Curt Hill

9. Domain Testing
Includes branch testing and but increases the test cases for comparisons
For any comparison such as a>b we should test three times
a<b
a=b
a>b
This should find relational errors
Copyright © Curt Hill

10. Combinatorics
It does not take too many decision statements to make the number of tests large
Whenever we can we do wish to reduce the number of tests without reducing effectiveness
We typically do this by limiting the tests in a way similar to short circuit evaluation
Copyright © Curt Hill

11. Limits
Suppose we have an if with a condition of: a && b where a and b are comparisons of Boolean variables
We would normally expect to do 4 tests
We may be able to omit the test where both a and b is false without losing effective coverage
Copyright © Curt Hill

12. Example Consider the following: if(x>4 || y < 2)
Each variable > = <
We could generate 9 tests
(x = 5, y = 1) => t, t, t
(x = 5, y = 2) => t, f, t
(x = 5, y = 3) => t, f, t
(x = 4, y = 1) => f, t, t
(x = 4, y = 2) => f, f, f
(x = 4, y = 3) => f, f, f
(x = 3, y = 1) => f, t, t
(x = 3, y = 2) => f, f, f
(x = 3, y = 3) => f, f, f
Copyright © Curt Hill

13. Data Flow Analysis
In data flow testing we consider the definition and access of variables along a path
Start with a flow graph and construct definition and use chains
Data flow analysis should lead us to develop test cases
Often better than basis path testing with fewer tests than every path
This is covered more fully in its own presentation
Copyright © Curt Hill

14. Static Analysis Data flow analysis is a form of static analysis
We know that static analysis cannot tell us the whole story
There may be error paths that are not possible to actually execute
Arrays and dynamic data structures are problematic because of one name with many values
Copyright © Curt Hill

15. Loop testing The whole point of computer software is looping
It is the ability to repeat some action very many times that makes programs useful
Unfortunately, bugs like to hide in and around loops
Loop testing is a white box technique to exterminate some of those bugs
Copyright © Curt Hill

16. Common Loop Errors The most common loop error is the off by one error
We want to go through M times and instead went through M-1 or M+1
Usually the condition has an extra equal or needs an equal or reversed condition
Uninitialized loop control variable
Often skips the loop
Loop control variable not changed in a loop
Usually gives an infinite loop
Copyright © Curt Hill

17. Common Loop Errors
Of those only the off by one has much of a chance of making it to unit testing
The others are usually detected in initial or ad hoc testing
Therefore we need tests that will detect the off by one loop
Copyright © Curt Hill

18. Loop Structures There are about four categories of loop Simple Nested
The approach depends on the category
Simple
Nested
Concatenated
Unstructured
Copyright © Curt Hill

19. Simple Loop Just what you might think
One loop construct
Single entry, single exit
Not in or adjacent to another loop
From a testing viewpoint you want tests that exercise:
Skipping the loop completely
If that is possible
Executing the loop once
Normal loop counts
Looking for off by one
Copyright © Curt Hill

20. Nested A loop in a loop Strategy:
Perhaps several deep
Strategy:
Minimize outer loops and test the innermost loop like before
Work outward until outermost loop is tested
Combinatorics may prevent us to applying the simple loop strategy
Copyright © Curt Hill

21. Concatenated Loops Two loops in succession
Independent loops can just use simple loop strategy
Related loops have something in common – often the control variable
These need a variation of the nested loop strategy
Verify the coupling between the two loops is properly handled
Copyright © Curt Hill

22. Unstructured Not the problem it once was
Characterized by either multiple entrances or multiple exits
Multiple entrances
Typically a GOTO has bypassed the loop initialization – hard to do otherwise
Multiple exits
Done with GOTOs, breaks, throws and returns
Copyright © Curt Hill

23. Unstructured Again
The multiple entrance, multiple exit loop is the most insidious control structure known
Extremely hard to analyze and test
Don’t do it
Rewrite into something structured
The multiple entrances are much worse than multiple exits
Breaks are most benign
GOTOs are the worst
Returns are somewhere in between
Copyright © Curt Hill

24. Finally Black box and white box testing complement each other
Use both
Often a black box test will cover one or more white box tests
For time’s sake eliminate the redundant tests
Copyright © Curt Hill