1. Automated Testing of Classes
Alessandro Orso
Politecnico di Milano - Georgia Institute of Technology
Ugo Buy
University of Illinois at Chicago
Mauro Pezzè
Politecnico di Milano – Università di Milano Bicocca

2. Automated class testing
Problems with class testing
Object = data + operations
Behavior of objects highly depends on their state
Encapsulation and Information hiding  the state is hidden
Breaking of the encapsulation
Generation of message sequences
Breaking of the encapsulation
Generation of message sequences
Formal specs
Scripting
Source code
Formal specs
Scripting
Source code
The overall goal is the automated, code-based generation of sequences of messages to exercise an object (i.e., an instance of a class)

3. Class and state: The CoinBox example
Class CoinBox {
int total;
int curQtrs;
int allowVend;
public:
CoinBox() {
total=0;
allowVend=0;
curQtrs=0; }
void addQtr();
void returnQtrs();
void vend(); };
allowVend=0;
void CoinBox::returnQtrs() {
curQtrs=0; }
void CoinBox::addQtr() {
curQtr=curQtr+1;
if (curQtr>1)
allowVend=1;
void CoinBox::vend() {
if (allowVend) {
total=total+curQtrs;
allowVend=0;
Fault: method returnQtrs does not reset variable allowVend
Possible failure:
CoinBox()
addQtr()
returnQtr()
vend()
 free drink!

4. Steps of the technique Data-flow analysis Symbolic execution
to identify “paths of interest” within the class
Symbolic execution
to express the semantics of the class in terms of pre- and post-conditions
Automated deduction techniques
to generate sequences of messages for the class that exercise those paths

5. Step I: Data flow analysis
Class CoinBox
18 vend()
18 vend()
1 CoinBox()
13 addQtr()
10 returnQtrs()
19 if(allowVend)
19 if(allowVend)
14a t=curQtrs+1
2 total=0
11 curQtrs=0
DEF
11 curQtrs=0 T F
14b curQtrs=t
20a t=total+curQtrs
USE
20a t=total+curQtrs
3 allowVend=0
12 exit
12 exit
20b total=tmp
20b total=t
15 if(curQtrs>1)
4 curQtrs=0 T
16 allowVend=1
21 curQtrs=0
21 curQtrs=0
5 exit F
17 exit
22 allowVend=0
22 allowVend=0
Virtual Node
Virtual Node
23 exit
23 exit Vend()

6. DU associations Step I DUA# variabile definition use
curQtrs CoinBox (4) addQtr (14a)
curQtrs CoinBox (4) vend (20a)
allowVend CoinBox (3) vend (19)
total CoinBox (2) vend (20a)
curQtrs returnQtrs (11) addQtr (14a)
allowVend vend (22) vend (19)
curQtrs returnQtrs (11) vend (20a)
curQtrs addQtr (14b) addQtr (14a)
curQtrs addQtr (14b) addQtr (15)
curQtrs addQtr (14b) vend (20a)
allowVend addQtr (16) vend (19)
total vend (20b) vend (20a)
curQtrs vend (21) addQtr (14a)

7. Step II: Symbolic execution
For each path in each method of the class:
Execution conditions: ( curQtrs>0 )
Relation between input and output variables: allowVend'=1, curQtrs'=curQtrs+1
Set of defined attributes def={curQtrs,allowVend}
Limitations:
Approximations required for loops
Restrictions on the program constructs that symbolic execution can handle

8. Conditions for the CoinBox example
Step II
addQtr
(curQtrs>0)  allowVend'=1, curQtrs'=curQtrs+1
def={curQtrs,allowVend}
(curQtrs==0)  curQtrs’=1
def={curQtrs}
curQtrs=curQtrs+1
exit
addQtr()
allowVend=1 T F
if(curQtrs>1)
(curQtrs == 0)
curQtrs'=1 T F
curQtrs=curQtrs+1
exit
addQtr()
allowVend=1
if(curQtrs>1)
(curQtrs>0)
allowVend'=1, curQtrs'=curQtrs+1
curQtrs=curQtrs+1
exit
addQtr()
allowVend=1 T F
if(curQtrs>1)

9. Step III: Automated deduction
For each DU association <def,use, v>, identified with data flow analysis, build the method sequence:
<constructor,...,mi,..., mD,...,mj,...,mU>
Method that contains the definition of v
Method that contains the use of v
Def-clear path w.r.t. v

10. The method invocation tree
Step III
node m = a path in m and the associated pre-condition cond(m)
node m = a path in m and the associated pre-condition cond(m)
successors m’ (of m) = all paths pj whose post-conditions do not contradict cond(m)
cond(m’) is the conjunction of
cond(m), without clauses implied by the post condition of pi
precondition of pi m
(a>0) &&
(b>=3) m’
exit
m()
... F
if(a>0)
if(b<3) T
(b>=3) &&
(c > d)
(a>0) &&
(c==d) m’
(c > d)  (a’ = 1)
(c == d)  (b’ = 9)
(c < d)  (b’ = 2)

11. Building the tree root = a path in mu that traverses the use
Step III
root = a path in mu that traverses the use
If possible, add md to the tree and explore only the sub-tree rooted at md, else start exploring the paths that do not imply the condition of their predecessor
Once md has been added, the next goal is to add a constructor
Stop when:
Both md and a constructor have been added: test case
No more nodes can be added: infeasible DUA
Tree depth reaches a user-provided threshold : fail

12. Building the deduction tree for DUA #7
Step III
vend
(AllowVend!=0)
use
var: curQtrs
def: returnQtr node 11
use: vend node 20a
returnQtrs
def
(AllowVend!=0)
addQtr
returnQtrs
returnQtrs
(true)  curQtrs'=0
def={curQtrs}
(curQtrs>0)
vend
addQtr
(curQtrs = 0)
(curQtrs>0)
(curQtrs > 0) && (AllowVend == 0)
(curQtrs == 0) && (AllowVend!=0)
(AllowVend!=0)
addQtrs
(curQtrs>0)  allowVend'=1
def={curQtrs,allowVend} curQtrs'=curQtrs+1
(curQtrs==0)  total'=total
def={curQtrs} curQtrs'=1
returnQtrs
(true)  curQtrs'=0
def={curQtrs}
CoinBox
(true)
constructor
Test case: CoinBox(), addQtr(), addQtr(), returnQtr(), vend()

13. Characteristics of the technique
Focuses on problems related to the state (instance variables)
Generates test cases from the source code (no specification required)
Uses a stepwise automated approach (partial results can be used)

14. The experimental setting
EDG
C++ Code
C++ Parser
IL Analyzer IL
Data Flow Analyzer
Abstract Graph
DF Tool
Symbolic Executor
DUAs
DUAs + Pre/Post Conditions
SICStus
Message
Sequences
Automated Deduction

15. Summary and future work
Technique for automated generation of test data for classes
Combination of existing techniques in a new context: data-flow analysis, symbolic execution, and automated deduction
Useful intermediate and partial results
DUAs
Pre/Post
Subset of test cases
Results of the preliminary experiments are encouraging
A tool is under development (first prototype due by the end of the year)
Future work
Experimentation
Extension to non-primitive attributes

16. Questions?

17. Some statistics
Percentage of classes containing only scalar attributes w.r.t. the total number of classes in the system
83.9%
78.9%
77.7%
53.6%
46.9%
37%
31.3%
jflex
javacup
jasmin
antlr
javasim
jasjas
jfe