1. IPOG: A general strategy for t-way software testing by Lei, Yu, IEEE 2007
5/2/2019
Mehra N Borazjany

2. Introduction IPOG, From 2-way(pairwise) to t-way testing Fireeye tool
Major Challenge of IPOG: dealing with the combinatorial growth in the number of combinations of parameter values
Fireeye tool
5/2/2019
Mehra N Borazjany

3. 2-way testing (pair-wise)
Create test suites to detect faults caused by interactions between any two parameters
3 Boolean parameters {a[0,1], b[0,1], c[0,1]}
2-way test set: Exhaustive test set: a b c 1 a b c 1
5/2/2019
Mehra N Borazjany

4. t-way testing (Combinatorial)
faults can also be caused by interactions involving more than two parameters
Create test suites by selecting values for input parameters and combining these parameter values
5/2/2019
Mehra N Borazjany

5. Cont. t-way testing
Studies have shown t-way can effectively detect faults.
Reduce the number of tests
t is referred to as the strength of coverage
t takes a small value. (2 to 6)
5/2/2019
Mehra N Borazjany

6. IPOG algorithm Algorithm IPOG-Test (int t, parameterset ps) {
Initialize test set ts = empty set
Sort parameters ps (non-increasing) and denote them as P1,…,Pk
Add into ts a test for each combination of values for first t parm.
For (int i = t+1 ; i<= k; i++){
let П be the set of all t-way combinations of values involving Pi and any group of (t-1) params among the first i-1 params
5/2/2019
Mehra N Borazjany

7. IPOG algorithm (Cont) // horizontal extension for Pi
For (each test t’=(v1,…vi-1) in test set ts) {
Choose a value vi of Pi and replace t’ with t’’ = (v1,..,vi) so that t’’ covers the most number of combinations of values in П
Remove from П the combinations of values covered by t’’
} // end for at line 7
5/2/2019
Mehra N Borazjany

8. IPOG algorithm(Cont) // vertical extension for Pi
For (each combination z in set П){
If (exists a test t’ in ts such that it can be changed to cover z){
Change test t’ to cover z
} else {
Add new test to cover z
}// end if at line 13
}// end for at line 12
}// end for at line 4
Return ts; }
5/2/2019
Mehra N Borazjany

9. IPOG algorithm (Cont) Example
Number of parameters in ps are greater or equal to t
Example : 3-way testing
t=3
Ps = {p1[0,1], p2[0,1], p3[0,1], p4[0,1,2]}
Number of parameter = 4
4 > 3
IPO is deterministic
5/2/2019
Mehra N Borazjany

10. IPOG algorithm (Cont) Example
ts : p1 p2 p3 p4 1
П : 1 p1 p2 p4 1 2 p1 p3 p4 1 2 p2 p3 p4 1 2 1 1 1 * 2 1 * 2 1 * 1 2 1 1 * 2 1
5/2/2019
Mehra N Borazjany

11. Other Approaches to t-way testing
AETG (Automatic Efficient Test
Generator) by Cohen
complete test at a time until all the combinations are covered
What is complete test?
Covering one parameter at a time allows the IPO strategy to achieve a lower order of complexity than AETG
5/2/2019
Mehra N Borazjany

12. Other Approaches to t-way testing
AI-based search techniques such as hill climbing
Unlike AETG and IPO, AI-based search techniques start from a randomly generated test set
AI-based search techniques can produce smaller test sets than AETG and IPO,
Significantly longer time to complete
5/2/2019
Mehra N Borazjany

13. Complexity of IPOG Algorithm
Enumeration all possible t-way combinations (Large Number)
Space for storing
Time to enumerate
IPOG-D
Reduce the number of combination needed to enumerate
Recursive construction
Deterministic
5/2/2019
Mehra N Borazjany

14. Constraint Satisfaction Problem (CSP)
Values {green, blue, red}
Variables {f,a,b,c,e,g}
Constraints {no same color as a neighbor} f a b e c g
5/2/2019
Mehra N Borazjany

15. Forward checking
One way to make a better use of constraints during search
After assign green to b, a and e forced to be blue
{f =red, b =green, e= blue} -> inconsistent f a b e c g f a b c e g
Initial
5/2/2019
Mehra N Borazjany

16. Fireeye (ACTS)
demo
5/2/2019
Mehra N Borazjany

17. Question?
5/2/2019
Mehra N Borazjany

18. Thank You
&
Happy Valentine’s day
5/2/2019
Mehra N Borazjany