IPOG: A general strategy for t-way software testing by Lei, Yu, IEEE 2007 5/2/2019 Mehra N Borazjany

Introduction IPOG, From 2-way(pairwise) to t-way testing Fireeye tool Major Challenge of IPOG: dealing with the combinatorial growth in the number of combinations of parameter values Fireeye tool 5/2/2019 Mehra N Borazjany

2-way testing (pair-wise) Create test suites to detect faults caused by interactions between any two parameters 3 Boolean parameters {a[0,1], b[0,1], c[0,1]} 2-way test set: Exhaustive test set: a b c 1 a b c 1 5/2/2019 Mehra N Borazjany

t-way testing (Combinatorial) faults can also be caused by interactions involving more than two parameters Create test suites by selecting values for input parameters and combining these parameter values 5/2/2019 Mehra N Borazjany

Cont. t-way testing Studies have shown t-way can effectively detect faults. Reduce the number of tests t is referred to as the strength of coverage t takes a small value. (2 to 6) 5/2/2019 Mehra N Borazjany

IPOG algorithm Algorithm IPOG-Test (int t, parameterset ps) { Initialize test set ts = empty set Sort parameters ps (non-increasing) and denote them as P1,…,Pk Add into ts a test for each combination of values for first t parm. For (int i = t+1 ; i<= k; i++){ let П be the set of all t-way combinations of values involving Pi and any group of (t-1) params among the first i-1 params 5/2/2019 Mehra N Borazjany

IPOG algorithm (Cont) // horizontal extension for Pi For (each test t’=(v1,…vi-1) in test set ts) { Choose a value vi of Pi and replace t’ with t’’ = (v1,..,vi) so that t’’ covers the most number of combinations of values in П Remove from П the combinations of values covered by t’’ } // end for at line 7 5/2/2019 Mehra N Borazjany

IPOG algorithm(Cont) // vertical extension for Pi For (each combination z in set П){ If (exists a test t’ in ts such that it can be changed to cover z){ Change test t’ to cover z } else { Add new test to cover z }// end if at line 13 }// end for at line 12 }// end for at line 4 Return ts; } 5/2/2019 Mehra N Borazjany

IPOG algorithm (Cont) Example Number of parameters in ps are greater or equal to t Example : 3-way testing t=3 Ps = {p1[0,1], p2[0,1], p3[0,1], p4[0,1,2]} Number of parameter = 4 4 > 3 IPO is deterministic 5/2/2019 Mehra N Borazjany

IPOG algorithm (Cont) Example ts : p1 p2 p3 p4 1 П : 1 p1 p2 p4 1 2 p1 p3 p4 1 2 p2 p3 p4 1 2 1 1 1 * 2 1 * 2 1 * 1 2 1 1 * 2 1 5/2/2019 Mehra N Borazjany

Other Approaches to t-way testing AETG (Automatic Efficient Test Generator) by Cohen complete test at a time until all the combinations are covered What is complete test? Covering one parameter at a time allows the IPO strategy to achieve a lower order of complexity than AETG 5/2/2019 Mehra N Borazjany

Other Approaches to t-way testing AI-based search techniques such as hill climbing Unlike AETG and IPO, AI-based search techniques start from a randomly generated test set AI-based search techniques can produce smaller test sets than AETG and IPO, Significantly longer time to complete 5/2/2019 Mehra N Borazjany

Complexity of IPOG Algorithm Enumeration all possible t-way combinations (Large Number) Space for storing Time to enumerate IPOG-D Reduce the number of combination needed to enumerate Recursive construction Deterministic 5/2/2019 Mehra N Borazjany

Constraint Satisfaction Problem (CSP) Values {green, blue, red} Variables {f,a,b,c,e,g} Constraints {no same color as a neighbor} f a b e c g 5/2/2019 Mehra N Borazjany

Forward checking One way to make a better use of constraints during search After assign green to b, a and e forced to be blue {f =red, b =green, e= blue} -> inconsistent f a b e c g f a b c e g Initial 5/2/2019 Mehra N Borazjany

Fireeye (ACTS) demo 5/2/2019 Mehra N Borazjany

Question? 5/2/2019 Mehra N Borazjany

Thank You & Happy Valentine’s day 5/2/2019 Mehra N Borazjany