Coverage Criteria for Testing of Object Interactions in Sequence Diagrams Atanas (Nasko) Rountev Scott Kagan Jason Sawin Ohio State University

UML Sequence Diagrams Shows messages exchanged among objects Contains control information relevant to message passing

Sequence Diagrams and Testing Object interaction testing Coverage criteria for sequence diagrams  all paths, all branches, etc. Most existing work utilizes the all paths criterion

Sequence Diagrams and Testing Most existing work focuses on design time diagrams Diagrams can also be reverse engineered from code  more paths due to less abstraction Do existing sequence diagram based testing approaches generalize to reverse engineered diagrams?  should the all paths criterion be considered?

Our Work Investigated 4 different coverage criteria for reverse engineered sequence diagrams  defined each criterion in terms of an RCFG tree  developed a method for comparing criteria  performed a comparison of the criteria on real world Java code

RCFG Tree Restricted Control Flow Graph (RCFG)  similar to a traditional CFG  for message passing only RCFG tree represents a sequence diagram  set of all start to end paths for both structures is the same Root RCFG is top level method Edges connecting RCFGs represent method invocations  RCFG n is a child of RCFG m means method m may invoke method n

m1 start end m2 m6 m5 end m4 startm3 m5 end m4 startm3 m6 start end m2 a:Ad:Dc:Cb:B [c1] m3 m2 [c1] m3 m4 [c2] m5 m1 m4 [c2] m5 [c3] m6 m2 start end m4

RCFG Tree Details Straightforward to construct when reverse- engineering sequence diagrams Can be traversed at run time to collect coverage statistics

Coverage Criteria Considered All start to end paths All RCFG paths All RCFG branches All unique branches m1 start end m2 m6 m5 end m4 startm3 m5 end m4 startm3 m6 start end m2

Criterion Comparison Technique Each start to end path corresponds to one test case For each criterion, calculate minimum number of start to end paths required to achieve complete coverage  indirect estimate of testing effort

Minimum Number of Paths All start to end paths: 20 All RCFG paths: 5 All RCFG branches: 3 All unique branches: 2 m1 start end m2 m6 m5 end m4 startm3 m5 end m4 startm3 m6 start end m2

Calculating Minimum Number of Paths All start to end paths  calculate the number of paths from the start node of an RCFG to the end node of the RCFG must consider paths in children  start with the leaves, propagate up to the root All RCFG paths, all RCFG branches, all unique branches  integer linear programming

Valid Path Property If node n represents a method call and edge (m, n) is part of a path p,  p must include an inter-method edge (n, start)  p must include an intra-method edge (n, r) n m r start q

Valid Path Property If node n represents a method call and edge (n, start) is part of a path p,  p must include an intra-method edge (start, q) n m r start q

RCFG Branches Property Consider some set of paths S that satisfies the RCFG branch property  every RCFG edge appears in at least one path n m r start q

Integer Linear Programming Formulation ILP equations encode the valid path property and the RCFG branches property  variables are edge frequencies  constraints each frequency is at least 1 frequency sums are equal (red, green, blue) n m r start q

Proof Obligation Solution to the integer linear programming problem is a frequency count for each edge Need to prove  There exists a set of paths that meets the coverage criterion and has the same edge frequencies  No smaller set of paths satisfies the criterion

Remaining Criteria Integer linear programming formulation  valid paths property  a criterion specific property All unique branches  the sum of the edges frequencies for equivalent edges is at least one All RCFG paths  each edge is covered at least as many times as its local edge frequency

Experimental Study Implemented as part of the RED tool (Reverse Engineering of sequence Diagrams) Implementation used lp_solve linear programming tool Input code:  code from the Java standard libraries  1104 RCFG trees  average 36 RCFGs per tree

Future Work Implement dynamic analyses for obtaining coverage statistics Measure coverage achieved by real world test suites