1. Testing OO Programs (7.1, 2.2.2, 2.4.2) Course Software Testing & Verification 2014/15 Wishnu Prasetya

2. Plan The goal is to discuss how to test OO programs; to that end we also need some concepts from: Data flow perspective to testing Integration testing 2

3. Basic idea The two paths of P above will carry the same value of x “defined” at the beginning to the return statement. A plain control flow graph does not describe how “data” flows through the program; so we can’t infer which execution paths need to be tested (or not) to test the influence of a certain variable on the program 3 P(x,y) { if (y<x) y = x+1 ; return x*y }

4. Data-flow based approach (sec. 2.2.2) Data flow graph : CFG, where the nodes and edges are decorated with information about the variables used and defined by each node/edge. In my examples I avoid decorating on the edges. Some subtlety on the granularity of the nodes/edges  later. 4 P1(x,y) { if (y<x) y = x+1 ; return x*y } n1 n2 n3 (then) use(n1) = {x,y} use{n2) = {x} def(n2) = {y} use(n3) = {x,y} n0 def (n0) = { x,y }

5. Terminologies Def/use graph: see above. In general, def/use labels are allowed on the edges as well. A path p = [i,...,k] is def-clear wrt v iff v  def(j) and v  def(e) for all nodes j and edges e in p between i and k. It (def-clear) is a du-path wrt v iff v  def(i) and v  use(k), and furthermore it is a simple path. 5 n1 n2 n3 use = {x,y} def = { x,y } use = {y} def = {y} n4 n5 use = {x,y} def =  use = {x,y} def =  use = {x,y} def ={x} du-paths wrt x: 13, 134, 135 12, 1234, 1235, 45 Not du-path wrt x: 1345 (not def-clear)

6. Block’s granularity When a node both defines and uses the same variable x, we will assume that it uses x first, then assigns to it, as in x =... x... If this assumption would be broken (middle example)  split the node. Else this will conflict the intention of your definition of du-path. Check 2.3.2 on how to map a source code to a def/use graph. 6 x = x+1 x = 0 y = x+1 ; x = 0 y = x+1 n use(n) = {x} def (n) = { x } n use(n) = {x} def (n) = { x,y } n1 use(n1) = {} def (n1) = { x} n2 use(n2) = {x} def (n2) = { y}

7. Data-flow based coverage (Rapps-Weyuker 1985, Frankl-Weyuker 1988) Def-path set du(i,v): the set of all du-paths wrt v, that starts at i. Def-pair set du(i,j,v): the set of all du-paths wrt v, that starts at i, and ends at j. Same concept of touring. (C 2.9, All Defs Covrg, ADC) For each def-path set S = du(i,v), TR includes at least one member of S. 7 n1 n2 n3 use = {x} def = { x} use = {x} def ={x} n4 use = {x} def =  use ={x} def = 

8. Data-flow based coverage (C 2.10, All Uses Covrg, AUC) For each def-pair set S = du(i,j,v), TR includes at least one member of S. (C2.11, All du-paths Covrg, ADUPC) For each def-pair set S = du(i,j,v), TR includes all members of S. Note: the above example only has one variable, namely x; consequently all TRs above only concern x. 8 ADC TR: 12, 234 AUC TR: 12 23, 232, 234 ADUC TR: 12 23, 232, 234, 24 n1 n2 n3 use = {x} def = { x } use = {x} def ={x} n4 use = {x} def =  use ={x} def = 

9. Overview of subsumption relations PPC  ADUPC, because every simple path is a subpath of some prime path. AUC  EC.... under the following assumptions: – every use (of v) is preceded by a def. – every def reaches at least one use. – multiple branches e.g. as in if(g) then S1 else S2 use the same variables (if S1 uses x, then S2 at some point also uses x, vice versa). 9 complete path covrg prime path covrg edge-pair covrg edge covrg node covrg ADUPC AUC ADC

10. Summary We learned another set of white-box test-coverage criteria, namely data-flow based coverage an alternative to prime-path-based testing, if the latter becomes too expensive. 10

11. Integration Test Integration test: to test if a composition of program units works correctly. “Hidden” in Section 2.4 about using design elements (e.g. your UML models) as your coverage criterion. 2.4.1 says: “testing software based on design elements is usually associated with integration testing”. 2.4.2 is actually more about integration testing rather than design-element-based testing. 11

12. Data flow approach to Integration Test (2.4.2) Imagine method f from module A calls method/API g from module B. Idea 1: construct the combined CFG of both, then use prime path coverage criteria. This blows up. Idea 2: focus on the flow of the “coupling data” around the call point. 12 f (a) {... g(e)... } g(x) {... } B A

13. Some terminologies caller, callee, actual parameters, formal parameters parameter coupling, shared variable coupling, external device coupling More general concept: coupling variable  a ‘variable’ defined in one ‘unit’, and used in the other. 13 (1) a =... ;b =... (3) a =... (2) a>b (4) b = g(b) (1) g(x) (2) x<=a (3) x==a (4) return... (5) return...

14. Coupling du-path (Jin-Offutt, 1998) (Def 2.40) Last-def of v : set of nodes defining v, from which there is a def-clear path (wrt v) through the call-site, to a use in the other unit. Can be at the caller or callee! Example: f1, f3, g4, g5. (Def 2.41) First-use (of v) E.g.: g2, f4 Coupling du-path of v is a path from v’s last-def to its first-use. 14 (1) a =... ;b =... (3) a =... (2) a>b (4) b = g(b) (1) g(x) (2) x<=a (3) x==a (4) return... (5) return...

15. First use at the callee.. When you just pass a variable as in g(b), it makes sense to define g’s first use to be the first time g uses the parameter (location g.3 in the previous example). But when you pass a complex expression like g(a+b)  what are g’s first uses of a and b?  in this case it make sense to define g’s first uses (of a and b) to be at the call location itself. 15

16. Integration Test-Requirement ADC  All Coupling Def Cov. : for every last-def d of each coupling var v, TR includes one coupling du-path starting from d. AUC  All Coupling Use Cov. : for every last-def d and first-use u of v, TR includes one couple du-path from d to u. 16 Coupling vars: a, b, g.return Coupling du-paths : a: [f1,f2,g1,g2] [f3,f4,g1,g2] b: [f1,f2,g1,g2] [f1,f2,f3,g1,g2] g.return: [g4,f4] [g5,f4] (1) a =... ;b =... (3) a =... (2) a>b (4) b = g(b) (1) g(x) (2) x<=a (3) x==a (4) return... (5) return...

17. OO is powerful, but also comes new sources of errors A function behaves “cleanly”; a procedure does not. An OO program is more complicated: – side effects on instance variables and static variables – access control (priv, pub, default, protected,...) – dynamic binding – interaction through inheritance 17

18. Anomalies Simply mean non-standard situations. Not necessarily errors, but they tend to be error prone. AO list 9 anomalies related to inheritance and dynamic binding: – Inconsistent Type Use (ITU) – State definition anomaly (SDA),... 7 more AO also implicitly discuss two more: deep yoyo and data-flow anomaly. 18

19. Inconsistent Type Use (ITU) anomaly 19 List +insertElemAt(i:int) +remElemAt(i:int) Stack +push(x) +pop() This is a quite common subclassing architecture that people use. However note that this does not prevent a Stack to be used as a List, which is error prone (it allows you to call insertElemAt or remElemAt on a Stack).

20. State Definition Anomaly (SDA) ScalableItem (subclass) directly manipulates the part of its state inheritted from UnitItem (rather than through UnitItem’s method). This error prone: may break UnitItem’s class invariant. 20 ScalableItem setScale(s) { scale = s } UnitItem scale = 1 price() { return 1/scale }

21. State definition inconsistency due to state variable hiding (SDIH) The shadowing of price done by OnSaleItem is error prone; notice the method netPrice (that it also inherits!) would use the old price. 21 Item price tax netPrice() { return price * (1 + tax) } setPrice(p) { price = p } OnSaleItem price sale(p) { price = 0.8 p }

22. State Visibility Anomaly (SVA) 22 A - x + incr() BC + incr2() C’s needs a method incr2 that would increase x by 2. However it can’t get to x, because it is private to A. It can however calls super.incr() twice. So far it is ok. Now imagine someone else changes B by overriding incr. This suddenly changes the behavior of C; as it now calls B’s incr instead.

23. Anomalous Construction Behavior 1 (ACB1) Inheritence may radically change the orginal behavior. Item’s constructor calls reset(). ExpensiveItem’s constructor calls Item’s constructor, but implicitly forces it to call the new reset() ! Called a Yoyo-effect phenomenon. 23 Item price tax Item() { reset() ; tax=0 } reset() { price = 1 } ExpensiveItem baseprice ExpensiveItem() { super() ; baseprice=10 } reset() { price = baseprice }

24. Call graph Here, it is a graph describing which program “units” call which.. 24 f gh Here, we’ll define f  g means f contains a call to g in its body (note that it does not mean that in the runtime f will actually call g; it may depend on its parameters) The two branches from f in the example says nothing about the order in which the children will be called by f.

25. The call graph of d() in ABC yoyo graph, Alexander-Offut 2000 25 C +j () { k() } B +h () { i()...} + i() { super.i()...} +k() { } A +d () { g() } +g () { h() } +h () { i() } +i () { j() } +j () {} C j() B h()i() A d()j()g()h()i() A d()j()g()h()i() B h()i() k() A d()j()g()h()i()

26. Data flow anomaly 26 B +h () +i () A +d () +g () +h () +i () +j () B h()i() A d()j()g()h()i() A d()j()g()h()i() def xuse x Not necessarily an error. E.g. if at the runtime the use of x in j() when it is called from B’s i() is actually unfeasible. But this is still error prone (anomaly) def xuse x

27. Check the remaining anomalies yourself State Defined Incorrectly (SDI) Indirect Inconsistent State Definition (IISD) Anomalous Construction Behavior 2 (ACB2) Incomplete Construction (IC) 27

28. Testing your OO programs Intra-method testing (method-level unit-testing) Intra-class testing, as done by class-level unit testing. Inter-class testing. The first two levels are usually not enough to expose errors related to access control, interaction through inheritance, and dynamic binding. Let’s now take a look at inter-class testing. 28

29. Inter-class testing Testing the interaction of class A and E : treat this as an integration testing problem  we already have a solution (data-flow based integration test), but additionally we need to quantify over the subclasses of A, due to dynamic binding (the exact m called in f depends on the type of actual type of o ). 29 class E { f(x) { A o = FactoryA(x ) ;... o.m() } }

30. Testing more complicated interaction The way f(x) interacts with o is more complicated. There may be various combinations of which m and n are actually called, leading to complex overall behavior. Let’s extend our idea of integration test, to also cover such interaction. 30 class E { f(x) { A o = FactoryA(x ) ; // (1) o.m() ; // (2)... o.n() } // (3)

31. Terminologies f itself is here called the coupling method. The field o.x is “indirectly” defined (i-def) if the field is updated. Analogously, use and i-use. Focus on coupling within f due to variables o.v (indirectly) def. by o.m(), and used by o.n().  antecedent and consequent. 31 f(x) { (1) A o = FactoryA(x ) ; (2) o.m() ;... (3) o.n() }

32. Terminologies Coupling sequence C: a pair of antecedent m, and consequence n, such that there is at least 1x def-clear path from m to n with respect to some field o.x defined in m and used in n. o.x is called a coupling var (of C) The set of all coupling vars of C  coupling set. 32 f(x) { (1) A o = FactoryA(x ) ; (2) o.m() ;... (3) o.n() }

33. 33 With polymorphism the situation will now look like this... Notice how different types of the context o may give different coupling variables as well as coupling paths.

34. Binding triple Each coupling sequence induces one or more binding “triples”; each specifying the coupling variables for various types of the context-var o. For example, this could be the binding triples of sequence o.m()  o.n() 34 A B C type of ocoupling vars A{ o.v } B{ o.u } C{ o.u, o.v } binding triples of c = o.m()  o.n()

35. Inter-class coverage test criteria (Def 7.53) All-Coupling-Sequences (ACS): for every coupling sequence c in the coupling method f, TR includes at least one coupling path of c.  ignore polymorphism. (Def 7.54) All-Poly-Classes (APC): for every binding triple t of c, TR includes at least one coupling path of t.  ignore that c may have multiple coupling vars. 35 A B C type(o)coupling varscoupling paths A{ o.v } o.v : { p1 } B{ o.u }o.u : { q1, q2 } C{ o.u, o.v }o.u : { r1 }, o.v :{r2} binding triples of c = o.m()  o.n(), extended with infos over coupling paths.

36. Inter-class coverage test criteria (Def 7.55) All-Coupling-Defs-Uses (ACDU): for every coupling var v of c, TR includes at least one of v’s coupling path. (Def 7.56) All-Poly-Coupling-Defs-Uses (APCDU): for every coupling var v of every binding triple of c, TR includes at least one of v’s coupling path. 36 A B C type(o)coupling varscoupling paths A{ o.v } o.v : { p1 } B{ o.u }o.u : { q1, q2 } C{ o.u }o.u : { r1 }