1. Data-flow Based Testing (A&O Ch. 2. 2. 2, 2. 4. 2, 7. 1) (2nd Ed. Ch 7
Data-flow Based Testing (A&O Ch , 2.4.2, 7.1) (2nd Ed. Ch 7.2.3, Unfortunately, there is no equivalent of 7.1 in the 2nd Ed.)
Course Software Testing & Verification
2018/19
Wishnu Prasetya & Gabriele Keller

2. Plan Data flow driven testing Integration testing
Testing inter-class interactions in an OO setup
Note: an important criticism to control-flow based testing (covered in the lecture on graph-based testing) is that it ignores how data actually flows through the control flows, and we cannot distinguish between which data flows might be more critical to test. Data-flow based testing tries to address this shortcoming. Later, it also gives us a useful instrument to handle integration level testing. OO poses another kind of challenges for testing, in particular with respect to error prone patterns due to inheritance and dynamic binding. We will also discuss how to address such situations; we will build our approach based on the previously discussed integration testing approach.

3. Basic idea
P(x,y) { if (y<0) y = -y else y = 1 ; if (x<0) x = -x + y else x = x + y ; return x }
Imagine the test set (leaving out the oracles): { P(-1,-1) , P1(0,0) } . This two tests would give us full edge coverage.
However, this ignores data flow between each branch of the first “if” to each branch of the second “if”. Each data flow triggers different behavior, and arguably should have been tested.
Arguably, dataflow is what actually determines the behavior of a program, rather than control flow. From this perspective, dataflow based testing is more complete than control flow based.
E.g. we may then decide to only test one of them.

4. Data-flow based approach (sec. 2.2.2/ 2nd Ed: 7.2.3)
use = 
def = { x,y }
use = {y}
def = {y} n4 n5
def = 
use = {x,y}
def = {retval}
use = {x}
def ={x} n1
P(x,y) { if (y<0) y = -y if (x<0) x = -x return y+x }
Data flow graph : CFG, where the nodes and edges are decorated with information about the variables used and defined by each node/edge.
In my examples I avoid decorating on the edges.
Some subtlety on the granularity of the nodes/edges  later.

5. Terminologies
use = {y}
def = {y}
use = {x}
def ={x}
du-paths wrt x:
013, 0123
0134, 01234,
0135,
Not du-path wrt x:
01345 (not def-clear)
use = 
def = { x,y } n2 n4 n0 n1 n3 n5
use = {y}
def = 
use = {x}
def = 
use = {x,y}
def = {retval}
Def/use graph: see above. In general, def/use labels are allowed on the edges as well.
A path p = [i,...,k] is def-clear wrt v iff v  def(j) and v  def(e) for all nodes j and edges e in p between i and k.
A du-path wrt v = a simple and def-clear path from i to k with vdef(i) and vuse(k).
If p is def-clear wrt v, then it does not re-define v in between; implying that v is only defined at the start of p (if it is defined there at all). Thus, this value of v is guaranteed to reach p’s final node.
Not du-path wrt to x: (the path is not clear wrt x)

6. Block’s granularity x = 0 y = x+1 ; x = 0 y = x+1 x = x+1
usen1 = {}
defn1 = { x} n1
use = {x}
def = { x }
use = {x}
def = { x,y } n n
usen2 = {x}
defn2 = { y} n2
When a node both defines and uses the same variable x, we will assume that it uses x first, then assigns to it, as in x = ... x ...
If this assumption is broken (middle example)  split the node. Else this will conflict the intention of your definition of du-path.
Check (2nd Ed ) on how to map a source code to a def/use graph.

7. Data-flow based coverage (Rapps-Weyuker 1985, Frankl-Weyuker 1988)
use = 
def = { x}
use = {x}
def ={x} n4
def =
use ={x}
def = 
Def-path set du(i,v): the set of all du-paths wrt v, that starts at i.
Def-pair set du(i,j,v): the set of all du-paths wrt v, that starts at i, and ends at j.
Same concept of touring.
(C 2.9/2nd Ed. C7.15, All Defs Covrg, ADC) For each def-path set S = du(i,v), TR includes at least one member of S.
The intuition of ADC is to cover for each variable, all the places where the variable is assigned a value. However, we only care about it when the new value assigned is actually used somewhere. This comes from the fact that the definition requires the existence of a du-path from the node.

8. Data-flow based coverage
use = {x}
def ={x}
use = {x}
def =
ADC TR:
12,
234
AUC TR: 12
23 , 232,
234
ADUC TR: 12
23 , 232,
234, 24
use ={x}
def =  n1 n2 n3 n4
use = 
def = { x }
(C 2.10/2nd Ed. C7.16, All Uses Covrg, AUC) For each def-pair set S = du(i,j,v), TR includes at least one member of S.
(C2.11/2nd Ed. C7.17, All du-paths Covrg, ADUPC) For each def-pair set S = du(i,j,v), TR includes all members of S.
Note: the above example only has one variable, namely x; consequently all TRs above only concern x.
Recall that in ADC we want to cover, for each variable, all the places where the variable is assigned a value. However, we only care about it when the new value assigned is actually used somewhere. However, for each node i where v is defined, there may be multiple nodes where i is subsequently used. ADC doesn’t distinguish those. In AUC we do. In AUC, for each pair of def/use we require at least one path is tested. There may however be multiple paths between the same def/use pair. In ADUPC we require to cover them all.

9. Overview of subsumption relations
complete path covrg
prime path covrg
ADUPC
edge-pair covrg
AUC
edge covrg
ADC
node covrg
PPC  ADUPC, because every simple path is a subpath of some prime path.
AUC  EC .... under the following assumptions:
there is at least one def
every def reaches at least one use.
every use (of v) is preceded by a def.
every branch uses a variable
(Def 1.24) C1 subsumes C2 if every test-set that satisfies C1 also satisfies C2 (so, C1 is stronger)
The assumption “arrows are labelled; arrows that leave the node i must use at least one var, and they all must use the same set of vars” basically wants to say, that all branches of e.g. if(g) then S1 else S2 will use all the variables used in g.

10. Summary
We learned another set of white-box test-coverage criteria, namely data-flow based coverage.
An alternative to prime-path-based testing, if the latter becomes too expensive, while being more aware of the semantic of the program under test.

11. Integration Test
Integration test: to test if a composition of program units works correctly.
“Hidden” in Section 2.4 (2nd Ed. 7.4) about using design elements (e.g. your UML models) as your coverage criterion (2nd Ed. 7.4 p146) says: “testing software based on design elements is usually associated with integration testing”.
2.4.2 (2nd Ed ) is actually more about integration testing rather than design-element-based testing.

12. Integration Test, example
Class A
Server
Design:
Class A
Mock Server
During unit test of A :
Class A
Actual
Server
During integration test of A :
We can test the integration between A and the Server by re-running the unit test of A, but replacing the mock server with a real server. Question: what should we use to determine the adequacy of this test?

13. Data flow approach to Integration Test (2.4.2/2nd Ed. 7.4.2)
module A
module B
g(x) { ... }
f (a) { g(e) }
Imagine a method f from module A uses a method/API g from module B. How to test such a composition?
Idea 1: construct the combined CFG of both, then use prime path coverage criteria. This blows up.
Idea 2: focus on the flow of the “coupling data” around the call point.

14. Terminology caller, callee, actual parameters, formal parameters
(1) g(x,y)
(1) a = ... ;b = ...
(2) x<=y
(4) return ...
(2) a>b
(3) a = ...
(3) x==y
(4) b = g(a,b)
(5) return ...
caller, callee, actual parameters, formal parameters
parameter coupling, shared variable coupling, external device coupling
More general concept: coupling variable  a ‘variable’ defined in one ‘unit’, and used in the other.

15. Coupling du-path (Jin-Offutt, 1998)
(1) g(x,y)
(1) a = ... ;b = ...
(2) x<=y
(4) return ...
(2) a>b
(3) a = ...
(3) x==y
(4) b = g(a,b)
(5) return ...
(Def 2.40/2nd Ed. 7.39) Last-def of v : set of nodes defining v, from which there is a def-clear path (wrt v) through the call-site, to a use in the other unit.
Can be at the caller or callee! Example:
(Def 2.41/2nd Ed. 7.40) First-use (of v) E.g.:
Coupling du-path of v is a path from v’s last-def to its first-use.
f1,f3 are last-defs of f’s a and b.
g4,g5 are last-defs of g’s return.
(Def 2.41) First-use of v : set of nodes using v, to which there is a def-clear and use-clear path (wrt v) coming from the other unit, and through the call point. E.g.: g1, g2, f4
g2 is first use of f’s a
g3 is first use of f’s b
f4 is first use of g’s return

16. Note on the first use at the callee
When you just pass a variable as a in g(a,b), it makes sense to define g’s first use to be the first time g uses the parameter (location g.2 in the previous example).
But when you pass a complex expression like g(a+b,...), it makes more sense to define g’s first uses (of a and b) to be the entry node of g (location g.1).

17. Integration Test-Requirementf
Coupling vars: a, b, g.return
Coupling du-paths : a: [f1,f2,f4,g1,g2] [f3,f4,g1,g2] b: [f1,f2,f4,g1,g2] [f1,f2,f3,f4,g1,g2] g.return: [g4,f4] [g5,f4]
(1) g(x,y)
(1) a = ... ;b = ...
(2) x<=y
(4) return ...
(2) a>b
(3) a = ...
(3) x==y
(4) b = g(a,b)
(5) return ...
f1 and f2 are last-defs of a at the f-side
f1 is the last-def of b at the f-side
g4,g5 are last-defs of g’s return.
g2 is first use of a at the g-side
g2 is first use of f’s b at the g-side
f4 is first use of g’s return at the f-side
(ACDC) All Coupling Defs Cov. : for every last-def d of each coupling var v, TR includes one coupling du-path starting from d.
(ACUC) All Coupling Uses Cov. : for every last-def d and first-use u of v, TR includes one couple du-path from d to u.

18. The next slides are based on A&O Ch 7. 1
The next slides are based on A&O Ch 7.1. Unfortunately, this chapter does not appear in the 2nd Ed.

19. OO is powerful, but also comes new sources of concerns
A function behaves “cleanly”. In contrast, a procedure may have side effects.
An OO program is more complicated:
access control (priv, pub, default, protected, ...)
side effects on instance variables and static variables
dynamic binding
interaction through inheritance

20. Inheritance-related error prone programming patterns
A&O call them inheritance-related “anomalies”: use of error prone programming patterns. AO list 9 anomalies related to inheritance and dynamic binding:
Inconsistent Type Use (ITU)
State definition anomaly (SDA) , more
A&O also implicitly discuss two more: deep yoyo and data-flow anomaly.

21. Inconsistent Type Use (ITU) anomaly
List
+insertAt(i,x)
+removeAt(i)
Stack
+push(x)
+pop()
This is a quite common subclassing pattern. However note that in OO a Stack s can also be used as a List, which leads to an error prone situation, since a stack should not allow elements to be inserted at or removed from an arbitrary position. But a user method may not be aware that s is actually a stack, and call insertAt and removeAt! The class Stack should have overridden the methods, and e.g. make them throw an illegal operation exception.

22. State Definition Anomaly (SDA)
UnitItem
scale = 1
+price() { return 1/scale }
Should maintain the class invariant: scale > 0.
ScalableItem
+setScale(s) { scale = s }
ScalableItem directly manipulates the part of its state which it inherits from UnitItem (the assignment “scale=s”), rather than doing so through UnitItem’s method. This error prone as it may unwittingly break UnitItem’s class invariant.

23. State definition inconsistency due to state variable hiding (SDIH)
Item
price
tax
+netPrice() { return price * (1 + tax) }
+setPrice(p) { price = p }
OnSaleItem
price
+sale(p) { price = 0.8 p }
The class OnSaleItem declares it own “price” field, which then shadows the original “price” field inherited from the class Item. Notice that OnSaleItem does not override netPrice, which uses “price” in its calculation. This leads to an error prone situation. When we have an instance o of OnSaleItem, and we call o.netPrice(), in Java the method netPrice would use the old price rather than the newly declared price.
Or perhaps, the conclusion s that shadowing instance vars is just not a good programming practice.

24. State Visibility Anomaly (SVA)x
+ incr() B C
+ incr2()
C’s needs a method incr2 that would increase x by 2. However it can’t get to x, because it is private to A. It can however calls incr() twice. So far it is ok. Now imagine someone else changes B by overriding incr. This suddenly changes the behavior of C; as it now calls B’s incr instead.

25. Anomalous Construction Behavior 1 (ACB1)
Item
price
tax
Item() { reset() ; tax=0 }
reset() { price = 1 }
ExpensiveItem
baseprice
ExpensiveItem() { super() ; baseprice=10 }
reset() { price = baseprice }
In the above design, Item’s constructor calls reset(). ExpensiveItem overrides reset(). So far so good. But then, ExpensiveItem’s constructor calls Item’s constructor. This leads to an error prone situation. The latter would then call reset(). In Java, it will call the new reset() rather than the old one (which may or may not be the intended behavior).
C# use base() instead of super() and base.m(x) instead of super.m(x)

26. Check the remaining anomalies yourself
State Defined Incorrectly (SDI)
Indirect Inconsistent State Definition (IISD)
Anomalous Construction Behavior 2 (ACB2)
Incomplete Construction (IC)

27. Error prone situations due to inheritance
In previous examples we have seen how inheritance can change the behavior of a class, and can create error prone situations.
These are not necessarily errors (could be the intended behavior), but there are definitely needs to verify them.
Also notice that methods from subclasses and superclasses can implicitly call each other, called “yoyo” effect.

28. The call graph of d() in A,B,C yoyo graph, Alexander-Offut 2000
+d () { g() }
+g () { h() }
+h () { i() }
+i () { j() }
+j () {} A
d()
g()
h()
i()
j() A
d()
g()
h()
i()
j() B
+h () { i() ...}
+ i() { super.i() ...}
+k() { } B
h()
i()
In C#: assume i() to be a virtual method. A
d()
g()
h()
i()
j() B
h()
i()
k() C
+j () { k() } C
j()

29. Testing OO programs
Of course: every method/class should be unit-tested.
However, it is hard to cover previously mentioned inheritance-related issue from just unit testing, because it depends a lot on how a class C is being used by other classes. It is not easy to say upfront how other classes will use C; even then we need to check that the using classes use C in a proper way.
The proposed approach in A&O is to check the latter in the using classes instead. To do this we can reuse the previously discussed integration test approach, and enhance it to cover for inheritance as well.

30. Inter-class testing 1
class A
+ m(...)
class E { f(x) { A o = Factory(x ) ;
o.m(...) } }
class B
+ m(...)
Testing the usage of class A by class E : treat this as an integration testing problem  we already have a solution (data-flow based integration test), but additionally now we need to quantify over the subclasses of A, due to dynamic binding (the exact m called in f depends on the type of actual type of o ).

31. Inter-class testing 2 class A class E2 { + m(...)
f(x) { A o = Factory(x ) ;
o.m(...) ;
...
y = o.n(...) }
class A
+ m(...)
+ n(..)
class B
+ m(...)
A class E2 that call methods of A multiple times within the same method (f). This may creates additional dynamic: the behavior of o.m() may affect o.n() through side effect on o’s fields. Some of this behavior could be critical. We will extend the previous approach of integration testing to also cover this kind of cross-method interactions through objects’ state.

32. Terminology f(x) { (1) A o = Factory(x ) ; (2) o.m(..) ; ...
(3) y = o.n(..) }
f itself is here called the coupling method.
The field o.x is “indirectly” defined (i-def) if the field is updated in some o.method() called by f. Analogously, we define i-use.
Focus on coupling within f due to fields o.x indirectly defined by o.m(), and used by o.n(); o.m() is then called antecedent of this coupling and o.n() the consequent.

33. Terminology f(x) { (1) A o = Factory(x ) ; (2) o.m(..) ; ...
(3) y = o.n(..) }
Coupling sequence c: a pair of antecedent m, and consequence n, such that there is at least 1x coupling path: a def-clear path from m to n with respect to some field o.x defined in m and used in n.
o is called the context var of the coupling sequence c
o.x is called a coupling var (of c)
The set of all coupling vars of c  coupling set.

34. Example with multiple coupling sequences
f2(x) { (1) A o = Factory(x ) ;
(2) o.m(..)
(3) o.n(..) (4) o.n(..) }
idef = { o.x , o.y }
iuse = { o.x, o.y } idef = { o.x }
iuse = { o.x , o.y } idef = { o.x }
Coupling sequence
Coupling vars
note
2 ➝ 3
o.x , o.y
3 ➝ 4
o.x
2 ➝ 4
o.y
Assuming there is no def. clear path between the idef of o.x in 2 and its iuse in 4

35. With polymorphism the situation will now look like this...
To note on the above diagram:
Different types of the context o gives different coupling variables as well as coupling paths.
Notice how different types of the context o may give different coupling variables as well as coupling paths.

36. Binding triple
Each coupling sequence c induces one or more binding “triples”; each specifying the coupling variables for various types of the context-var o. For example, this could be the binding triples of some coupling sequence c = o.m() o.n() A
coupling seq
type of o
coupling vars c A
{ o.v } B
{ o.u } C
{ o.u , o.v } B C
binding triples of c = o.m() o.n()

37. Inter-class coverage test criteria
Imagine the following coupling sequences within some method f : A
coupling seq. in f
type(o)
coupling vars
coupling paths c A
{ o.v }
o.v : { 𝜎1 } B
{ o.u }
o.u : {𝜏1, 𝜏2 } C
{ o.u , o.v }
o.u : { 𝜋1 } , o.v :{𝜋2 , 𝜋3} d
o.u : { 𝜌1 } , o.v :{𝜌2}
binding triples of c = o.m() o.n() , extended with infos over coupling paths. B C
(Def 7.53) All-Coupling-Sequences (ACS): for every coupling sequence in the coupling method f , TR includes at least one coupling path of c.  ignore polymorphism. In the above example, the TR would consist of two paths, one for c and one for d.

38. Inter-class coverage test criteria
coupling seq. in f
type(o)
coupling vars
coupling paths c A
{ o.v }
o.v : { 𝜎1 } B
{ o.u }
o.u : {𝜏1, 𝜏2 } C
{ o.u , o.v }
o.u : { 𝜋1 } , o.v :{𝜋2, 𝜋3 } d
o.u : { 𝜌1 } , o.v :{𝜌2} A B C
(Def 7.54) All-Poly-Classes (APC): for every coupling sequence c, and every binding triple t of c, TR includes at least one coupling path of t.  ignore that c may have multiple coupling vars. In the above example, the TR would consist of 3 paths for c, and 1 for d.

39. Inter-class coverage test criteria
coupling seq. in f
type(o)
coupling vars
coupling paths c A
{ o.v }
o.v : { 𝜎1 } B
{ o.u }
o.u : {𝜏1, 𝜏2 } C
{ o.u , o.v }
o.u : { 𝜋1 } , o.v :{𝜋2 , 𝜋3} d
o.u : { 𝜌1 } , o.v :{𝜌2} A B C
(Def 7.55) All-Coupling-Defs-Uses (ACDU): for every coupling sequence c and every coupling var v of c, TR includes at least one of v’s coupling path.
(Def 7.56) All-Poly-Coupling-Defs-Uses (APCDU): for every coupling sequence c and every coupling var v of every binding triple of c, TR includes at least one of v’s coupling path.