1. An Eternal Triangle Martin Woodward, U. Liverpool, UK
Saturday, February 23, 2019
An Eternal Triangle
Fault Size
Testability
D-to-R Ratio
Martin Woodward, U. Liverpool, UK
Zuhoor Al-Khanjari, SQU, Oman
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland
ISSTA 2000, Portland

2. Outline of Talk Functional view of software
Domain-to-range ratio (DRR)
Domain testability and DRR
PIE testability and fault size
PIE testability via mutation
Fault size and DRR
Related work
Conclusions
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland

3. A Functional View of Software
Correct program P: D ® R
Suppose single fault f exists.
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland

4. Domain-to-Range Ratio (DRR)
DRR = |D| / |R|, where:
|D| = cardinality of inputs
|R| = cardinality of outputs
High DRR, i.e. |D|>|R| Þ
“greater potential to hide faults”
“internal state collapse”
DRR is “a rough approximation of the software’s testability”
|D|>|R|
|D|=|R|
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland

5. Domain Testability (Freedman, 1991)
Observability: “distinct outputs are generated from distinct inputs”
" a, b ÎD · P(a)¹P(b) Þ a¹b
Controllability: “given any desired output value, an extra input exists which ‘forces’ the component to that value”
" r Î R · $ d Î D · P(d) = r
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland

6. Domain Testability Extensions
Observable extensions:
introduce n new input variables with types Ti
Ob = log2( |T1| ´ ´ |Tn| )
Controllable extensions:
reduce the range by modifying m output variable types to Tk
Ct = log2( |T1| ´ ´ |Tm| )
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland

7. Domain Testability and DRR
Extend D by D giving D’
Reduce R by R giving R’
For testable component:
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland

8. PIE Testability and Fault Size
Pr[Failure]=Pr[E]Pr[I |E]Pr[P |I]
E=Execute, I=Infect, P=Propagate
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland

9. PIE Testability via Mutation
Let ML be representative non-equivalent mutant of location L
TestabilityL » min (semantic size of ML)
Find proportion of large number of random test cases that ‘kills’ ML
Traditional mutation stops testing a mutant once ‘dead’
c.f. size estimates for mutation operators (Offutt & Hayes, 96)
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland

10. Fault Size and DRR Define:
input fault size = |Df | / |D |
output fault size = |Rf | / |R |
DRR for correct program P under full input domain D :
Can be related to DRR for faulty version under Df :
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland

11. Related Work PIE (Voas,92), DRR(Voas+,95)
Fault size (Offutt & Hayes, 96)
Fault-based testing theory (Morell, 90)
RELAY model (Richardson et al., 93).
Dynamic impact analysis (Goradia, 93)
Prioritizing fault-exposing tests (Rothermel et al., 99)
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland

12. Conclusions Testability: Domain testability and DRR
Saturday, February 23, 2019
Conclusions
Testability:
is an important attribute
has various guises
and links with other concepts
Domain testability and DRR
PIE testability and fault size
Validation possible by:
empirical studies
analytic approach (How Tai Wah, 2000)
Saturday, February 23, 2019Saturday, February 23, 2019Saturday, February 23, 2019
ISSTA 2000, Portland
ISSTA 2000, Portland