1. Software Testing Methodologies (STM) Unit 4 – Domain Testing
Compiled with reference from:
Software Testing Techniques: Boris Beizer
Craft of Software Testing: Brain Marrick
Narasimha Rao.P

2. Domain Testing Domains & Paths Nice & Ugly Domains Domain Testing
We will see in this part of Unit 3:
Domains & Paths
Nice & Ugly Domains
Domain Testing
Domains & Interfaces Testing
Domains & Testability
ref boris beizer

3. Transaction-Flow TestingU3
Contents of the unit
Domains
Model, as a set, paths
Predicates
closure, dimensionality, assumptions
Nice & ugly domains
specified & implemented domains
Nice domains – properties, testing implications
Ugly domains, simplification by programmers & testers
Domain testing
Bugs & testing for those bugs
Testing procedure
Variations, tools, effectiveness
Domains & Interface Testing
Range, closure, span
Interface range, domain compatibility
Finding the values
Domains & Testability
Linearizing, coordinate transformations. Canonical program form
ref boris beizer

4. Domain Testing - Domains & PathsU3
Domain Testing Model D1 D2
INPUT
CLASSIFY
DO CASE 1
OUTPUT D3
(x,y,z,…)
{ Outcome } Dn
DO CASE 2
(1, 2, 3, 4, 5,…)
DO CASE 3
DO CASE n
ref boris beizer

5. Domain Testing Domain Testing Views Programs as input data classifiers
Verifies if the classification is correct
ref boris beizer

6. Domain Testing - Domains & PathsU3
Domain Testing Model
Two Views
Based on Specs
Functional Testing
Based on Implementation information
Structural Technique
ref boris beizer

7. Domain Testing - Domains & PathsU3
Domain Testing Model
Variables
Simple combinations of two variables
Numbers from - to +
Input variables as numbers
Loop-free Programs
ref boris beizer

8. Domain Testing - Domains & PathsU3
Domain Testing Model
Structural Knowledge is not needed. Only Specs.
For each Input Case,
A Hypothetical path for Functional testing
An Actual path for Structural testing
ref boris beizer

9. Domain Testing - Domains & PathsU3
Domain (simplified)
A Single Connected Set of numbers
No arbitrary discrete sets
Defined by the boundaries
One or more boundaries
Specified by predicates
Bugs likely at the boundaries
One or more variables D1
ref boris beizer

10. Domain Testing - Domains & PathsU3
Predicates
Interpretation
Structural Testing - CFG
Functional Testing - DFG
Specifies the Domain Boundary
n Predicates in Sequence  2n domains
Or, just two domains
A .AND. B .AND. C
ref boris beizer

11. Domain Testing - Domains & PathsU3
Paths
At least One PATH thru the Program
More paths for disconnected domains
ref boris beizer

12. Domain Testing - Domains & PathsU3
Summary:
Domain for a loop-free program corresponds to a set of numbers defined over the input vector
For every domain, there is at least one path thru the routine, along which that domain’s processing is done
The set of interpreted predicates traversed on that path (ie., the path’s predicate expression) defines the domain’s boundaries.
ref boris beizer

13. Domain Testing - Domains & Paths - Domain Closure
Boundary: Closed / Open
X >= MIN
X <= MAX
MAX
MIN D2 D3 D1
Closed D2
X >= MIN
X < MAX D3 D1
Closed
Open
MIN
MAX
ref boris beizer

14. Domain Testing - Domains & Paths - Domain Closure
Boundary: Closed / Open
MIN
MAX D1 D2 D3
X > MIN
X < MAX
Open
ref boris beizer

15. Domain Testing - Domains & Paths - Domain DimensionalityU3
One dimension per variable
At least one predicate
Slices thru previously defined Domain
Slicing Boundary
N-spaces are cut by Hyperplanes
ref boris beizer

16. Domain Testing - Domains & Paths – Bug Assumptions
Processing is OK. Domain definition may be wrong.
 Boundaries are wrong.
 Predicates are wrong.
Once input vector is set on the right path, it’s correctly processed.
More bugs causing domain errors …
ref boris beizer

17. Domain Testing - Domains & Paths – Bug Assumptions..
Double-zero representation
Floating point zero check
Contradictory domains
Ambiguous domains
defined
undefined
ref boris beizer

18. Domain Testing - Domains & Paths – Bug Assumptions..
Over-specified domains
Boundary errors
Boundary closure
Shifted
X > 3 .AND. X < 2 .AND. Y > 3
ref boris beizer

19. Domain Testing - Domains & Paths - Bug Assumptions
Boundary errors….
Tilted
Missing
Extra boundary
ref boris beizer

20. Domain Testing - Domains & Paths - Bug Assumptions
Closure Reversal
Faulty Logic
Simplification of compound predicates
X >= 3
ref boris beizer

21. Domain Testing - Domains & Paths - Bug Assumptions
Strategy for domain testing in 2-dim
ref boris beizer

22. Domain Testing - Domains & Paths - RestrictionsU3
In General…
Coincidental Correctness
DT cannot detect
Example
Representative outcome
Partition testing
Input Equivalence
F1(x) : +1 D1 D2
F2(x) : -1
ref boris beizer

23. Domain Testing - Domains & PathsU3
Domain Testing Model
INPUT
CLASSIFY
DO CASE 1
OUTPUT
DO CASE 2
DO CASE 3
DO CASE n D1 D3
(x, y, z, …)
(1, 2, 3, 4, 5,…)
{ Outcome } D2 Dn
Function f1 f2 f3 f4
ref boris beizer

24. Domain Testing - Domains & Paths - - RestrictionsU3
Simple boundaries & Compound predicates D1 X 16
X >=0 .AND. X <= 16
Y = 1 .AND. X >=0 .AND. X <= 16 X 16 Y D1
ref boris beizer

25. Domain Testing - Domains & Paths - - RestrictionsU3
Compound predicates….
Impact of .OR.
Concave, Disconnected
Adjacent domains with same function
Example
A B C + D E F
Eliminate compound predicates
ref boris beizer

26. Domain Testing - Domains & Paths - RestrictionsU3
Functional Homogeneity of Bugs
Functional form still Retained
a x + b y >= c
Bugs are only in a, b, c
ref boris beizer

27. Domain Testing - Domains & Paths - - RestrictionsU3
Linear Vector Space
Linear boundary predicate, Interpreted
Simple relational operators
Conversion to linear vector space
2-d Polar co-ordinates
Polynomials
Problems with Non-linear Boundaries
ref boris beizer

28. Domain Testing - Domains & Paths - RestrictionsU3
Loop-free Software
Predicate for each iteration
Loop over the entire transaction
Definite loop
ref boris beizer

29. Domain Testing - Domains & Paths – Nice & Ugly domains
Nice Domains
Requirements
Bugs  ill-defined domains
Before DT
Analyze specs
Make the Boundary Specs Consistent & Complete
ref boris beizer

30. Domain Testing - Domains & Paths – Nice domainsU3
Implemented Domains
Complete, Consistent & Process all inputs
Specified Domains
Incomplete, Inconsistent
Programmer’s / Designer’s Effort
ref boris beizer

31. Domain Testing - Domains & Paths – Nice domainsU3
Nice Domains
Linear, Complete, Systematic, Orthogonal, Consistently Closed, & Convex U1 U2 V1 V2
D11
D12
D21
D22
ref boris beizer

32. Domain Testing - Domains & Paths – Nice domainsU3
Nice Domains
Advantages
Ease of DT
Fewer Bug Occurrences
ref boris beizer

33. Domain Testing - Domains & Paths – Nice domainsU3
Nice Domains
Boundaries are
Linear
Complete
Systematic
Orthogonal
Closure consistency
Convex
Simply connected
ref boris beizer

34. Domain Testing - Domains & Paths – Nice domainsU3
Linear Boundaries
Interpreted linear inequalities
n-dim Hyperplane:
n+1 Points
n Test Cases
Non-Linear
Transform
ref boris beizer

35. Domain Testing - Domains & Paths – Nice domainsU3
Complete Boundaries
Span the total number space (-, +)
One set of Tests
Incomplete…
Reasons
Tests
ref boris beizer

36. Domain Testing - Domains & Paths - Nice domainsU3
Systematic Boundaries
Linear Inequalities differing by a constant
fj (X) ≥ kj or, fj (X) ≥ g (j, c) g (j, c) = j + k * c
Parallel lines
Identical Sectors in a Circle DT
Test a domain  Tests for other Domains
ref boris beizer

37. Domain Testing - Domains & Paths - Nice domainsU3
Orthogonal Boundaries
Two boundaries or, boundary sets
Parallel to axes DT
Each Set Independently
# Tests  O (n) Uj Vj
ref boris beizer

38. Domain Testing - Domains & Paths - Nice domainsU3
Orthogonal Boundaries
Tilted sets
transformation
Test cases: O(n)
Concentric circles with radial lines
Rectangular coordinates
Polar
r ≥ aj .AND. r < aj+1 .AND.
≥ j .AND.  < j+1
ref boris beizer

39. Domain Testing - Domains & Paths - Nice domainsU3
Non-Orthogonal Boundaries
Test Intersections  O ( n2 ) + O (n)
ref boris beizer

40. Domain Testing - Domains & Paths - Nice domainsU3
Closure Consistency
A Simple pattern in all boundary closures
Example
Same relational operator for systematic boundaries D3 D1 D2 D4
ref boris beizer

41. Domain Testing - Domains & Paths - Nice domainsU3
Convex Domain
Line joining any two points lies with in the domain DT
n on-points & 1 off-pt
Concave
“ But, However, Except, Or … “ in Specs
Handle with special care
ref boris beizer

42. Domain Testing - Domains & Paths - Nice domainsU3
Simply Connected Domain
In a single piece
2 complementary domains
D1: Convex  D2: Concave, not-Connected
Programmer / Designer
Convex part First D2 D1 D3 D1 D2
ref boris beizer

43. Domain Testing - Domains & Paths – Ugly Domains
Generally,
From Bad Specs
Programmer / Designer Simplifies => Ugly to Good
possibility of Introduction of more bugs ?!?
ref boris beizer

44. Domain Testing - Domains & Paths – Ugly Domains
Causes
Non-linear Boundaries
Ambiguities & Contradictions
Simplifying the topology
Rectifying boundary closures
ref boris beizer

45. Domain Testing - Domains & Paths – Ugly Domains
Non-linear Boundaries
Transform
ref boris beizer

46. Domain Testing - Domains & Paths – Ugly Domains
Ambiguities
Holes in input vector space.
Missing boundary
Detected by Specification languages & tools.
ref boris beizer

47. Domain Testing - Domains & Paths – Ugly Domains
Contradictions..
Overlapping of
Domain Specs
Closure Specs D3 D1 D2
ref boris beizer

48. Domain Testing - Domains & Paths – Ugly Domains
Simplifying the Topology….
Complexity !
Concavity, Holes, Disconnectedness
ref boris beizer

49. Domain Testing - Domains & Paths – Ugly Domains
Simplifying the Topology….
Smoothing out concavity
Filling in Holes
ref boris beizer

50. Domain Testing - Domains & Paths – Ugly Domains
Simplifying the Topology….
Joining the pieces
Correct:
Connect disconnected boundary segments
Extend boundaries to infinity
ref boris beizer

51. Domain Testing - Domains & Paths – Ugly Domains
Rectifying Boundary Closures.
make closures in one direction for parallel boundaries with closures in both directions
Force a Bounding Hyperplane to belong to the Domain.
Consistent Direction
Inclusion / Exclusion Consistency
ref boris beizer

52. Domain Testing - Domains & Paths – Domain TestingU3
General DT Strategy
Select test points near the boundaries.
Define test strategy for each possible bug related to boundary
Test points for a domain useful to test its adjacent domain.
Run the tests. By post test analysis determine if any boundaries are faulty & if so how?
Run enough tests to verify every boundary of every domain
ref boris beizer

53. Domain Testing - Domains & Paths – Domain TestingU3
DT for Specific Domain Bugs
Generally,
Interior point
Exterior point
Epsilon neighborhood
Extreme point
On point
ref boris beizer

54. Domain Testing - Domains & Paths – Domain TestingU3
DT for Specific Domain Bugs
Off Points
Extreme
point
Boundary point
Epsilon neighborhood
On Points
Domain D1
ref boris beizer

55. Domain Testing - Domains & Paths – Domain TestingU3
1-d Domains
2-d Domains
Equality & inequality Predicates
Random Testing
Testing n-dimensional Domains
ref boris beizer

56. Domain Testing - Domains & Paths – Domain TestingU3
Testing 1-d Domains : Bugs with open boundaries A B x A B x
Closure Bug A B x x1
Shift left Bug A B x x1
Shift Right Bug
ref boris beizer

57. Domain Testing - Domains & Paths – Domain TestingU3
Testing 1-d Domains : Bugs with open boundaries A B x A B x
Missing Boundary
Extra Boundary A B x1 x C
Bugs with Closed Boundaries : Similar to the above
ref boris beizer

58. Domain Testing - Domains & Paths – Domain TestingU3
Testing 2-d Domains
Closure bug
Boundary Shift : up / down
Tilted Boundary
Extra Boundary
Missing Boundary
ref boris beizer

59. Domain Testing - Domains & Paths – Domain TestingU3
Domain Testing - Domains & Paths – Domain Testing
2 n : Domains share tests
3 n : no sharing of tests by domains
Strategy for domain testing in 2-dim
ref boris beizer

60. Domain Testing - Domains & Paths – Domain TestingU3
Equality & Inequality Predicates
An Equality predicate defines a line in 2-d a A c’ c c d B b
To avoid bugs
ref boris beizer

61. Domain Testing - Domains & Paths – Domain TestingU3
Random Testing
A Point in the center : verifies computation
ref boris beizer

62. Domain Testing - Domains & Paths – Domain TestingU3
Testing n-Dimensional Domains (strategy)
n-dimensions, p boundary segments
(n+1)*p test cases : n on points & 1 off point
Extreme pt shared : 2 * p points
Equalities over m-dimensions create a subspace of n-m dimensions
Orthogonal domains with consistent boundary closures, orthogonal to the axes & complete boundaries
Independent testing
ref boris beizer

63. Domain Testing - Domains & Paths – Domain TestingU3
Procedure for solving for values
Simple procedure. Need tools.
Identify input variables
Identify variables which appear in domain-defining predicates, such as control-flow predicates
ref boris beizer

64. Domain Testing - Domains & Paths – Domain TestingU3
Procedure
Interpret all domain predicates in terms of input variables:
Transform non-linear to linear
Find data flow path
Predicate expression with p # predicates.
Find # domains : < 2 p
Solve inequalities for extreme points
Use extreme points to solve for nearby on points …
ref boris beizer

65. Domain Testing - Domains & Paths – Domain TestingU3
Effectiveness
Cost effective
Bugs on boundaries, extreme points
Hardware logic testing – tool intensive
ref boris beizer

66. Domain Testing - Domains & Paths – Domain TestingU3
Domains & Interface Testing
Domain
Range
Range
Variable
Domain
Classify
Function /
Routine
ref boris beizer

67. Domain Testing - Domains & Paths – Domain TestingU3
Domains & Interface Testing
Span compatibility
Routine 2
Routine 1
Domain
Variable
Range for
Routine2
Domain for
Routine1
ref boris beizer

68. Domain Testing - Domains & Paths – Domain TestingU3
Domains & Interface Testing
Closure compatibility
ref boris beizer

69. Domain Testing - Domains & Paths – Domain TestingU3
Interface Range/Domain Compatibility Testing
Test each var independently
Find an inverse function
Build a super domain
ref boris beizer

70. Domain Testing - Domains & Paths – Domain TestingU3
Finding the values / solving inequalities
ref boris beizer

71. Domain Testing - Domains & Paths – Domain TestingU3
Domains and Testability
Linearizing transformations
Polynomial
Rectangular to polar
Generic & rational => Taylor series
ref boris beizer

72. Domain Testing - Domains & Paths – Domain TestingU3
Domains and Testability
Perform transformations for better testing.
Co-ordinate transformations
ref boris beizer

73. Domain Testing - Domains & Paths – Domain TestingU3
Domains and Testability
Canonical Program Form
ref boris beizer

74. Domain Testing – Questions from the previous year’s exams
Explain Nice & Ugly Domains.
What is domain testing? Discuss applications of domain testing
Explain the domain boundary bugs for two dimensional domains
What is the purpose of Domain Testing? Give its schematic representation
Define the following concepts.
Domains 2. Domain closure Domain dimensionality Bug assumptions for Domain Testing
Explain simple domain boundaries & compound predicates.
Classify what can go wrong with boundaries, then define a test strategy for each case in domain testing.
ref boris beizer

75. Domain Testing To Unit 5…Paths, Path Products & Regular Expressions
ref boris beizer