1. COSC 4506/ITEC 3506 Software Engineering
Classic Testing (2)

2. Outline
Testing in the small
Black box testing

3. Black-box testing
Black-box testing is testing without having an insight into the details of the underlying code.
Dynamic, because the program is running
Black-box, because testing is done without knowledge of how the program is implemented.
Sometimes referred to as behavioral testing.
Document required?
Test
Test case t

4. Black-box testing Characteristics of Black-box testing:
Program is treated as a black box.
Implementation details do not matter.
Requires an end-user perspective.
Criteria are not precise.
Test planning can begin early. (why)

5. Black-box testing
Why do we need it? ?

6. Black-box testing

7. Equivalence Partitioning
Equivalence partitioning is the process of methodically reducing the huge (or infinite) set of possible test cases into a small, but equally effective, set of test cases.

8. Equivalence Partitioning
An equivalence class is a set of test cases that test the same thing or reveal the same bug
Calculator: 1+1, 1+2, 1+3..
Equivalence partition is to divide the input domain into several equivalence classes, from which the test cases can be derived

9. Equivalence Partitioning
An equivalence class can be defined with following guidelines
If an input condition specifies a range, one valid and two invalid equivalence classes are defined
If an input condition requires a specific value, one value and two invalid equivalence classes are defined
If an input condition specifies a number of a set, one value and one invalid equivalence classes are defined
If an input condition is Boolean, one value and one invalid equivalence classes are defined

10. Example
With automated banking application, the user can access the bank using a PC by providing phone-number as ID, 6-digit password, following with a series of type Commands that trigger various banking functions. During log-on sequence, s/w accepts data in form:
Area-code: blank or three-digit number
Prefix: three digit number, not beginning with 0 or 1
Suffix: four digit number
Password: six alphanumic string
Command: check, deposit, bill pay, and the like

11. Example Area-code: Prefix Postfix
Input condition: boolean (the area code may or may not be present)
Input condition: range (value defined between with some exception)
Prefix
Input condition: range (>200)
Postfix
Input condition: value – four digit length

12. Example Password Input condition: boolean (may or may not be present)
Input condition: value (six character string)
Command
Input condition: set (containing commands noticed previously)

13. Example Test Part Input Conditions Test cases Area code Boolean,
Blank, 245
range
158, 245, 1001
Prefix
123, 445, 1002
Postfix
value
123, 3456, 23456
Password
boolean
Blank, 12390h
1234, 12390h,
Command
set
Check, doing

14. Equivalence Partitioning (or Equivalence Class Testing)
The use of equivalence class testing has two motivations:
Sense of complete testing
Avoid redundancy
Equivalence classes form a partition of a set that is a collection of mutially disjoint subsets whose union is the entire set.
Two important implications for testing:
The fact that the entire set is represented provides a form of completeness
The disjointness assures a form of non-redundency

15. Equivalence Classes
The idea of equivalence class testing is to identify test cases by using one element from each equivalence class.
The key point in equivalence class testing is the choice of the equivalence relation that determines the classes (partitions).
If the equivalence classes are chosen wisely this greatly reduces the potential redundancy among test cases.
We will differentiate below, between weak and strong equivalence class testing.

16. Example
Let us consider a program P with 3 inputs: a, b and c and the corresponding input domains are A, B, and C.
let there be defined the partition:

17. define a1, a2 and a3 as:
let ai be a “representative” or “typical” value within its respective equivalence class (e.g. the midpoint in a linear equivalence class).
similarly define bi and ci.
test cases can be stated for the inputs <a,b,c> in terms of the representative points.
the basic idea behind the techniques is that one point within an equivalence class is just as good as any other point within the same class.(?)

18. Weak Equivalence Class Testing
Weak equivalence class testing is accomplished by using one variable from each equivalence class in a test case.
From the previous example, we have:
Test Case
Variable a
Variable b
Variable c
WE1 a1 b1 c1
WE2 a2 b2 c2
WE3 a3 b3
WE4 b4

19. Strong Equivalence Class Testing
Strong equivalence class testing is based on the Cartesian Product of the partition subsets.
From the previous example, this would generate:
3 * 4 * 2 = 24 test cases
Generates more test cases which test for any interaction between the representative values from each of the subsets.
For either method, it may be possible to define equivalence relations for the program output, then test cases can also be based on these.

20. Test Case # a b c SE1 a1 b1 c1 SE2 c2 SE3 b2 SE4 C2 SE5 b3 C1 SE6 SE7

21. Traditional View
The traditional view of equivalence class testing defines equivalence in terms of validity that is, test cases determined from the valid and invalid values for each input variable.
For each input variable there are valid and invalid values.
In the traditional approach, these are identified and numbered, and then incorporated into test cases in the weak sense as presented above.

22. Traditional View
Given valid and invalid sets of inputs, the traditional equivalence testing strategy identifies test cases as follows:
For valid inputs, use one value from each valid class (as in what we have called weak equivalence class testing). In this context, each input in these test cases will be valid.
For invalid inputs, a test case will have one invalid value and the remaining values will be valid. In this context, a “single failure” should cause the test case to fail.
If the input variables have defined ranges, then the test cases from traditional equivalence class testing will always be a subset of those that would be generated by robustness testing.

23. Example
For example consider a program with two input variables size and weight:
valid ranges:
S1: 0 < size < 200
W1: 0 < weight < 1500
corresponding invalid ranges might be:

24. Test Cases Example (Traditional View)
size
weight
Expected Output
TE1
100
750
what ever it should be
TE2 -1
invalid input
TE3
1500
TE4
TE5
200

25. Equivalence Test Cases for the NextDate Problem (Input Domain)
Nextdate is a function of three variables, month, day, and year and these have ranges defined as:
1 ≤ month ≤ 12
1 ≤ day ≤ 31
1812 ≤ year ≤ 2012
We will examine below the valid, invalid equivalence classes, strong, and weak equivalence class testing.

26. Traditional Test Cases
The valid equivalence classes are:
M1= {month | 1 ≤ month ≤ 12}
D1= {day | 1 ≤ day ≤ 31}
Y1= {year | 1812 ≤ year ≤ 2012}
The invalid equivalence classes are:
M2= {month | month < 1}
M3= {month | month > 12}
D2= {day | day < 1}
D3= {day | day > 31}
Y2= {year | year < 1812}
Y3= {year | year > 2012}
These classes yield the following test cases, where the valid inputs are mechanically selected from the approximate middle of the valid range:

27. Traditional Test Cases
Case ID
Month
Day
Year
Expected
Output
TE1 6 15
1912
6/16/1912
TE2 -1
Invalid
TE3 13
TE4
TE5 32
TE6
1811
TE7
2013

28. Choice of Equivalence Classes
If we more carefully chose the equivalence relation, the resulting equivalence classes will be more useful
M1= {month | month has 30 days}
M2= {month | month has 31 days}
M3= {month | month is February}
D1= {day | 1 ≤ day ≤ 28}
D2= {day | day = 29}
D3= {day | day = 30}
D4= {day | day=31}
Y1= {year | year = 1900}
Y2= {year | 1812 ≤ year ≤ 2012 AND year ≠ 1900 AND (year mod 4 = 0 }
Y3= {year | 1812 ≤ year ≤ 2021 AND year mod 4 ≠ 0 }

29. Weak Equivalence Class Test Cases
Case ID
Month
Day
Year
Expected Output
WE1 6 14
1900
6/15/1900
WE2 7 29
1912
7/30/1912
WE3 2 30
1913
Invalid
WE4 31

30. Strong Equivalence Test Cases
CASE ID
Month
Day
Year
Output
SE1 6 14
1900
6/15/1900
SE2
1912
6/15/1912
SE3
1913
6/15/1913
SE4 29
6/30/1900
SE5
6/30/1912
SE6
6/30/1913
SE7 30
7/1/1900
SE8
7/1/1912
SE9
7/1/1913
SE10 31
ERROR
SE11
SE12
SE13 7
7/15/1900
SE14
7/15/1912
SE15
7/15/1913
SE16
7/30/1900
SE17
7/30/1912
SE18
7/30/1913

31. Strong Equivalence Test Classes
CASE ID
Month
Day
Year
Output
SE19 7 30
1900
7/31/1900
SE20
1912
7/31/1912
SE21
1913
7/31/1913
SE22 31
8/1/1900
SE23
8/1/1912
SE24
8/1/1913
SE25 2 14
2/15/1900
SE26
2/15/1912
SE27
2/15/1913
SE28 29
ERROR
SE29
3/1/1912
SE30
SE31
SE132
SE33
SE34
SE35
SE36

32. Guidelines and Considerations
Equivalence class testing is appropriate when input data is defined in terms of ranges and sets of discrete values.
Strong equivalence takes the presumption that variables are independent, otherwise it generates some “error” test cases
Can be strengthened by using it with domain testing (boundary value) – reuse the work to define the ranges.
The traditional form of equivalence testing is generally not as thorough as weak equivalence testing, and in its turn, not as thorough as strong equivalence testing
If error conditions is a priority we can extend strong equivalence testing to include invalid classes

33. Boundary Value Analysis
Boundary value analysis focuses on the boundary of the input space to identify test cases.
a test case design technique
complements to equivalence partition
The rationale behind boundary value analysis is that errors tend to occur near the extreme values of an input variable.

34. Boundary Value Analysis
In our discussion we will assume a program P accepting two inputs x1 andx2 such that a ≤ y1 ≤ b and c ≤ y2 ≤ d

35. Valid Input for Program P
consider the following function:
boundary inequalities of n input variables define an n-dimensional input space:

36. Boundary Value Analysis
Guidelines:
1. If an input condition specifies a range bounded by values a and b,
test cases should be designed with value a and b, just above and below a and b.
Example: Integer D with input condition [-3, 10],
test values: -3, 10, 11, -2, 0
2. If an input condition specifies a number values, test cases should be developed to exercise the minimum and maximum numbers. Values just above and below minimum and maximum are also tested.
Example: Enumerate data E with input condition: {3, 5, 100, 102}
test values: 3, 102, -1, 200, 5
The basic idea in boundary value analysis is to select input variable values at their:
Minimum
Just above the minimum
A nominal value
Just below the maximum
Maximum

37. Boundary Value Analysis
3. Apply the guideline 1 and 2 to output condition
assume a temperature and pressure table is required as output from a program, test case should be designed to create an output that produce the max (min) allowable number of table entries
4. If internal program data structures have prescribed boundaries, be certain to design a test case to exercise the data structure at its boundary
Such as data structures:
- array input condition:
empty, single element, full element, out-of-boundary
search for element:
- element is inside array or the element is not inside array
You can think about other data structures:
- list, set, stack, queue, and tree

38. Boundary Value Analysis for Program P. . . .
T = { <y1nom, y2min>, <y1nom, y2min+>, <y1nom, y2nom>, <y1nom, y2max->,
<y1nom, y2max+>, <y1min, y2nom>, < 1nin+, y2nom>, <y1max-, y2nom>,
<y1max, y2nom> }

39. Limitations of Boundary Value Analysis
Boundary value analysis works well when the program to be tested is a function of several independent variables that represent bounded physical quantities.
Boundary value analysis selected test data with no consideration of the function of the program, nor of the semantic meaning of the variables.
The basic boundary value analysis can be generalized in two ways:
By the number of variables - (4n +1) test cases for n variables
By the kinds of ranges of variables
Programming language dependent
Bounded discrete
Unbounded discrete (no upper or lower bounds clearly defined)
Logical variables

40. Decision table-based Testing (DTT)
Applicable to the software requirements written using “if-then” statements
Can be automatically translated into code
Assume the independence of inputs
Example
If c1 AND c2 OR c3 then A1

41. DTT
Decision tables, like if-then-else and switch-case statements, associate conditions with actions to perform
A decision table lists causes and effects in a matrix. Each column represents a unique combination.
Purpose is to structure logic
Cause = condition Effect = action = expected results

42. DTT
Example

43. Step 1: List all causes Hints:
Write down the values the cause/condition can assume
Cluster related causes
Put the most dominating cause first

44. Step 2: Calculate combinations
If all causes are simply Y/N values: 2number of causes
If 1 cause with 3 values and 3 with 2: 31 * 23 = 24
Or, use the Values column and multiply each value down the column, eg. 3*2*2*2=24
Number of Values to the power of the number of causes with these values

45. Step 3: Fill columns Algorithm:
Determine Repeating Factor (RF): divide remaining combinations by the number of possible values for that cause
Write RF times the first value, then RF times the next etc. until row is full
Next row, go to 1.

46. Step 4: Reduce combinations
Find indifferent combinations – place a ‘-’
Often there are dependencies between causes. For example cause 1 can test if a field is entered (Y/N) and cause 2 can test if the same field is valid (Y/N). It is no use testing if the field is valid if it has not been entered.
Join columns: When in a column a cause is of no interest any more to determine the effects, a dash ‘-’ is placed in the column for that cause. This has to be done for both the column with a ‘Y’ and the column with a ‘N’. This makes the 2 columns identical and one of them can be deleted.
Join columns where columns are identical
Tip: ensure the effects are the same

47. Step 5: Check covered combinations
Checksum
For each column calculate the combinations it represents
A ‘-’ represents as many combinations as the cause has
Multiply for each ‘-’ down the column
Add up total and compare with step 2

48. Step 6: Add effects to table
Read column by column and determine the effects
One effect can occur in multiple test combinations

49. Exercise: Specification Create a decision table
A mailing is to be sent out to customers. The content of the mailing is about the current level of discounting and potential levels of discounting. The content is different for different types of customers.
Customer Types A, B and C get a normal letter except Customer Type C, who get a special letter. Any customer with 2 or more current lines or with a credit rating of ‘X’ get a special paragraph added with an offer to subscribe to another level of discounting.
Ask questions from the analyst for clarification

50. Exercise: possible solution
“2 or more current lines OR credit rating X”. What if both: AND?
Other customer types? See “O-Other” above.
What about non current lines?

51. Decision tables as a basis for test case design
The use of decision-table model to design test cases is applicable
The spec is given by table or is easily converted to one
The order in which the conditions are evaluated does not affect the interpretation of rules or the resulting action
The order in which the rules are evaluated does not affect the resulting action
Once a rule has been satisfied and an action selected, no other rule need be examined

52. Summery: Black Box Testing
a.k.a. behavioral, functional, closed
Test against the specification
does not need the program source
internal implementation is unknown

53. Black Box Testing Advantages: user's point of view less biased
can discover if part of the specification has not been fulfilled.
Do not need to know specific language
Disadvantages:
may be redundant
can be difficult to design.
testing every possible input stream is unrealistic
Advantages:
The test is unbiased because the designer and the tester are independent of each other.
The tester does not need knowledge of any specific programming languages.
The test is done from the point of view of the user, not the designer.
Test cases can be designed as soon as the specifications are complete.