Data Analysis for Testing In order to test a program or system, it usually requires : –Starting the system (program) –Inputting some data –Let it execute –Observe the results (outputs and state) What data do we “input?” –Note that inputting data is not limited to keying in information. It includes clicking on a “button’ or making a choice from a “drop down” list Clicking on a button is a binary event – clicked or not clicked If there are n buttons on a screen, then we have 2 n permutations of “inputs” that need to be considered.

Consider an Example Key in # of people (max. 12): Cruise Meal Reservations Breakfast LunchDinner For this meal reservation, we have 4 input fields. - key in number of people - 3 buttons to click for the meal or meals What type and how many “inputs should we consider for the test ?

Example (cont.) Breakfast button Dinner button Lunch button True False X X X X X X X X X X X X X X X X X X X X X X X X -First, the numerical input field allows “integer” inputs. -Second, the integer must not be greater than 12 -Third, there are 2 3 = 8 permutations for the 3 meal buttons These 8 combinations equates to 8 input test cases.

Numerical Field Test (from example) The “illegal” input test for numerical field. In this case “number of people” has to be positive integers. Input tests should include: –Non-integer (decimal number) –Negative integer –non-numeric (characters) The “legal” integer is “upper” bounded by 12: –Integer 12 (inclusive boundary) –Integer 13 (one over the boundary) –Integer 11 (one inside the boundary) Is there a “lower” bound? not clear –Assume integer 1 (inclusive boundary) –0 (one outside the boundary) –2 (inside the boundary) The different inputs considered for the numerical field equates to 9 test cases.

More on the Example Should we just add up the two separate sets of test cases and get 17 input tests? –8 from the 3 buttons –9 from the numerical field We can probably reduce some from the numerical field tests: –Just do one inside the boundary instead of both sides of the boundary. –One outside of the lower bound instead of 0 and a negative number. Do we need to ask if there is any relationship among the input fields? –Should we ensure that the combination of legitimate numerical input (e.g. 8) is combined with all three buttons not clicked to test the inter-relationship? –What type of error message, if any, should be issued if the user keys in 0 people and no buttons clicked? –Should we worry about some other possible relationship that we did not consider and thus perform 8 X 9 = 72 test cases?

1-Dimensional Boundary Value Analysis Our previous example of numerical field had an upper and a lower bound of 12 and 1 respectively. There are two way to consider the boundary value analysis. 1.Legitimate to illegitimate includes : a) 2 (legitimate-inside boundary) b) 1 (legitimate – on boundary) c) 0 (illegitimate – outside the boundary) d) 11 (legitimate –inside the boundary) e) 12 (legitimate – on boundary) f) 13 (illegitimate – outside the boundary) legitimate 2. Illegitimate to legitimate includes : a) -1 (illegitimate-inside boundary) b) 0 (illegitimate – on boundary) c) 1 (legitimate – outside the boundary) d) 14 (illegitimate – intside the boundary) e) 13 (illegitimate – on boundary) f) 12 (legitimate – outside the boundary) illegitimate May choose to reduce to the set with 5 inputs { 0, 1, 2, 12, 13} May choose to reduce to the set with 5 inputs {-1, 0, 1, 13, 14 } or with only 4 inputs { 0, 1, 12, 13} uncomfortable ?

Multi-Dimensional Boundary Value Analysis (Case 1 – Partially Dependent) Book check-out Date; Book check-in Date: - Assume that for these two fields we have performed the boundary analysis and estimated the test cases: - check-out date : n1 input test cases - check-in date : n2 input test cases -Then the total test cases will be (n1 + n2). - But is there any inter-relationship? - Yes, book checkout date must be before or same as book check-in (return) date. - ensure that that the 3 cases of =, check-out > check-in, and check-out< check-in are in the ( n1+n2 ) input tests or one needs to add these.

Multi-Dimensional Boundary Value Analysis (Case 2 – Dependent) Flight Time : Flight Date : Consider the situation where a discount is given to flights between: - 6 PM to 12AM (inclusive) and - 6/1/2005 through 12/31/2005 (inclusive) Aside from the individual field level 1-dimensional boundary value analysis, we need to take into account the “AND” logic for getting a discount. Individually, this logic requires: - {5:59pm, 6:00pm, 6:01pm, 11:59PM, 12:00AM, 12:01AM} for Flight Time - {5/31/2005, 6/1/2005, 6/2/2005, 12/30/2005, 12/31/2005, 1/1/2006} for Flight Dates ( note that we have reduced the one duplication inside the boundary )

Multi-Dimensional Boundary Value Analysis (Case 2 – Dependent) – cont. Because of the ‘AND’ logic for the time and dates, we would consider the (5 “ time” inputs X 5 “date” inputs) or a total of 25 combinations. 5:59 PM 6:00 PM 6:01 PM 12:00 AM 12:01 AM 5/31/05 6/01/05 6/02/05 12/31/05 1/01/06 (ill, ill) (ill, leg) (ill, ill) (leg, ill) (ill, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (ill, leg) (ill, ill) the entries are in the form : (date, time) where - ill= illegitimate - leg = legitimate

Multi-Dimensional Boundary Value Analysis (Case 2 – Dependent) – cont. 5:59 PM 6:00 PM 6:01 PM 12:00 AM 12:01 AM 5/31/05 6/01/05 6/02/05 12/31/05 1/01/06 (ill, ill) (ill, leg) (ill, ill) (leg, ill) (ill, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (ill, leg) (ill, ill) Do we need to run all 25 combinations ---- can we reduce the test cases? Cover all the borders scheme with a very limited saving - 24 test cases: (ill,ill)  (leg,leg); (ill, leg)  (leg,leg); (leg,ill)  (leg,leg)

Multi-Dimensional Boundary Value Analysis (Case 2 – Dependent) – cont. 5:59 PM 6:00 PM 6:01 PM 12:00 AM 12:01 AM 5/31/05 6/01/05 6/02/05 12/31/05 1/01/06 (ill, ill) (ill, leg) (ill, ill) (leg, ill) (ill, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (ill, leg) (ill, ill) Do we need to run all 25 combinations ---- can we reduce the test cases? Cover some of the borders scheme with more savings – 12 test cases: (ill,ill)  (leg,leg); (ill, leg)  (leg,leg); (leg,ill)  (leg,leg)

Multi-Dimensional Boundary Value Analysis (Case 2 – Dependent) – cont. 5:59 PM 6:00 PM 6:01 PM 12:00 AM 12:01 AM 5/31/05 6/01/05 6/02/05 12/31/05 1/01/06 (ill, ill) (ill, leg) (ill, ill) (leg, ill) (ill, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (ill, leg) (ill, ill) Do we need to run all 25 combinations ---- can we reduce the test cases more? Try Partitioned Data sets of (ill, ill); (ill, leg); (leg,ill); (leg,leg) – 4 test cases: ( NOT a good choice to use partitioned data sets on this 2-dimensional boundary value analysis )

Multi-Dimensional Boundary Value Analysis We have seen in our example of 2 inter-related data fields, each having 5 potential inputs from boundary value analysis, would result in 5 x 5 = 25 test cases (even though the number may be reduced with risk) In general, if we have Z inter-related data fields, each with n test cases from individual boundary value analysis, then we would have: – n x n x x n (Z times) or – n Z test cases if n= 5 and z = 3, that would mean 5 3 = 125 test cases If we have three independent screens like this it would mean 3 x 125 = 375 test cases ! If the 3 screens were inter-dependent, then we could have as much as 125 x 125 x 125 = 1,953,125 test cases (imagine that!)