1. 841f07frankl12oct21 Evaluating Testing Methods by Delivered Reliability Frankl, Hamlet, Littlewood, Strigini IEEE TOSE Aug98

2. 841fall06 exam1 question 1 (25 pts) Calculations – given the following table, estimate the E(Q) using the MFR est and the subdomain formula. Assume that subdomains a and c are twice as likely as the other subdomains. You can simplify the table by projections” on each axis. Show your work. State and justify any assumptions that you must make. Show your work. State and justify any assumptions that you must make. subdomainF1F2F3Domain Size a100 50 b5102150 c150 50 d04210 e04420 f010020

4. 841fall06 exam1 question 2a 2. (30 pts) Modeling Assume that boundary testing randomly picks 2 test cases from each subset that makes the two sides in a relational expression equal (e.g. if the decision was “x < y”, the set would be those points where the value of x was equal to the value of y.} Boundary testing for the whole program will do this for each relational expression. Use Frankl’s formulas to estimate the E(Q) for this approach.

5. 841f06 exam1 question 2b cin >> a >> b; out = “X”; if (a > b + 1) out = “Y”; if (a < 2*b ) out = “Z”; cout<< out; Seed two faults: Change “b+1” to “b+2” Change “2*b” to “b” Consider the following program. Assume the operation profile is uniform and consists of all pairs of integers between 1 and 5. E.g. (1,1), (1,2), etc

6. 841f06 exam1 question 3 3. (30 pts) Analysis Consider the following code. Find the largest set of mutually exclusive subdomains that might be useful for thorough testing of the code. Justify why they would be useful. Express the sets with relational conditions on a and b and draw a 2D map of the subdomains. Consider these three faults: 1) change “b+3” to “b+2” ; 2) change “a b” ; 3) change “a+2” to “a+3” Can these subdomains reveal these faults? cin >> a >> b; out = “X”; if (a > b + 3) out = “Y”; a = a + 2; if (a < b ) out = “Z”; cout<< out;

7. 841f06 exam1 question 4 4. (15 pts) Discussion Suppose that your testing effort is restricted to n tests and you have identified 2*n important subdomains in the product. How do you decide which subdomains to test? Can you use seeded faults to help select? What faults would you seed?

8. 841f07frankl12oct28 Empirical Tool – version 3

9. 841f07frankl12oct29 Triangle Example cin >> a >> b >> c ; type = “scalene”; if (a == b || a == c || b == c) type= “isosceles”; if (a == b && a == c) type = “equilateral”; if (a >= b+c || b >= a+c || c >= a+b) type=“not a triangle”; if (a <= 0 || b <= 0 || c <= 0) type=“bad input”; cout<< type;

10. Triangle Domain all combinations of integers 0 – 5 Faults flt 1 - line 3: change a==b to a==a flt 2 – line 4: change “equilateral” to “isosceles” flt 3 – line 5: change a >= to a> flt 4 – line 5: change a+b to b+b

11. 841f07frankl12oct211 Types of testing u What subdomains? u Boundary testing?

12. 841f07frankl12oct212 For Tuesday, Oct 2 u Failure sets for triangle and faults 1-4

13. 841f07frankl12oct213 Empirical Tool u For Tues, Oct 9 u Use your empirical tool to calculate E(theta) for subdomain testing of the triangle problem with the given faults shown in lecture 11 and 12. u Compare with Frankl’s formulas. Make the comparision as fair as possible. Choose the number of tests and subdomains carefully u Turn in hardcopy at start of class, Tues 10/9