2. CS 325: Software Engineering March 31, 2015 Software Testing Black-Box Testing White-Box Testing Regression Testing

3. CS 325 March 31, 2015 Page 203 Black-Box Testing Recall that black-box testing is used to determine whether a software system meets its functional specifications. Black-box testing comes in multiple flavors: Equivalence Partitioning Divide the input and output domains into disjoint subsets and test one selection from each subset Boundary Value Analysis Set up the Equivalence Partitioning subsets; test selections at and near their boundaries Cause-Effect Analysis Map the different combinations of system input values to their corresponding resultant output values

4. CS 325 March 31, 2015 Page 204 Black-Box Testing: Equivalence Partitioning For the function below, equivalence partitioning would involve three input pairs: one producing underflow, one producing uinderflow, and one producing neither. int safe_add( int a, int b ) { int c = a + b; int c = a + b; if ( (a >= 0) && (b >= 0) && (c = 0) && (b >= 0) && (c < 0) ) cout << "ERROR: Overflow!\n" ); cout << "ERROR: Overflow!\n" ); else if ( (a = 0) ) else if ( (a = 0) ) cout << "ERROR: Underflow!\n" ); cout << "ERROR: Underflow!\n" ); return c; return c;}

5. CS 325 March 31, 2015 Page 205 Black-Box Testing: Boundary-Value Analysis For the function below, boundary values would be where a+b=INT_MAX and where a+b=INT_MIN, while values just beyond the boundaries would be where a+b=IKNT_MAX+1 an d where a+b=INT_MIN-1. int safe_add( int a, int b ) { int c = a + b; int c = a + b; if ( (a >= 0) && (b >= 0) && (c = 0) && (b >= 0) && (c < 0) ) cout << "ERROR: Overflow!\n" ); cout << "ERROR: Overflow!\n" ); else if ( (a = 0) ) else if ( (a = 0) ) cout << "ERROR: Underflow!\n" ); cout << "ERROR: Underflow!\n" ); return c; return c;}

6. CS 325 March 31, 2015 Page 206 Black-Box Testing: Cause-Effect Analysis Consider a loan application software system, where users can enter the amount of the monthly repayment or the number of years they want to take to pay back the loan (i.e., the term of the loan). If a user enters both, the system will make a compromise between the two if they conflict. The two conditions are the loan amount and the term, which are put in a table, along with their different combinations of possible values. Conditions Rule 1 Rule 2 Rule 3 Rule 4 Repayment amount has been entered TTFF Term of loan has been entered TFTFConditions Rule 1 Rule 2 Rule 3 Rule 4 Repayment amount has been entered TTFF Term of loan has been entered TFTF Outcome: Process loan amount YY Outcome: Process term YY Outcome Error message Y Test cases would then be developed to test all four of the cause-effect rules.

7. CS 325 March 31, 2015 Page 207 Black-Box Testing: Cause-Effect Analysis (continued) Consider a credit card system that offers discounts. Conditions Rule 1 Rule 2 Rule 3 Rule 4 Rule 5 Rule 6 Rule 7 Rule 8 New customer (15%) TTTTFFFF Loyalty card (10%) TTFFTTFF Coupon (20%) TFTFTFTF Note that Rules 1 and 2 represent impossible cases (new customers with loyalty cards). Also, Rule 3 assumes that customers always select the larger discount (when they have to select). New customers who want to open a credit card account will get a 15% discount on the day they open their accounts. Existing customers who hold a loyalty card, always get a 10% discount. Finally, customers who have a coupon, can get 20% off today (but this coupon discount can’t be used with the ‘new customer’ discount). Discount amounts are added, if applicable.Conditions Rule 1 Rule 2 Rule 3 Rule 4 Rule 5 Rule 6 Rule 7 Rule 8 New customer (15%) TTTTFFFF Loyalty card (10%) TTFFTTFF Coupon (20%) TFTFTFTF Effect: Discount NANA20%15%30%10%20%0%

8. CS 325 March 31, 2015 Page 208 White-Box Testing White-box testing derives test cases via knowledge of the source code itself. Essentially, the minimum number of test cases needed for white-box testing is the number of distinct paths from the start to the finish of the code (with loops being traversed no more than once). This number is called the cyclomatic complexity of the program.

9. CS 325 March 31, 2015 Page 209 White-Box Testing: Cyclomatic Complexity Consider the various paths to implement the following rule in the game Monopoly. If a player lands on a property owned by an opponent, the player needs to pay the rent. If the player doesn’t have enough money, the player is out of the game. If the property is not owned, and the player has enough money, the player may buy the property for the price associated with the property.

10. CS 325 March 31, 2015 Page 210 White-Box Testing: Cyclomatic Complexity (continued) There are six independent paths through this code: 1. 1-2-3-4-5-10 (property owned by others, no money for rent) 2. 1-2-3-4-6-10 (property owned by others, pay rent) 3. 1-2-3-10 (property owned by the player) 4. 1-2-7-10 (property available, don’t have enough money) 5. 1-2-7-8-10 (property available, have money, don’t want to buy it) 6. 1-2-7-8-9-10 (property available, have money, and buy it) So, the minimum number of test cases needed to test this would be six.

11. CS 325 March 31, 2015 Page 211 Regression Testing When features are added or altered, the changes might cause previously working code to fail Common methods of regression testing include re- running previously run tests and checking whether previously fixed bugs have reemerged