1. Software Testing

2. Testing Levels (McConnel) Unit Testing Single programmer involved in writing tested code Component Testing Multiple programmers involved in writing test code Integration Testing Testing two or more independently developed, interacting modules together Regression Testing Use a set of tests that can be rerun when changes are made to the system

3. Testing Transparency White-box Testing Tester has access to the code Black-box Testing Tester has no access to the code Relies on functional descriptions

4. Test Plan Test plan consists of a set of tests and the expected results

5. Code coverage What parts of the code are actually executed by a test? Statement coverage What combinations of sections of code are executed by a test? Conditional Coverage (all conditions are evaluated as true at least once and as false at least once)

6. Boundary Values Identify equivalence classes Array declaration example: Good test cases usually test the boundary values of inputs Empty strings Max number of array elements

7. Finding Equivalence Classes: C++ Array Declaration Input ConditionValid Equivalence classesInvalid Equivalence classes Size of array name<255 chars (1) >255 truncates to 255 (3) <0 char (2) Array nameHas letters (4), has digits(5), has underscore(6) Has something else (7) Number of dimensionsHas 1 dimension (8) Has a lot of dimensions(9) Has 0 dimensions SpacesSpaces between name and bracket (10), Spaces between brackets(11), No spaces between name and bracket (12), No spaces between brackets(13) Spaces within name (14)

8. Boundary-Value Analysis Successful test cases frequently include values that are “directly on, above or beneath the edges of input equivalence classes and output equivalence classes.” (Myers) Each edge of an equivalence class is subject to test Result space is considered as well as input space

9. Boundary-Value Analysis (Myers) Write test cases for the ends of input ranges Input range is [-1.0, 1.0] Write cases for -1.0, 1.0, -1.0001, 1.0001 Number of possible inputs is [1,255] Write cases for 0, 1, 255, 256 Write test cases for the ends of output ranges Calculated SS Tax (FICA) is [$0,$6621.60] Write cases that yield results: $0, $.01, $6621.60, and $6621.61 Number of possible output is [1,255] Write cases for 0,1 255, 256 outputs Use “your ingenuity” to search for other boundary conditions. Boundaries are not always so obvious

10. When to write tests? “All in all, I think test-first programming is one of the most beneficial software practices to emerge during the past decade…” Steve McConnell If you have to write tests eventually anyway, why not write them first so they can be useful longer. Detect bugs earlier Think more about requirements Think more about how others will use your code

11. Research Results on Tests Two studies 80% of errors are found in 20% of the classes 50% of errors are found in 5% of classes Most of the cost of a system might be represented by a very small amount of code. Error prone routines should be redesigned and/or re- implemented

12. Characterizing Errors 85% corrected within a single routine Three most common causes of errors Thin-domain knowledge Fluctuating/Conflicting Requirements Communication/Coordination Breakdown Almost always the programmer’s fault Typos/Spelling errors are very common Three most expensive errors involved the change of a single character ($1.6 Billion, $900 Million, $245 Million) Misinterpretation of Design 85% of errors can be fixed in a few hours It pays to know where you make errors (personal process) and where your project makes errors

13. Number of Errors to Expect Industry average is estimated at 1-25 errors per 1000 lines of code MS Applications Division gets 10-20/1000 lines in in-house tests:.5/1000 in released code. Space shuttle claims 0 defects in 500,000 lines of code