Testing Strategy Big Picture Planning for Software Testing

Why Plan Your Tests? Testing can take a significant amount of (often unplanned) time There is more to be tested than possible Haphazard testing is likely to miss errors Haphazard testing is likely to duplicate effort Haphazard testing is likely to generate confusion

The Testing Keyword: “Systematic” Plan what you will test, when, and how. Design tests with a purpose (diagnosis) Use TRs as part of your test plan Start small, but test the “all” Test logic, function, behavior, and requirements. Test your deployment. Test your documentation.

Types (Levels) of Testing Unit: Tests individual modules and classes, generally with knowledge of program logic (white-box or glass-box testing). Integration: Tests collaborating classes or modules, also with knowledge of program structure. Validation: Tests requirements, functions and behavior (black-box or functional testing) System: Tests application in context.

Things We Can Do to Make Testing Easier Specify requirements quantifiably: e.g., GUI response time should be less than 1 sec State test objectives explicitly: e.g., this test verifies data structure initialization Create a user profile: determine expected pattern of use (and then also test for unexpected) Build bullet-proof software: expect exceptions Use TRs to think deeply about the program logic Let someone else review your test cases for completeness Instrument your classes to show their state Use unit testing frameworks where available

Common Errors In Computation Misunderstood operator precedence Mixed data types (casting) Incorrect initialization Precision problems Incorrect equations (or derivation of the same) In Logic (control flow) Comparison errors (type mismatch, wrong operator) Misunderstood precedence Missed equality Improper loop initialization or termination Improper loop maintenance Boundary values misunderstood or just missed (off-by-one) Data structures or their operations misunderstood Overly complicated Boolean expressions (e.g., double negation) In Error Handling Error description is meaningless Error reported is not error encountered Error not caught Error handling not tested

Top-Down Versus Bottom-Up Testing Top-Down Advantages: Main control structure exercised early Early demonstration of capability builds confidence in app Disadvantages: Requires stub code May degenerate into “big bang” testing approach Bottom-Up Advantages: Easier to comprehend Helps create solid program foundation Supported by testing frameworks Disadvantages: “Glue” code required to test cooperating modules or classes

Additional Points of Emphasis Testing and quality (pg 451 7/e, pg 356 6/e) Testing and the programmer’s “conflict of interest” (pg 451 7/e, pg 357 6/e) Other types of testing: regression and smoke testing (pg /e, /e) Verification &Validation (pg 451 7/e, pg 356 6/e) Still other types of testing: recovery, security, etc. (pgs /e, pgs /e) Comments on Debugging (pgs /e, pgs /e)