1. Software Engineering Lecture 13 Software Testing Strategies 1

2. Verification and Validation Verification refers to the set of activities that ensure that software correctly implements a specific function. It asks the question: “Are we building the product right?” Validation refers to a different set of activities that ensure that the software that has been built is traceable to customer requirements. It asks the question: “Are we building the right product?” 2

3. Testing Strategy 3 unit test integrationtest validationtest systemtest

4. Unit Testing 4 module to be tested test cases results softwareengineer

5. Unit Testing 5 interface local data structures boundary conditions independent paths error handling paths module to be tested test cases

6. Unit Test Environment 6 Module stub stub driver RESULTS interface local data structures boundary conditions independent paths error handling paths test cases

7. Integration Testing Strategies 7 Options: the “big bang” approachthe “big bang” approach an incremental construction strategyan incremental construction strategy

8. Top Down Integration 8 top module is tested with stubs stubs are replaced one at a time, "depth first" as new modules are integrated, some subset of tests is re-run A B C DE FG

9. Bottom-Up Integration 9 drivers are replaced one at a time, "depth first" worker modules are grouped into builds and integrated A B C DE FG cluster

10. Sandwich Testing 10 Top modules are tested with stubs Worker modules are grouped into builds and integrated A B C DE FG cluster

11. Validation Testing Validation Testing can be defined as ensuring that software functions in a manner that can be reasonably expected by the customer. Achieve through a series of black-box testings tha demonstrate conformity with requirements. A series of acceptance tests (include both alpha and beta testing) are conducted with the end users. 11

12. Alpha vs. Beta Testing Alpha Testing Is conducted at the Developer’s site by a customer The developer would supervise Is conducted in a controlled environment Beta Testing Is conducted at customer’s site by the end user of the software The developer is generally not present Is conducted in a “live” environment 12

13. System Testing System testing is actually a series of different tests whose primary purpose is to fully exercise the computer-based system Although each system test has a different purpose, all work to verify that all system elements have been integrated and perform allocated functions 13

14. System Testing Examples Recovery Testing A system test that forces software to fail in a variety of ways and verifies that recovery is properly performed Security Testing Attempts to verify tha protection mechanism built into a system will in fact protect it from improper penetration Stress Testing Is designed to confront programs with abnormal situation where unusual quantity, frequency, or volume of resources are demanded Performance Testing Seeks to test the run-time performance of software within the context of an integrated system 14

15. Debugging: A Diagnostic Process 15

16. The Debugging Process 16 test cases results Debugging suspectedcauses identifiedcauses corrections regressiontests new test cases

17. Debugging Effort 17 time required to diagnose the symptom and determine the cause time required to correct the error and conduct regression tests

18. Symptoms & Causes 18 symptom cause symptom and cause may be geographically separated symptom may disappear when another problem is fixed cause may be due to a combination of non-errors cause may be due to a system or compiler error cause may be due to assumptions that everyone believes symptom may be intermittent

19. Debugging Techniques 19 brute force / testing backtracking Cause elimination

20. Debugging: Final Thoughts 20 Don't run off half-cocked, think about the about the symptom you're seeing. Use tools (e.g., dynamic debugger) to gain (e.g., dynamic debugger) to gain more insight. If at an impasse, get help from someone else. from someone else. Be absolutely sure to conduct regression tests when you do "fix" the bug. 1. 2. 3. 4.

21. References Pressman, Chapter 18 21