1. Chapter 11: Testing The dynamic verification of the behavior of a program on a finite set of test cases, suitable selected from the usually infinite execution domain, against the expected behavior

2. Testing Fundamentals  Testing is the one step in software engineering process that could be viewed as destructive rather than constructive. A successful test is one that breaks the software.  A successful test is one that uncovers an as yet undiscovered defect.  Testing can not show the absence of defects, it can only show that software defects are present.  For most software exhaustive testing is not possible.

3. Testing  Unit testing White-box Code walkthroughs and inspections  Integration testing Bottom-up Top-down Sandwich Big Bang  Functional testing (black-box)  Performance testing  Acceptance testing  Installation testing

5. System testing  Functional testing  Performance testing  Acceptance testing  Installation testing

6. System Testing Process  Functional testing: does the integrated system perform as promised by the requirements specification?  Performance testing: are the non- functional requirements met?

7. Functional Testing  Test cases derived from requirements specification document Black box testing Independent testers Test both valid and invalid input and the success of the test is determined by the produced output  Equivalence partitioning Boundary values

8. Equivalence partitioning  Reduces the number of test cases to a necessary minimum and select the right test cases to cover all possible scenarios  Example: method accepts an int value for month (1..12) as a parameter

9. Performance Testing  Stress tests  Volume tests  Recovery tests  Security tests  Timing tests  Environmental tests  Quality tests  Maintenance tests  Documentation tests  Human factors (usability) tests

10. Acceptance Tests  Enable the customers and users to determine if the built system meets their needs and expectations  Written, conducted, and evaluated by the customers

11. Types of Acceptance Tests  Pilot test: install on experimental basis  Alpha test: in-house test  Beta test: customer pilot  Parallel testing: new system operates in parallel with old system

12. Installation test  Does the system run at the customer site(s)?

13. System Testing

14. Pop quiz  How does system testing differ from unit and integration testing?  How does unit testing differ from integration testing?

15. In a method, a variable did not get initialized properly. Which type of testing would most likely expose this defect? A. Unit testing B. Integration testing C. Functional testing D. Performance testing E. Acceptance testing F. Installation testing

16. A gas pump system is supposed to allow the user to choose whether or not a receipt is printed, but the print function has not been implemented. Which type of testing is most likely to expose this defect? A. Unit testing B. Integration testing C. Functional testing D. Performance testing E. Acceptance testing F. Installation testing

17. A configuration file used by the reporting subsystem is not placed in the correct directory in the customer's environment. Which type of testing is most likely to expose this defect? A. Unit testing B. Integration testing C. Function testing D. Performance testing E. Acceptance testing F. Installation testing

18. The customer is unhappy with the number of screens that must be traversed before getting to the parts list screen, a screen accessed frequently when using the system. Which type of testing is most likely to expose this defect? A. Unit testing B. Integration testing C. Function testing D. Performance testing E. Acceptance testing F. Installation testing

19. Short answer question  What is the difference between verification and validation testing?

20. Test Documentation  Test plan: describes system and plan for testing all functions and characteristics  Test case specification: details each test and defines criteria for evaluating each feature  Test incident report: results of each test  Test report summary: lists all failures from the tests that need to be investigated

21. Test Plan (part of 480 QA plan)  Define the subsystems to be tested (and not tested)  Describe the process to follow for unit, integration, system, and acceptance testing  List the test cases  Plan the test schedule

22. Test Documentation

23. Defect Tracking Form

24. Regression Testing  Identifies new faults that may have been introduced as current ones are being corrected  Verifies that a new version or release still performs the same functions in the same manner as an older version or release

25. Quality Assurance concerns  Software reliability: operating without failure under given condition for a given time interval  Software availability: operating successfully according to specification at a given point in time  Software maintainability: for a given condition of use, a maintenance activity can be carried out within stated time interval, procedures and resources

26. Testing Safety-Critical Systems  Recognize that testing cannot remove all faults or risks  Assume that every mistake users can make will be made Do not assume that low-probability, high- impact events will not happen  Emphasize requirements definition, testing, code and specification reviews, and configuration control Cleanroom testing

27. Different Levels of Failure Severity  Catastrophic: causes death or system loss  Critical: causes severe injury or major system damage  Marginal: causes minor injury or minor system damage  Minor: causes no injury or system damage

28. One-minute quiz  What is meant by regression testing?  Is regression testing used for verification or validation?