1. Creator: ACSession No: 12 Slide No: 1Reviewer: CSE300Advanced Software EngineeringJanuary 2006 Testing Strategy CSE300 Advanced Software Engineering University of Sunderland © 2006 Anne Comer anne.comer@sunderland.ac.uk

2. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 2Reviewer: Aim of this Session To consider Verification and Validation at the strategic level – what do the terms mean, what is the difference between static and dynamic V&V, what do we mean by testing, how does testing relate to the development process?

3. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 3Reviewer: Verification & Validation Any engineered product can be checked in one of two ways: check vs. the specified functions - Validation “doing the right thing” check vs. the internal workings - Verification “doing the thing right”

4. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 4Reviewer: Static and dynamic V&V Static V&V - software inspections - concerned with analysis of the static system representation, in order to discover problems. –May be supplemented by tool-based document and code analysis. Dynamic V&V - software testing - concerned with exercising and observing product behaviour. –The system is executed with test data and its operational behaviour is observed.

5. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 5Reviewer: What is Testing? Testing is the process of executing a program with the intent of finding errors A good test case has a high probability of finding an as-yet undiscovered error A successful test is one that uncovers an as-yet undiscovered error. - G. Myers

6. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 6Reviewer: Types of testing Defect testing –Tests designed to discover system faults. –A successful test is one which reveals the presence of faults in a system. Statistical testing –Tests designed to reflect the frequency of user inputs (i.e., the operational profile). Is used for reliability estimation.

7. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 7Reviewer: Testing Shows… “Testing cannot show the absence of faults, it can only show that software faults are present.” However, testing also shows function and performance. Testing is also used to indicate the quality of the software.

8. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 8Reviewer: Independent Testing The DEVELOPERS understand the system - because they built it! Unsurprisingly, they test to show it works; they test gently,and are driven by ‘delivery’. INDEPENDENT testers have to learn about the system, but will attempt to break it. They are driven by quality.

9. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 9Reviewer: Exhaustive Testing Even in small programs the number of possible logical paths can be enormous. Take a program about 100 lines long, with a couple of nested loops executing 20 times each. There are approximately 10 14 possible paths that may be executed. At one test per millisecond, that would be 3170 years alone! Exhaustive testing is a ‘non-starter’.

10. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 10Reviewer: Selective Testing Even if exhaustive testing is a non-starter, white box testing should not be dismissed. Important logical paths and loops should be tested. Selective testing validates the interfaces and gives confidence in the internal workings of the software.

11. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 11Reviewer: Where Errors Occur Requirements Analysis System design & specfn Program design Program implementation Unit and integration test System test Maintenance Incorrect or unclear requirements, incorrect or unclear translation Incorrect or unclear design spec. Misinterpretation of system design Misinterpretation of prog. Design incorrect documentation incorrect syntax or semantics Incomplete test procedures adding new errors when fixing old User documentation, human factors, changing requirements,.....

12. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 12Reviewer: The Testing ‘V’ Lifecycle (see next slide for details of the test plans) rquirment s Reqrts and Specification System design Detailed design Module code and test Subsystem integration test System integration test Acceptance test Subsystem integration test plan System integration test plan Acceptance test plan

13. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 13Reviewer: Test Plans Test process Requirements traceability Items under test Test schedules Test recording procedures Hardware and software requirements Constraints

14. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 14Reviewer: Test Case Design “ Bugs lurk in corners and congregate at boundaries.....”- Boris Beizer OBJECTIVEto uncovers errors CRITERIAin a complete manner CONSTRAINTwith a minimum of effort & time

15. CSE300Advanced Software EngineeringJanuary 2006 Creator/Editor: ACSession No:12 Slide No: 15Reviewer: Prior Reading Prior to the next session students are required to read the software testing chapters of Pressman.