1. Integration and System Testing CSSE 376, Software Quality Assurance Rose-Hulman Institute of Technology March 29, 2007

2. 2 Lifecycle Testing Relationships Requirements High-level Design Low-level Design Code Unit Testing Integration Testing System Testing Acceptance Testing

3. 3 Testing in the Large 1. Integration testing: putting the pieces together 2. System testing: function and performance

4. 4 1. Integration Testing

5. 5 Integration Testing Objectives n Gain confidence in the integrity of overall system design n Ensure proper interaction of components n Run simple system-level tests

6. 6 Integration Strategies n Top-down n Bottom-up n Critical-first n Function-at-a-time n As-delivered n Big bang

7. 7 Top-Down Integration n Start with top-level modules n Use stubs for lower- level modules n As each level is completed, replace stubs with next level of modules

8. 8 Top-Down Issues n Pros: –Always have a top-level system –Stubs can be written from interface specifications n Cons: –May delay performance problems until too late –Stubs can be expensive

9. 9 Bottom-Up Integration n Start with bottom- level modules n Use drivers for upper-level modules n As each level is completed, replace drivers with next level of modules

10. 10 Bottom-Up Issues n Pros: –Primitive functions get most testing –Drivers are usually cheap n Cons: –Only have a complete system at the end

11. 11 Critical-First Integration n Integrate the most critical components first n Add remaining pieces later n Issues: –guarantees that most important components work first –may be difficult to integrate

12. 12 Function-at-a-Time Integration n Integrate all modules needed to perform one function at the same time n For each function, add another group of modules n Issues: –makes for easier test generation –may postpone function interaction too long

13. 13 As-delivered Integration n Integrate all of the modules that can be integrated as they become available n Issues –efficient: just-in-time integration –lazy: may lead to missed schedules

14. 14 Big Bang Integration n Wait until all the components are ready, then put everything together at once n Issues: –avoids cost of scaffolding (stubs or drivers) –does not provide any locality for finding faults

15. 15 Summary of Integration Strategies n Top-down n Bottom-up n Critical-first n Function-at-a-time n As-delivered n Big bang

16. 16 Question: What should be used as integration test cases?

17. 17 2. System Testing

18. 18 System Testing Objectives n Gain confidence in the integrity of the system as a whole n Ensure compliance with functional requirements n Ensure compliance with performance requirements

19. 19 Testing Functional Requirements 1. Prepare a test plan from the functional specification of the system 2. Prepare tests for all areas of functionality 3. Review test plan and tests 4. Execute tests 5. Monitor fault rate

20. 20 Fault Rate 0 20 40 60 80 100 120 1/1/001/2/001/3/001/4/001/5/001/6/001/7/001/8/001/9/00 1/10/001/11/001/12/001/13/001/14/001/15/001/16/001/17/001/18/001/19/001/20/00 Total Faults

21. 21 Testing Performance Requirements 1. Identify stress points of system 2. Create or obtain load generators –might use existing system –might buy/make special purpose tools 3. Run stress tests 4. Monitor system performance –usually need instrumentation