1. CS 49000-ST0 Software Testing Spring 2011 Review Last updated: April 26, 2011 Aditya P. Mathur Purdue University

2. Learning Objectives Various test processes and continuous quality improvement Types of errors and fault models Methods of test generation from requirements Behavior modeling using UML: Finite state machines (FSM) Test generation from FSM models 2

3. Learning Objectives (contd) Input space modeling using combinatorial designs Combinatorial test generation Test adequacy assessment using: control flow, data flow, and program mutations The use of various test tools Application of software testing techniques in commercial environments 3

4. Test processes and continuous quality improvement 4 Test metrics: Chapter 1.6 Test plan: Chapter 1.5 Defect management: Chapter 1.9 Walkthroughs and inspection: Chapter 1.12 Test process models: Chapter 1.18.5

5. Types of errors and fault models 5 Chapter 1.1

6. Methods of test generation from requirements 6 Equivalence partitioning: Chapter 2.3 Boundary value analysis: Chapter 2.4

7. Behavior modeling using UML: Finite state machines (FSM) 7 Microsoft project AEC project: parsing FSM basics: Chapter 3.2

8. Test generation from FSM models 8 Spec Explorer W-method: Chapter 3.6 State, transition, and switch covers: Chapter 3.9

9. Input space modeling using combinatorial designs 9 Test configurations and modeling the input space: Chapter 4.1 Fault model: Chapter 4.3

10. Combinatorial test generation 10 MOLS: 4.8 Covering arrays: 4.9, 4.10, 4.11

11. Test adequacy assessment using: control flow, data flow, and program mutations 11 Basics: 6.1 Criteria based on control flow: 6.2 (excluded 6.2.7) Criteria based on data flow: 6.3, 6.4.1, 6.4.2, 6.4.3

12. The use of various test tools 12 Load and performance testing: JMeter GUI testing: Abbot, Guitar Combinatorial testing: AETG, FireEye Unit testing: Achoo, JUnit, Pex/Moles, PyUnit Model based testing testing: Spec Explorer Coverage analysis: Corbertura Defect management: Bugzilla Test automation: TestComplete Concurrency: ConTest

13. Application of software testing techniques in commercial environments 13 AEC project Microsoft project Some Spec Explorer Statistics [MS]: Team Size test cases 600 798 158 8178 100100 60 164 40 40 30 900 20 629 18 3000 13 1900 10 150 10 60 8 28 Some Spec Explorer Statistics [Customers]:

14. What did we not cover [1]? 14 Regression testing (Chapter 5) Mutation testing (for test adequacy and enhancement; Chapter 7) Test generation from statecharts Test generation from timed-automata for real-time systems Test generation from formal specification (e.g.; Z) Testing concurrent programs

15. What did we not cover [2]? 15 Test process Test automation Error types and classification method

16. Final Examination 16 Tuesday May 3. 8-10am. Smith 208 Part A: Multiple choice, comprehensive [Closed book/notes] Part B: Few questions (3 or 4) [Open book/notes] FSM-based test generation, Combinatorial designs, Test adequacy

17. 17 Good luck! Best wishes!