1. Testing Integral part of the software development process

2. Testing Testing process Testing strategies – Top-down integration – Bottom-up integration Testing techniques – White box – Black box Testing process Testing strategies – Top-down integration – Bottom-up integration Testing techniques – White box – Black box

3. Commonly used terminologies Testing – Verification – Validation Debugging Defects Bug Failure Testing – Verification – Validation Debugging Defects Bug Failure

4. Software Testing definition Testing is a verification and validation activity that is performed by executing program code. Testing is the process of executing a program with the intent of finding errors. - Glen Myers Testing is a verification and validation activity that is performed by executing program code. Testing is the process of executing a program with the intent of finding errors. - Glen Myers

5. Verification vs validation Verification: "Are we building the product right?" The software should conform to its specification Validation: "Are we building the right product?" The software should do what the user really requires Verification: "Are we building the product right?" The software should conform to its specification Validation: "Are we building the right product?" The software should do what the user really requires

6. Testing Vs Debugging Testing – Verification and validation [Testing] is concerned with establishing the existence of defects in a program – Testing cannot show the absence of defects, it can only show that software defects are present. Debugging – It is concerned with locating and repairing these errors – Debugging always occurs as a consequence of testing – Debugging process Testing – Verification and validation [Testing] is concerned with establishing the existence of defects in a program – Testing cannot show the absence of defects, it can only show that software defects are present. Debugging – It is concerned with locating and repairing these errors – Debugging always occurs as a consequence of testing – Debugging process

7. Defects, Bugs & Failures Defect – Variance from product specifications Human error E.g. Maximum no. of books issued for students [Specified – 3 but implemented for 5] – Variance from customer/user expectation Difference between the actual output of software and the correct output E.g. Login - accepts the invalid member Defect – Variance from product specifications Human error E.g. Maximum no. of books issued for students [Specified – 3 but implemented for 5] – Variance from customer/user expectation Difference between the actual output of software and the correct output E.g. Login - accepts the invalid member

8. Defects, Bugs & Failures Bug/Fault – Condition that causes a system to fail in performing its required function – Defect/Bug/fault – No explicit difference but slightly differs in concept. Failure – Inability of the system or component to perform a required function according to its specification – Negatively impacts a customer/user Presence of fault –> has a potential to cause a failure to occur Bug/Fault – Condition that causes a system to fail in performing its required function – Defect/Bug/fault – No explicit difference but slightly differs in concept. Failure – Inability of the system or component to perform a required function according to its specification – Negatively impacts a customer/user Presence of fault –> has a potential to cause a failure to occur

9. Testing Relationship Definitions - Testing Lifecycle Concept Testing phase – after coding [Mainly to find defects] Operational or Business Need Define Requirements Design System Build System Unit Test Integration Test System Test Acceptance Test

10. What to verify? & What to validate? Testing Relationship Definitions - Continuous Life Cycle Testing Operational or Business Need Define Requirements Design System Build System Unit Test Integration Test System Test Acceptance Test Verify Validate

11. Testing objective A good test is one that has a high probability of finding an as yet undiscovered error. A successful test is one that uncovers an as yet undiscovered error. To improve the quality of the software The objective is to design tests that systematically uncover different classes of errors and do so with a minimum amount of time and effort. E.g. Testing objective in correcting SE paper: SE knowledge & not grammatical mistakes A good test is one that has a high probability of finding an as yet undiscovered error. A successful test is one that uncovers an as yet undiscovered error. To improve the quality of the software The objective is to design tests that systematically uncover different classes of errors and do so with a minimum amount of time and effort. E.g. Testing objective in correcting SE paper: SE knowledge & not grammatical mistakes

12. Test case A test plan is a software testing document, which serves as the roadmap for testing the system.test plan A test case is a software testing document, which consists of event, action, input, output, expected result, and actual result.test case A test case is like a check list to make sure that all the requirements are covered in the/this testing.test case Each scenario of a usecase will be a test casetest case The most common term for a collection of test cases is a test suite. The test suite often also contains more detailed instructions or goals for each collection of test cases.test suite A test plan is a software testing document, which serves as the roadmap for testing the system.test plan A test case is a software testing document, which consists of event, action, input, output, expected result, and actual result.test case A test case is like a check list to make sure that all the requirements are covered in the/this testing.test case Each scenario of a usecase will be a test casetest case The most common term for a collection of test cases is a test suite. The test suite often also contains more detailed instructions or goals for each collection of test cases.test suite

13. White Box Testing

14. Agenda White-box vs Black-box Program Flow Controls White-box Test Methods Exercises Complexity Q&A

15. What is White-box Testing? Looking at the internal structure of a program and deriving test cases based on the logic or control flow. Test cases can be designed to reach every branch in the code and to exercise each condition Typically done during unit testing Also known as: –Structural Testing –Glass-Box Testing

16. What is Black-box Testing? Looking at the program from an external point of view and deriving test cases based on the specification. The only criteria upon which the program is judged is if it produces the correct output for a given input.

17. Why Do Both? Black-box Impossible to write a test case for every possible set of inputs and outputs Some of the code may not be reachable without extraordinary measures Specifications are not always complete White-box Does not address the question of whether or not the program matches the specification Does not tell you if all of the functionality has been implemented Does not discover missing program logic

18. Basic Program Flow Controls IF IF-Then-Else FOR While Do-While Case

19. IF Diagram

20. IF-THEN-ELSE Diagram

21. FOR or WHILE Diagram

22. DO-WHILE Diagram

23. CASE Diagram

24. Example Code Fragment

25. Example Control Flow Graph Source: The Art of Software Testing – Glenford Myers

26. White-box Test Methods Statement Coverage Decision/Branch Coverage Condition Coverage Decision/Condition Coverage Path Coverage

27. Example Code Fragment If ((A>1) & (B=0)) then Do; X=X/A; END; If ((A==2) | (X>1)) then Do; X=X+1; END; Source: The Art of Software Testing – Glenford Myers

28. Statement Coverage Exercise all statements at least once How many test cases?  A=2 and B=0 (ace) Source: The Art of Software Testing – Glenford Myers

29. Decision/Branch Coverage Each decision has a true and a false outcome at least once How many test cases?  A=2 and B=0 (ace)  A=1 and X=1 (abd) Source: The Art of Software Testing – Glenford Myers

30. Condition Coverage Each condition in a decision takes on all possible outcomes at least once Conditions: A>1, B=0, A=2, X>1 How many test cases?  A=2, B=0, and X=4 (ace)  A=1, B=1, and X=1 (abd) Source: The Art of Software Testing – Glenford Myers

31. Decision/Condition Coverage Each condition in a decision takes on all possible outcomes at least once, and each decision takes on all possible outcomes at least once How many test cases?  A=2, B=0, and X=4 (ace)  A=1, B=1, and X=1 (abd) What about these?  A=1, B=0, and X=3  A=2, B=1, and X=1 (abe)

32. Multiple Condition Coverage Exercise all possible combinations of condition outcomes in each decision Conditions: Source: The Art of Software Testing – Glenford Myers A>1, B=0 A>1, B<>0 A<=1, B=0 A 0 A=2, X>1 A=2, X<=1 A<>2, X>1 A<>2, X<=1

33. Multiple Condition Coverage How many test cases?  A=2, B=0, X=4  A=2, B=1, X=1  A=1, B=0, X=2  A=1, B=1, X=1 Source: The Art of Software Testing – Glenford Myers (ace) (abe) (abd)

34. Path Coverage Every unique path through the program is executed at least once How many test cases?  A=2, B=0, X=4 (ace)  A=2, B=1, X=1 (abe)  A=3, B=0, X=1 (acd)  A=1, B=1, X=1 (abd) Source: The Art of Software Testing – Glenford Myers

35. McCabe’s Cyclomatic Complexity Software metric Developed by Tom McCabe (circa 1976) Directly measures the number of linearly independent paths through a program’s source code, taking into account the various decision points Independent of implementation language Source: Wikipedia

36. Calculating Complexity

38. How Complex Should Code Be? <10: Simple module, not much risk 10-20: More Complex; moderate risk 20-50: Complex; high risk >50: Untestable; extremely high risk Source: Carnegie Mellon Software Engineering Institute

39. Complexity Caveats As code is broken into smaller modules to decrease cyclomatic complexity, structural complexity increases Some modules may have high complexity but are very easy to comprehend and easy to test High complexity numbers are only an indicator of something to investigate