1. Prof. Mohamed Batouche batouche@ksu.edu.sa

2. Software Testing

3.  Software testing is a popular risk management strategy. It is used to verify that functional requirements were met.  The limitation of this approach, however, is that by the time testing occurs, it is too late to build quality into the product 3

4.  Risks are potential problems that may affect successful completion of a software project.  Risks involve uncertainty and potential losses.  Risk analysis and management are intended to help a software team understand and manage uncertainty during the development process. 4

5.  Reactive strategies  very common, also known as fire fighting.  project team sets resources aside to deal with problems.  team does nothing until a risk becomes a problem.  Proactive strategies  risk management begins long before technical work starts, risks are identified and prioritized by importance.  team builds a plan to avoid risks if they can or to minimize risks if they turn into problems. 5

6.  Component testing  Testing of individual program components;  Usually the responsibility of the component developer (except sometimes for critical systems);  Tests are derived from the developer’s experience.  System testing  Testing of groups of components integrated to create a system or sub-system;  The responsibility of an independent testing team;  Tests are based on a system specification. 6

7. 7 Component testing System testing Software developerIndependent testing team

8.  Validation testing  To demonstrate to the developer and the system customer that the software meets its requirements;  A successful test shows that the system operates as intended.  Defect testing  To discover faults or defects in the software where its behavior is incorrect or not in conformance with its specification;  A successful test is a test that makes the system perform incorrectly and so exposes a defect in the system. 8

9. Software Testing Process 9

10.  Involves integrating components to create a system or sub-system.  May involve testing an increment to be delivered to the customer.  Two phases:  Integration testing - the test team have access to the system source code. The system is tested as components are integrated.  Release testing - the test team test the complete system to be delivered as a black-box. 10

11.  There are two groups of software integration strategies: - Non Incremental software integration - Incremental software integration Non Incremental software integration: Big bang integration approach Incremental software integration: Top- down software integration Bottom-up software integration Sandwich integration 11

12.  Involves building a system from its components and testing it for problems that arise from component interactions.  Top-down integration  Develop the skeleton of the system and populate it with components. Use stubs to replace real components.  Bottom-up integration  Integrate infrastructure components then add functional components. Use drivers to test components  To simplify error localisation, systems should be incrementally integrated. 12

13.  The process of testing a release of a system that will be distributed to customers.  Primary goal is to increase the supplier’s confidence that the system meets its requirements.  Release testing is usually black-box or functional testing  Based on the system specification only;  Testers do not have knowledge of the system implementation. 13

14.  Only exhaustive testing can show a program is free from defects. However, exhaustive testing is impossible,  Testing policies define the approach to be used in selecting system tests:  All functions accessed through menus should be tested;  Combinations of functions accessed through the same menu should be tested;  Where user input is required, all functions must be tested with correct and incorrect input. 14

15. Software Testing Process 15

16.  Black-box or behavioral testing  knowing the specified function a product is to perform and demonstrating correct operation based solely on its specification without regard for its internal logic  White-box or glass-box testing  knowing the internal workings of a product, tests are performed to check the workings of all possible logic paths 16

17. Black-Box Testing Functional Testing

18.  Input data and output results often fall into different classes where all members of a class are related.  Each of these classes is an equivalence partition or domain where the program behaves in an equivalent way for each class member.  Test cases should be chosen from each partition. 18

19.  Black-box technique divides the input domain into classes of data from which test cases can be derived.  An ideal test case uncovers a class of errors that might require many arbitrary test cases to be executed before a general error is observed. 19

20.  Black-box technique  focuses on classes and also on the boundaries of the input domain.  Guidelines: 1. If input condition specifies a range bounded by values a and b, test cases should include a and b, values just above and just below a and b 2. If an input condition specifies a number of values, test cases should exercise the minimum and maximum numbers, as well as values just above and just below the minimum and maximum values 20

21. 21 Factorial n n!

22.  Equivalence partitioning – break the input domain into different classes:  Class1: n<0  Class2: n>0 and n! doesn’t cause an overflow  Class3: n>0 and n! causes an overflow  Boundary Value Analysis:  n=0 (between class1 and class2) 22

23.  Test case = ( ins, expected outs)  Equivalence partitioning – break the input domain into different classes: 1. From Class1: ((n = -1), “ function not defined for n negative”) 2. From Class2: ((n = 3), 6) 3. From Class3: ((n=100), “ input value too big”)  Boundary Value Analysis: 4. ((n=0), 1) 23

24. White-Box Testing Structural Testing

25.  The objective of path testing is to ensure that the set of test cases is such that each path through the program is executed at least once.  The starting point for path testing is a program flow graph that shows nodes representing program decisions and arcs representing the flow of control.  Statements with conditions are therefore nodes in the flow graph. 25

26.  White-box technique is based on the program flow graph (CFG) Many paths between 1 (begin) and 6 (end) 1, 2, 5, 6 1, 2, 3, 4, 2, 6 1, 2, 3, 4, 2, 3, 4, 2, 5, 6 …  Prepare test cases that will force the execution of each path in the basis set.  Test case : ((inputs …), (expected outputs …)) 26 2 3 4 5 1 6

27. 27 1 2 A sequence: X = 1; Y = X * 10; 1 4 23 If condition: If … Then … Else … End if 2 3 4 5 1 While loop: While … do … statements … End while

28. 28

29. 29

30. 30

31.  1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 14  1, 2, 3, 4, 5, 14  1, 2, 3, 4, 5, 6, 7, 11, 12, 5, …  1, 2, 3, 4, 6, 7, 2, 11, 13, 5, …  Test cases should be derived so that all of these paths are executed 31

32.  McCabe’s cyclomatic number, introduced in 1976, is, after lines of code, one of the most commonly used metrics in software development.  The cyclomatic complexity of the program is computed from its control flow graph (CFG) using the formula: V ( G ) = Edges – Nodes + 2 or by counting the conditional statements and adding 1  This measure determines the basis set of linearly independent paths and tries to measure the complexity of a program 32

33. How to evaluate quality? Fix a Threshold value  An important aspect of a metric is guidance about when the values are reasonable and when the values are not reasonable.  McCabe analyzed a large project and discovered that for modules with cyclomatic number over 10, the modules had histories of many more errors and many more difficulties in maintenance.  Thus, 10 has been accepted as the threshold value for the cyclomatic number in a module. If the cyclomatic number is greater than 10, efforts should be made to reduce the value or to split the module. 33

34. V ( G ) = Edges – Nodes + 2 V(G) = 6 – 6 + 2 = 2 V(G) = conditional statements + 1 = 1 + 1 = 2 Two linearly independent paths: 1, 2, 5, 6 1, 2, 3, 4, 2, 5, 6 Complexity = 2 good quality 34 2 3 4 5 1 6

35. V ( G ) = Edges – Nodes + 2 V(G) = 16 – 14 + 2 = 4 V(G) = conditional statements + 1 = 3 + 1 = 4 four linearly independent paths Complexity = 4 good quality 35

36.  K. N AIK AND P. T RIPATHY : “S OFTWARE T ESTING AND Q UALITY A SSURANCE ”, W ILEY, 2008.  I AN S OMMERVILLE, S OFTWARE E NGINEERING, 8 TH EDITION, 2006.  A DITYA P. M ATHUR,“F OUNDATIONS OF S OFTWARE T ESTING ”, P EARSON E DUCATION, 2009.  D. G ALIN, “S OFTWARE Q UALITY A SSURANCE : F ROM T HEORY TO I MPLEMENTATION ”, P EARSON E DUCATION, 2004  D AVID G USTAFSON, “T HEORY AND P ROBLEMS OF S OFTWARE E NGINEERING ”, Schaum’s Outline Series, McGRAW-HILL, 2002. 36