Software Testing The process of operating a system or component under specified conditions, observing and recording the results, and making an evaluation of some aspect of the system or component. IEEE Std

Testing versus Other Approaches Testing versus Software Quality Management Testing versus Formal Inspections Testing versus Proofs of Correctness Testing versus Formal Verification Testing versus Debugging Testing versus Programming

Black-box Testing Tests the software without understanding how the internal code is implemented. Tests functionality based on requirements Advantages? Disadvantages?

White-box Testing Tests based on the internal logic and structure of the code Advantages? Disadvantages?

What makes a good test? Expected test outcome is known Has a high probability of exposing an error Independence from coding Tests boundary conditions Automated Testing can not show the absence of defects, it can only show that software defects are present

Test Levels Unit testing Integration testing – Finding errors when integrating unit-tested modules System testing – Testing the entire system (as a whole) running on the target hardware Acceptance testing – Performed by the customer – Beta testing Regression testing – Testing changes to make sure the old code still works with the new changes

Types of Tests

Black-box Testing Techniques Equivalence partitioning – Reduces the number of test cases and selects the right test cases to cover all possible scenarios. Boundary-value analysis – Test cases chosen at boundaries of equivalence partitions. Helps with off-by-one errors Robustness testing – Test values outside the domain Ad hoc testing, Exploratory testing – Based on the tester’s skill, intuition, and experience

Equivalence Partitions X < = -2valid -2 < X < 1invalid X >= 1valid Integer.MIN_VALUE, -2 -1, 0 1, Integer.MAX_VALUE

White-box Testing Techniques Basis Path testing 1.Convert the unit into a flow graph A flow graph is a directed graph with a start node and a terminal node 2.Compute the cyclomatic complexity of the unit's logical complexity 3.Use the measure to derive a basis set of execution paths 4.Prepare test cases that will force execution of each path in the basis set.

Flow Graph Example 1.GET(A); GET(B); 2.if A > 15 then 3. if B < 10 then 4. B := A + 5; 5. else 6. B := A - 5; 7. end if 8. else 9. A := B + 5; 10. end if;

Cyclomatic Complexity The cyclomatic complexity gives a quantitative measure of logical complexity Its value is the number of independent paths in the basis set V(G) = E - N + 2 where E = number of edges in G and N = number of nodes in G V(G) provides an upper bound on the number of tests needed to ensure that all paths are executed at least once Studies have shown: – V(G) is directly related to the number of errors in source code – V(G) = 10 is a practical upper limit for testing

Draw the Flow Graph 1:WHILE NOT EOF LOOP 2: Read Record; 2: IF field1 equals 0 THEN 3: Add field1 to Total 3: Increment Counter 4: ELSE 4: IF field2 equals 0 THEN 5: Print Total, Counter 5: Reset Counter 6: ELSE 6: Subtract field2 from Total 7: END IF 8: END IF 8: Print "End Record" 9: END LOOP 9: Print Counter Cyclomatic complexity #edges - #nodes – = 4