1. Software Testing Mistake in coding is called error ,
Error found by tester is called defect,
Defect accepted by development team is called bug ,
Product does not meet the requirements then it Is failure.”

2. The Importance of Test
There are hundreds of stories about failures of computer systems that have been attributed to errors in software
There are many reasons why systems fail but the issue that stands out the most is the lack of adequate testing.
Example Pepsi - $42 Billion Error Philippines -1992
Due to a software error, 800,000 bottle caps were produced with number 349 instead of one.
It was equivalent to $32 billion in prize money instead of $40,000

3. Testing Levels

4. Testing Levels
Unit
Module
Each of testing activities is performed during the process of building an application
Unit
…..
Unit
Build
Application
on final
platform
Application
on test bed
…..
…..
Module
Interface testing
…..
system testing
installation testing
usability testing
Unit
Build
acceptance testing
Unit
Module
integration testing
…..
Unit
Regression testing throughout

5. Testing Goals Based on Test Process Testing Levels (Bezier)
Level 0 There is no difference between testing and debugging
Level 1 The purpose of testing is to show the correctness (software works).
Level 2 The purpose of testing is to show that the software doesn’t work.(the purpose is to show failures)
Level 3 Testing shows the presence , not absence or failure. If we use software, we expose some risk. The risk may be small and unimportant, or the risk may be great and the consequences catastrophic, but risk is always there
(The purpose of testing is to reduce the risk of software ; both testers and developers work together to reduce risk.
Level 4 The purpose of testing is to improve the ability of the developers to produce high quality software.
(Level 4 testing shows that testers and developers are on the same team)

6. Debugging and Testing
Debugging is done in the development phase by the developer
In debugging phase identified bugs are fixed
Testing is done in the tester phase by the tester.
In testing, locating and identifying of the bugs are included.

7. Software Bug
A software bug is an error, flaw, mistake, failure, or fault in a computer program or system
Bug is terminology of Tester
Software bug produces an incorrect or unexpected result, or causes it to behave in unintended ways

8. Software Fault
A static (physical)defect*, imperfection, flaw in the software.
short between wires
break in transistor
infinite program loop
A fault means that there is a problem within the code of a program which causes it to behave incorrectly.
*defect: Mismatch between the requirements

9. Software Error
Error is an incorrect internal state that is the demonstration of some fault.
In other words; Error is a deviation from correctness or accuracy.
Example
Suppose a line is physically shortened to 0. (there is a fault).
As long as the value on line is supposed to be 0, there is no error.
Errors are usually associated with incorrect values in
the system state.

10. Software Failure
A failure means that the program has not performed as expected, and the actual performance has not met the minimum standards for success.
External, incorrect behavior with respect to the requirements or other description of the expected behavior
Example
Suppose a circuit controls a lamp (0 = turn off, 1 = turn on) and the output is physically shortened to 0 (there is a fault).
As long as the user wants the lamp off, there is no failure.

11. Example: Failure & Fault & Error
Consider a medical doctor making a diagnosis for a patient.
The patient enters the doctor’s office with a list of failures (that is, symptoms).
The doctor then must discover the fault, or root cause of the symptom.
To aid in the diagnosis, a doctor may order tests that look for anomalous internal conditions.
In our terminology, these anomalous internal conditions correspond to errors.

12. Cause-and-Effect Relationship
Faults can result in errors.
Errors can lead to system failures
Errors are the effect of faults.
Failures are the effect of errors
Bug in a program is a fault.
Possible incorrect values caused by this bug is an error.
Possible crush of the operating system is a failure

13. Program Example
class numZero { public static int numZero (int[] x) { // if x == null throw NullPointerException // else return the number of occurrences of 0 in x int count = 0; for (int i = 1; i < x.length; i++) { if (x[i] == 0) count++; } } return count; } }

14. The Fault ,Error and Failure in the Example
The fault is that the program starts looking for zeroes at index 1 instead of index 0
For example, numZero ([2, 7, 0]) correctly results with 1,
while numZero ([0, 7, 2]) incorrectly results with 0.
In both of these cases the fault is executed.
Both of the cases result in an error (because of next slide)
But the first case results with correct value.
Only the second case results in failure.

15. Error States in the Example
To understand the error states, we need to identify the state for the program.
The state for numZero consists of values for the variables x, count , i , program counter (PC).
For the first case: The state at the if statement on the first iteration ( x = [2, 7, 0], count = 0, i = 1, PC = if)
This state is in error because the value of i should be zero on the first iteration.
But the value of count is correct, the error state does not propagate to the output, and the software does not fail.
In other words, a state is in error if it is not the expected state, even if all of the values in the state are acceptable.

16. Error States in the Example
In the second case the corresponding (error) state is (x = [0, 7, 2], count = 0, i = 1, PC = if). In this case, the error propagates to the variable count and is present in the return value of the method. Hence a failure results.

17. Distinguish Testing from Debugging
The definitions of fault and failure allow us to distinguish testing from debugging.
The difference is that debugging is conducted by a programmer and the programmer fix the errors during debugging phase.
Tester never fixes the errors, but rather find them and return to programmer.

18. Testing versus Debugging
Testing activity is carried down by a team of testers, in order to find the defect* in the software.
Testers run their tests on the piece of software and if they encounter any defect,they report it to the development team.
Testers also have to report at what point the defect occurred and what happened due the occurrence of that defect.
All this information is used by development team to DEBUG the defect.
After finding out the bug, he tries to modify that portion of code and then he rechecks if the defect has been removed
After fixing the bug, developers send the software back to testers.
*actual results don't match expected results

19. What is Software Quality ?
According to the IEEE Software Quality is:
The degree to which a system, component, or process meets specified requirements.
The degree to which a system, component, or process meets customer or user needs or expectations.

20. Software Quality Factors

21. Software Quality Assurance
Verification
– are we building the product right ?
– performed at the end of a phase to
ensure that requirements established
during previous phase have been
met
Validation
– are we building the right product ?
– performed at the end of the development
process to ensure compliance with product
requirements

22. Verification & Validation
Verification: The process of determining whether the products of a given phase of the software development process fulfill the requirements established during the previous phase.
Validation: The process of evaluating software at the end of software development to ensure compliance with intended usage.

23. Difference Between Verification &Validation- I
Verification is preventing mechanism to detect possible failures before the testing begin.
It involves reviews, meetings, evaluating documents, plans, code, inspections, specifications etc.
Validation occurs after verification and it's the actual testing to find defects against the functionality or the specifications

24. The Difference between Verification &Validation- II
Verification is usually a more technical activity that uses knowledge about the individual software artifacts, requirements, and specifications.
Validation usually depends on domain knowledge; that is, knowledge of the application for which the software is written.
For example, validation of software for an airplane requires knowledge from aerospace engineers and pilots.

25. Software Testing Types
Black box testing : You don't need to know the internal design in detail or have a good knowledge about the code for this test.
It's mainly based on functionality and specifications, requirements.
White box testing : This test is based on detailed knowledge of the internal design and code.
Tests are performed for specific code statements and coding styles.

26. Traditional Testing Levels
System testing : Test the overall functionality of the system
main Class P
Acceptance testing : Is the software acceptable to the user?
Module testing (developer testing) : Test each class, file, module or component
Integration testing : Test how modules interact with each other
Class A
method mA1()
method mA2()
Class B
method mB1()
method mB2()
Unit testing (developer testing) : Test each unit (method) individually

27. Object-Oriented Testing Levels
Inter-class testing : Test multiple classes together
Class A
Class B
Intra-class testing : Test an entire class as sequences of calls
method mA1()
method mB1()
method mA2()
method mB2()
Inter-method testing : Test pairs of methods in the same class
Intra-method testing : Test each method individually

28. Coverage Criteria
Even small programs have too many inputs to fully test them all
private static computeAverage (int A, int B, int C)
On a 32-bit machine, each variable has over 4 billion possible values
Over 80 octillion possible tests!!
Input space might as well be infinite
Testers search a huge input space
Trying to find the fewest inputs that will find the most problems
Coverage criteria give structured, practical ways to search the input space
Search the input space thoroughly
Not much overlap in the tests

29. Test Requirements and Criteria
Test Criterion : A collection of rules and a process that define test requirements
Cover every statement
Cover every functional requirement
Test Requirements : Specific things that must be satisfied or covered during testing
Each statement is a test requirement
Each functional requirement is a test requirement
Testing researchers have defined dozens of criteria, but they are all really just a few criteria on four types of structures …
Graphs
Logic expressions
Input domains
Syntax descriptions

30. Coverage Criterion
A coverage criterion is simply a recipe for generating test requirements in a systematic way: Coverage Criterion: A coverage criterion is a rule or collection of rules that impose test requirements on a test set.

31. Structural Coverage Criteria -I
Node Coverage (Formal Definition): For each node n ∈ reachG(N0),
TR contains the predicate “visit n.”
Node Coverage (NC): TR contains each reachable node in G.
Node Coverage (NC) (Standard Definition) Test set T satisfies node coverage on graph G if and only if for every syntactically reachable node n in N, there is some path p in path(T) such that p visits n.
TR = { 0, 1, 2 }
path(t1) = [0, 1 , 2] T1={t1}
T1 satisfies node coverage on the graph 1 2

32. Structural Coverage Criteria - II
Edge coverage is slightly stronger than node coverage
Edge Coverage (EC) : TR contains each reachable path of length up to 1, inclusive, in G.
The “length up to 1” allows for graphs with one node and no edges
Edge Coverage : TR = { (0,1), (0, 2), (1, 2) }
path (t1 ) = [ 0,1,2]
path (t2 ) = [ 0, 2 ]
T2={t1, t2} Test Paths = [ 0, 1, 2 ] [ 0, 2 ]
T2 satisfies edge coverage on the graph 1 2

33. Edge-Pair Coverage
TR = { [0,1,2], [0,2,3], [0,2,4], [1,2,3], [1,2,4], [2,3,6],
[2,4,5], [2,4,6], [4,5,4], [5,4,5], [5,4,6] }
path (t1 )= [ 0, 1, 2, 3, 6 ]
path (t2 )= [ 0, 1, 2, 4, 6 ]
path (t3 )=[ 0, 2, 3, 6 ]
path (t4 )= [ 0, 2, 4, 5, 4, 5, 4, 6 ]
T={t1 , t2 , t3 , t4 } 1 2
Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 6 ] [ 0, 2, 3, 6 ]
[ 0, 2, 4, 5, 4, 5, 4, 6 ] 3 4 5 6

34. Graph Coverage for Source Code
Graph : Usually the control flow graph (CFG)
Node coverage : Execute every statement
Edge coverage : Execute every branch
Loops : Looping structures such as for loops, while loops, etc.
Data flow coverage : Augment the (CFG) control flow graph
Defs are statements that assign values to variables
Uses are statements that use variables

35. Control Flow Graphs : The if Statement
if (x < y) { y = 0; x = x + 1; } else x = y; 4 1 2 3
x >= y
x < y
x = y
y = 0
x = x + 1 3 1 2
x >= y
x < y
y = 0
x = x + 1
if (x < y) {
y = 0;
x = x + 1; }

36. Control Flow Graphs : while and for Loops1
x = 0
x = 0; while (x < y) { y = f (x, y); x = x + 1; } 2
for (x = 0; x < y; x++) {
y = f (x, y); }
x >= y
x < y
x >= y
x < y 4 3
y =f(x,y)
x = x + 1 1 2 3 5
x >= y
x < y
y = f (x, y) 4
x = x + 1

37. Control Flow Graphs : do Loop, break and continue1
x = 0
x = 0; do { y = f (x, y); x = x + 1; } while (x < y); println (y)
x = 0;
while (x < y) {
y = f (x, y);
if (y == 0)
break;
} else if y < 0)
y = y*2;
continue; }
x = x + 1;
print (y); 2 3
y =f(x,y)
y == 0 4
break 1
x = 0 5
y < 0 2
y = f (x, y)
x = x+1 6
y = y*2
continue
x >= y
x < y 7
x = x + 1 3 8

38. The Application of Graph Criteria: Source Code
It is usually defined with the control flow graph (CFG)
Node coverage is used to execute every statement
Edge coverage is used to execute every branch
Loops are used to execute looping structures such as for loops, while loops, etc.
Data flow coverage is the extended form of control flow graph (CFG) with defs and uses
defs are statements that assign values to variables
uses are statements that use variables

39. System.out.println ("length: " + length);
public static void computeStats (int [ ] numbers) {
int length = numbers.length;
double med, var, sd, mean, sum, varsum;
sum = 0;
for (int i = 0; i < length; i++)
sum += numbers [ i ]; }
med = numbers [ length / 2];
mean = sum / (double) length;
varsum = 0;
varsum = varsum + ((numbers [ I ] - mean) * (numbers [ I ] - mean));
var = varsum / ( length );
sd = Math.sqrt ( var );
System.out.println ("length: " + length);
System.out.println ("mean: " + mean);
System.out.println ("median: " + med);
System.out.println ("variance: " + var);
System.out.println ("standard deviation: " + sd); } 1 2
i = 0 3 4 5 6
i = 0 7 8
i >= length
i < length
i++

40. Post-Unit Testing Interface testing system testing
Module
Each of post-unit testing activities is performed during
The process of building an application
Unit
…..
Unit
Build
Application
on final
platform
Application
on test bed
…..
…..
Module
Interface testing
…..
system testing
installation testing
usability testing
Unit
Build
acceptance testing
Unit
Module
integration testing
…..
Unit
Regression testing throughout

41. How to select input values? Scenario I
Random values
For each input parameter we randomly select the values
Tester Experience
For each input we use our experience to select relevant values to test
Domain knowledge
We use requirements information or domain knowledge information to identify relevant values for inputs

42. How to select input values? Scenario II
Equivalence classes (for black-box testing)
We subdivide the input domain into a small number of sub-domains
The equivalence classes are created assuming that they exhibits the same behavior on all elements
Few values for each classes can be used for our testing
Boundary values (for black-box testing)
Is a test selection technique that targets faults in applications at the “boundaries” of equivalence classes
Experience indicates that programmers make mistakes in processing values at and near the boundaries of equivalence classes

43. Black-Box Testing Example: Person a rents « the Matrix» on May5
Person b rents «Gone with the Wind» on May 6
Person a returns «the Matrix» on May 10
It does not take the manner in which the application was designed or implemented
This is analogous to building an automobile and then testing it by driving it under various conditions.
BUT
If the car is built with a new design for its automatic transmission , we would be wise to use this knowledge
White box testing is required
White box tests are based on design and implementation

44. Parameter Space of computeFine()
Each element represents a pair: (Days late, Movie name)
new releases
old releases
one-of-a-kind
The penalty also depends
on the whether the movie
is a «new» or «old» release or
«one of a kind.» -5 5 10 15
Days late
e.g., (6 days late, “Casablanca”)