2. Mutation Testing
Faults are introduced into the program by creating many versions of the program called mutants.
Each mutant contains a single fault.
Test cases are applied to the original program and to the mutant program.
The goal is to cause the mutant program to fail, thus demonstrating the effectiveness of the test case.

3. Test Case Adequacy
A test case is adequate if it is useful in detecting faults in a program.
A test case can be shown to be adequate by finding at least one mutant program that generates a different output than does the original program for that test case.
If the original program and all mutant programs generate the same output, the test case is inadequate.

4. Mutant Programs
Mutation testing involves the creation of a set of mutant programs of the program being tested.
Each mutant differs from the original program by one mutation.
A mutation is a single syntactic change that is made to a program statement.

5. Example of a Program Mutation
1 int max(int x, int y)
2 {
3 int mx = x;
4 if (x > y)
5 mx = x;
6 else
7 mx = y;
8 return mx;
9 }
1 int max(int x, int y)
2 {
3 int mx = x;
4 if (x < y)
5 mx = x;
6 else
7 mx = y;
8 return mx;
9 }

6. Categories of Mutation Operators
Operand Replacement Operators:
Replace a single operand with another operand or constant. E.g.,
if (5 > y) Replacing x by constant 5.
if (x > 5) Replacing y by constant 5.
if (y > x) Replacing x and y with each other.
E.g., if all operators are {+,-,*,**,/} then the following expression a = b * (c - d) will generate 8 mutants:
4 by replacing *
4 by replacing -.

7. Categories of Mutation Operators
Expression Modification Operators:
Replace an operator or insert new operators. E.g.,
if (x == y)
if (x >= y) Replacing == by >=.
if (x == ++y) Inserting ++.

8. Categories of Mutation Operators
Statement Modification Operators:
E.g.,
Delete the else part of the if-else statement.
Delete the entire if-else statement.
Replace line 3 by a return statement.

9. Example of Testing By Mutation
function MAX(M<N:INTEGER)
return INTEGER is
begin
if M>N then
return M;
else
return N;
end if:
end MAX;
First test data set--M=1, N=2
the original funtion returns 2
five mutants: replace”>“ operator in if satements by (>,<,<=or=)
executing each mutant:
Mutants Outputs Comparison
if M>=N then 2 live
if M<N then 1 dead
if M<=N then 1 dead
if M=N then 2 live
if M< >N then 1 dead
adding test data M=2, N=1 will eliminate the latter live mutant, but the former live mutant remains live because it is equivalent to the original function. No test data can eliminate it.

10. Mutation Testing Example
1. What is an equivalent mutant and why are equivalent mutants a problem?
2. For the program fragment x= x+y; give five examples of mutants which are equivalent and five which are not equivalent
3. Give examples of test cases which kill your five non-equivalent mutants
4. Now make up some simple program fragments and try to think of some more stubborn mutants of these fragments. That is, mutants which are hard to kill but which are not equivalent.

11. Equivalent mutant
An equivalent mutant is one which cannot be killed by any test case because it gives identical output to the original program for every possible input.
That is, it is possible to create a syntactic variant of a program (a mutant) which is syntactically different to the original, but which behaves identically to it.
This is a problem because one cannot tell whether the failure of test data to kill a mutant stems from inadequate test data or the fact that the mutant is equivalent.

12. Equivalent mutants x = x + y ; is equivalent to:- x = y + x ;
x = -z + x + y + z ;
x = x+ (1==1)?y:z ;
x = 2*x + y – x ;
x = x + y + 0 ;

13. Non Equivalent mutants
x = x + y ;
is not equivalent to:-
x = y * x ;
x = x + z ;
x = x - y;
x = x + 1 ;
x = 2 + y ;

14. Killing the Non Equivalent mutants
x = y * x ; is killed by anything other than y=2, x=2
x = x + z ; is killed by any non-equal values for y and z
x = x - y; is killed by any non zero value for y
x = x + 1 ; is killed by any value of y other than 1
x = 2 + y ; is killed by any value of x other than 2

15. Stubborn mutant Consider if (a==b) then z=1; else z=2;
Suppose we mutate this to
if (a==b+1) then z=1; else z=2;
This is only killed by test cases where a and b are equal or where b one more than a.

16. Program-based Grammars
Original Method
int Min (int A, int B) {
int minVal;
minVal = A;
if (B < A)
minVal = B; }
return (minVal);
} // end Min
With Embedded Mutants
int Min (int A, int B) {
int minVal;
minVal = A;
∆ 1 minVal = B;
if (B < A)
∆ 2 if (B > A)
∆ 3 if (B < minVal)
minVal = B;
∆ Bomb ();
∆ minVal = A;
∆ minVal = failOnZero (B); }
return (minVal);
} // end Min
Replace one variable with another
Changes operator
Immediate runtime failure … if reached
6 mutants
Each represents a separate program
Immediate runtime failure if B==0 else does nothing

17. Syntax-Based Coverage Criteria
Mutation Coverage (MC) : For each m  M, TR contains exactly one requirement, to kill m.
The RIP model:
Reachability : The test causes the faulty statement to be reached (in mutation – the mutated statement)
Infection : The test causes the faulty statement to result in an incorrect state
Propagation : The incorrect state propagates to incorrect output
The RIP model leads to two variants of mutation coverage …

18. Syntax-Based Coverage Criteria
1) Strongly Killing Mutants:
Given a mutant m  M for a program P and a test t, t is said to strongly kill m if and only if the output of t on P is different from the output of t on m
2) Weakly Killing Mutants:
Given a mutant m  M that modifies a location l in a program P, and a test t, t is said to weakly kill m if and only if the state of the execution of P on t is different from the state of the execution of m immediately on t after l
Weakly killing satisfies reachability and infection, but not propagation

19. Example consider the following C++ code fragment:
if (a && b)
{ c = 1; }
else { c = 0; }
The condition mutation operator would replace && with || and produce the following mutant:
if (a || b)
Now, for the test to kill this mutant, the following three conditions should be met:
A test must reach the mutated statement.
Test input data should infect the program state by causing different program states for the mutant and the original program. For example, a test with a = 1 and b = 0would do this.
The incorrect program state (the value of 'c') must propagate to the program's output and be checked by the test.
These conditions are collectively called the RIP model.

20. Weak mutation testing (or weak mutation coverage) requires that only the first and second conditions are satisfied. Strong mutation testing requires that all three conditions are satisfied. Strong mutation is more powerful, since it ensures that the test suite can really catch the problems. Weak mutation is closely related to code coverage methods. It requires much less computing power to ensure that the test suite satisfies weak mutation testing than strong mutation testing.
However, there are cases where it is not possible to find a test case that could kill this mutant. The resulting program is behaviorally equivalent to the original one. Such mutants are called equivalent mutants.
Equivalent mutants detection is one of biggest obstacles for practical usage of mutation testing. The effort needed to check if mutants are equivalent or not can be very high even for small programs. 

21. Weak Mutation
Weak Mutation Coverage (WMC) : For each m  M, TR contains exactly one requirement, to weakly kill m.
“Weak mutation” is so named because it is easier to kill mutants under this assumption
Weak mutation also requires less analysis
A few mutants can be killed under weak mutation but not under strong mutation (no propagation)
In practice, there is little difference

22. Mutant 1 in the Min( ) example is:
Weak Mutation Example
Mutant 1 in the Min( ) example is:
minVal = A;
∆ 1 minVal = B;
if (B < A)
minVal = B;
The complete test specification to kill mutant 1:
Reachability : true // Always get to that statement
Infection : A ≠ B
Propagation: (B < A) = false // Skip the next assignment
Full Test Specification : true  (A ≠ B)  ((B < A) = false)
≡ (A ≠ B)  (B ≥A)
≡ (B > A)
(A = 5, B = 3) will weakly kill mutant 1, but not strongly

23. Equivalent Mutation Example
Mutant 3 in the Min() example is equivalent:
minVal = A;
if (B < A)
∆ 3 if (B < minVal)
The infection condition is “(B < A) != (B < minVal)”
However, the previous statement was “minVal = A”
Substituting, we get: “(B < A) != (B < A)”
This is a logical contradiction !
Thus no input can kill this mutant

24. Strong Versus Weak Mutation
1 boolean isEven (int X)
2 {
if (X < 0)
X = 0 - X;
∆ X = 0;
if (double) (X/2) == ((double) X) / 2.0
return (true);
else
return (false);
9 }
Reachability : X < 0
Infection : X != 0
(X = -6) will kill mutant 4 under weak mutation
Propagation :
((double) ((0-X)/2) == ((double) 0-X) / 2.0)
!= ((double) (0/2) == ((double) 0) / 2.0)
That is, X is not even …
Thus (X = -6) does not kill the mutant under strong mutation