Mutation Testing Presented by Sharath Kumar Garlapati Vinesh Thummala.

Learning Objectives What is Mutation testing History of Mutation Traditional Process of Mutation testing Kinds of Mutations Mutation Operators An Example for Mutation Testing Pros and Cons Conclusion

What is Mutation testing? Mutation Testing is a testing technique that focuses on measuring the adequacy of test cases Mutation Testing is NOT a testing strategy like path or data- flow testing. It does not outline test data selection criteria.

History of Mutation Proposed by Richard Lipton as a student in 1971. Developed and published by DeMillo, Lipton and Sayward in late 70’s. Implemented as a testing tool by Timothy Budd in 1980 from Yale University.

Mutation Process [2]

Mutation Process The process, given program P and test suite T, is as follows: Systematically apply mutations to program P to obtain a sequence,P1, P2,... Pn of mutants of P. Each mutant is derived by applying a single mutation operation to P. Run the test suite T on each of the mutants T is said to kill mutant Pj, if it detects an error. If k out of n mutants killed, the adequacy of T is measured by the quotient k/n implies T is mutation adequate, if k = n.

Kinds of Mutations Value Mutation - Involves changing the values of constants or parameters(by adding or subtracting values etc). Decision Mutation- Involves in modifying conditions to reflect potential slips(replacing a > by a < in a comparison). Statement Mutation - Deleting or Swapping the lines of codes.

Mutation Operators Statement deletion Boolean expression changes (true and false) Arithmetic operation changes (+ and *, - and /) Boolean operator changes (> and >=, == and <=) Declared in the same scope variables changes (variable types should be the same)

Class level mutation operators Polymorphic Method Overloading Method Overriding/Hiding in Inheritance Field Variable Hiding in Inheritance Information Hiding (Access Control) Static/Dynamic States of Objects Exception Handling

Polymorphic mutations class T { … } interface K { … } class S extends T implements K { … } class U extends S { … } The original code: S s = new S(); Mutants: T s = new S(); K s = new S(); S s = new T(); S s = new U();

Method overloading mutations public LogMessage(int level, String logKey, Object[]inserts){…} public LogMessage(int level, String logKey, Object insert) {…} Mutant: public LogMessage(String logKey, int level, Object[]inserts{…}

Method Overriding/Hiding in Inheritance public class Animal { public void speak(){..} public void bite(){..} } public class Cat extends Animal { public void speak() {..} Mutant: //public void speak() {..}

Field Variable Hiding in Inheritance public class Animal { public String kind; public int legCount; } public class Cat extends Animal { Mutant: //public int legCount;

Information Hiding (Access Control) The original code: protected Address address; Mutants: public Address address; private Address address; Address address; //default (no access modifier defined)

Static/Dynamic States of Objects The original code: public static int VALUE = 100; private String s; Mutants: public int VALUE = 100; //static is removed. private static String s; //static is added.

Exception Handling The original code: String formatMsg(LogMessage msg){ … try { … } catch(MissingResourceException mre){… } … Mutant: String formatMsg(LogMessage msg) throws MissingResourceException { … //try-catch block removed

Weak Mutation Compares the internal states of the mutant and original program immediately after execution of the mutated portion of the program. Requires much less computing power to ensure that the test suite satisfies weak mutation testing.

Strong Mutation When strong mutation testing is performed, the values produced by mutated code and changes (or not) the output of the module or subroutine. Requires much more computing power.

Automation of Mutation Testing List of tools available – Ninja Turtles- .NET mutation testing tool Mutagenesis- PHP mutation testing framework Heckle- Ruby Mutation Testing Tool Jester- Mutation Testing Tool for Java

Advantages It is a powerful approach to attain high testing coverage of the source program. This testing is capable of comprehensively testing the mutant program. Mutation testing brings a good level of error detection to the software developer. Customers are benefited from this testing by getting a more reliable and stable system.

DisAdvantages Costly and Time consuming. Many test cases are required. Requires a lot of testing before a dependable data is obtained. Needs an automated tool to reduce testing time. Not an applicable method for black box testing. Very complicated to use without an automated tool.

Conclusion Not just a testing method, more of an analytical method. An effective program for improving the quality of testing software but its widespread use is prevented by the difficulties encountered in using them. It would be very helpful to the developers of complex software, if testing methods could be improved

Reference [1] Anguswamy,R., Frakes,W., A Comparison of Database Fault Detection Capabilities Using Mutation Testing, Proceedings of ESEM,12,(Sept,2012,NY,USA),323-326. [2] Praphamontripong,U., Offutt,J., Applying Mutation Testing to Web Applications, Proceeding of ICSTW ,10 ,(Apr,2010, Paris, [3] Schuler,D., Zeller,A., Efficient Mutation Testing for Java, Proceedings of ESEC/FSE, 06 (Aug. 2009., NY, USA),297-298. [4] Madeyski, L., Nguyen, Q., Problems of Mutation Testing and Higher Order Mutation Testing , Proceedings of ICCSAMA, 12,(2014, Switzerland),157-172. [5] Khurshid, S., Marinov, D., Zhang, L., Zhang,Lu., Regression Mutation Testing, Proceeding of ISSTA ,12,(July.2012, MN, USA),331-341.

Thank You