A Test Case + Mock Class Generator for Coding Against Interfaces

Slides:



Advertisements
Similar presentations
Satisfiability Modulo Theories (An introduction)
Advertisements

PLDI’2005Page 1June 2005 Example (C code) int double(int x) { return 2 * x; } void test_me(int x, int y) { int z = double(x); if (z==y) { if (y == x+10)
Automatic test case generation for programs that are coded against interfaces and annotations or use native code Mainul Islam Supervisor: Dr. Christoph.
Problem Solving 5 Using Java API for Searching and Sorting Applications ICS-201 Introduction to Computing II Semester 071.
1 Symbolic Execution for Model Checking and Testing Corina Păsăreanu (Kestrel) Joint work with Sarfraz Khurshid (MIT) and Willem Visser (RIACS)
1/20 Generalized Symbolic Execution for Model Checking and Testing Charngki PSWLAB Generalized Symbolic Execution for Model Checking and Testing.
SOFTWARE TESTING. INTRODUCTION  Software Testing is the process of executing a program or system with the intent of finding errors.  It involves any.
Clearly Visual Basic: Programming with Visual Basic 2008 Chapter 9 Decisions, Decisions, Decisions.
Hybrid Concolic Testing Rupak Majumdar Koushik Sen UC Los Angeles UC Berkeley.
CSE503: SOFTWARE ENGINEERING SYMBOLIC TESTING, AUTOMATED TEST GENERATION … AND MORE! David Notkin Spring 2011.
Pexxxx White Box Test Generation for
DART Directed Automated Random Testing Patrice Godefroid, Nils Klarlund, and Koushik Sen Syed Nabeel.
DART: Directed Automated Random Testing Koushik Sen University of Illinois Urbana-Champaign Joint work with Patrice Godefroid and Nils Klarlund.
Tao Xie (North Carolina State University) Nikolai Tillmann, Jonathan de Halleux, Wolfram Schulte (Microsoft Research, Redmond WA, USA)
CUTE: A Concolic Unit Testing Engine for C Technical Report Koushik SenDarko MarinovGul Agha University of Illinois Urbana-Champaign.
DySy: Dynamic Symbolic Execution for Invariant Inference.
Tao Xie Automated Software Engineering Group Department of Computer Science North Carolina State University
DEPARTMENT OF COMPUTER SCIENCE & TECHNOLOGY FACULTY OF SCIENCE & TECHNOLOGY UNIVERSITY OF UWA WELLASSA 1 CST 221 OBJECT ORIENTED PROGRAMMING(OOP) ( 2 CREDITS.
Teaching and Learning Programming and Software Engineering via Interactive Gaming Tao Xie University of Illinois at Urbana-Champaign In collaboration with.
1 Conditions Logical Expressions Selection Control Structures Chapter 5.
Tao Xie North Carolina State University Nikolai Tillmann, Peli de Halleux, Wolfram Schulte Microsoft Research.
Working with arrays (we will use an array of double as example)
CSC 211 Data Structures Lecture 13
IDENTIFYING SEMANTIC DIFFERENCES IN ASPECTJ PROGRAMS Martin Görg and Jianjun Zhao Computer Science Department, Shanghai Jiao Tong University.
Xusheng Xiao North Carolina State University CSC 720 Project Presentation 1.
jFuzz – Java based Whitebox Fuzzing
Proving Non-Termination Gupta, Henzinger, Majumdar, Rybalchenko, Ru-Gang Xu presentation by erkan.
Learning Symbolic Interfaces of Software Components Zvonimir Rakamarić.
A Test Case + Mock Class Generator for Coding Against Interfaces Mainul Islam, Christoph Csallner Software Engineering Research Center (SERC) Computer.
SOFTWARE TESTING. Introduction Software Testing is the process of executing a program or system with the intent of finding errors. It involves any activity.
CUTE: A Concolic Unit Testing Engine for C Koushik SenDarko MarinovGul Agha University of Illinois Urbana-Champaign.
( = “unknown yet”) Our novel symbolic execution framework: - extends model checking to programs that have complex inputs with unbounded (very large) data.
Week 6 MondayTuesdayWednesdayThursdayFriday Testing III Reading due Group meetings Testing IVSection ZFR due ZFR demos Progress report due Readings out.
Symbolic Execution in Software Engineering By Xusheng Xiao Xi Ge Dayoung Lee Towards Partial fulfillment for Course 707.
CSE 331 SOFTWARE DESIGN & IMPLEMENTATION SYMBOLIC TESTING Autumn 2011.
Symstra: A Framework for Generating Object-Oriented Unit Tests using Symbolic Execution Tao Xie, Darko Marinov, Wolfram Schulte, and David Notkin University.
© 2011 Pearson Education, publishing as Addison-Wesley Chapter 1: Computer Systems Presentation slides for Java Software Solutions for AP* Computer Science.
CS223: Software Engineering Lecture 26: Software Testing.
Chapter 8: Recursion Data Structures in Java: From Abstract Data Types to the Java Collections Framework by Simon Gray.
Modern Programming Tools And Techniques-I
Software Testing.
Control Flow Testing Handouts
Dynamic Symbolic Execution
Testing and Debugging.
Outline of the Chapter Basic Idea Outline of Control Flow Testing
Chapter 5 Decisions. Chapter 5 Decisions ssential uestion: How are Boolean expressions or operators used in everyday life?
Algorithm Analysis CSE 2011 Winter September 2018.
RDE: Replay DEbugging for Diagnosing Production Site Failures
Comparing Genetic Algorithm and Guided Local Search Methods
Microsoft Visual Basic 2005 BASICS
White-Box Testing Using Pex
New Ideas Track: Testing MapReduce-Style Programs Christoph Csallner, Leonidas Fegaras, Chengkai Li Computer.
Dynamic Symbolic Data Structure Repair
null, true, and false are also reserved.
CS18000: Problem Solving and Object-Oriented Programming
Computers & Programming Languages
Chapter 1: Computer Systems
Automatic Test Generation SymCrete
Homework Any Questions?.
Basics of Recursion Programming with Recursion
Focus of the Course Object-Oriented Software Development
Teaching with angr: A Symbolic Execution Curriculum and CTF
CMSC 202 Exceptions 2nd Lecture.
Diagnostic Evaluation
CUTE: A Concolic Unit Testing Engine for C
Introduction to Programming
Whitebox Testing.
CMSC 202 Exceptions 2nd Lecture.
Review for Midterm 3.
CS 151: Object-Oriented Design October 8 Class Meeting
Presentation transcript:

A Test Case + Mock Class Generator for Coding Against Interfaces Mainul Islam, Christoph Csallner Software Engineering Research Center (SERC) Computer Science and Engineering Department University of Texas at Arlington, USA February 07, 2011. Published at: 8th International Workshop on Dynamic Analysis (WODA), 2010

Mock Class Generator for Coding Against Interfaces Talk Outline Test case generation Dynamic Symbolic Execution Mock Class Motivating examples to generate Mock Class Algorithm to generate Mock Class Results Demonstration Mock Class Generator for Coding Against Interfaces

Mock Class Generator for Coding Against Interfaces Test Case Generation What is the goal behind testing ? public void Test(int x) { if (x > 0) { if (x == 1234567890) { throw new Exception(“Bug”); } Probability to reach the bug (with a random test input) is: ½ * 1/2^31 What can be a good test case to test this method ? Mock Class Generator for Coding Against Interfaces

Mock Class Generator for Coding Against Interfaces Symbolic Execution Explore all feasible execution paths Initialize the input with symbolic values and execute the program over symbolic values At conditional statements check if either of the branches can be taken (using a constraint solver) For each path we get an accumulated path condition If (s) then … else … C s true false C` = C ⋀ s C` = C ⋀⌝s Dynamic symbolic execution: Combine Concrete and Symbolic execution Mock Class Generator for Coding Against Interfaces

Dynamic Symbolic Execution by Example Choose Next Path void TestMe (int a[]) { if (a == null) return; if (a.length > 0) if (a[0] == 123) throw new Exception(“Error”); } Solve Execute Constraints to Solve Input Observed Constraints null a == null a == null a != null {} a != null && !(a.length > 0) a == null T F a != null && a.length > 0 {0} a != null && a.length > 0 && a[0] != 123 a.length > 0 T a != null && a.length > 0 && a[0] == 123 {123} a != null && a.length > 0 && a[0] == 123 F a[0] == 123 T F Done: No path left This example is taken from the slides of Nikolai Tillman (with permission)

Introduction to Mock Class public interface A { // … } public class C { public void foo (A a, int x) { Mock Class Generator for Coding Against Interfaces

When we might need to generate mock-classes ? At the initial stage of any development Mock Class Generator for Coding Against Interfaces

Mock Class Generator for Coding Against Interfaces Why is this important ? Test case generation is important Interfaces are very important Existing techniques are not very good at this Pex - can generate an instance of a class that implements an interface that is used in the code * * http://research.microsoft.com/en-us/projects/pex/ Mock Class Generator for Coding Against Interfaces

Mock Class Generator for Coding Against Interfaces Goal: Support interfaces in test case generation for object-oriented programs Mock Class Generator for Coding Against Interfaces

Mock Class Generator for Coding Against Interfaces Motivating Example 1 public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) { B b = (B) a; int z = b.m2(x); } public interface A { int m1(int x); } public interface B { int m2(int x); To reach this block of code, ‘a’ must be an instance of A, as well as an instance of B Mock Class Generator for Coding Against Interfaces

Mock Class Generator for Coding Against Interfaces Motivating Example 2 public class K implements A, B { // … } public class C { public void bar (A a, int x) { if (a == null) return; if ( !(a instanceof B) ) { B b = (B) a; int z = b.m2(x); public interface A { int m1 (int x); } public interface B { int m2 (int x); K implements all of the interfaces referred by the bar method, but fails reach the code block Mock Class Generator for Coding Against Interfaces

Mock Class Generator for Coding Against Interfaces Framework DSC – Dynamic Symbolic Execution Engine (for Java) Z3 – Constraint Solver http://research.microsoft.com/en-us/um/redmod/projects/z3/ Mock Class Generator for Coding Against Interfaces

Our approach to generate Mock Class Determining the type of the Mock Class Generate meaningful method body if needed Mock Class Generator for Coding Against Interfaces

Sub-/Supertype relation in Java Java defines a binary subtype relation If type B implements/extends type A then, - A is a direct super type of B - B is a direct sub type of A Reflexive: A is also a subtype of itself Transitive: if B is a subtype of A and C is a subtype of B then C is also a subtype of A Mock Class Generator for Coding Against Interfaces

Sub-/Supertype relation in Java (Cont.) A class has one direct class super type and arbitrarily many interface super types. Exceptions: type object – has no direct super type type null – has no direct sub type Mock Class Generator for Coding Against Interfaces

Subtype constraints Object public interface A { int m1 (int x); } public interface B { int m2(int x); public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) { B b = (B) a; int z = b.m2(x); Object A B C A B C M null Initial types in the system null A desired solution: with new type M Constraints: type(a) subtypeof A type(a) != null type type(a) subtypeof B Mock Class Generator for Coding Against Interfaces

Subtype Relation Matrix public interface A { int m1 (int x); } public interface B { int m2(int x); public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) { B b = (B) a; int z = b.m2(x); Null Object A B C x 1 2 3 4 Null Object A B C M x 1 2 3 4 5 Null Object A B C M x 1 2 3 4 5 MA MB MC Null Object x 1 Solution: MA = true MB = true MC = false Mock Class Generator for Coding Against Interfaces

Algorithm: Generating Mock Class Program execution Initialization: NO_MC = 0 MAX_MC = 2 Build Subtype Relation matrix (Generate the constraints) Yes Found Solution? No Solution not found NO_MC < MAX_MC No Yes So, here is how our algorithm works. Once in a while the normal execution of the program is suspended and the correctness of the data structure is checked. If this check by calling repOk() method fails, our By the way, in this slide I-repOk means Instrumented repOk Yes Introduce a new Mock Class, N0_MC++ Encode the Mock Class in the type system Generate Test Case Dynamic Symbolic Data Structure Repair

Generating Method Body: for Example 1 public interface A { int m1 (int x); } public interface B { int m2(int x); public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) { B b = (B) a; int z = b.m2(x); if (z >= 10) { //… public class M implements A, B { int m1 (int x){ return 0; } int m2(int x){ Outline of the source code for Mock Class M return 10; Mock Class Generator for Coding Against Interfaces

Results We have run our solution on JMS (Java messaging system) These are all hand-written test cases Interfaces used from JMS Number of goals Number of goals reached w/o mock-classes Number of goals reached with mock-classes Message 6 ByteMessage 5 MapMessage 4 StreamMessage TextMessage ObjectMessage Mock Class Generator for Coding Against Interfaces

Comparison with Pex: Motivating Example 1 public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) { B b = (B) a; int z = b.m2(x); } public interface A { int m1(int x); } public interface B { int m2(int x); Goal 1 Goal 2 PEX reaches 1/2 goal(s) DSC reaches 2/2 goal(s) Mock Class Generator for Coding Against Interfaces

Mock Class Generator for Coding Against Interfaces Future Work Extend the solution to support more types, e.g. ‘Array’ Determine a upper bound to the number of mock classes needed to solve a system Evaluation Mock Class Generator for Coding Against Interfaces

Mock Class Generator for Coding Against Interfaces Future Work (Cont.) Extend method body generation for complex types public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) { B b = (B) a; int z = b.m2(x); if (z >= 10) { //… } public class M implements A, B { int m1 (int x){ return 0; } int m2(int x){ Outline of the source code for Mock Class M return 10; Mock Class Generator for Coding Against Interfaces

Mock Class Generator for Coding Against Interfaces Demo Mock Class Generator for Coding Against Interfaces

Mock Class Generator for Coding Against Interfaces Thank you! Mock Class Generator for Coding Against Interfaces