Introduction Previous work Test Suite Minimization reduce the total number of tests to execute by identifying duplicate tests Typically using coverage measures Test Suite Prioritization

Introduction Previous work Test Suite Minimization Test Suite Prioritization Re-order test cases by a given priority Useful to identify faults earlier but does not actually reduce the total execution time Example of Test Suite Prioritization: When changes occur, the most relevant test cases to the changes are executed first

Unit Test Virtualization Rather than focusing on the number of test cases, minimize the total amount of time on execution instead

Test Suite Execution Loop In large applications, each test executes in its own process, rather than in the same process The isolation is to prevent the tests cases in one suite from having side effects on others

Test Suite Execution Loop

Test Suite Execution Loop Removing the initialization from the loop, instead perform it only at the beginning of the test suite Fork the initialized process for test cases(just like pre-test and post-test written by programmers)

Key Insight No need to reinitialized the entire application Maintain the isolation

Unit Test Virtualization Automatically identifies the code segments that may create side-effects and isolates them in a container similar to a light weight virtual machine

VMVM Transforms application byte code directly without requiring modification to the JVM or application source code Integrated with Junit, ant and maven

Motivation MQ1: Do developers isolate their unit tests? MQ2: Why do developers isolate their unit tests? MQ3: What is the overhead of the isolation technique that developers use?

MQ1: Do developers isolate tests?

Motivation MQ1: Do developers isolate their unit tests? MQ2: Why do developers isolate their unit tests? MQ3: What is the overhead of the isolation technique that developers use?

MQ2: Why isolate tests? In order to make the environment “clean” Example:

Motivation MQ1: Do developers isolate their unit tests? MQ2: Why do developers isolate their unit tests? MQ3: What is the overhead of the isolation technique that developers use?

MQ3: The overhead of isolation Time to execute in separate process Time to execute in the same process

Approach Determines which parts of the application will need to be reset before the next execution Reset the affected memory at the end of each test execution

Approach Static analysis Dynamic analysis Runtime performance Precision

Static Analysis Placing each addressed memory region into one of the two categories: Ms(“safe”) and Mu(“unknown”) Area in Ms means it is never accessed, or it is always reset to its starting condition at last

Dynamic Analysis Areas in Mu are left to a runtime checker to identify which one is written but not cleared

Implementation No modification to JVM Architecturally, VMVM consists of a static bytecode instrumenter and a dynamic runtime Static analyzer and instrmenter Identify and insert codes for reinitialization Dynamic runtime Tracks what needs to be reset and perform the reinitialization

Static Analysis For static fields, those not only hold a constant value, but its value must not be dependent on any non-constant values can be placed in Ms Instead of reinitializing individual static fields of a class, the entire class needs to be handle No field in Mu, no side effects to other classes

Static Instrumentation Preface each access with a check to see whether needs reinitialization by force Trigger the initialization when Creating a new instance of a class Accessing a static method of the class Accessing a static field of the class Explicitly requested via reflection

Logging Class Initializations Insert logging code in <clinit> Two places Initialization state of the class in a static field for quick lookup on determining whether the class needs reinitialization Index that contains all the classes that have been initialized to see whether the reinitialization occurs

Dynamically Reinitializing Classes Clears the flag that indicates that the class has been initialized so that next time the class is accessed, the initialization process is called Only application code(not Java core library set) are instrumented(causes leaked data) Using scripts to see possible candidates and verifying by hand to identify false positives Logging the values

Test Automation Integration Plugs directly into Junit and build automation systems ant and maven

Supporting Class Reinitialization Housekeeping Renaming <clinit> to configurable name(<_vmvm_clinit_>) Adding a shell <clinit> including logging code segments and former <clinit> Supporting ‘final’ field Remove final field Check to ensure that they do not reference a previously final field

Experimental Results RQ1: RQ2: RQ3: VMVM compared to TSM on performance and fault-finding ability RQ2: VMVM compared to creating a new process for each test on performance RQ3: VMVM compared to traditional isolation on fault-finding ability

RQ1 RF = 0

RQ2 & RQ3

Limitation Experiment results are not convincing enough Based on Java The selection of subjects used can be a potential threat RT & RS Based on Java Only isolate in-memory state(not on disk or in databases)