2. Introduction Previous work Test Suite Minimization
reduce the total number of tests to execute by identifying duplicate tests
Typically using coverage measures
Test Suite Prioritization

3. Introduction Previous work Test Suite Minimization
Test Suite Prioritization
Re-order test cases by a given priority
Useful to identify faults earlier but does not actually reduce the total execution time
Example of Test Suite Prioritization:
When changes occur, the most relevant test cases to the changes are executed first

4. Unit Test Virtualization
Rather than focusing on the number of test cases, minimize the total amount of time on execution instead

5. Test Suite Execution Loop
In large applications, each test executes in its own process, rather than in the same process
The isolation is to prevent the tests cases in one suite from having side effects on others

6. Test Suite Execution Loop

7. Test Suite Execution Loop
Removing the initialization from the loop, instead perform it only at the beginning of the test suite
Fork the initialized process for test cases(just like pre-test and post-test written by programmers)

8. Key Insight No need to reinitialized the entire application
Maintain the isolation

9. Unit Test Virtualization
Automatically identifies the code segments that may create side-effects and isolates them in a container similar to a light weight virtual machine

10. VMVM
Transforms application byte code directly without requiring modification to the JVM or application source code
Integrated with Junit, ant and maven

11. Motivation MQ1: Do developers isolate their unit tests?
MQ2: Why do developers isolate their unit tests?
MQ3: What is the overhead of the isolation technique that developers use?

12. MQ1: Do developers isolate tests?

13. Motivation MQ1: Do developers isolate their unit tests?
MQ2: Why do developers isolate their unit tests?
MQ3: What is the overhead of the isolation technique that developers use?

14. MQ2: Why isolate tests? In order to make the environment “clean”
Example:

15. Motivation MQ1: Do developers isolate their unit tests?
MQ2: Why do developers isolate their unit tests?
MQ3: What is the overhead of the isolation technique that developers use?

16. MQ3: The overhead of isolation
Time to execute in separate process
Time to execute in the same process

17. Approach
Determines which parts of the application will need to be reset before the next execution
Reset the affected memory at the end of each test execution

18. Approach Static analysis Dynamic analysis Runtime performance
Precision

19. Static Analysis
Placing each addressed memory region into one of the two categories: Ms(“safe”) and Mu(“unknown”)
Area in Ms means it is never accessed, or it is always reset to its starting condition at last

20. Dynamic Analysis
Areas in Mu are left to a runtime checker to identify which one is written but not cleared

21. Implementation No modification to JVM
Architecturally, VMVM consists of a static bytecode instrumenter and a dynamic runtime
Static analyzer and instrmenter
Identify and insert codes for reinitialization
Dynamic runtime
Tracks what needs to be reset and perform the reinitialization

22. Static Analysis
For static fields, those not only hold a constant value, but its value must not be dependent on any non-constant values can be placed in Ms
Instead of reinitializing individual static fields of a class, the entire class needs to be handle
No field in Mu, no side effects to other classes

23. Static Instrumentation
Preface each access with a check to see whether needs reinitialization by force
Trigger the initialization when
Creating a new instance of a class
Accessing a static method of the class
Accessing a static field of the class
Explicitly requested via reflection

24. Logging Class Initializations
Insert logging code in <clinit>
Two places
Initialization state of the class in a static field for quick lookup on determining whether the class needs reinitialization
Index that contains all the classes that have been initialized to see whether the reinitialization occurs

25. Dynamically Reinitializing Classes
Clears the flag that indicates that the class has been initialized so that next time the class is accessed, the initialization process is called
Only application code(not Java core library set) are instrumented(causes leaked data)
Using scripts to see possible candidates and verifying by hand to identify false positives
Logging the values

26. Test Automation Integration
Plugs directly into Junit and build automation systems ant and maven

27. Supporting Class Reinitialization
Housekeeping
Renaming <clinit> to configurable name(<_vmvm_clinit_>)
Adding a shell <clinit> including logging code segments and former <clinit>
Supporting ‘final’ field
Remove final field
Check to ensure that they do not reference a previously final field

28. Experimental Results RQ1: RQ2: RQ3:
VMVM compared to TSM on performance and fault-finding ability
RQ2:
VMVM compared to creating a new process for each test on performance
RQ3:
VMVM compared to traditional isolation on fault-finding ability

29. RQ1
RF = 0

30. RQ2 & RQ3

31. Limitation Experiment results are not convincing enough Based on Java
The selection of subjects used can be a potential threat
RT & RS
Based on Java
Only isolate in-memory state(not on disk or in databases)