1. Test Case Purification for Improving Fault Localization
Jifeng Xuan Martin monperrus
Source: aCM FSE
G 처그

2. Context Abstract Introduction Motivation Test Case Atomization
Test Case Slicing
Rank Refinement
Experimental Result
Conclusion

3. Abstract
Failing test cases and their assertions form test oracles for the failing behavior of the system under analysis.
Test case purification is to separate existing test cases in to small fractions (purified test cases) and to enhance the oracles to further localize faults.
Combining with an original fault localization technique (e.g., Tarantula).
Test case purification results in better ranking the program statements.

4. Introduction
The test oracle in a test case is implemented as a set of executable assertions for verifying the correctness of the program behavior.
Each test case includes 5.8 assertions on average.
If an assertion in a test case is violated, the unit test framework aborts the execution of this test case and report the test result (i.e., the test case is failed)
Our key intuition is that recovering the execution of those omitted assertions can lead to more test cases and further enhance the ability of fault localiztion.

5. Introduction
The goal of test case purification is to generate purified versions of failing test cases.
Which include only one assertion per test and excludes unrelated statements of this assertion.

6. Introduction
Test case purification for fault localization is consists of three major phases:
Test case atomization
Test case slicing
Rank refinement
Test case atozimation generates a set of single assertion test cases for each failed test case.
Test case slicing removes the unrelated statements in all the failing single assertion test cases
Rank refinement combines the spectra of purified test cases with an existing fault localization technique (e.g., Tarantula) and sorts the statements as the final result.

7. Introduction
We evaluate our work on six real world open source Java projects with 1800 seeded bugs.
We compare our results with six mature fault localization technique.
Test case purification can effectively improve the results of existing techniques.
Test case purification on Tarantula obtains the best result among all the techniques we have considered.

8. Motivation

9. Test Case Atomization
Given failing test case which has k assertions, create k copies for this test case.
Transform k-1 assertions into regular test case statements for each copy.

10. Test Case Atomization
To transform an assertion into regular test case statement, we surround this assertion with try-catch structure

11. Test Case Atomization
After that, compile and execute all generated test cases.
Records failing test cases and broken statement that is either an assertion or a statement which throws an unexpected error.

12. Test Case Slicing
The goal of test case slicing is to generate purified test cases before collecting their spectra.
Spectra mean the execution trace for each purified test case.
Dynamic slicing
A slicing criterion is defined as a pair <b, V>, where b is broken statement in the target program and V is set of variables to be observed at b.
Compared static slicing , dynamic slicing may lead to more removal statements.

13. Test Case Slicing
After that, compile and execute all purified test cases.
Finally, collect execution trace for each purified test case.

14. Rank Refinement
The goal of rank refinement is to re-rank the statement by an existing fault localization technique with the spectra in the phase of test case slicing.
The statements covered by failing test case is re-ranked.
Let S be a set of the statements.
For statement s ∈ S, get a ratio of executions by purified test cases.
Where βef(s) and βnf(s) is the number of test cases covering and non covering s.

15. Rank Refinement
For statement s ∈ S, normalize suspiciousness of statements calculated by existing fault localizing techniques (e.g., Tarantula).
Where susp(s) is the suspiciousness value, min(S) and max(S) denotes the minimum susp/value and maximum susp/value of all the statements in S.

16. Rank Refinement
Refine the ranking of a statement s ∈ S by combining ratio(s) and norm(s).
where norm(s) is the normalized suspiciousness value, ratio(s) is the ratio of execution by purified test cases.

17. Framework of Test Case Purification

18. Subject Programs Six open source programs for our experiments
We compute the size metric (Source Line of Code - SLoC) with CLOC.
The subject programs in our selection are provided with large test suites written in JUnit

19. Subject Programs
We use existing work [14],[31] and use mutation testing tools to create faulty version
A mutant of a program is a copy of the original program with a single change.
Mutant may contain one change of negating a conditional statement.

20. Experimental Result
The number changes of the suspiciousness score of the faulty statement in faults after applying the approach.
Better fault localization on 18 to 43% of faults.
Worse on only 1.3 to 2.4%of faults
Tarantula-Purification performs better than Tarantula on 43.28% of the faults with an average fault-localization improvement of statements.

21. Experimental Result
Presents the wasted efforts with or without applying test case purification on 1800 faults
The wasted effort measured with the absolute number of statements to be examined before finding the faulty one.
Purification test case improves the wasted effort from statements (Tarantula) to statements (Tarantula-purification).

22. Experimental Result Computation time.
The paper claims that the computation time of test case purification is acceptable since the whole process can be executed automatically.

23. Conclusion
The paper proposed a test case purification approach for improving existing fault localizing techniques.
In the experimental result, the authors have shown that the fault localization applying the test case purification performs better on 18-43% of faults while performing worse on only % of faults.