5/7/03ICSE Fragment Class Analysis for Testing of Polymorphism in Java Software Atanas (Nasko) Rountev Ohio State University Ana Milanova Barbara Ryder Rutgers University

5/7/03ICSE Testing of Polymorphism  Testing of object-oriented software  Inheritance, polymorphism, object state, etc.  Polymorphism  A variable may refer to objects of different classes  Existing work on testing of polymorphism  Basic idea: exercise all possible polymorphic bindings  Our focus: bindings at polymorphic calls

5/7/03ICSE Two Coverage Criteria for Calls  RC criterion: exercise each call site with all possible classes of the receiver object  At x.m(), x may refer to A, B, or C objects  TM criterion: exercise each call site with all possible target methods  x.m() may invoke A.m or C.m  Goal: build a tool for Java that measures RC/TM coverage achieved by test suites  The approach is also applicable to other criteria for testing of polymorphism

5/7/03ICSE Coverage Tool for Testing of Polymorphism  Step 1: analyze the tested code  What are all possible receiver classes and target methods at polymorphic call sites?  Need class analysis  Step 2: insert instrumentation  Step 3: run tests and report coverage  What were the receiver classes and target methods actually observed while running the tests? compare

5/7/03ICSE Outline  Class analysis  Class analysis of partial programs  Contribution 1: General approach that enables existing analyses to handle partial programs  Analysis imprecision  Experimental results  Contribution 2: Determining the imprecision of four popular class analyses  Conclusions and future work

5/7/03ICSE Class Analysis  Essential static analysis for object-oriented languages; large body of existing work  Given a reference variable x, what kinds of objects may x refer to?  e.g. “x may refer to instances of A, B, or C”  Uses of class analysis  Compiler optimizations: e.g. devirtualization  Software understanding and restructuring  Testing: e.g. to compute the RC/TM criteria

5/7/03ICSE Problem 1: Partial Programs  Many existing class analyses  Typically defined as whole-program analyses  Cannot be used directly when testing is done on partial programs  Analysis input  Component under test (CUT): set of classes  CUT interface that is being tested  Methods and fields against which tests have been written  Server classes of CUT classes

5/7/03ICSE Fragment Class Analysis  Goal: adapt whole-program class analyses  Solution: fragment class analysis  Step 1: Create additional code that “simulates” the possible effects of unknown external code  Located inside an artificial main  Step 2: Run a whole-program class analysis starting from main  Use the resulting solution to determine RC and TM at polymorphic call sites in the CUT

5/7/03ICSE Analysis Solution  If it is possible to write a test that executes x.m() with a receiver of class Y, the analysis solution must report Y  Need to “simulate” all possible (infinitely many) tests for the component  This property provably holds for a large category of whole-program class analyses  Flow insensitive  Context insensitive and context sensitive

5/7/03ICSE Placeholder Variables  main contains placeholders for all reference variables from unknown external code class A { public A() {...} public Y m(Z z) {...} } class B extends A { public B m2(A a) {…} } Placeholder variables main() { A ph_a; B ph_b; Y ph_y; Z ph_z; … } CUT

5/7/03ICSE Placeholder Statements class A { public A() {...} public Y m(Z z) {...} } class B extends A { public B m2(A a) {…} } Placeholder statements main() { … ph_a = new A(); ph_y = ph_a.m(ph_z); ph_b = ph_b.m2(ph_a); … ph_a = ph_b; ph_b = (B) ph_a; … }

5/7/03ICSE Placeholder Statements class A { public A() {...} public Y m(Z z) {...} } class B extends A { public B m2(A a) {…} } Placeholder statements main() { … ph_a = new A(); ph_y = ph_a.m(ph_z); ph_b = ph_b.m2(ph_a); … ph_a = ph_b; ph_b = (B) ph_a; … }

5/7/03ICSE Placeholder Statements class A { public A() {...} public Y m(Z z) {...} } class B extends A { public B m2(A a) {…} } Placeholder statements main() { … ph_a = new A(); ph_y = ph_a.m(ph_z); ph_b = ph_b.m2(ph_a); … ph_a = ph_b; ph_b = (B) ph_a; … }

5/7/03ICSE Problem 2: Analysis Imprecision  Among all classes reported by the analysis, which ones are impossible at run time?  Coverage requirements that cannot be satisfied  Goal: determine the imprecision of four fragment class analyses, derived from:  Class Hierarchy Analysis (CHA)  Rapid Type Analysis (RTA)  0-CFA  Andersen’s points-to analysis (AND)  No such data is available in previous work

5/7/03ICSE Experiments  Considered 8 CUTs  From publicly available Java libraries  CUT size: between 8 and 24 classes  Wrote tests that achieve as much RC/TM coverage as possible  Two people wrote tests independently  Compared differences and merged  Metric of imprecision: coverage reported by the tool for these tests  Report of 100% means no analysis imprecision

5/7/03ICSE RC Coverage

5/7/03ICSE Results  Bad news: CHA and RTA have significant imprecision and should be avoided  Good news: 0-CFA and AND have much lower imprecision  Close to 100% for 5 of the 8 components  Good candidates for future investigations  Open questions  Will we see the same trend for other CUTs?  What is the improvement for analyses that are theoretically more precise than 0-CFA and AND?

5/7/03ICSE Future Work  More datapoints  More class analyses (e.g. context-sensitive)  When do analyses fail?  Common sources of imprecision  Generalize to flow-sensitive analyses  Apply to existing test suites  Measure achieved coverage  Identify weaknesses and add more test cases