1. B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE TESTING OF JAVA PROGRAMS

2. STRUCTURAL SOFTWARE TESTING Assure quality, robust software Collect coverage information about the program E.g., branch coverage records which edges are hit Uses program instrumentation

3. ..Continued.. Automatically apply multiple test strategies Multiple languages and platforms Handle large programs

4. Static instrumentation approaches Not scalable: Instrumentation left in place Time, memory Inflexible: Only certain tests, languages, platforms Generally not detailed

5. Approach Specification driven: User written test Test plans: Recipe of how & where to test Path specific: Instrument only what is needed Dynamic: Insert & remove instrumentation

6. TEST FRAMEWORK INSTANCE FOR JAVA Wizard for creating the test specification From test specification determine how to test Test plan uses instrumentation primitives Source level coverage results

7. Apply different tests to various code regions. may be applied with different coverage criteria A GUI for What tests to apply, where to apply, under what conditions Select code regions Conditions can be specified TEST SPECIFICATION

8. public class Example { public int foo(int x) { if(x==200) x = x+100; else x = x-100; return x; } DEFINITIONS { NAME: X, REGION_D, LOCATION: FILE Example.java { CLASS example, METHOD foo } } BODY { DO BRANCH_TEST ON REGION X UNTIL:90% } Example test specification

9. TEST PLANNER Test plan: Where and how to test program Test table Where to instrument in the target machine code Test information (results) Branch coverage: when to insert & remove instrumentation, records which edges are hit/not hit

10. Test payload – What to do at an instrumentation point – Target machine code – Branch coverage: Update edge coverage

11. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3N,N 24N 34N 41N Instrumentation Hit

12. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3N,N 24N 34N 41N Instrumentation Hit

13. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3N,N 24N 34N 41N Instrumentation Hit

14. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3N,N 24N 34N 41N Instrumentation Hit

15. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3Y,N 24N 34N 41N Instrumentation Hit

16. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3Y,N 24N 34N 41N Instrumentation Hit

17. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3Y,N 24Y 34N 41N Instrumentation Hit

18. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3Y,N 24Y 34N 41N Instrumentation Hit

19. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3Y,N 24Y 34N 41Y Instrumentation Hit

20. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3Y,Y 24Y 34N 41Y Instrumentation Hit

21. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3Y,Y 24Y 34N 41Y Instrumentation Hit

22. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3Y,Y 24Y 34Y 41Y Instrumentation Hit

23. BRANCH COVERAGE EXAMPLE Test Table Test Payload Mark edge hit Insert at next point Remove instrumentation 1 1 2 2 3 3 4 4 BlockNextHit 12,3Y,Y 24Y 34Y 41Y Instrumentation Hit

24. Example CFG with block as its own successor

25. TEST VIRTUAL MACHINE Provides functionality to insert and remove instrumentation at run-time. Implements an interface for inserting and removing instrumentation with fast breakpoints*. API provides primitives, such as the placement of successor breakpoints, storing test- specific data removal of breakpoints, for constructing fast breakpoints with varying payloads

26. TEST ANALYZER Displays the results of tests For branch coverage, the analyzer displays the CFG for a method CFG for target machine code

27. Observations SoftTest significantly reduces the number of tests resulting in savings in scripting effort. Significant impact on the acceptability of code coverage. Provides cost effective approach to testing. Low performance and memory overhead. For the benchmarks, the memory overhead was 178 to 822 bytes for the test table. The performance overhead for coverage varied from 1% to 5.5% of the total execution time.

28. Preliminary Results: Branch Coverage Testing Overhead includes: 1.Test Planner 2.Instrumentation Run-time performance overhead [3]

29. REFERENCES [1]http://veryoldwww.cs.pitt.edu/copa/eclipse.html [2]SoftTest: A Framework for Software Testing of Java Programs [3]http://www.cs.pitt.edu/coco/papers/SoftTest- Etx-2003 [4]http://www.softwaretestingmentor.com/types-of- testing/static-testing.php

30. QUESTIONS?