1. Carnegie Mellon University Java PathFinder and Model Checking of Programs Guillaume Brat, Dimitra Giannakopoulou, Klaus Havelund, Mike Lowry, Phil Oh, Corina Pasareanu, Charles Pecheur, John Penix, Willem Visser NASA Ames Research Center Automated Software Engineering Group Alex Groce, Flavio Lerda Carnegie Mellon University School of Computer Science Matt Dwyer, John Hatcliff Kansas State University Department of Computing and Information Sciences

2. Carnegie Mellon University Outline Motivation Model Checking and Testing Java PathFinder Program Model Checking

3. Carnegie Mellon University Motivation Software errors are expensive –Mars Polar Lander –Ariane 501 Software bugs in space do not fly

4. Carnegie Mellon University Model Checking Verification and Validation are crucial –Model checking has been shown effective OK Error trace or Finite-state model Temporal logic formula Model Checker  Line 5: … Line 12: … Line 15:… Line 21:… Line 25:… Line 27:… … Line 41:… Line 47:…

5. Carnegie Mellon University The dream Model Check Programs OK Error trace or Program Temporal logic formula Model Checker  Line 5: … Line 12: … Line 15:… Line 21:… Line 25:… Line 27:… … Line 41:… Line 47:… void add(Object o) { buffer[head] = o; head = (head+1)%size; } Object take() { … tail=(tail+1)%size; return buffer[tail]; }

6. Carnegie Mellon University Some of the Issues Semantics Gap –Programming Languages vs. Modeling Languages Complexity Not Automated void add(Object o) { buffer[head] = o; head = (head+1)%size; } Object take() { … tail=(tail+1)%size; return buffer[tail]; } Gap

7. Carnegie Mellon University Outline Motivation Model Checking and Testing Java PathFinder Program Model Checking

8. Carnegie Mellon University Model Checking and Testing Software complexity is too high Some of the presented methods are not sound This is not model checking anymore It is “automated” testing

9. Carnegie Mellon University The assumption Programs have bugs –Knowing that there are doesn’t mean knowing where they are Testing is not always effective –Requires a lot of knowledge of the system Model checking can be used to find bugs systematically –If no bug is found we have a non-result

10. Carnegie Mellon University Coverage Metrics Testing has coverage metrics –They tell you how good your testing is –They can be used to measure confidence Testing is not very effective for concurrent systems –You don’t just have to guess the inputs but also the timing of the inputs and the scheduling Model checking can address these issues –We are still missing metrics for concurrent programs

11. Carnegie Mellon University Bug hunting Bug hunting instead of trying to prove something correct –We can accept unsound methods –We may be able to handle real world examples –If we allow for modeling we are still not checking the correctness of the system itself

12. Carnegie Mellon University Outline Motivation Model Checking and Testing Java PathFinder Program Model Checking

13. Carnegie Mellon University Model Checking for Java Explicit State Model Checker Java Bytecode as Input Language Assertions, Deadlock Freedom, LTL Properties Source Level Error Trace Special JVM –Allows guided execution Special JVM Special JVM Model Checker Model Checker State Space State Space Classes Bytecode Classes Bytecode

14. Carnegie Mellon University Architecture Generic Verification Environment Generic C++ C C Java Search Algorithms (model checking, testing) Search Algorithms (model checking, testing) Storage Subsystem (hash table, bitstate hashing) Storage Subsystem (hash table, bitstate hashing) Special JVM Special JVM Class Loader Class Loader Expression Evaluator

15. Carnegie Mellon University Outline Motivation Model Checking and Testing Java PathFinder Program Model Checking

16. Carnegie Mellon University Programs are complex Enabling Technologies –Slicing –Abstractions –State Compression –Partial Order Reduction –Heuristic Search

17. Carnegie Mellon University Property-directed Slicing Slicing criterion automatically generated Backwards slicing automatically finds dependencies Resulting slice Slice Source program mentioned in property indirectly relevant

18. Carnegie Mellon University Abstractions Remove behaviors but preserve errors –manual or partially automated Over-approximation –Preserve correctness –Type-based abstractions –Predicate abstraction –Semi-automated

19. Carnegie Mellon University JPF Predicate Abstraction Annotation used to indicate abstractions Source-to-source translation Java PathFinder can find abstract error traces … Abstract.remove(x); Abstract.remove(y); Abstract.addBoolean(“EQ”, x==y); … Abstract.remove(x); Abstract.remove(y); Abstract.addBoolean(“EQ”, x==y); …

20. Carnegie Mellon University Choice-bounded Search An abstract trace that does not contain any non-deterministic choice correspond to at least one concrete trace Bias the model checker to look only choice- free traces

21. Carnegie Mellon University Storing the States States are complex objects –Classes, Instances, Threads, Stack Frames Classes Objects Threads Thread Stack Frame (Locals, Stack) Thread Stack Frame (Locals, Stack) Class Fields/Methods Object Fields/Methods Object Fields/Methods Class Fields/Methods

22. Carnegie Mellon University State Compression Instructions modify only part of a state Different states share common subparts X0X0X1X1 X = X + 1 X  11 Y  27 Z  75 T  45 W  11 X  11 Y  27 Z  75 T  45 W  11

23. Carnegie Mellon University State Compression Class Fields Object Fields Class Monitors Object Monitors Thread Data Stack Frames State Pools Array Compression is very effective: up to 94%!

24. Carnegie Mellon University Partial Order Reduction Do not explore “equivalent” traces Requires analysis before model checking X=11 Y=28 X=12 Y=27 X=11 Y=27 X=12 Y=28 X++ Y++X++ Y++ Access to local variable is perfect candidate for partial order reduction. Java does not provide enough information. Assume that every access to a shared object is made in mutual exclusion. Massive use of partial order reduction. Use lockset algorithm to check that mutual exclusion is actually present.

25. Carnegie Mellon University Heuristic Search Depth first search leads to very long counter examples Reactive system often exhibit periodic behavior It is possible to discover errors at a shorter depth Heuristic Search –Breadth first like state generation –Priority queue for the states based on some heuristic The challenge –Find good heuristics: Based on the property being checked Based on the program structure JPF offers an API for user-defined heuristics

26. Carnegie Mellon University An example DEOS –Real time OS from Honeywell –1500 lines of code –Subtle concurrency error Testing did not reveal it We (re)discovered the bug! –Dependency analysis –Type abstraction –Choice-free heuristic

27. Carnegie Mellon University Conclusion Model check programs poses some specific issues –Some we can deal with –Some we looked for a way around Model checking can be used for systematic testing –Can be automated –Can handle concurrent systems This is still work in progress!

28. Carnegie Mellon University Future directions Apply the same techniques to C/C++ –Next summer internship proposal Combine property and heuristic specification –Allow the model checker to direct the search Combine coverage, model checking and runtime analysis –Develop metrics –Check the system under certain assumptions