1. Efficient Software Model Checking of Soundness of Type Systems Michael Roberson, Melanie Harries, Paul T. Darga, Chandrasekhar Boyapati University of Michigan

2. Type Soundness All Possible Programs defined by the syntax Good Programs do not cause certain kinds of runtime errors e.g. memory errors, race conditions, etc. defined by the semantics

3. Type Soundness Type System designed to statically prevent errors must accept only good programs Well Typed Programs defined by the type system

4. Type Soundness implies well typed implies good class of errors eliminated at compile time entire class of errors eliminated at compile time

5. Type Soundness Much research in new type systems Much research in new type systems Proving type soundness is important Proving type soundness is important –Becoming its own research area –POPLmark workshop Proving type soundness is tedious Proving type soundness is tedious –Manual proofs –Machine checked proofs –Both require intensive human effort Our technique is completely automatic Our technique is completely automatic Our technique provides counterexamples Our technique provides counterexamples

6. Outline Motivation Motivation Example Example Approach Approach Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

7. Example Syntax: e :== true | false | 0 | succ e | pred e | iszero e | if e 1 then e 2 else e 3 Semantics: e  e’ succ e  succ e’ e  e’ pred e  pred e’ e  e’ iszero e  iszero e’ e1  e1’e1  e1’e1  e1’e1  e1’ if e 1 then e 2 else e 3  if e 1 ’ then e 2 else e 3 pred 0  0 pred succ nv  nv iszero 0  true iszero succ nv  false if true then e 2 else e 3  e 2 if false then e 2 else e 3  e 3 Good program: iszero 0 Bad program: iszero true Final States: v :== true | false | nvnv :== 0 | succ nv

8. Example Type system: T :== Nat | Bool Typing rules: true : Bool false : Bool 0 : Nat e : Nat succ e : Nat e : Nat pred e : Nat e : Nat iszero e : Bool e 1 : Bool ; e 2 : T ; e 3 : T if e 1 then e 2 else e 3 : T Type soundness: Every well typed program is a good program.

9. Soundness Proof Parts to a Soundness Proof: Parts to a Soundness Proof: –Progress Theorem If wellTyped(t) then t is a final state or t transitions to t’ If wellTyped(t) then t is a final state or t transitions to t’ –Preservation Theorem If wellTyped(t) and t transitions to t’ then wellTyped(t’) If wellTyped(t) and t transitions to t’ then wellTyped(t’) –Together, these two theorems imply Type Soundness

10. Software Model Checking Software Model Checking Software Model Checking –For each well typed program state Check Progress Check Progress Check Preservation Check Preservation –If a check fails, show a counterexample

11. Challenges Search space is infinite! Search space is infinite! –Impose finite bounds on the search space –Small Scope Hypothesis justifies this approach [Jackson96][Offutt00][Marinov03] [Jackson96][Offutt00][Marinov03] Search space is huge! Search space is huge! –Find similarities and eliminate redundancy

12. State Space Reduction Many software model checkers Many software model checkers –Verisoft, JPF, CMC, SLAM, Blast, Magic, … Many state space reduction techniques Many state space reduction techniques –Partial order reduction –Predicate abstraction Not effective for checking type soundness Not effective for checking type soundness

13. Our Key Idea if iszero 0 then true else false if iszero true false 0 if true false if iszero pred succ 0 if true 00 pred succ 00 if iszero if false 0 if true false true if false true PRUNED

14. Outline Motivation Motivation Example Example Approach Approach –Defining the language –Searching the state space –Representing the state space –Reducing the state space Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

15. Language Definition class ExpressionLanguage implements Language { … static class Expression { int kind; // TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, IF @Tree Expression e1, e2, e3; @Declarative boolean wellTyped() { return syntaxOk() && (kind == TRUE ==> true ) && (kind == FALSE ==> true ) && (kind == ZERO ==> true ) && (kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && (kind == PRED ==> e1.wellTyped() && e1.type() == INT) && (kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && (kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type()); } Expression smallStep() throws StuckException { if (isValue()) return this; if (!e1.isValue()) { e1 = e1.smallStep(); return this; } if (kind == PRED && e1.kind == ZERO ) return e1; if (kind == PRED && e1.kind == SUCC ) return e1.e1; if (kind == ISZERO && e1.kind == ZERO ) return True(); if (kind == ISZERO && e1.kind == SUCC ) return False(); if (kind == IF && e1.kind == TRUE ) return e2; if (kind == IF && e1.kind == FALSE) return e3; throw new StuckException(); } // Helper Functions … } Expression root; @Declarative public boolean wellTyped() { return root.wellTyped(); } public void smallStep() throws StuckException { root = root.smallStep(); } public boolean isFinalState() { return root.isValue(); } }

16. Language Definition class ExpressionLanguage implements Language { … static class Expression { int kind; // TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, IF @Tree Expression e1, e2, e3; @Declarative boolean wellTyped() { return syntaxOk() && (kind == TRUE ==> true ) && (kind == FALSE ==> true ) && (kind == ZERO ==> true ) && (kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && (kind == PRED ==> e1.wellTyped() && e1.type() == INT) && (kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && (kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type()); } Expression smallStep() throws StuckException { if (isValue()) return this; if (!e1.isValue()) { e1 = e1.smallStep(); return this; } if (kind == PRED && e1.kind == ZERO ) return e1; if (kind == PRED && e1.kind == SUCC ) return e1.e1; if (kind == ISZERO && e1.kind == ZERO ) return True(); if (kind == ISZERO && e1.kind == SUCC ) return False(); if (kind == IF && e1.kind == TRUE ) return e2; if (kind == IF && e1.kind == FALSE) return e3; throw new StuckException(); } // Helper Functions … } Expression root; @Declarative public boolean wellTyped() { return root.wellTyped(); } public void smallStep() throws StuckException { root = root.smallStep(); } public boolean isFinalState() { return root.isValue(); } } static class Expression { int kind; // TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, IF @Tree Expression e1, e2, e3; … }

17. class ExpressionLanguage implements Language { … static class Expression { int kind; // TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, IF @Tree Expression e1, e2, e3; @Declarative boolean wellTyped() { return syntaxOk() && (kind == TRUE ==> true ) && (kind == FALSE ==> true ) && (kind == ZERO ==> true ) && (kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && (kind == PRED ==> e1.wellTyped() && e1.type() == INT) && (kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && (kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type()); } Expression smallStep() throws StuckException { if (isValue()) return this; if (!e1.isValue()) { e1 = e1.smallStep(); return this; } if (kind == PRED && e1.kind == ZERO ) return e1; if (kind == PRED && e1.kind == SUCC ) return e1.e1; if (kind == ISZERO && e1.kind == ZERO ) return True(); if (kind == ISZERO && e1.kind == SUCC ) return False(); if (kind == IF && e1.kind == TRUE ) return e2; if (kind == IF && e1.kind == FALSE) return e3; throw new StuckException(); } // Helper Functions … } Expression root; @Declarative public boolean wellTyped() { return root.wellTyped(); } public void smallStep() throws StuckException { root = root.smallStep(); } public boolean isFinalState() { return root.isValue(); } } Language Definition Expression smallStep() throws StuckException { if (isValue()) return this; if (!e1.isValue()) { e1 = e1.smallStep(); return this; } if (kind == PRED && e1.kind == ZERO ) return e1; if (kind == PRED && e1.kind == SUCC ) return e1.e1; if (kind == ISZERO && e1.kind == ZERO ) return True(); if (kind == ISZERO && e1.kind == SUCC ) return False(); if (kind == IF && e1.kind == TRUE ) return e2; if (kind == IF && e1.kind == FALSE) return e3; throw new StuckException(); }

18. class ExpressionLanguage implements Language { … static class Expression { int kind; // TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, IF @Tree Expression e1, e2, e3; @Declarative boolean wellTyped() { return syntaxOk() && (kind == TRUE ==> true ) && (kind == FALSE ==> true ) && (kind == ZERO ==> true ) && (kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && (kind == PRED ==> e1.wellTyped() && e1.type() == INT) && (kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && (kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type()); } Expression smallStep() throws StuckException { if (isValue()) return this; if (!e1.isValue()) { e1 = e1.smallStep(); return this; } if (kind == PRED && e1.kind == ZERO ) return e1; if (kind == PRED && e1.kind == SUCC ) return e1.e1; if (kind == ISZERO && e1.kind == ZERO ) return True(); if (kind == ISZERO && e1.kind == SUCC ) return False(); if (kind == IF && e1.kind == TRUE ) return e2; if (kind == IF && e1.kind == FALSE) return e3; throw new StuckException(); } // Helper Functions … } Expression root; @Declarative public boolean wellTyped() { return root.wellTyped(); } public void smallStep() throws StuckException { root = root.smallStep(); } public boolean isFinalState() { return root.isValue(); } } Language Definition @Declarative boolean wellTyped() { return syntaxOk() && (kind == TRUE ==> true ) && (kind == FALSE ==> true ) && (kind == ZERO ==> true ) && (kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && (kind == PRED ==> e1.wellTyped() && e1.type() == INT) && (kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && (kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type()); }

19. Approach Defining the language Defining the language Searching the state space Searching the state space Representing the state space Representing the state space Reducing the state space Reducing the state space

20. Searching Select an unchecked well typed state Select an unchecked well typed state if iszero if false 0 true

21. Searching Check progress and preservation Check progress and preservation –Progress: Run one small step void smallStep() throws StuckException { … } if iszero if false 0 if true false true if false true

22. Searching Check progress and preservation Check progress and preservation –Preservation: Check if the result is well typed @Declarative boolean wellTyped() { … } if iszero if false 0 if true false true if false true

23. Searching Identify and prune similar states Identify and prune similar states if iszero true false 0 if true false if iszero pred succ 0 if true 00 pred succ 00 if iszero if false 0 true if true if false true PRUNED

24. Search Algorithm Let W be the well typed state space While W is not empty Choose a state s in W Run one step of evaluation on s If Progress or Preservation fail Print out s as a bug trace Let P be the set of states similar to s W = W – P Need efficient representation and operations for these sets!

25. Approach Defining the language Defining the language Searching the state space Searching the state space Representing the state space Representing the state space Reducing the state space Reducing the state space

26. Representing the State Space Represent a set as a boolean formula Represent a set as a boolean formula –Encode each field as bits (b0, b1, …) –Constrain the bits using boolean operations n1.kind == IF && n1.e1.kind == TRUE ۸ b1 ۸¬2) ۸۸ (¬b6۸ ¬b7۸ ¬b8) (b0 ۸ b1 ۸ ¬b2) ۸ b3 ۸ (¬b6 ۸ ¬b7 ۸ ¬b8) kind = {b0,b1,b2} e1 = {b3} e2 = {b4} e3 = {b5} kind = {b6,b7,b8} e1 = {} e2 = {} e3 = {} … At AST height 5, a formula representing 2,504,790,381 well typed states is only 24,117 terms long. n1 n2

27. Representing the State Space Initialize to the set of well typed states Initialize to the set of well typed states –Construct a formula describing the type system @Declarative boolean wellTyped() { return syntaxOk() && ( kind == TRUE ==> true ) && ( kind == FALSE ==> true ) && ( kind == ZERO ==> true ) && ( kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && ( kind == PRED ==> e1.wellTyped() && e1.type() == INT) && ( kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && ( kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type() ); } boolean formula

28. Representing the State Space Initialize to the set of well typed states Initialize to the set of well typed states –Construct a formula describing the type system Declarative methods Declarative methods –May not use object creations, assignments, or loops –May use,, operators –May use ==>, forall, exists operators Declarative methods translate to small formulas Declarative methods translate to small formulas @Declarative boolean wellTyped() { return syntaxOk() && ( kind == TRUE ==> true ) && ( kind == FALSE ==> true ) && ( kind == ZERO ==> true ) && ( kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && ( kind == PRED ==> e1.wellTyped() && e1.type() == INT) && ( kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && ( kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type() ); }

29. Search Algorithm Let W be the well typed state space While W is not empty Choose a state s in W Run one step of evaluation on s If Progress or Preservation fail Print out s as a bug trace Let P be the set of states similar to s W = W – P Use a SAT solver Add ¬P to the SAT solver

30. Approach Defining the language Defining the language Searching the state space Searching the state space Representing the state space Representing the state space Reducing the state space Reducing the state space –Dynamic analysis –Static analysis –Other optimizations

31. Dynamic Analysis Discover and prune states that are similar Discover and prune states that are similar –Touched analysis Track which fields are accessed Track which fields are accessed –Symbolic execution Track which fields are accessed and how they are used Track which fields are accessed and how they are used if iszero if false 0 true if true if false true (root.kind == IF || root.kind == ISZERO || root.kind == SUCC || root.kind == PRED) && root.e1.kind == ISZERO && root.e1.e1.kind == ZERO root.kind == IF && root.e1.kind == ISZERO && root.e1.e1.kind == ZERO

32. Static Analysis

33. class WhyStaticAnalysis { private boolean a, b; public void smallStep() { a = !a; } @Declarative public boolean wellTyped() { return a || b; }

34. Static Analysis a = true b = true Choose a well typed state class WhyStaticAnalysis { private boolean a, b; public void smallStep() { a = !a; } @Declarative public boolean wellTyped() { return a || b; }

35. Static Analysis a = true b = true a = false b = true Choose a well typed state Check the transition class WhyStaticAnalysis { private boolean a, b; public void smallStep() { a = !a; } @Declarative public boolean wellTyped() { return a || b; }

36. Static Analysis a = true b = true a = false b = true Choose a well typed state Check the transition a = true b = false a = false b = false Prune similar transitions well typed class WhyStaticAnalysis { private boolean a, b; public void smallStep() { a = !a; } @Declarative public boolean wellTyped() { return a || b; } PRUNED

37. Static Analysis Dynamic analysis alone is not always correct! Dynamic analysis alone is not always correct! Need to perform additional checks Need to perform additional checks Could check each pruned transition separately Could check each pruned transition separately –Negates the benefits of pruning We use static analysis to ensure correctness We use static analysis to ensure correctness –All pruned transitions checked together –Any error within finite bounds is caught

38. Static Analysis a = true b = true a = false b = true Prestate of a pruned transition Poststate of the pruned transition a = true b a = false b Not valid when b = false! class WhyStaticAnalysis { private boolean a, b; public void smallStep() { a = !a; } @Declarative public boolean wellTyped() { return a || b; } wellTyped = (a || b) wellTyped Pre wellTyped Pre = (a || b) a=true = true wellTyped Post = (a || b) a=false = b wellTyped Pre  wellTyped Post = b

39. Static Analysis Partially evaluate wellTyped formula Partially evaluate wellTyped formula Use SAT solver to check the above implication Use SAT solver to check the above implication If the implication holds then pruning is sound If the implication holds then pruning is sound If not, we have a counterexample If not, we have a counterexample Often, wellTyped Pre and wellTyped Post are identical Often, wellTyped Pre and wellTyped Post are identical Checking the implication is trivial then Checking the implication is trivial then wellTyped Pre wellTyped Post 

40. Other Optimizations Isomorphism pruning Isomorphism pruning –Prune more when checking heaps Handling term cloning Handling term cloning –Perform symbolic copy Handling term substitution Handling term substitution –Perform incremental substitution

41. Outline Motivation Motivation Example Example Approach Approach Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

42. Experimental Results We verified type soundness of five languages: We verified type soundness of five languages: –Expression Language –Imperative Language –Featherweight Java –Mini Java –Mini Java with Ownership Types

43. Experimental Results Imperative Language Imperative Language –Simple language from [Winskel93, Chapter 2] –Features variables, assignment, conditionals, loops –Features arithmetic, relational, logical operators int a; a := 5; while (a < 7) a := a+1;

44. Experimental Results Featherweight Java [Igarashi99] Featherweight Java [Igarashi99] –Subset of Java –Features classes, inheritance, methods, fields class A extends Object { A() { super(); } } class B extends Object { B() { super(); } } class Pair extends Object { Object fst; Object snd; Pair(Object fst, Object snd) { super(); this.fst=fst; this.snd=snd; } Pair setfst(Object newfst) { return new Pair(newfst, this.snd); }

45. Experimental Results Mini Java Mini Java –Extends Featherweight Java –Allows mutation of objects –Features an explicit heap –Features integers and booleans class Node extends Object { int value; Node(int value) { super(); this.value = value; } Node mutate() { return this.value := this.value + 1; }

46. Experimental Results Mini Java with Ownership Types Mini Java with Ownership Types –[Clarke98;Boyapati03;Aldrich02] –Types are parameterized with owners –Enforces object encapsulation class Node extends Object { Object data; Node next; Node(Object data, Node next) { super(); this.data = data; this.next = next; } class Stack extends Object { Node root; Stack(Node root) { super(); this.root = root; } Stack push(Object data) { return this.root := new Node (data, this.root); }

47. Experimental Results Benchmark Max Expression Size well typed States (approx.) States Checked Time (s) Expression Language 1234…134012136410933280 2 1 2 2 2 4 … 2 10 2 31 2 93 2 280 2 841 2 2523 1335…1117232935410.0680.0930.1050.122…0.2460.5511.3763.63310.83338.543 IMP 1234567…153163127255511 2 3 2 5 2 8 2 9 2 12 2 14 2 17 … 2 36 2 75 2 157 2 327 2 679 2 1402 171119343434…61961472303776520.1020.1850.2560.4080.7100.7390.816…2.1585.10710.06621.01352.208331.138 Checks 2 157 well typed IMP programs in about 10 seconds

48. Experimental Results Benchmark Max Expression Size States Checked Time (s) Featherweight Java 12345…2185341379913…7029812101.1481.5941.6501.8992.151…6.90543.756475.022 Mini Java 12345…2185341521405359…275113345652.7213.1173.8975.7506.191…37.354342.4355981.114 Mini Java with Ownership Types 12345…212529331373110135157…7338771021116550.81877.135103.230231.328247.954…2760.7343963.8365271.5096255.260

49. Experimental Results We inserted 25 errors into Mini Java with Ownership Types Max Expression Size Percentage of Errors Caught 12345678084068768084100

50. Experimental Results We inserted 25 errors into Mini Java with Ownership Types Max Expression Size Percentage of Errors Caught Time to Check the Entire Search Space (s) 12345678…21252933084068768084100…100100100100 50.818 77.135 103.230 231.328 247.954 339.632 403.249 836.568…2760.7343963.8365271.5096255.260

51. Related Work State space reduction techniques State space reduction techniques –Abstraction & refinement [SLAM; Blast; Magic] –Partial order reduction [Godefroid97; Flanagan05] –Heap canonicalization [Musuvathi05; Iosif02] –Symmetry reduction [Ip93]

52. Related Work Software model checkers Software model checkers –Verisoft [Godefroid97] –Java PathFinder [Visser00] –CMC [Musuvathi02] –Bandera [Corbett00] –Bogor [Dwyer05] –SLAM [Ball01] –Blast [Henzinger02] –Magic [Chaki03] –Jalloy [Vaziri03] –Miniatur [Dolby07]

53. Conclusions Fully automatic technique to check type soundness Fully automatic technique to check type soundness Counterexamples provided if soundness fails Counterexamples provided if soundness fails Useful to check small to medium sized languages Useful to check small to medium sized languages –Research languages –Core subsets of complex languages