Efficient Software Model Checking of Soundness of Type Systems Michael Roberson, Melanie Harries, Paul T. Darga, Chandrasekhar Boyapati University of Michigan

Type Soundness All Possible Programs defined by the syntax Good Programs do not cause certain kinds of runtime errors e.g. memory errors, race conditions, etc. defined by the semantics

Type Soundness Type System designed to statically prevent errors must accept only good programs Well Typed Programs defined by the type system

Type Soundness implies well typed implies good class of errors eliminated at compile time entire class of errors eliminated at compile time

Type Soundness Much research in new type systems Much research in new type systems Proving type soundness is important Proving type soundness is important –Becoming its own research area –POPLmark workshop Proving type soundness is tedious Proving type soundness is tedious –Manual proofs –Machine checked proofs –Both require intensive human effort Our technique is completely automatic Our technique is completely automatic Our technique provides counterexamples Our technique provides counterexamples

Outline Motivation Motivation Example Example Approach Approach Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

Example Syntax: e :== true | false | 0 | succ e | pred e | iszero e | if e 1 then e 2 else e 3 Semantics: e  e’ succ e  succ e’ e  e’ pred e  pred e’ e  e’ iszero e  iszero e’ e1  e1’e1  e1’e1  e1’e1  e1’ if e 1 then e 2 else e 3  if e 1 ’ then e 2 else e 3 pred 0  0 pred succ nv  nv iszero 0  true iszero succ nv  false if true then e 2 else e 3  e 2 if false then e 2 else e 3  e 3 Good program: iszero 0 Bad program: iszero true Final States: v :== true | false | nvnv :== 0 | succ nv

Example Type system: T :== Nat | Bool Typing rules: true : Bool false : Bool 0 : Nat e : Nat succ e : Nat e : Nat pred e : Nat e : Nat iszero e : Bool e 1 : Bool ; e 2 : T ; e 3 : T if e 1 then e 2 else e 3 : T Type soundness: Every well typed program is a good program.

Soundness Proof Parts to a Soundness Proof: Parts to a Soundness Proof: –Progress Theorem If wellTyped(t) then t is a final state or t transitions to t’ If wellTyped(t) then t is a final state or t transitions to t’ –Preservation Theorem If wellTyped(t) and t transitions to t’ then wellTyped(t’) If wellTyped(t) and t transitions to t’ then wellTyped(t’) –Together, these two theorems imply Type Soundness

Software Model Checking Software Model Checking Software Model Checking –For each well typed program state Check Progress Check Progress Check Preservation Check Preservation –If a check fails, show a counterexample

Challenges Search space is infinite! Search space is infinite! –Impose finite bounds on the search space –Small Scope Hypothesis justifies this approach [Jackson96][Offutt00][Marinov03] [Jackson96][Offutt00][Marinov03] Search space is huge! Search space is huge! –Find similarities and eliminate redundancy

State Space Reduction Many software model checkers Many software model checkers –Verisoft, JPF, CMC, SLAM, Blast, Magic, … Many state space reduction techniques Many state space reduction techniques –Partial order reduction –Predicate abstraction Not effective for checking type soundness Not effective for checking type soundness

Our Key Idea if iszero 0 then true else false if iszero true false 0 if true false if iszero pred succ 0 if true 00 pred succ 00 if iszero if false 0 if true false true if false true PRUNED

Outline Motivation Motivation Example Example Approach Approach –Defining the language –Searching the state space –Representing the state space –Reducing the state space Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

Language Definition class ExpressionLanguage implements Language { … static class Expression { int kind; // TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, Expression e1, e2, boolean wellTyped() { return syntaxOk() && (kind == TRUE ==> true ) && (kind == FALSE ==> true ) && (kind == ZERO ==> true ) && (kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && (kind == PRED ==> e1.wellTyped() && e1.type() == INT) && (kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && (kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type()); } Expression smallStep() throws StuckException { if (isValue()) return this; if (!e1.isValue()) { e1 = e1.smallStep(); return this; } if (kind == PRED && e1.kind == ZERO ) return e1; if (kind == PRED && e1.kind == SUCC ) return e1.e1; if (kind == ISZERO && e1.kind == ZERO ) return True(); if (kind == ISZERO && e1.kind == SUCC ) return False(); if (kind == IF && e1.kind == TRUE ) return e2; if (kind == IF && e1.kind == FALSE) return e3; throw new StuckException(); } // Helper Functions … } Expression public boolean wellTyped() { return root.wellTyped(); } public void smallStep() throws StuckException { root = root.smallStep(); } public boolean isFinalState() { return root.isValue(); } }

Language Definition class ExpressionLanguage implements Language { … static class Expression { int kind; // TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, Expression e1, e2, boolean wellTyped() { return syntaxOk() && (kind == TRUE ==> true ) && (kind == FALSE ==> true ) && (kind == ZERO ==> true ) && (kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && (kind == PRED ==> e1.wellTyped() && e1.type() == INT) && (kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && (kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type()); } Expression smallStep() throws StuckException { if (isValue()) return this; if (!e1.isValue()) { e1 = e1.smallStep(); return this; } if (kind == PRED && e1.kind == ZERO ) return e1; if (kind == PRED && e1.kind == SUCC ) return e1.e1; if (kind == ISZERO && e1.kind == ZERO ) return True(); if (kind == ISZERO && e1.kind == SUCC ) return False(); if (kind == IF && e1.kind == TRUE ) return e2; if (kind == IF && e1.kind == FALSE) return e3; throw new StuckException(); } // Helper Functions … } Expression public boolean wellTyped() { return root.wellTyped(); } public void smallStep() throws StuckException { root = root.smallStep(); } public boolean isFinalState() { return root.isValue(); } } static class Expression { int kind; // TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, Expression e1, e2, e3; … }

class ExpressionLanguage implements Language { … static class Expression { int kind; // TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, Expression e1, e2, boolean wellTyped() { return syntaxOk() && (kind == TRUE ==> true ) && (kind == FALSE ==> true ) && (kind == ZERO ==> true ) && (kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && (kind == PRED ==> e1.wellTyped() && e1.type() == INT) && (kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && (kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type()); } Expression smallStep() throws StuckException { if (isValue()) return this; if (!e1.isValue()) { e1 = e1.smallStep(); return this; } if (kind == PRED && e1.kind == ZERO ) return e1; if (kind == PRED && e1.kind == SUCC ) return e1.e1; if (kind == ISZERO && e1.kind == ZERO ) return True(); if (kind == ISZERO && e1.kind == SUCC ) return False(); if (kind == IF && e1.kind == TRUE ) return e2; if (kind == IF && e1.kind == FALSE) return e3; throw new StuckException(); } // Helper Functions … } Expression public boolean wellTyped() { return root.wellTyped(); } public void smallStep() throws StuckException { root = root.smallStep(); } public boolean isFinalState() { return root.isValue(); } } Language Definition Expression smallStep() throws StuckException { if (isValue()) return this; if (!e1.isValue()) { e1 = e1.smallStep(); return this; } if (kind == PRED && e1.kind == ZERO ) return e1; if (kind == PRED && e1.kind == SUCC ) return e1.e1; if (kind == ISZERO && e1.kind == ZERO ) return True(); if (kind == ISZERO && e1.kind == SUCC ) return False(); if (kind == IF && e1.kind == TRUE ) return e2; if (kind == IF && e1.kind == FALSE) return e3; throw new StuckException(); }

class ExpressionLanguage implements Language { … static class Expression { int kind; // TRUE, FALSE, ZERO, SUCC, PRED, ISZERO, Expression e1, e2, boolean wellTyped() { return syntaxOk() && (kind == TRUE ==> true ) && (kind == FALSE ==> true ) && (kind == ZERO ==> true ) && (kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && (kind == PRED ==> e1.wellTyped() && e1.type() == INT) && (kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && (kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type()); } Expression smallStep() throws StuckException { if (isValue()) return this; if (!e1.isValue()) { e1 = e1.smallStep(); return this; } if (kind == PRED && e1.kind == ZERO ) return e1; if (kind == PRED && e1.kind == SUCC ) return e1.e1; if (kind == ISZERO && e1.kind == ZERO ) return True(); if (kind == ISZERO && e1.kind == SUCC ) return False(); if (kind == IF && e1.kind == TRUE ) return e2; if (kind == IF && e1.kind == FALSE) return e3; throw new StuckException(); } // Helper Functions … } Expression public boolean wellTyped() { return root.wellTyped(); } public void smallStep() throws StuckException { root = root.smallStep(); } public boolean isFinalState() { return root.isValue(); } } Language boolean wellTyped() { return syntaxOk() && (kind == TRUE ==> true ) && (kind == FALSE ==> true ) && (kind == ZERO ==> true ) && (kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && (kind == PRED ==> e1.wellTyped() && e1.type() == INT) && (kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && (kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type()); }

Approach Defining the language Defining the language Searching the state space Searching the state space Representing the state space Representing the state space Reducing the state space Reducing the state space

Searching Select an unchecked well typed state Select an unchecked well typed state if iszero if false 0 true

Searching Check progress and preservation Check progress and preservation –Progress: Run one small step void smallStep() throws StuckException { … } if iszero if false 0 if true false true if false true

Searching Check progress and preservation Check progress and preservation –Preservation: Check if the result is well boolean wellTyped() { … } if iszero if false 0 if true false true if false true

Searching Identify and prune similar states Identify and prune similar states if iszero true false 0 if true false if iszero pred succ 0 if true 00 pred succ 00 if iszero if false 0 true if true if false true PRUNED

Search Algorithm Let W be the well typed state space While W is not empty Choose a state s in W Run one step of evaluation on s If Progress or Preservation fail Print out s as a bug trace Let P be the set of states similar to s W = W – P Need efficient representation and operations for these sets!

Approach Defining the language Defining the language Searching the state space Searching the state space Representing the state space Representing the state space Reducing the state space Reducing the state space

Representing the State Space Represent a set as a boolean formula Represent a set as a boolean formula –Encode each field as bits (b0, b1, …) –Constrain the bits using boolean operations n1.kind == IF && n1.e1.kind == TRUE ۸ b1 ۸¬2) ۸۸ (¬b6۸ ¬b7۸ ¬b8) (b0 ۸ b1 ۸ ¬b2) ۸ b3 ۸ (¬b6 ۸ ¬b7 ۸ ¬b8) kind = {b0,b1,b2} e1 = {b3} e2 = {b4} e3 = {b5} kind = {b6,b7,b8} e1 = {} e2 = {} e3 = {} … At AST height 5, a formula representing 2,504,790,381 well typed states is only 24,117 terms long. n1 n2

Representing the State Space Initialize to the set of well typed states Initialize to the set of well typed states –Construct a formula describing the type boolean wellTyped() { return syntaxOk() && ( kind == TRUE ==> true ) && ( kind == FALSE ==> true ) && ( kind == ZERO ==> true ) && ( kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && ( kind == PRED ==> e1.wellTyped() && e1.type() == INT) && ( kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && ( kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type() ); } boolean formula

Representing the State Space Initialize to the set of well typed states Initialize to the set of well typed states –Construct a formula describing the type system Declarative methods Declarative methods –May not use object creations, assignments, or loops –May use,, operators –May use ==>, forall, exists operators Declarative methods translate to small formulas Declarative methods translate to small boolean wellTyped() { return syntaxOk() && ( kind == TRUE ==> true ) && ( kind == FALSE ==> true ) && ( kind == ZERO ==> true ) && ( kind == SUCC ==> e1.wellTyped() && e1.type() == INT) && ( kind == PRED ==> e1.wellTyped() && e1.type() == INT) && ( kind == ISZERO ==> e1.wellTyped() && e1.type() == INT) && ( kind == IF ==> e1.wellTyped() && e1.type() == BOOL && e2.wellTyped() && e3.wellTyped() && e2.type() == e3.type() ); }

Search Algorithm Let W be the well typed state space While W is not empty Choose a state s in W Run one step of evaluation on s If Progress or Preservation fail Print out s as a bug trace Let P be the set of states similar to s W = W – P Use a SAT solver Add ¬P to the SAT solver

Approach Defining the language Defining the language Searching the state space Searching the state space Representing the state space Representing the state space Reducing the state space Reducing the state space –Dynamic analysis –Static analysis –Other optimizations

Dynamic Analysis Discover and prune states that are similar Discover and prune states that are similar –Touched analysis Track which fields are accessed Track which fields are accessed –Symbolic execution Track which fields are accessed and how they are used Track which fields are accessed and how they are used if iszero if false 0 true if true if false true (root.kind == IF || root.kind == ISZERO || root.kind == SUCC || root.kind == PRED) && root.e1.kind == ISZERO && root.e1.e1.kind == ZERO root.kind == IF && root.e1.kind == ISZERO && root.e1.e1.kind == ZERO

Static Analysis

class WhyStaticAnalysis { private boolean a, b; public void smallStep() { a = !a; public boolean wellTyped() { return a || b; }

Static Analysis a = true b = true Choose a well typed state class WhyStaticAnalysis { private boolean a, b; public void smallStep() { a = !a; public boolean wellTyped() { return a || b; }

Static Analysis a = true b = true a = false b = true Choose a well typed state Check the transition class WhyStaticAnalysis { private boolean a, b; public void smallStep() { a = !a; public boolean wellTyped() { return a || b; }

Static Analysis a = true b = true a = false b = true Choose a well typed state Check the transition a = true b = false a = false b = false Prune similar transitions well typed class WhyStaticAnalysis { private boolean a, b; public void smallStep() { a = !a; public boolean wellTyped() { return a || b; } PRUNED

Static Analysis Dynamic analysis alone is not always correct! Dynamic analysis alone is not always correct! Need to perform additional checks Need to perform additional checks Could check each pruned transition separately Could check each pruned transition separately –Negates the benefits of pruning We use static analysis to ensure correctness We use static analysis to ensure correctness –All pruned transitions checked together –Any error within finite bounds is caught

Static Analysis a = true b = true a = false b = true Prestate of a pruned transition Poststate of the pruned transition a = true b a = false b Not valid when b = false! class WhyStaticAnalysis { private boolean a, b; public void smallStep() { a = !a; public boolean wellTyped() { return a || b; } wellTyped = (a || b) wellTyped Pre wellTyped Pre = (a || b) a=true = true wellTyped Post = (a || b) a=false = b wellTyped Pre  wellTyped Post = b

Static Analysis Partially evaluate wellTyped formula Partially evaluate wellTyped formula Use SAT solver to check the above implication Use SAT solver to check the above implication If the implication holds then pruning is sound If the implication holds then pruning is sound If not, we have a counterexample If not, we have a counterexample Often, wellTyped Pre and wellTyped Post are identical Often, wellTyped Pre and wellTyped Post are identical Checking the implication is trivial then Checking the implication is trivial then wellTyped Pre wellTyped Post 

Other Optimizations Isomorphism pruning Isomorphism pruning –Prune more when checking heaps Handling term cloning Handling term cloning –Perform symbolic copy Handling term substitution Handling term substitution –Perform incremental substitution

Outline Motivation Motivation Example Example Approach Approach Experimental Results Experimental Results Related Work Related Work Conclusions Conclusions

Experimental Results We verified type soundness of five languages: We verified type soundness of five languages: –Expression Language –Imperative Language –Featherweight Java –Mini Java –Mini Java with Ownership Types

Experimental Results Imperative Language Imperative Language –Simple language from [Winskel93, Chapter 2] –Features variables, assignment, conditionals, loops –Features arithmetic, relational, logical operators int a; a := 5; while (a < 7) a := a+1;

Experimental Results Featherweight Java [Igarashi99] Featherweight Java [Igarashi99] –Subset of Java –Features classes, inheritance, methods, fields class A extends Object { A() { super(); } } class B extends Object { B() { super(); } } class Pair extends Object { Object fst; Object snd; Pair(Object fst, Object snd) { super(); this.fst=fst; this.snd=snd; } Pair setfst(Object newfst) { return new Pair(newfst, this.snd); }

Experimental Results Mini Java Mini Java –Extends Featherweight Java –Allows mutation of objects –Features an explicit heap –Features integers and booleans class Node extends Object { int value; Node(int value) { super(); this.value = value; } Node mutate() { return this.value := this.value + 1; }

Experimental Results Mini Java with Ownership Types Mini Java with Ownership Types –[Clarke98;Boyapati03;Aldrich02] –Types are parameterized with owners –Enforces object encapsulation class Node extends Object { Object data; Node next; Node(Object data, Node next) { super(); this.data = data; this.next = next; } class Stack extends Object { Node root; Stack(Node root) { super(); this.root = root; } Stack push(Object data) { return this.root := new Node (data, this.root); }

Experimental Results Benchmark Max Expression Size well typed States (approx.) States Checked Time (s) Expression Language 1234… … … … IMP … … … … Checks well typed IMP programs in about 10 seconds

Experimental Results Benchmark Max Expression Size States Checked Time (s) Featherweight Java 12345… … … Mini Java 12345… … … Mini Java with Ownership Types 12345… … …

Experimental Results We inserted 25 errors into Mini Java with Ownership Types Max Expression Size Percentage of Errors Caught

Experimental Results We inserted 25 errors into Mini Java with Ownership Types Max Expression Size Percentage of Errors Caught Time to Check the Entire Search Space (s) … … …

Related Work State space reduction techniques State space reduction techniques –Abstraction & refinement [SLAM; Blast; Magic] –Partial order reduction [Godefroid97; Flanagan05] –Heap canonicalization [Musuvathi05; Iosif02] –Symmetry reduction [Ip93]

Related Work Software model checkers Software model checkers –Verisoft [Godefroid97] –Java PathFinder [Visser00] –CMC [Musuvathi02] –Bandera [Corbett00] –Bogor [Dwyer05] –SLAM [Ball01] –Blast [Henzinger02] –Magic [Chaki03] –Jalloy [Vaziri03] –Miniatur [Dolby07]

Conclusions Fully automatic technique to check type soundness Fully automatic technique to check type soundness Counterexamples provided if soundness fails Counterexamples provided if soundness fails Useful to check small to medium sized languages Useful to check small to medium sized languages –Research languages –Core subsets of complex languages