Writing specifications for object-oriented programs K. Rustan M. Leino Microsoft Research, Redmond, WA, USA 21 Jan 2005 Invited talk, AIOOL 2005 Paris, France joint work with Mike Barnett, Robert DeLine, Manuel Fähndrich, Wolfram Schulte, Herman Venter, Peter Müller, David A. Naumann, Bor-Yuh Evan Chang, Bart Jacobs, Xinming Ou, and Qi Sun

Software engineering problem Building and maintaining large systems that are correct

Using tools to program better Specifications record design decisions –bridge intent and code Tools amplify human effort –manage details –find inconsistencies –ensure quality

Spec# Experimental mix of contracts and tool support Aimed at real programmers, real programs Superset of C# –non-null types –enhanced exception support –pre- and postconditions –object invariants Tool support –more type checking –compiler-emitted run-time checks –static program verification

Spec# demo

Basic architecture of a program verifier verification condition generator theorem prover verification condition program with specifications correct or list of errors

Spec# program verifier architecture V.C. generator automatic theorem prover verification condition Spec# correct or list of errors Spec# compiler MSIL translator intermediate program representation abstract interpreter Spec# program verifier

Modular verification Dont assume we have the entire program –for example, we want to be able to verify a library without having access to its clients Verify a given portion of the program (e.g., a class) against its specifications, assuming that the rest of the program lives up to its specifications Analogous to separate compilation

Specification challenges Object invariants Frame conditions (modifies clauses) Class initialization and class variables Model fields Delegates …

Objects have invariants class C { private int x; private int y; public void M() { int t := 100 / (y – x); x := x + 1; P(t); y := y + 1; } … } division by zero

Pre/post are not enough class C { private int x; private int y; public void M() requires x < y; { int t := 100 / (y – x); x := x + 1; P(t); y := y + 1; } … }

Object invariants class C { private int x; private int y; invariant x < y; public void M() { int t := 100 / (y – x); x := x + 1; P(t); y := y + 1; } … }

When do object invariants hold? class C { private int x; private int y; invariant x < y; public C() { x := 0; y := 1; } public void M() { int t := 100 / (y – x); x := x + 1; P(t); y := y + 1; } … } invariant assumed to hold on entry to method invariant checked to hold on exit from method invariant checked to hold at end of constructor invariant may be temporarily broken here invariant is restored here what if P calls back into M?

Object state: valid vs. mutable introduce a special object field st : { Mutable, Valid } idea: program invariant ( o: C o.st = Mutable Inv C (o)) field updates are allowed only for mutable objects: a field-update statement o.f := E; has the precondition o.st = Mutable holds at every program point! for any o: C, we write Inv C (o) o.x < o.y class C { int x, y; invariant x < y;

pack/unpack statements st is changed by special statements pack and unpack pack o as C –check that Inv C (o) holds –then change o.st from Mutable to Valid unpack o from C –change o.st from Valid to Mutable

Example, with pack/unpack class C { int x, y; invariant x < y; public void M() { int t := 100 / (y – x); unpack this from C; x := x + 1; P(t); y := y + 1; pack this as C; } … } invariant checked to hold here invariant may be temporarily broken here

Specifying methods and constructors class C { int x, y; invariant x < y; public C() ensures this.st = Valid; { x := 0; y := 1; pack this as C; } public void M() requires this.st = Valid; { int t := 100 / (y – x); unpack this from C; x := x + 1; P(t); y := y + 1; pack this as C; } … } Note: if P calls back into M, then P must first make this valid in order to satisfy Ms precondition, so no dangerous reentrancy can occur

Summary, so far invariant … st : { Mutable, Valid } pack, unpack updates of o.f require o.st = Mutable Inv C (o) can mention only the fields of o ( o: C o.st = Mutable Inv C (o))

Aggregate objects class Set { Hashtable h; invariant …; public void Add(Element e) requires this.st = Valid; { unpack this from Set; h.Add(e, e); pack this as Set; } … } class Hashtable { invariant …; public void Add(object key, object val) requires this.st = Valid; … } how do we know h is valid here?

Aggregate objects class Set { Hashtable h; invariant …; public void Add(Element e) requires this.st = Valid; { unpack this from Set; h.Add(e, e); pack this as Set; } public Hashtable Leak() { return h; } } class Hashtable { invariant …; public void Add(object key, object val) requires this.st = Valid; … } how do we know h is valid here? Perhaps it isnt! void Violation(Set s) requires s.st = Valid; { Hashtable g := s.Leak(); unpack g from Hashtable; g.x := …; s.Add(…); }

Ownership class Set { owned Hashtable h; invariant …; public void Add(Element e) requires this.st = Valid; { unpack this from Set; h.Add(e, e); pack this as Set; } … } class Hashtable { invariant …; public void Add(object key, object val) requires this.st = Valid; … } For any s: Set, s uniquely owns s.h validity of s implies validity of s.h ownership of h temporarily relinquished here ownership of h re-obtained here

Object state: mutable, valid, committed st : { Mutable, Valid, Committed } program invariant ( o: C o.st = Mutable Inv C (o)) and for every owned field f ( o: C o.st = Mutable o.f.st = Committed) pack o as C –check that Inv C (o) holds, –check that o.f.st = Valid, –then change o.f.st from Valid to Committed –and change o.st from Mutable to Valid unpack o from C –change o.st from Valid to Mutable, and –change o.f.st from Committed to Valid Committed means valid and owned class C { owned T f; … for every owned field f

Object states: a picture of the heap Mutable Valid Committed s: Set Hashtable ownership h unpack s from Set

Object states: a picture of the heap Mutable Valid Committed s: Set Hashtable ownership h unpack s from Set

Object states: a picture of the heap Mutable Valid Committed s: Set Hashtable ownership h pack s as Set

Object states: a picture of the heap Mutable Valid Committed s: Set Hashtable ownership h pack s as Set

Summary of object invariants invariant … owned T f; st : { Mutable, Valid, Committed } pack, unpack updates of o.f require o.st = Mutable Inv C (o) can mention only the fields of o and the fields of owned fields –example: invariant this.n = this.h.Count; ( o: C o.st = Mutable Inv C (o)) ( o: C o.st = Mutable o.f.st = Committed)

Frame conditions To be useful to the caller, a postcondition must say what goes unchanged –ensures x = old(x) y = old(y) … A modifies clause says what is allowed to change, implicitly indicating what goes unchanged –modifies z

What do modifies clauses mean? modifies M; = modifies Heap; ensures ( o,f Heap[o,f] = old(Heap(o,f)) (o,f) old(M) ¬old(Heap[o,alloc])

Modifying underlying representation Mutable Valid Committed s: Set Hashtable ownership h

Fields of committed objects may change modifies M; = modifies Heap; ensures ( o,f Heap[o,f] = old(Heap(o,f)) (o,f) old(M) ¬old(Heap[o,alloc]) old(Heap[o,Committed]))

So: common modifies clause modifies this.*;

Quirk? void M(T p) requires p.st = Committed p.x = 12; { int y := Math.Sqrt(…); assert p.x = 12; } assertion failure

Default specifications I have showed the building blocks A programming language would use defaults, for example: –a method has precondition o.st=Valid for every parameter o –public method bodies start with unpack and end with pack –a field is owned unless declared as shared

Further research challenges Extend specification methodology to more complicated programming idioms Develop abstract interpretations that work with object-oriented specifications Combine abstract interpretation with theorem proving Use programming system among developers

Conclusions Because of tool support, were ready for programming at the next level of rigor Rigor can be enforced by type checking, by run-time checking, and by static verification Specifications give programmers a way to record their design decisions Methodology is underdeveloped –Can programming theory yet fully explain why real big programs work? –programming theory has not kept up with practice