1. Spec# K. Rustan M. Leino Senior Researcher Programming Languages and Methods Microsoft Corporation Joint work with: Mike Barnett, Robert DeLine, Manuel Fähndrich, Wolfram Schulte, Herman Venter And interns: Bor-Yuh Evan Chang, Ádám Darvas, Bart Jacobs, Daan Leijen, Angela Wallenburg And visiting researchers: Francesco Logozzo, Peter Müller, David A. Naumann, Arnd Poetzsch-Heffter

2. Software Engineering Problem Building and maintaining large systems that are correct

3. Approach Specifications record design decisions Bridge intent and code Tools amplify human effort Manage details Find inconsistencies Ensure quality

4. Research Goals Build the best such system we can build today Experiment with the system to get a feel for what it is like to use Advance the state of the art

5. Spec# Experimental mix of contracts and tool support Aimed at experienced developers who know the high cost of testing and maintenance Superset of C# Non-null types Pre- and postconditions Object invariants Tool support More type checking Compiler-emitted run-time checks Static program verification C# Contracts everywhere type checking static verification into the future run-time checks degree of checking, effort familiar

6. Spec# Demo: Chunker

7. Some Design Issues 1. Non-null types 2. C# compatibility 3. Preconditions 4. Object invariants 5. Program verifier architecture 6. Verification-condition generation

8. 0. Non-null Types T x; The value of x is null or a reference to an object whose type is a subtype of T T ! y; The value of y is a reference to an object whose type is a subtype of T, not null

9. Non-null Escape Hatch: Cast object o; string s; … string! a = (string!)o; string! b = (!)s;

10. public void M( T x ) { if (x == null) { … } else { int y = ((!)x).f; … } } Comparing Against Null

11. public void M( T x ) { if (x == null) { … } else { int y = x.f; … } } Comparing Against Null Spec# performs a data-flow analysis to allow this (similar to definite assignment)

12. Non-null Instance Fields class C : B { T ! x; public C(T ! y) :base() { this.x = y; } public override int M() { return x.f; } class C : B { T ! x; public C(T ! y) :base() { this.x = y; } public override int M() { return x.f; }} Is this code type safe? No! abstract class B { public B() { this.M(); } public abstract int M(); abstract class B { public B() { this.M(); } public abstract int M();} null dereference

13. Non-null Instance Fields class C : B { T ! x; public C(T ! y) { this.x = y; base(); } public override int M() { return x.f; } class C : B { T ! x; public C(T ! y) { this.x = y; base(); } public override int M() { return x.f; }} Spec# allows x to be assigned before base constructor is called

14. Other Non-null Issues Comparing a field against null if (this.f != null) { // … this.f.M(…); } Static fields static T g = new T(); Arrays T![ ] a = new T![100]; Generics List myList = new List ();

15. Spec# is superset of C# From C# to Spec# Accept every C# program Compile it to have the same behavior Consequences “Possible null dereference” is just a warning “Must initialize non-null fields before calling base constructor” is an error Support for out-of-band contracts 1. C# Compatibility

16. From Spec# To C# Or: Leveraging wiz-bang features of Visual Studio 2005 class B : A { string! src; public B(string! source, int x) //^ requires 0 <= x; { this.src = source; base(x); }

17. From Spec# To C# Or: Leveraging wiz-bang features of Visual Studio 2005 class B : A { string/*!*/ src; public B(string/*!*/ source, int x) //^ requires 0 <= x; : base(x) { this.src = source; //^ base; }

18. 2. Preconditions

19. StringBuilder.Append Method (Char[ ], Int32, Int32) Appends the string representation of a specified subarray of Unicode characters to the end of this instance public StringBuilder Append(char[] value, int startIndex, int charCount); Parameters value A character array startIndex The starting position in value charCount The number of characters append Return Value A reference to this instance after the append operation has occurred Exceptions Contracts Today Exception Type Condition ArgumentNullException value is a null reference, and startIndex and charCount are not zero ArgumentOutOfRangeException charCount is less than zero -or- startIndex is less than zero -or- startIndex + charCount is less than the length of value

20. Contract In Spec# public StringBuilder Append(char[ ] value, int startIndex, int charCount ); requires value == null ==> startIndex == 0 && charCount == 0; requires value == null ==> startIndex == 0 && charCount == 0; requires 0 <= startIndex; requires 0 <= startIndex; requires 0 <= charCount; requires 0 <= charCount; requires value == null || startIndex + charCount <= value.Length; requires value == null || startIndex + charCount <= value.Length; Exception Type Condition ArgumentNullException value is a null reference, and startIndex and charCount are not zero ArgumentOutOfRangeException charCount is less than zero -or- startIndex is less than zero -or- startIndex + charCount is less than the length of value

21. Otherwise Clauses public StringBuilder Append(char[ ] value, int startIndex, int charCount ); requires value == null ==> startIndex == 0 && charCount == 0 otherwise ArgumentNullException; requires value == null ==> startIndex == 0 && charCount == 0 otherwise ArgumentNullException; requires 0 <= startIndex otherwise ArgumentOutOfRangeException; requires 0 <= startIndex otherwise ArgumentOutOfRangeException; … Exception Type Condition ArgumentNullException value is a null reference, and startIndex and charCount are not zero ArgumentOutOfRangeException charCount is less than zero -or- startIndex is less than zero -or- startIndex + charCount is less than the length of value

22. Inheriting Contracts interface J { void M(int x); requires P; } class A { public abstract void M(int x); requires Q; } class B : A, J { public override void M(int x) { … } }

23. 3. Object Invariants

24. When Do Object Invariants Hold? class C { private int x; private int y; invariant x < y; public C() { x = 0; y = 1; } public void M() { int t = 100 / (y – x); x = x + 1; P(t); y = y + 1; } …} invariant assumed to hold on entry to method invariant checked to hold on exit from method invariant checked to hold at end of constructor invariant may be temporarily broken here invariant is restored here what if P calls back into M?

25. Object States Mutable Object invariant may not hold Field updates allowed Valid Object invariant holds Field updates not allowed

26. Valid Versus Mutable Objects class C { private int x; private int y; invariant x < y; public void M() requires this.inv == Valid; { expose (this) { int t = 100 / (y – x); x = x + 1; P(t); y = y + 1; } } …} represent explicitly that invariant holds (without revealing what the invariant is) change this.inv from Valid to Mutable check invariant; then, change this.inv from Mutable to Valid field updates allowed only on Mutable objects

27. Summary Of Object Invariants invariant … inv : { Mutable, Valid } expose updates of o.f require o.inv = Mutable (  o ･ o.inv = Mutable  Inv (o))

28. 4. Spec# Verifier Architecture V.C. generator automatic theorem prover verification condition Spec# “correct” or list of errors Spec# compiler MSIL (“bytecode”) bytecode translator Boogie PL inference engine Spec# program verifier (aka Boogie)

29. BoogiePL Intermediate language Theory part Imperative part Semantics of Spec# is encoded in BoogiePL Can be used for other program- verification tasks, like verifying other source languages

30. Example BoogiePL var $Heap: [ref,name]any where IsHeap($Heap); function IsHeap(h: [ref,name]any) returns (bool); const Chunker: name; axiom Chunker <: System.Object; const Chunker.n: name; function DeclType(field: name) returns (class: name); axiom DeclType(Chunker.n) = Chunker; const $allocated: name; axiom (  h: [ref,name]any, o: ref, f: name IsHeap(h)  h[o, $allocated]  h[h[o, f],$allocated]); const $inv: name; axiom (  $oi: ref, $h: [ref,name]any IsHeap($h)  $h[$oi, $inv] <: Chunker  0 < $h[$oi, Chunker.ChunkSize]  0 ≤ $h[$oi, Chunker.n]  $h[$oi, Chunker.n] ≤ $Length($h[$oi, Chunker.src]));

31. Example BoogiePL procedure Chunker.NextChunk(this: ref) returns ($result: ref); requires $Heap[this, $inv] = Chunker; requires $Heap[this, $inv] = Chunker; requires $Heap[this, $ownerFrame] = $PeerGroupPlaceholder  ¬($Heap[$Heap[this, $ownerRef], $inv] <: $Heap[this, $ownerFrame]); requires $Heap[this, $ownerFrame] = $PeerGroupPlaceholder  ¬($Heap[$Heap[this, $ownerRef], $inv] <: $Heap[this, $ownerFrame]); free requires $Heap[this, $allocated] = true  $IsNotNull(this, Chunker); free requires $Heap[this, $allocated] = true  $IsNotNull(this, Chunker); modifies $Heap; modifies $Heap; ensures $Length($result) ≤ $Heap[this, Chunker.ChunkSize]; ensures $Length($result) ≤ $Heap[this, Chunker.ChunkSize]; ensures (  $pc: ref $pc ≠ null  $Heap[$pc, $allocated] = true  $Heap[$pc, $ownerRef] = $Heap[$result, $ownerRef]  $Heap[$pc, $ownerFrame] = $Heap[$result, $ownerFrame]  ($Heap[$pc, $ownerFrame] = $PeerGroupPlaceholder  ¬($Heap[$Heap[$pc, $ownerRef], $inv] <: $Heap[$pc, $ownerFrame]))  $Heap[$pc, $inv] = $typeof($pc)); ensures (  $pc: ref $pc ≠ null  $Heap[$pc, $allocated] = true  $Heap[$pc, $ownerRef] = $Heap[$result, $ownerRef]  $Heap[$pc, $ownerFrame] = $Heap[$result, $ownerFrame]  ($Heap[$pc, $ownerFrame] = $PeerGroupPlaceholder  ¬($Heap[$Heap[$pc, $ownerRef], $inv] <: $Heap[$pc, $ownerFrame]))  $Heap[$pc, $inv] = $typeof($pc)); free ensures $Heap[$result, $allocated] = true  $IsNotNull($result, System.String); free ensures $Heap[$result, $allocated] = true  $IsNotNull($result, System.String); free ensures (  $o: ref $o ≠ null  ¬old($Heap)[$o, $allocated]  $Heap[$o, $allocated]  $Heap[$o, $inv] = $typeof($o)); free ensures (  $o: ref $o ≠ null  ¬old($Heap)[$o, $allocated]  $Heap[$o, $allocated]  $Heap[$o, $inv] = $typeof($o)); ensures (  $o: ref $o ≠ null  old($Heap)[$o, $allocated] = true  old($Heap)[$Heap[$o, $ownerRef], $allocated] = true  old($Heap)[$o, $ownerRef] = $Heap[$o, $ownerRef]  old($Heap)[$o, $ownerFrame] = $Heap[$o, $ownerFrame]); ensures (  $o: ref $o ≠ null  old($Heap)[$o, $allocated] = true  old($Heap)[$Heap[$o, $ownerRef], $allocated] = true  old($Heap)[$o, $ownerRef] = $Heap[$o, $ownerRef]  old($Heap)[$o, $ownerFrame] = $Heap[$o, $ownerFrame]); free ensures (  $o: ref, $f: name $f ≠ $inv  $o ≠ null  old($Heap)[$o, $allocated] = true  (old($Heap)[$o, $ownerFrame] = $PeerGroupPlaceholder  ¬(old($Heap)[old($Heap)[$o, $ownerRef], $inv] <: old($Heap)[$o, $ownerFrame]))  (¬IsStaticField($f)  ¬IsDirectlyModifiableField($f))  old($Heap)[$o, $f] = $Heap[$o, $f]); free ensures (  $o: ref, $f: name $f ≠ $inv  $o ≠ null  old($Heap)[$o, $allocated] = true  (old($Heap)[$o, $ownerFrame] = $PeerGroupPlaceholder  ¬(old($Heap)[old($Heap)[$o, $ownerRef], $inv] <: old($Heap)[$o, $ownerFrame]))  (¬IsStaticField($f)  ¬IsDirectlyModifiableField($f))  old($Heap)[$o, $f] = $Heap[$o, $f]); free ensures (  $o: ref old($Heap)[$o, $inv] = $Heap[$o, $inv]  old($Heap)[$o, $allocated] ≠ true); free ensures (  $o: ref old($Heap)[$o, $inv] = $Heap[$o, $inv]  old($Heap)[$o, $allocated] ≠ true); free ensures (  $o: ref old($Heap)[$o, $allocated]  $Heap[$o, $allocated]); free ensures (  $o: ref old($Heap)[$o, $allocated]  $Heap[$o, $allocated]);

32. Example BoogiePL implementation Chunker.NextChunk(this: ref) returns ($result: ref) { var temp0: ref, local4: ref, stack0i: int, stack1i: int, stack1o: ref, stack0b: bool, stack0o: ref, stack2i: int, s: ref, return.value: ref, SS$Display.Return.Local: ref; var temp0: ref, local4: ref, stack0i: int, stack1i: int, stack1o: ref, stack0b: bool, stack0o: ref, stack2i: int, s: ref, return.value: ref, SS$Display.Return.Local: ref; entry: entry: // … // … // ----- load field ----- Chunker.ssc(14,7) // ----- load field ----- Chunker.ssc(14,7) assert this ≠ null; assert this ≠ null; stack1o := $Heap[this, Chunker.src]; stack1o := $Heap[this, Chunker.src]; // … // … // ----- binary operator ----- Chunker.ssc(14,7) // ----- binary operator ----- Chunker.ssc(14,7) stack0b := stack0i > stack1i; stack0b := stack0i > stack1i; // ----- branch ----- Chunker.ssc(14,7) // ----- branch ----- Chunker.ssc(14,7) goto true5814to5848, false5814to5831; goto true5814to5848, false5814to5831; true5814to5848: true5814to5848: assume stack0b = true; assume stack0b = true; …

33. Example BoogiePL // ----- call ----- Chunker.ssc(17,9) // ----- call ----- Chunker.ssc(17,9) assert stack0o ≠ null; assert stack0o ≠ null; call s := System.String.Substring$System.Int32(stack0o, stack1i); call s := System.String.Substring$System.Int32(stack0o, stack1i); // … // … // ----- store field ----- Chunker.ssc(19,7) // ----- store field ----- Chunker.ssc(19,7) assert this ≠ null; assert this ≠ null; assert ¬($Heap[this, $inv] <: Chunker); assert ¬($Heap[this, $inv] <: Chunker); $Heap[this, Chunker.n] := stack0i; $Heap[this, Chunker.n] := stack0i; // … // … // ----- return // ----- return $result := stack0o; $result := stack0o; return; return;}

34. 5. Verification Conditions Automatic theorem prover Can be hidden from programmer Generates counterexamples Interactive theorem prover Requires gurus Not limited by built-in decision procedures

35. Performance Considerations Generate verification conditions that the theorem prover can handle quickly “Efficient” encodings of axioms “Efficient” weakest preconditions

36. Initial Verifier Experience Pilot production project “It changes how you think” Several smaller (300-1500 lines) case studies Parts of Spec# program verifier External academic use

37. Conclusions Because of tool support, we’re ready for programming at the next level of rigor Current work Specification/programming/verification methodology Performance Technology transfer Engineering effort Technology sharing Teaching Case studies BoogiePL as common intermediate logic http://research.microsoft.com/~leino http://research.microsoft.com/specsharp Download Spec# from here

38. © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.