Challenges in increasing tool support for programming K. Rustan M. Leino Microsoft Research, Redmond, WA, USA 23 Sep 2004 ICTAC Guiyang, Guizhou, PRC joint work with Mike Barnett, Bor-Yuh Evan Chang, Robert DeLine, Manuel Fähndrich, Peter Müller, David A. Naumann, Wolfram Schulte, and Herman Venter

Software engineering problem Building and maintaining large systems that are correct

Approach Specifications record design decisions –bridge intent and code Tools amplify human effort –manage details –find inconsistencies –ensure quality

Spec# demo

Challenge areas 0.Program verification 1.A better language 2.Methodology

0. Program verification an extreme form of managing the details

Basic architecture of a verifier verification condition generator theorem prover verification condition program with specifications correct or list of errors

Generating verification conditions int AbsX(Robot c) requires c null; ensures 0 result; { if (0 c.x) { a := c.x; } else { a := c.x; } return a; } Weakest preconditions 0 result 0 a c null 0 sel(c,x) c null (0 c.x c null 0 sel(c,x)) (¬(0 c.x) c null 0 sel(c,x)) c null c null (0 c.x c null 0 sel(c,x)) (¬(0 c.x) c null 0 sel(c,x))

Analyzing verification conditions Automatic theorem prover –can be hidden from programmer –generates counterexamples Interactive theorem prover –requires gurus –not limited by built-in decision procedures

Theory opportunities Developing good encodings of verification conditions Combining inference techniques for different domains Combining inference and proving

Spec# verifier architecture V.C. generator automatic theorem prover verification condition Spec# correct or list of errors Spec# compiler MSIL (bytecode) translator Boogie PL inference engine Boogie

the programming language is the software engineer's primary thinking and working tool 1. A better language

Designing more rigor into the language Type system –impose coarse-grained restrictions –easy to understand and use –sound efficient checking General specifications –enforce by mix of dynamic and static checking –dont worry about completeness or decidability Develop good defaults non-null types requires Pre; modifies Frame; ensures Post; Examples requires this.Valid; modifies this.*; ensures true;

Enforcing specifications Dynamic checking –may leave out expensive checks, if not required for soundness example: modifies clauses Static checking –requires more effort from the user example: modifies clauses –modular checking a necessity

Language design problem: example procedure P (Robot c) { int a := c.x; error: possible null dereference

Language design problem: example procedure P (Robot c) { Robot d := new Robot(); int a := d.x; if (c null) { a := c.x; if (c.f null) { a := sqrt(a); OpenFile(b.txt); c.M(); a := c.f.g; } } error?

Migration problems To the new language –how does the language compare to existing ones? From (!) the new language –development organizations may have process requirements

2. Programming methodology light-weight rules for simpler reasoning

Pre/post are not enough class C { private int x; private int y; public void Update(int z) requires 0 z && x y; { T[ ] a := new T[z]; T[ ] b := new T[y – x]; … } … }

Object invariants class C { private int x; private int y; invariant x y; public void Update(int z) requires 0 z; { T[ ] a := new T[z]; T[ ] b := new T[y – x]; … } … }

The reentrance problem class C { int x; int y; invariant x y; public void M() { x++; P(…); y++; } … } invariant may be temporarily broken here invariant is restored here what if P calls back into M? invariant assumed to hold on entry invariant checked to hold on exit

The reentrance solution class C { int x; int y; invariant x y; public void M() requires this.inv = Valid; { expose (this) { x++; P(…); y++; } } … } represent explicitly that invariant holds (without revealing what the invariant is) change this.inv from Valid to Mutable check invariant; then, change this.inv from Mutable to Valid field updates allowed only on Mutable objects

The abstraction problem class Set { Hashtable h; invariant h null; public void Add(Element e) requires this.inv = Valid; { expose (this) { h.Add(e, e);} … } class Hashtable { public void Add(object o) requires this.inv = Valid; … } how do we know h is Valid here?

The abstraction solution class Set { owned Hashtable h; invariant h null; public void Add(Element e) requires this.inv = Valid; { expose (this) { h.Add(e, e);} … } class Hashtable { public void Add(object o) requires this.inv = Valid; … } set uniquely owns set.h set.inv=Valid implies set.h.inv=Valid ownership temporarily relinquished here ownership re-obtained here

Wanted: methodology for advanced invariants multiple owners delegates (closures) events …

Conclusions Because of tool support, were ready for programming at the next level of rigor Still requires theory support –language design –program semantics –specification techniques –inference algorithms –decision procedures –… Methodology is underdeveloped –Can programming theory yet fully explain why real big programs work? –programming theory has not kept up with practice Some papers at