1. Technologies for finding errors in object-oriented software K. Rustan M. Leino Microsoft Research, Redmond, WA Lecture 3 Summer school on Formal Models of Software 5 Sep 2003, Tunis, Tunisia

2. Review: An object-oriented programming notation C::=w := E |assert P |var w in C end |C 0 ; C 1 |if P then C 0 else C 1 end |o.f := E |x := new(T) |w := o.m(E 0, E 1 )

3. Invariants: straw man class T { // field declarations... invariant J ; T(...) requires P modifies v ensures Q { … } method m(...) requires R modifies w ensures T { … } … class T { // field declarations... T(...) requires P modifies v ensures Q J { … } method m(...) requires R J modifies w ensures T J { … } … Object invariants hold on public method boundaries and are shorthands for pre/post-conditions

4. Invariants, example class T { private int x, y ; invariant 0 x < y ; public T() { x = 0 ; y = 1 ; } public method m() modifies x, y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … class T { private int x, y ; public T() ensures 0 x < y { x = 0 ; y = 1 ; } public method m() requires 0 x < y modifies x, y ensures 0 x < y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } …

5. Invariants, problems class T { private int x, y ; invariant 0 x < y ; public T() { x = 0 ; y = 1 ; } public method m() modifies x, y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … class T { private int x, y ; public T() ensures 0 x < y { x = 0 ; y = 1 ; } public method m() requires 0 x < y modifies x, y ensures 0 x < y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … callers are expected to establish property about internal data structure

6. Invariants, problems class T { private int x, y ; invariant 0 x < y ; public T() { x = 0 ; y = 1 ; } public method m() modifies x, y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … class T { private int x, y ; public T() ensures 0 x < y { x = 0 ; y = 1 ; } public method m() requires 0 x < y modifies x, y ensures 0 x < y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … possible solution (?): callers dont need to be checked for this precondition it holds automatically!

7. Invariants, problems class T { private int x, y ; invariant 0 x < y ; public T() { x = 0 ; y = 1 ; } public method m() modifies x, y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … class T { private int x, y ; public T() ensures 0 x < y { x = 0 ; y = 1 ; } public method m() requires 0 x < y modifies x, y ensures 0 x < y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … invariant does not hold here

8. Invariants, problems class T { private int x, y ; invariant 0 x < y ; public T() { x = 0 ; y = 1 ; } public method m() modifies x, y { assert y-x 0 ; x = x + 3 ; y = 4 * y ; } … class T { private int x, y ; public T() ensures 0 x < y { x = 0 ; y = 1 ; } public method m() requires 0 x < y modifies x, y ensures 0 x < y { assert y-x 0 ; x = x + 3 ; p(...) ; y = 4 * y ; } … invariant does not hold here, so what if p calls m?!

9. The problem of specifying modifications class Kitchen { private Dishes d ; private bool hasGarbage ; private Stove s ; private Light l ;... public method SpiffUp() modifies hasGarbage, s.foodPieces, s.big.sufaceColor, … { d.Clean() ; hasGarbage = false ; s.Wipe() ; l.TurnOff() ;... } class Stove { private Burner big ; private Burner small ; private int foodPieces ; private Knob[] knobs ;... public method Wipe() modifies foodPieces, big.surfaceColor, … { big.Polish() ; small.Polish() ; foodPieces = 0 ;... } these lists are long, and they mention private state

10. The problem of specifying modifications class Kitchen { private Dishes d ; private bool hasGarbage ; private Stove s ; private Light l ;... public method SpiffUp() modifies hasGarbage, s.foodPieces, s.big.sufaceColor, … { d.Clean() ; hasGarbage = false ; s.Wipe() ; l.TurnOff() ;... } class Stove { private Burner big ; private Burner small ; private int foodPieces ; private Knob[] knobs ;... public method Wipe() modifies foodPieces, big.surfaceColor, … { big.Polish() ; small.Polish() ; foodPieces = 0 ;... } possible solution (?): dont need to declare modifications of private state it cant be observed anyhow

11. The problem of specifying modifications class Kitchen { private Dishes d ; private bool hasGarbage ; private Stove s ; private Light l ;... public method SpiffUp() modifies { d.Clean() ; hasGarbage = false ; s.Wipe() ; assert hasGarbage ; l.TurnOff() ;... } class Stove { private Burner big ; private Burner small ; private int foodPieces ; private Knob[] knobs ;... public method Wipe() modifies { big.Polish() ; small.Polish() ; foodPieces = 0 ;... } SpiffUp treats Wipe as if Wipe modified nothing, so what if Wipe calls a method in Kitchen that sets hasGarbage to true?!

12. Soundness of verification Soundness = verification finds all errors Soundness follows from: – pre- and postconditions are the same for caller and callee Note: In addition to soundness, we want something usable

13. Methodology object invariant declaration – class T { int x, y ; invariant x < y ; special variable st: {Invalid, Valid} Idea: program invariant (o o.st = Invalid Inv(o)) st is changed by commands pack and unpack holds at every program point! for any o: T, we write Inv(o) o.x < o.y

14. pack and unpack pack o assert o.st = Invalid ; assert Inv(o) ; o.st := Valid unpack o assert o.st = Valid ; o.st := Invalid

15. Example class T { int x, y ; invariant 0 x < y ; method init(t) requires t.st = Invalid modifies t.st, t.x, t.y ensures t.st = Valid { t.x := 0 ; t.y := 1 ; pack t } method m(t) requires t.st = Valid modifies t.x, t.y { unpack t ; t.x := t.x + 3 ; t.y := 4 * t.y ; pack t } receiver parameter (this, self, current)

16. Program invariant (o o.st = Invalid Inv(o)) x := new(T)... ; assume x.st = Invalid pack o... ; assert Inv(o) ; o.st := Valid unpack o... ; o.st := Invalid o.f := E assert o.st = Invalid ;... Inv(o) can mention only the fields of o

17. Methodology, summary invariant... st: {Invalid, Valid} pack, unpack modifications of o.f require o.st=Invalid Inv(o) can mention only the fields of o (o o.st = Invalid Inv(o))

18. Methodology, extended component declarations – class Kitchen { component Stove s ; st: {Invalid, Valid, Committed} Idea: program invariant (o o.st=Invalid (Inv(o) (pComp(o) p.st=Committed))) pack o and unpack o change st for o and o's components for any k: Kitchen, we have k.s Comp(k)

19. pack and unpack, extended pack o assert o.st = Invalid Inv(o) ; assert (p Comp(o) p.st=Valid) ; o.st := Valid ; change st such that (p (p Comp(o) p.st=Committed) (p Comp(o) p.st=p.st 0 )) unpack o assert o.st = Valid ; change st such that (p (p Comp(o) p.st=Valid) (p Comp(o) p.st=p.st 0 )) ; o.st := Invalid ;

20. Example class Kitchen { method SpiffUp(k) requires k.st=Valid … { unpack k ; k.d.Clean() ; k.s.Wipe() ; pack k } class Stove { method Wipe(s) requires s.st=Valid … s d Kitchen k Stove Dishes Valid Committed

21. Valid Invalid Example class Kitchen { method SpiffUp(k) requires k.st=Valid … { unpack k ; k.d.Clean() ; k.s.Wipe() ; pack k } class Stove { method Wipe(s) requires s.st=Valid … s d Kitchen k Stove Dishes Committed Valid

22. Program invariant (o o.st=Invalid (Inv(o) (pComp(o) p.st=Committed))) x := new(T)... ; assume x.st=Invalid pack o, unpack o o.f := E assert o.st=Invalid ;... Inv(o) can mention only the fields of o and of o.p for any component field p

23. Extended methodology, summary invariant... component... st: {Invalid, Valid, Committed} pack, unpack modifications of o.f require o.st=Invalid Inv(o) can mention only the fields of o and of o.p for any component field p (o o.st=Invalid (Inv(o) (pComp(o) p.st=Committed)))

24. Verification system We let st be used in method specifications (requires, modifies, ensures) We must address the Problem of Specifying Modifications

25. An alternative heap model class T { f: U; g: V;... } In my previous lecture: – many one-dimensional map variables (one per field) – x := o.f= x := f[o] = x := select(f, o) – o.f := E= f[o] := E = f := store(f, o, E) Now: – one two-dimensional map variable – x := o.f= x := Heap[o, f] = x := select(Heap, o, f) – o.f := E= Heap[o, f] := E = Heap := store(Heap, o, f, E)

26. Meaning of modifies modifies w = modifies Heap ensures (o,f Heap[o,f] = Heap 0 [o,f] (o,f) w 0 ) viewed as set of object/field-name pairs

27. Meaning of modifies modifies w = modifies Heap ensures (o,f Heap[o,f] = Heap 0 [o,f](o,f) w 0Heap 0 [o,alloc] )

28. Meaning of modifies modifies w = modifies Heap ensures (o,f Heap[o,f] = Heap 0 [o,f](o,f) w 0Heap 0 [o,alloc]Heap 0 [o,st]=Committed )

29. Example class Kitchen { method SpiffUp(k) requires k.st=Valid modifies k.hasGarbage { unpack k ; k.d.Clean() ; k.hasGarbage := false ; k.s.Wipe() ; assert k.hasGarbage ; pack k } class Stove { method Wipe(s) requires s.st=Valid modifies s.foodPieces

30. Another example method m(p) requires p.st=Committed { var y, z in y := p.x ; z := sqrt(49) ; assert y = p.x end }

31. Yet another example method m(p) requires p.st=Committed { p.x := p.x + 1 } Is this field update allowed?

32. Soundness Pre- and postconditions are the same for callers and callees, so verification system is sound!

33. Methodology conclusions Invariants different from pre/post-conditions Resulting program invariants hold at every program point Uses pack/unpack commands, but good defaults can be constructed for these No linear-type system Components are not unique references objects (pointers) can freely be copied Fields can freely be read No additional features of abstraction needed to support the specification of modifications Sound For more details, see Barnett, DeLine, Fahndrich, Leino, Schulte, 2003 (reference [65] in my lecture-notes bibliography).

34. Research challenges experience, understanding of limits extensions to support more good programs

35. Summary of lectures Automatic high-precision programming tools can be built for object-oriented languages – need more research to explore other design trade-offs Semantics of programming languages (including object-oriented languages) can be defined from the semantics of simple commands Object-oriented programming requires methodology – needs further research See lecture notes for selected bibliographic references. http://research.microsoft.com/leino