1. Chair of Software Engineering 1 Doing more with contracts Bertrand Meyer, Ilinca Ciupa, Andreas Leitner, Lisa Liu SOFSEM, Harrachov, January 2007 ETH Zurich, Chair of Software Engineering

2. 2 The three questions  What does it expect?  What does it promise?  What does it maintain?

3. 3 Progress in software quality Tools, techniques, methods, languages, software quality in general… Market forces work not for very high quality but for good enough software Collective decision to turn IT into a service business … have been improving steadily… … but not enough!

4. 4 Where best to apply our efforts? End-user applications Compilers & operating systems General-purpose components Specialized components Relies on

5. 5 Components should be contracted Definition of what each element of the functionality:  Expects (precondition)  Promises (postcondition)  Maintains (invariant) Does not have to be complete (but wait)

6. 6 Design by Contract Every software element is intended to satisfy a certain goal, for the benefit of other software elements (and ultimately of human users). This goal is the element’s contract. The contract of any software element should be  Explicit  Part of the software element itself

7. 7 A class with contracts class BANK_ACCOUNT create make feature make (n : STRING) -- Set up with name n require n /= Void do name := n balance := 0 ensure name = n end name : STRING balance : INTEGER deposit ( v : INTEGER) -- Add amount v do balance := balance + v ensure balance = old balance + v end invariant name /= Void balance >= 0 end ensure name = n require n /= Void ensure balance = old balance + v invariant name /= Void balance >= 0

8. 8 The correctness of a class For every creation procedure cp : {pre cp } do cp {INV and post cp } For every exported routine r : {INV and pre r } do r {INV and post r } a. h (…) create a. make (…) S1 S2 S3 S4 a. g (…) a. f (…)

9. 9 What we do with contracts today Write better software  Analysis  Design  Reuse  Implementation  Bug avoidance Document software automatically Help project managers do their job Perform systematic testing Guide the debugging process (with run-time monitoring)

10. 10 deferred class VAT inherit TANK feature in_valve, out_valve: VALVE fill is -- Fill the vat. require in_valve. open out_valve. closed deferred ensure in_valve. closed out_valve. closed is_full end empty, is_full, is_empty, gauge, maximum,... [Other features]... invariant is_full = (gauge >= 0.97  maximum) and (gauge <= 1.03  maximum) end Contracts for analysis, specification

11. 11 Seamless development with Eiffel & contracts Single notation, tools, concepts, principles Continuous, incremental development Keep model, implementation, documentation consistent Reversibility Example classes: PLANE, ACCOUNT, TRANSACTION… STATE, COMMAND… HASH_TABLE… TEST_DRIVER… TABLE… Analysis Design Implemen- tation V&V Generali- zation

12. 12 Documentation: the contract view of a class class BANK_ACCOUNT create make feature make (n : STRING) -- Set up with name n require n /= Void do name := n balance := 0 ensure name = n end name : STRING balance : INTEGER deposit ( v : INTEGER) -- Add amount v do balance := balance + v ensure balance = old balance + v end invariant name /= Void balance >= 0 end ensure name = n require n /= Void ensure balance = old balance + v invariant name /= Void balance >= 0

13. 13 Lists with cursors "Prague" Cursor index count1 forth afterbefore

14. 14 Command “forth″

15. 15 Where the cursor may go count+1 Valid cursor positions item count1 afterbefore 0

16. 16 From the invariant of class LIST Valid cursor positions

17. 17 Contracts and inheritance r is require  ensure  r is require  ensure  a1: A a1.r (…) … Correct call in C: if a1.  then a1. r (...) -- Here a1.  holds end r ++ CADB Client Inheritance ++ Redefinition

18. 18 Assertion redeclaration rule When redeclaring a routine, we may only:  Keep or weaken the precondition  Keep or strengthen the postcondition

19. 19 A simple language rule does the trick! Redefined version may have nothing (assertions kept by default), or require else new_pre ensure then new_post Resulting assertions are:  original_precondition or new_pre  original_postcondition and new_post Assertion redeclaration rule in Eiffel

20. 20 Contracts as a management tool High-level view of modules for the manager:  Follow what’s going on without reading the code  Enforce strict rules of cooperation between units of the system  Control outsourcing

21. 21 Contracts for testing and debugging Contracts provide the right basis:  Testing is there to find bugs  A bug is a discrepancy between intent and reality  Contracts describe intent A contract violation always signals a bug:  Precondition violation: bug in client  Postcondition violation: bug in routine In EiffelStudio: select compilation option for run-time contract monitoring at level of class, cluster or system.

22. 22 Anecdotal & non-anecdotal evidence HP 1: invariant r = 2 ^ i HP 2: Eiffel messes up our great system! Axa Rosenberg: postcondition fails in deep_clone of TWO_WAY_LIST ! Patrice Chalin study (Concordia): Eiffel programmers do use contracts day in, day out.

23. 23 Should we test software? Some evidence of why we should:  Limitations of proof technology  Properties not formally specified Tests as complement of proofs?

24. From a survey of 240 companies in N. America and Europe:  8% release software to beta sites without testing  83% of their developers don't like to test code.  53% don't like to test their own code because they find it tedious.  30% don't like to test because they find testing tools inadequate. Testing is tedious

25. 25 “Automated testing” What can be automated:  Test execution  Robust execution  Regression testing  Test case generation (test suites)  Test result verification (test oracles)  Test scheduling  Test case minimization

26. Automated vs. manual unit tests Automatically generated test cases complement manual ones:  Contracts may be “wrong”  Implementation is unlikely to establish postcondition on input satisfying A -> ¬B.  Test cases will likely only be written for B (and not for A -> ¬B ).

27. 27 AutoTest: an automatic test framework  Input is a set of classes  AutoTest generates instances and calls features with automatically selected arguments  Automatic tests rely on contracts:  Precondition violations: skip  Postcondition/invariant violation: bingo!  Manual tests can be added explicitly:  Relevant ones are determined automatically  Any test (manual or automated) that fails becomes part of the test suite

28. 28 AutoTest demo

29. 29 Automated testing and slicing auto_test system.ace BANK_ACCOUNT STRING create {STRING} v1 v1. wipe_out v1. append_character (’c’) v1. append_double (2.45) create {STRING} v2 v1. append_string (v2) v2. fill (’g’, 254343)... create {BANK_ACCOUNT} v3.make (v2) v3. deposit (15) v3. deposit (100) v3. deposit (-8901)... class BANK_ACCOUNT create make feature make (n : STRING) require n /= Void do name := n balance := 0 ensure name = n end name : STRING balance : INTEGER deposit (v : INTEGER) do balance := balance + v ensure balance = old balance + v end invariant name /= Void balance >= 0 end

30. Robust execution

31. 31 Some AutoTest results (random strategy) Library TotalFailed TotalFailed EiffelBase 40,0003%20006% Data Structures 21,0005%14006% Gobo Math 15001%1406% TESTS ROUTINES

32. 32 Result analysis (Raluca Borga, U. of Cluj-Napoca, diploma work at ETH) Fault classification (manual):  Specification error  Implementation error  Inheritance  Don’t know About 50% implementation, 50% specification  Results polluted by “void” issue (Will no longer exist thanks to attached types, see ECOOP 2005 paper)

33. AutoTest strategies Random strategy  Use random input Planning strategy  Employ information from postcondition to satisfy preconditions...

34. 34 Basic strategy  Object pool  For each type, initialize through creation procedures (constructors)  This requires proper ordering of classes  Diversify through procedures  Routine arguments  Basic values: heuristics for each type  Objects: get from pool  Invoke selected routines  AutoTest tests all routines, including inherited ones (“Fragile base class” issue)

35. 35 Basic idea: random testing may find bugs faster if inputs evenly spread over range of possible values Example: r (m, n: INTEGER) do … end m n Random inputs m n ART Adaptive Random Testing (ART, Chen et al.)

36. 36 So far ART only applied to numeric inputs, on which a total order relation exists Multi-field objects are not members of a totally ordered set Need to define notion of distance between objects Extending ART to objects

37. 37 Object distance (Ilinca Ciupa) p ↔ q combination ( type_distance (p. type, q. type), field_distance (p, q), recursive_distance ( {[p. r ↔ q. r] | r  Reference_attributes } ) = Δ

38. 38 The role of argument-less boolean queries  Externally, boolean queries often serve as preconditions and appear in invariants  Internally, boolean queries often serve as branch conditions Lists: before, after, off, is_first, is_last, is_empty… Account: is_overdraft… "Prague" Cursor index count1 after before is_first is_last

39. 39 Boolean query conjecture The argument-less boolean queries of a well-written class yield a partition of the object state space that helps the testing process.

40. 40 Boolean query coverage (Lisa Liu) A set of tests for a class satisfies boolean query coverage if and only if the execution of these tests can cover all the reachable abstract object states for that class. Strategy:  Through constraint solving, generate states that satisfy boolean queries in invariant (e.g. after implies off)  Through theorem proving, remove states that do not satisfy rest of invariant (e.g. count < capacity)

41. Chair of Software Engineering 41 "Prague" Cursor index count1 after before is_first is_last

42. 42 Experimental results ClassRandomBoolean Queries Routine Coverage Bugs Found Routine Coverage Bugs Found INT_STACK 33% 1 100% 2 LINKED_LIST 85% 1 99% 7 BINARY_TREE 88% 5 100% 11 ARRAYED_SET 84% 1 100% 6 FIXED_LIST 93% 12 99% 12

43. 43 AutoTest developments  Large-scale extensive tests (cluster computing)  Comparison with manual efforts  Scaling up (memory, time) (mostly done)  Rigorous assessment of testing criteria & strategies  Complete integration with EiffelStudio environment  Background, unobtrusive, continuous testing  Automatic regression testing  Distributed cooperative testing (“Testi@home”)

44. 44 Towards a real framework  Pluggable strategies  Flexible execution mechanism  Robust execution mechanism  Unified interface  Integrate tightly with manual testing  Concise output

45. 45 Proving classes class LINKED_LIST [G] feature … remove_front -- Remove first item require not empty do first := first. right ensure count = old count – 1 first = old item (2) end … end first right

46. 46 Issues in proving classes 1.Providing full contracts 2.Devising models of the underlying world 3.Devising proof rules (i.e. semantics of the programming language) 4.Taking advantage of inheritance on the proof side 5.Doing the proofs, mechanically

47. 47 The contract language In Eiffel (and other formalisms based on Design by Contract): mostly boolean expressions of the language, plus some extensions such as old. Seems not expressive enough — but is!

48. 48 Beefing up expressive power Eiffel Model Library Components to prove (e.g. EiffelBase)

49. 49 Eiffel Model Library (also known as MML) (With Bernd Schoeller, Tobias Widmer) Classes correspond to mathematical concepts: SET [G], FUNCTION [G, H ], TOTAL_FUNCTION [G, H ], RELATION [G, H ], SEQUENCE [G ], … Completely applicative: no attributes (fields), no implemented routines (all completely deferred) Specified with contracts (unproven) reflecting mathematical properties Expressed entirely in Eiffel

50. 50 Example MML class class SEQUENCE [G] feature count : NATURAL -- Number of items last : G -- Last item extended (x) : SEQUENCE [G] -- Identical sequence except x added at end ensure Result. count = count + 1 Result. last = x Result. sub (1, count) ~ Current mirrored : SEQUENCE [G] -- Same items in reverse order ensure Result. count = count … end

51. 51 Specifying lists class LINKED_LIST [G] feature … remove_front -- Remove first item require not empty do first := first. right ensure end … end first right count = old count – 1 first = old item (2) model = old model. tail

52. 52 Principles Very simple mathematics only  Logic  Set theory

53. 53 Proofs: current progress Basic theory of O-O programming with pointers Progress towards frame issue Proof engine: Boogie (Microsoft Research) Several publications in progress

54. 54 Tests or proofs? Tests and proofs!  Failed proof: proof obligations remain – candidates for tests  Model checking counter-example: try to generate test data that exercises it TAP Tests And Proofs ETH Zurich, 11-13 Feb 2007 http://tap.ethz.ch

55. 55 Concurrency What do contracts become in a concurrent context?

56. 56 put

57. 57 store (b : [G ] ; v : G ) -- Store v into b. require not b. is_full do … ensure not b. is_empty end QUEUEBUFFER my_queue : [T ] … if not my_queue. is_full then store (my_queue, t ) end BUFFER QUEUE put

58. 58 On separate target, precondition becomes wait condition From preconditions to wait-conditions store (buffer : separate BUFFER [INTEGER ] ; v : INTEGER) -- Store v into buffer. require not buffer. is_full v > 0 do buffer. put (v) ensure not buffer. is_empty end... store (my_buffer, 10 )

59. 59 SCOOP in a nutshell  No intra-object-concurrency  One keyword: separate, indicates thread of control is “elsewhere”  Reserve one or more objects through argument passing  Preconditions become wait conditions  Automatic resynchronization on queries (lazy wait)  Exception-based mechanism to break lock  Asynchronous exceptions

60. 60 Dining philosophers class PHILOSOPHER inherit PROCESS rename setup as getup redefine step end feature {BUTLER} step do think ; eat (left, right) end eat (l, r : separate FORK) -- Eat, having grabbed l and r. do … end end

61. 61 Contract-Driven Development: demo (Andreas Leitner)

62. 62 Introductory programming course Since 2003 200-300 students Uses object technology, Eiffel, Design by Contract Outside-in approach (“inverted curriculum”): work from existing software  TRAFFIC library Textbook in progress, Touch of Class, available online google touch class meyer

63. 63 Traffic GUI example

64. 64 Traffic GUI example

65. 65 The Traffic software (Michela Pedroni) EiffelMedia - OpenGL (3D Drawing) - 2D Drawing Primitives - Sound - Collision detection Gobo - Data structures - XML parsing - File handling EWG - Library wrapping Traffic - 2D and 3D display of a city - Transportation lines, roads, schedules - Random buildings - Vehicles, passengers EiffelBase - Data structures - Kernel classes - Basic data types

66. 66 The background technology: Eiffel Method, language, IDE (EiffelStudio), libraries Constantly refined since 1985 ECMA standard (2005); ISO standard (2006)

67. 67 Some Eiffel projects Axa Rosenberg Investment management: from $2 billion to >$40 billion 2 million lines of Eiffel Chicago Board of Trade Price reporting system Eiffel + CORBA + Solaris + Windows + … Boeing (Xontech) Large-scale Eiffel simulations of missile defense system Swedish social security: accident reporting & management

68. Chair of Software Engineering 68 Open-Source EiffelStudio

69. 69 References Object-Oriented Software Construction, 2 nd edition (Prentice Hall) se.ethz.ch eiffelstudio.origo.ethz.ch