Deriving formal specifications (almost) automatically Glenn Ammons and Ras Bodik and James R. Larus

Three pillars of formal verification Model checkers and other verifiers –well automated (SLAM, Spin, type checkers, Vault) Program abstractors –getting there (SLAM, Engler’s metacompiler) Formal specifications –Written by hand –Our goal: bring automation to writing formal specifications

Deriving specs is feasible Well-debugged software exists –Good code obeys the rules, but doesn’t state them clearly Common behavior is good behavior –Because testing exposes common behavior Programmers exist –But they don’t want to write specs!

Rules describe good behavior A rule is a nondeterministic finite automaton: S T = XNextEventXSetSelectionOwner(T) XGetSelectionOwner F

Rules are derived from traces, with user guidance XtAppNextEvent() = event(type = 5, window = 22, time = 3:15) XtDispatchEvent(type = 5, window = 22, time = 3:15) XtFree(NULL) XtMalloc(size = 8) = 0x10 XmuInternStrings(names = 0x20, count = 2, atoms_return = 0x10) XtOwnSelection(widget = 0x30, selection = 1, time = 3:15) And so on: the more traces the better

Overview Traces Bugs ! Programs or traces (buggy?) Rule learner Matcher Program abstractor Seeds Abstraction prescription Rules Abstract programs or traces

Case study: selections in X11 The rule: SetSelectionOwner must be passed a timestamp from an Xevent 25 programs from the X11 distribution and the contrib directories (all used selections) Verification done over traces (not statically) Found two bugs in 29 static uses Found three benign violations

To do Static checking: typestates Better simplifier Better user interaction What else can we learn? –Protocols like socket/bind/accept/close –Operations on data structures

Power What else can we do with this stuff? Compare with Ernst

Detailed figure?

Running example

Testing vs. verification Examines the complete programs Examines some inputs For better coverage, write more test cases Examines only some aspects of programs Examines all inputs For better coverage, write more specs The practice sees writing test cases as easier than writing formal models and specifications, so testing dominates.