A Pragmatic View of Formal Methods The Hi-Lite Project Robert B K Dewar SSS ‘11 President & CEO, AdaCore Emeritus Professor of Computer Science, New York University Feb 10 th 2011

Slide: 2 Copyright © 2009 AdaCore An observation from Yogi Berra In theory there is no difference between theory and practice. In practice there is. Variously attributed to Yogi Berra, Einstein etc.

Slide: 3 Copyright © 2009 AdaCore

Slide: 4 Copyright © 2009 AdaCore Correctness and Reliability Not really the same thing Often merged in people’s thinking The issue of large reliable programs Correctness means adherence to some formal model Reliability means that a program works Many reasons why a correct program might be unreliable. And a reliable program may well be incorrect

Slide: 5 Copyright © 2009 AdaCore We can do Pretty Well in Practice No one has ever died on a commercial airplane as a result of a software bug. We have however had some hair raising close calls But a close call is a close call When it comes to avionics, software is not the weak link in the chain. But we could do better And most certainly best practices are not widely applied –Cars –Medical Equipment –Critical financial applications

Slide: 6 Copyright © 2009 AdaCore Safety And Security Safety means we don’t have bugs that kill people Security means bad guys can’t cause chaos We have lots of bad guys around who know a lot Any safety critical system is security critical now How vulnerable are we TV thinks we are very vulnerable –Alias and Avionics systems –24 and Air Traffic Control Don’t be too quick to dismiss fiction –Clancy crashed a plane long before 9/11

Slide: 7 Copyright © 2009 AdaCore Proving Entire Programs Correct A nice conceptual goal But definitely a case where theory and practice are not on the same page Fundamental problems stand in the way

Slide: 8 Copyright © 2009 AdaCore Producing a Full Formal Specification No use unless we find an appropriate level of abstraction. Every program is a full formal specification in some sense, but that’s not useful Every generation of programming languages has been sold as high level specification eliminating the need for programmers –Fortran, mathematicians will enter formulas directly –COBOL, people will stick business models into the computer directly –4G languages, no, honestly, they really will stick them in directly –Modeling languages, no more need for expensive programmers Expensive, but might be worth the cost if it was a panacea, but it is not.

Slide: 9 Copyright © 2009 AdaCore Difficulties in Producing Full Specifications No one understands formal specifications There are things that can’t be formally specified –Good error messages in a compiler –Intuitive user interfaces Deliberate incompleteness Difficulties of proving termination Difficulties of specifying programming languages Anyway, they will have errors A specification is basically a large program And often one that has never been tested

Slide: 10 Copyright © 2009 AdaCore Proving Properties of Programs Definitely a promising approach Not just promises, we can deliver –Freedom from run-time errors –Proof of security properties Security a particular important target –Secure programs do not have to work –But they do have to be secure

Slide: 11 Copyright © 2009 AdaCore The Role of Testing Right now, we largely rely on testing, e.g. in DO-178B Thorough testing definitely improves our confidence But testing is fundamentally incomplete DO-178B has no formal model, it just works So can we eliminate testing? Almost no one thinks we can eliminate 100% of testing I don’t care how much you have proved, I don’t want to fly on a plane with avionics that has never been tested. But if we can’t eliminate ANY testing, then bleak times ahead for formal methods, certainly in the safety area. So the idea (enshrined in DO-178C) is to figure out how to give some credit for formal methods, eliminating some testing requirements.

Slide: 12 Copyright © 2009 AdaCore Proofs Replacing Testing Specific Unit tests might be replaced by proofs, e.g. of the correctness of an algorithm, eliminating the need for complete domain testing. Object Oriented Programs and Dynamic Dispatch Thorough testing impractical (every call to every method). Require LSP in OO hierarchies OK, we can require it, but how to verify? Unit testing??? This is a place where proof techniques do make sense

Slide: 13 Copyright © 2009 AdaCore Reasoning About Existing Programs The use of static analysis Finding bugs is useful, but not so useful Guaranteeing absence of bugs is definitely useful But eliminating all false negatives can generate too many false positives Helps to start with an appropriate language, e.g. in Ada, range declarations add a focus for analysis. The Microsoft Vista story –Added 500,000 annotations –Found 100,000 potential buffer overflows Finding pre and post conditions Incremental Analysis

Slide: 14 Copyright © 2009 AdaCore Putting it All Together, the Hi-Lite Project In practice we will make use of a combination of techniques and tools –Proof of correctness and formal specification –Thorough testing –Use of static analysis tools All these tools can benefit from the addition of annotations about what should be going on. –Postconditions –Preconditions –Invariants But different tools have different views –Is a precondition something to be proved? –Or is it something to be tested? –Or is it there to give hints to a static analyzer

Slide: 15 Copyright © 2009 AdaCore Preconditions and Postconditions At one level, they are just formalizations of comments. Comments are a good thing if –They are precise –They are complete –They are kept up to date But out of date or incorrect comments are a menace So formalizing them can only help Not that much work once you get used to it But what language should be used Ada 2012 adds extensive capabilities for annotations including preconditions, postconditions, invariants, and predicates.

Slide: 16 Copyright © 2009 AdaCore Trying to Unify the Language Requirements If annotations/contracts are useful for proof, testing, and static analysis, how about we use the same annotation language, so we can integrate the different approaches within a common framework? Multiple approaches are problematic In AdaCore technology, four syntaxes for preconditions –Precondition pragma for Ada 95/2005 –Pre aspect in Ada 2012 –SPARK annotation for preconditions –CodePeer syntax for discovered precondition

Slide: 17 Copyright © 2009 AdaCore Hi-Lite One important goal of Hi-Lite is to create a unified language for annotations that can be used in all contexts. Then the second step is to create a unified development environment in which different tools can use the same source base in different contexts. The goal: to facilitate (i.e. make cheaper) the integration of formal methods and testing regimes. We need to create industrial workflows that incorporate verification on a routine basis These tools need to integrate different programming approaches as well (different languages, modeling …)

Slide: 18 Copyright © 2009 AdaCore Problems in the Unification The requirements for mathematical specifications and programming language expressions are different. Does A+B mean A+B or does it mean apply some weird programming notion of + with overflows or wrap around? –with Pre => A + B <= Integer’Last Does A or B mean A or B, or does it mean do some strange evaluation in which an overflow in B causes an exception even if A is True? Not insuperable, but such issues stand in the way of this being a simple job of getting people to agree on syntax.

Slide: 19 Copyright © 2009 AdaCore Conclusion We have a lot of industrial experience to feed in here We have creators of tools and users very interested in participating in creating a workable framework –AdaCore –Praxis –CEA –INRIA We have mportant industrial partners who need this technology now, so this is much more than just research. –Altran –Astrium –Thales –Rockwell