Automatic Software Verification: A Renaissance Andrew Ireland Dependable Systems Group School of Mathematical & Computer Sciences Heriot-Watt University Edinburgh

Outline A historical perspective New wave of software verification tools Cooperative reasoning Applications and challenges

Assertion Based Verification “Assertion boxes” Goldstine & von Neuman 1947 “Checking a large routine” Turing 1949 “Inductive assertion method” Floyd 1967 “Axiomatic basis” Hoare 1969 “A program verifier” King 1969

Floyd’s Summation Program

Floyd’s Summation Program Pre-condition Post-condition

Floyd’s Summation Program Loop invariant

Floyd’s Verifying Compiler Proposal “The Verifying Compiler” CMU Annual Report p18-19, 1967

First Wave of Verifiers “A Program Verifier” (King 1969) Stanford Pascal Verifier (Luckham et al 1973) Gypsy (Good et al 1974) VISTA (German & Wegbreit 1975) HDM (Levitt et al 1977) SPARK for Ada (Carre et al 1983) Penelope (Guaspari et al 1990) ESC/Java (Detlefs et al 1998)

So Why the Poor Uptake? Formalizing specifications is hard Verification requires auxiliary specifications and properties, e.g. loop invariants, inductive properties Mechanized proof is semi-automatic at best, i.e. skilled user interaction required Scalability issues associated with mainstream programming languages, e.g. reasoning about pointers

Times They Are Changing Static Device Verifier (SDV): http://www.microsoft.com/whdc/devtools/tools/sdv.mspx ESC/Java2: http://secure.ucd.ie/products/opensource/ESCJava2 Spec#: http://research.microsoft.com/specsharp The SPARK Approach: http://www.praxis-his.com/sparkada

What’s Changed? Generic property verification “Buffer overflows have been the most common form of security vulnerability for the past 10 years“ OGI/DARPA Programming language subsets Programming logics Automated reasoning tools Tool integrations

Cooperative Reasoning + =

Cooperative Reasoning + =

Cooperative Reasoning + = = Complementary techniques compensating for each other’s weaknesses

Reliable & Safe Ada Subsets “It is not too late! I believe that by careful pruning of the Ada language, it is still possible to select a very powerful subset that would be reliable and efficient in implementation and safe and economic to use” C.A.R. Hoare 1980 ACM Turing Award Lecture

The SPARK Approach A subset of Ada that eliminates potential ambiguities and insecurities (PRAXIS) Expressive enough for industrial applications, but restrictive enough to support rigorous analysis SPARK toolset supports data & information flow analysis and formal verification Partial correctness & exception freedom proof, e.g. proving code is free from buffer overflows, range violations, division by zero.

SPARK Applications Advanced avionics (Eurofighter Typhoon), transportation, air traffic control, security (MONDEX) Ship Helicopter Operating Limits Information System (SHOLIS): first system developed to meet UK MOD Defence Standard 00-55 Supports “correctness-by-construction” and is advocated by US National Cyber Security Partnership (NSA) as one of three software development processes that can deliver sufficient assurance for security critical systems

The SPARK Approach  SPARK code SPARK Examiner VCs SPADE Simplifier Proofs  UnprovenVCs Cmds SPADE Proof Checker

NuSPADE Project  Refinement Abstraction Bill J. Ellis (RA) SPARK code SPARK Examiner VCs SPADE Simplifier Proofs  UnprovenVCs Cmds SPADEase SPADE Proof Checker Refinement Abstraction Bill J. Ellis (RA) EPSRC Critical Systems programme (GR/R24081) EPSRC RAIS Scheme (GR/T11289) http://www.macs.hw.ac.uk/nuspade

SPADEase Program Analysis Proof Planning Exception freedom proofs Automatic loop invariant discovery

Proof Planning Conjecture Method Theory Critic Proof Planner Tactic Checker Proof

Proof Planning Reuse: strategies can be easily ported between checkers Robustness: critics and middle-out reasoning provide flexibility in how proof search is organized Cooperation: planning provides a natural level at which to combine multiple reasoning processes

SPADEase Unproven VCs Cmds Proof Program Planner Analyzer Annotations Abstract Predicates Cmds Proof Planner Program Analyzer Annotations Cooperative reasoning, e.g. “productive use of failure”

An Example subtype AR_T is Integer range 0..9; type A_T is array (AR_T) of Integer; ... procedure Filter(A: in A_T; R: out Integer) is begin R:=0; for I in AR_T loop if A(I)>=0 and A(I)<=100 then R:=R+A(I); end if; end loop; end Filter;

Exception Freedom VC ... H4: element(a, [loop__1__i]) >= 0 . H6: r >= integer__first . H7: r <= integer__last . -> C2: r + element(a,[loop__1__i]) <= integer__last . H1: for_all (i___1: integer, ((i___1 >= ar_t__first) and (i___1 <= ar_t__last)) -> ((element(a, [i___1]) >= integer__first) and (element(a, [i___1]) <= integer__last))) . H2: loop__1__i >= ar_t__first . H3: loop__1__i <= ar_t__last . H4: element(a, [loop__1__i]) >= 0 . H5: element(a, [loop__1__i]) <= 100 . H6: r >= integer__first . H7: r <= integer__last . -> C1: r + element(a, [loop__1__i]) >= integer__first . C2: r + element(a, [loop__1__i]) <= integer__last . C3: loop__1__i >= ar_t__first . C4: loop__1__i <= ar_t__last .

Exception Freedom VC ... H4: element(a, [loop__1__i]) >= 0 . H6: r >= integer__first . H7: r <= integer__last . -> C2: r + element(a,[loop__1__i]) <= integer__last H1: for_all (i___1: integer, ((i___1 >= ar_t__first) and (i___1 <= ar_t__last)) -> ((element(a, [i___1]) >= integer__first) and (element(a, [i___1]) <= integer__last))) . H2: loop__1__i >= ar_t__first . H3: loop__1__i <= ar_t__last . H4: element(a, [loop__1__i]) >= 0 . H5: element(a, [loop__1__i]) <= 100 . H6: r >= integer__first . H7: r <= integer__last . -> C1: r + element(a, [loop__1__i]) >= integer__first . C2: r + element(a, [loop__1__i]) <= integer__last . C3: loop__1__i >= ar_t__first . C4: loop__1__i <= ar_t__last . -32768 . 32767 . 32767 .

Exception Freedom VC ... H4: element(a, [loop__1__i]) >= 0 . H6: r >= integer__first . H7: r <= integer__last . -> C2: r + element(a,[loop__1__i]) <= H1: for_all (i___1: integer, ((i___1 >= ar_t__first) and (i___1 <= ar_t__last)) -> ((element(a, [i___1]) >= integer__first) and (element(a, [i___1]) <= integer__last))) . H2: loop__1__i >= ar_t__first . H3: loop__1__i <= ar_t__last . H4: element(a, [loop__1__i]) >= 0 . H5: element(a, [loop__1__i]) <= 100 . H6: r >= integer__first . H7: r <= integer__last . -> C1: r + element(a, [loop__1__i]) >= integer__first . C2: r + element(a, [loop__1__i]) <= integer__last . C3: loop__1__i >= ar_t__first . C4: loop__1__i <= ar_t__last . -32768 . If (32668 <= r <= 32767) then possible overflow, i.e. VC is unprovable 32767 . 32767 .

Proof-Failure Analysis Preconditions: 1. Unproven relational goal of the form E Rel C, e.g. r + element(a,[loop__1__i]) <= 32767 2. Given the constraints on E, a counter-example can be found that shows bounds are insufficient to prove exception freedom, e.g. r >= -32768 and r <= 32767 Patch: R >= ? and R <= ?

SPADEase Unproven VCs Proof Program Planner Analyzer Annotations subtype AR_T is Integer range 0..9; type A_T is array (AR_T) of Integer; ... procedure Filter(A: in A_T; R: out Integer)is begin R:=0; for I in AR_T loop if A(I)>=0 and A(I)<=100 then R:=R+A(I); end if; end loop; end Filter; R >= ? and R <= ? Abstract Predicates Proof Planner Program Analyzer Annotations

Program Analysis Solution: PURRS: Recurrence Relation Solver http://www.cs.unipr.it/purrs/ Solution:

Program Analysis

Program Analysis Solutions:

--# assert R >= 0 and R <= I*100 Combining Solutions --# assert R >= 0 and R <= I*100

--# assert R >= 0 and R <= I*100 SPADEase Unproven VCs subtype AR_T is Integer range 0..9; type A_T is array (AR_T) of Integer; ... procedure Filter(A: in A_T; R: out Integer)is begin R:=0; for I in AR_T loop --# assert R >= 0 and R <= I*100 if A(I)>=0 and A(I)<=100 then R:=R+A(I); end if; end loop; end Filter; subtype AR_T is Integer range 0..9; type A_T is array (AR_T) of Integer; ... procedure Filter(A: in A_T; R: out Integer)is begin R:=0; for I in AR_T loop if A(I)>=0 and A(I)<=100 then R:=R+A(I); end if; end loop; end Filter; R >= ? and R <= ? Abstract Predicates Proof Planner Program Analyzer Annotations --# assert R >= 0 and R <= I*100

Revised Filter Code subtype AR_T is Integer range 0..9; type A_T is array (AR_T) of Integer; ... procedure Filter(A: in A_T; R: out Integer) is begin R:=0; for I in AR_T loop --# assert R >= 0 and R <= I*100 if A(I)>=0 and A(I)<=100 then R:=R+A(I); end if; end loop; end Filter;

Revised Exception Freedom VC H1: r >= 0 . H2: r <= loop__1__i * 100 . ... H6: element(a, [loop__1__i]) >= 0 . H7: element(a, [loop__1__i]) <= 100 . … -> C2: r + element(a,[loop__1__i]) <= integer__last . H1: r >= 0 . H2: r <= loop__1__i * 100 . H3: for_all (i___1: integer, ((i___1 >= ar_t__first) and (i___1 <= ar_t__last)) -> ((element(a, [i___1]) >= integer__first) and (element(a, [i___1]) <= integer__last))) . H4: loop__1__i >= ar_t__first . H5: loop__1__i <= ar_t__last . H6: element(a, [loop__1__i]) >= 0 . H7: element(a, [loop__1__i]) <= 100 . H8: r >= integer__first . H9: r <= integer__last . -> C1: r + element(a, [loop__1__i]) >= integer__first . C2: r + element(a, [loop__1__i]) <= integer__last . C3: loop__1__i >= ar_t__first . C4: loop__1__i <= ar_t__last . Transitivity proof plan 

Loop Invariant VC  Rippling proof plan H1: r >= 0 . H2: r <= loop__1__i * 100 . ... H6: element(a, [loop__1__i]) >= 0 . H7: element(a, [loop__1__i]) <= 100 . … -> C1: r + element(a,[loop__1__i]) >= 0 . C2: r + element(a,[loop__1__i]) <= (loop__1__i + 1) * 100 . Rippling proof plan 

✗ ✓ ✓ Phase Goal Proof Planning Proof-Failure Analysis Program P1 P2

NuSPADE Results Our evaluation was based upon examples drawn from industrial data provided by Praxis, e.g. SHOLIS SPADE Simplifier is very effective on exception freedom VC, i.e. typical hit-rate of 92% SPADEase targeted the VCs which the SPADE Simplifier failed to prove, i.e. loop-based code While critical software is engineered to minimize the number and complexity of loops, we found that 80% of the loops we encountered were provable using SPADEase

Compensating for Weaknesses? constraints not available post-VCGen soundness equational reasoning extensibility Program Analysis Proof Planning

Limitations & Related Work Constraint solving fails when reasoning with “big numbers”, i.e. integers out with –(225)… 225-1 Precondition strengthening would improve our hit-rate, constraint solving may have a role to play. Integrating decision procedures with SPADEase would also improve hit-rate. Paul Jackson, Bill J. Ellis, Kathleen Sharp “Using SMT Solvers to Verify High-Integrity Programs” 2nd Int. Workshop on Automated Formal Methods (AFM-07) Boulton, Bundy, Gordon, Slind Clam/HOL Project (`96-`99)

SPADEase Impact? “… It increases the proportion of SPARK-generated verification conditions that can be proved automatically without introducing any new opaque, black-box processes. …” Peter Amey Chief Technical Officer Praxis High Integrity Systems

SPADEase Impact? “… The separation of proof planning from proof checking also acts as a talent multiplier by allowing proof experts to spend their time creating new and reusable methods and approaches rather than working on individual proofs.” Peter Amey Chief Technical Officer Praxis High Integrity Systems

CORE Project (EPSRC EP/F037597) Separation Logic: Reynolds (CMU) & O’Hearn (QMU) Focuses the reasoning effort on only those parts of the heap that are relevant to a program unit, i.e. local reasoning Building upon shape analysis tools, CORE aims to automate the verification of functional properties More details: http://www.macs.hw.ac.uk/core Tech Talks at Google via

SEAR (EPSRC EP/F037058) System Evolution via Animation and Reasoning Requirements Formal Proof Design Formal Proof Code Formal Proof System Evolution via Animation and Reasoning BAE Systems & Lemma1 Event-B and AntiPatterns Collaborators: Edinburgh & Imperial

Grand Challenges UK GC6: Dependable Systems Evolution (a collaboration with the International Verified Software Grand Challenge) Verifying Compiler Grand Challenge – Hoare The Verified Software Repository – Woodcock: accelerate the development of verification tools open access to challenge problems and results Industry engagement http://vsr.sourceforge.net/Introduction.htm

Conclusion A new wave of software verification tools is emerging within a range of niche markets Technological advances and the economics of software failures have been major drivers Cooperation: tools and communities Grand Challenge or Emerging Technology? As Automated Verification researchers we are challenged by a grand opportunity!

Verified Software: Theories, Tools & Experiments And Finally … 3rd IFIP Working Conference on Verified Software: Theories, Tools & Experiments hosted by Heriot-Watt University August 16th-19th, 2010 http://www.macs.hw.ac.uk/vstte10