1. Automatic Software Verification: A Renaissance
Andrew Ireland
Dependable Systems Group
School of Mathematical & Computer Sciences
Heriot-Watt University
Edinburgh

2. Outline A historical perspective
New wave of software verification tools
Cooperative reasoning
Applications and challenges

3. Assertion Based Verification
“Assertion boxes” Goldstine & von Neuman 1947
“Checking a large routine” Turing 1949
“Inductive assertion method” Floyd 1967
“Axiomatic basis” Hoare 1969
“A program verifier” King 1969

4. Floyd’s Summation Program

5. Floyd’s Summation Program
Pre-condition
Post-condition

6. Floyd’s Summation Program
Loop invariant

7. Floyd’s Verifying Compiler Proposal
“The Verifying Compiler” CMU Annual Report p18-19, 1967

8. First Wave of Verifiers
“A Program Verifier” (King 1969)
Stanford Pascal Verifier (Luckham et al 1973)
Gypsy (Good et al 1974)
VISTA (German & Wegbreit 1975)
HDM (Levitt et al 1977)
SPARK for Ada (Carre et al 1983)
Penelope (Guaspari et al 1990)
ESC/Java (Detlefs et al 1998)

9. So Why the Poor Uptake? Formalizing specifications is hard
Verification requires auxiliary specifications and properties, e.g. loop invariants, inductive properties
Mechanized proof is semi-automatic at best, i.e. skilled user interaction required
Scalability issues associated with mainstream programming languages, e.g. reasoning about pointers

10. Times They Are Changing
Static Device Verifier (SDV):
ESC/Java2:
Spec#:
The SPARK Approach:

11. What’s Changed? Generic property verification
“Buffer overflows have been the most common form of security vulnerability for the past 10 years“ OGI/DARPA
Programming language subsets
Programming logics
Automated reasoning tools
Tool integrations

12. Cooperative Reasoning+ =

13. Cooperative Reasoning+ =

14. Cooperative Reasoning+ = =
Complementary techniques compensating
for each other’s weaknesses

15. Reliable & Safe Ada Subsets
“It is not too late! I believe that by careful pruning of the Ada language, it is still possible to select a very powerful subset that would be reliable and efficient in implementation and safe and economic to use”
C.A.R. Hoare
1980 ACM Turing Award Lecture

16. The SPARK Approach
A subset of Ada that eliminates potential ambiguities and insecurities (PRAXIS)
Expressive enough for industrial applications, but restrictive enough to support rigorous analysis
SPARK toolset supports data & information flow analysis and formal verification
Partial correctness & exception freedom proof, e.g. proving code is free from buffer overflows, range violations, division by zero.

17. SPARK Applications
Advanced avionics (Eurofighter Typhoon), transportation, air traffic control, security (MONDEX)
Ship Helicopter Operating Limits Information System (SHOLIS): first system developed to meet UK MOD Defence Standard 00-55
Supports “correctness-by-construction” and is advocated by US National Cyber Security Partnership (NSA) as one of three software development processes that can deliver sufficient assurance for security critical systems

18. The SPARK Approach  SPARK code SPARK Examiner VCs SPADE Simplifier
Proofs 
UnprovenVCs
Cmds
SPADE
Proof Checker

19. NuSPADE Project  Refinement Abstraction Bill J. Ellis (RA)
SPARK
code
SPARK
Examiner
VCs
SPADE
Simplifier
Proofs 
UnprovenVCs
Cmds
SPADEase
SPADE
Proof Checker
Refinement
Abstraction
Bill J. Ellis (RA)
EPSRC Critical Systems programme (GR/R24081)
EPSRC RAIS Scheme (GR/T11289)

20. SPADEase Program Analysis Proof Planning Exception freedom proofs
Automatic loop invariant discovery

21. Proof Planning Conjecture Method Theory Critic Proof Planner Tactic
Checker
Proof

22. Proof Planning Reuse: strategies can be easily ported between checkers
Robustness: critics and middle-out reasoning provide flexibility in how proof search is organized
Cooperation: planning provides a natural level at which to combine multiple reasoning processes

23. SPADEase Unproven VCs Cmds Proof Program Planner Analyzer Annotations
Abstract
Predicates
Cmds
Proof
Planner
Program
Analyzer
Annotations
Cooperative reasoning, e.g. “productive use of failure”

24. An Example subtype AR_T is Integer range 0..9;
type A_T is array (AR_T) of Integer;
...
procedure Filter(A: in A_T; R: out Integer) is
begin
R:=0;
for I in AR_T loop
if A(I)>=0 and A(I)<=100 then
R:=R+A(I);
end if;
end loop;
end Filter;

25. Exception Freedom VC ... H4: element(a, [loop__1__i]) >= 0 .
H6: r >= integer__first .
H7: r <= integer__last .
->
C2: r + element(a,[loop__1__i]) <= integer__last .
H1: for_all (i___1: integer, ((i___1 >= ar_t__first) and (i___1 <= ar_t__last)) ->
((element(a, [i___1]) >= integer__first) and (element(a, [i___1]) <= integer__last))) .
H2: loop__1__i >= ar_t__first .
H3: loop__1__i <= ar_t__last .
H4: element(a, [loop__1__i]) >= 0 .
H5: element(a, [loop__1__i]) <= 100 .
H6: r >= integer__first .
H7: r <= integer__last .
->
C1: r + element(a, [loop__1__i]) >= integer__first .
C2: r + element(a, [loop__1__i]) <= integer__last .
C3: loop__1__i >= ar_t__first .
C4: loop__1__i <= ar_t__last .

26. Exception Freedom VC ... H4: element(a, [loop__1__i]) >= 0 .
H6: r >= integer__first .
H7: r <= integer__last .
->
C2: r + element(a,[loop__1__i]) <= integer__last
H1: for_all (i___1: integer, ((i___1 >= ar_t__first) and (i___1 <= ar_t__last)) ->
((element(a, [i___1]) >= integer__first) and (element(a, [i___1]) <= integer__last))) .
H2: loop__1__i >= ar_t__first .
H3: loop__1__i <= ar_t__last .
H4: element(a, [loop__1__i]) >= 0 .
H5: element(a, [loop__1__i]) <= 100 .
H6: r >= integer__first .
H7: r <= integer__last .
->
C1: r + element(a, [loop__1__i]) >= integer__first .
C2: r + element(a, [loop__1__i]) <= integer__last .
C3: loop__1__i >= ar_t__first .
C4: loop__1__i <= ar_t__last .

27. Exception Freedom VC ... H4: element(a, [loop__1__i]) >= 0 .
H6: r >= integer__first .
H7: r <= integer__last .
->
C2: r + element(a,[loop__1__i]) <=
H1: for_all (i___1: integer, ((i___1 >= ar_t__first) and (i___1 <= ar_t__last)) ->
((element(a, [i___1]) >= integer__first) and (element(a, [i___1]) <= integer__last))) .
H2: loop__1__i >= ar_t__first .
H3: loop__1__i <= ar_t__last .
H4: element(a, [loop__1__i]) >= 0 .
H5: element(a, [loop__1__i]) <= 100 .
H6: r >= integer__first .
H7: r <= integer__last .
->
C1: r + element(a, [loop__1__i]) >= integer__first .
C2: r + element(a, [loop__1__i]) <= integer__last .
C3: loop__1__i >= ar_t__first .
C4: loop__1__i <= ar_t__last .
If (32668 <= r <= 32767) then possible overflow, i.e. VC is unprovable

28. Proof-Failure Analysis
Preconditions:
1. Unproven relational goal of the form E Rel C, e.g.
r + element(a,[loop__1__i]) <= 32767
2. Given the constraints on E, a counter-example
can be found that shows bounds are insufficient
to prove exception freedom, e.g.
r >= and r <= 32767
Patch:
R >= ? and R <= ?

29. SPADEase Unproven VCs Proof Program Planner Analyzer Annotations
subtype AR_T is Integer range 0..9;
type A_T is array (AR_T) of Integer;
...
procedure Filter(A: in A_T; R: out Integer)is
begin
R:=0;
for I in AR_T loop
if A(I)>=0 and A(I)<=100 then
R:=R+A(I);
end if;
end loop;
end Filter;
R >= ? and R <= ?
Abstract Predicates
Proof
Planner
Program
Analyzer
Annotations

30. Program Analysis Solution: PURRS: Recurrence Relation Solver
Solution:

31. Program Analysis

32. Program Analysis
Solutions:

33. --# assert R >= 0 and R <= I*100
Combining Solutions
--# assert R >= 0 and R <= I*100

34. --# assert R >= 0 and R <= I*100
SPADEase
Unproven
VCs
subtype AR_T is Integer range 0..9;
type A_T is array (AR_T) of Integer;
...
procedure Filter(A: in A_T; R: out Integer)is
begin
R:=0;
for I in AR_T loop
--# assert R >= 0 and R <= I*100
if A(I)>=0 and A(I)<=100 then
R:=R+A(I);
end if;
end loop;
end Filter;
subtype AR_T is Integer range 0..9;
type A_T is array (AR_T) of Integer;
...
procedure Filter(A: in A_T; R: out Integer)is
begin
R:=0;
for I in AR_T loop
if A(I)>=0 and A(I)<=100 then
R:=R+A(I);
end if;
end loop;
end Filter;
R >= ? and R <= ?
Abstract Predicates
Proof
Planner
Program
Analyzer
Annotations
--# assert R >= 0 and R <= I*100

35. Revised Filter Code subtype AR_T is Integer range 0..9;
type A_T is array (AR_T) of Integer;
...
procedure Filter(A: in A_T; R: out Integer) is
begin
R:=0;
for I in AR_T loop
--# assert R >= 0 and R <= I*100
if A(I)>=0 and A(I)<=100 then
R:=R+A(I);
end if;
end loop;
end Filter;

36. Revised Exception Freedom VC
H1: r >= 0 .
H2: r <= loop__1__i * 100 .
...
H6: element(a, [loop__1__i]) >= 0 .
H7: element(a, [loop__1__i]) <= 100 . …
->
C2: r + element(a,[loop__1__i]) <= integer__last .
H1: r >= 0 .
H2: r <= loop__1__i * 100 .
H3: for_all (i___1: integer, ((i___1 >= ar_t__first) and (i___1 <= ar_t__last)) ->
((element(a, [i___1]) >= integer__first) and (element(a, [i___1]) <= integer__last))) .
H4: loop__1__i >= ar_t__first .
H5: loop__1__i <= ar_t__last .
H6: element(a, [loop__1__i]) >= 0 .
H7: element(a, [loop__1__i]) <= 100 .
H8: r >= integer__first .
H9: r <= integer__last .
->
C1: r + element(a, [loop__1__i]) >= integer__first .
C2: r + element(a, [loop__1__i]) <= integer__last .
C3: loop__1__i >= ar_t__first .
C4: loop__1__i <= ar_t__last .
Transitivity proof plan 

37. Loop Invariant VC  Rippling proof plan H1: r >= 0 .
H2: r <= loop__1__i * 100 .
...
H6: element(a, [loop__1__i]) >= 0 .
H7: element(a, [loop__1__i]) <= 100 . …
->
C1: r + element(a,[loop__1__i]) >= 0 .
C2: r + element(a,[loop__1__i]) <= (loop__1__i + 1) * 100 .
Rippling proof plan 

38. ✗ ✓ ✓ Phase Goal Proof Planning Proof-Failure Analysis Program P1 P2

39. NuSPADE Results
Our evaluation was based upon examples drawn from industrial data provided by Praxis, e.g. SHOLIS
SPADE Simplifier is very effective on exception freedom VC, i.e. typical hit-rate of 92%
SPADEase targeted the VCs which the SPADE Simplifier failed to prove, i.e. loop-based code
While critical software is engineered to minimize the number and complexity of loops, we found that 80% of the loops we encountered were provable using SPADEase

40. Compensating for Weaknesses?
constraints
not available
post-VCGen
soundness
equational reasoning
extensibility
Program Analysis
Proof Planning

41. Limitations & Related Work
Constraint solving fails when reasoning with “big numbers”, i.e. integers out with –(225)… 225-1
Precondition strengthening would improve our hit-rate, constraint solving may have a role to play.
Integrating decision procedures with SPADEase would also improve hit-rate.
Paul Jackson, Bill J. Ellis, Kathleen Sharp “Using SMT Solvers to Verify High-Integrity Programs” 2nd Int. Workshop on Automated Formal Methods (AFM-07)
Boulton, Bundy, Gordon, Slind Clam/HOL Project (`96-`99)

42. SPADEase Impact?
“… It increases the proportion of SPARK-generated verification conditions that can be proved automatically without introducing any new opaque, black-box processes. …”
Peter Amey
Chief Technical Officer
Praxis High Integrity Systems

43. SPADEase Impact?
“… The separation of proof planning from proof checking also acts as a talent multiplier by allowing proof experts to spend their time creating new and reusable methods and approaches rather than working on individual proofs.”
Peter Amey
Chief Technical Officer
Praxis High Integrity Systems

44. CORE Project (EPSRC EP/F037597)
Separation Logic: Reynolds (CMU) & O’Hearn (QMU)
Focuses the reasoning effort on only those parts of the heap that are relevant to a program unit, i.e. local reasoning
Building upon shape analysis tools, CORE aims to automate the verification of functional properties
More details:
Tech Talks at Google via

45. SEAR (EPSRC EP/F037058) System Evolution via Animation and Reasoning
Requirements
Formal Proof
Design
Formal Proof
Code
Formal Proof
System Evolution via Animation and Reasoning
BAE Systems & Lemma1
Event-B and AntiPatterns
Collaborators: Edinburgh & Imperial

46. Grand Challenges
UK GC6: Dependable Systems Evolution (a collaboration with the International Verified Software Grand Challenge)
Verifying Compiler Grand Challenge – Hoare
The Verified Software Repository – Woodcock:
accelerate the development of verification tools
open access to challenge problems and results
Industry engagement

47. Conclusion
A new wave of software verification tools is emerging within a range of niche markets
Technological advances and the economics of software failures have been major drivers
Cooperation: tools and communities
Grand Challenge or Emerging Technology?
As Automated Verification researchers we are challenged by a grand opportunity!

48. Verified Software: Theories, Tools & Experiments
And Finally …
3rd IFIP Working Conference on
Verified Software: Theories, Tools & Experiments
hosted by
Heriot-Watt University
August 16th-19th, 2010