Proofs and Counterexamples Rupak Majumdar. In the good old days… Decision Procedure  yes no.

Slides:



Advertisements
Similar presentations
The behavior of SAT solvers in model checking applications K. L. McMillan Cadence Berkeley Labs.
Advertisements

Exploiting SAT solvers in unbounded model checking
A practical and complete approach to predicate abstraction Ranjit Jhala UCSD Ken McMillan Cadence Berkeley Labs.
Exploiting SAT solvers in unbounded model checking K. L. McMillan Cadence Berkeley Labs.
Automated Theorem Proving Lecture 1. Program verification is undecidable! Given program P and specification S, does P satisfy S?
50.530: Software Engineering
SAT Based Abstraction/Refinement in Model-Checking Based on work by E. Clarke, A. Gupta, J. Kukula, O. Strichman (CAV’02)
Automatic Verification Book: Chapter 6. What is verification? Traditionally, verification means proof of correctness automatic: model checking deductive:
50.530: Software Engineering Sun Jun SUTD. Week 10: Invariant Generation.
Rigorous Software Development CSCI-GA Instructor: Thomas Wies Spring 2012 Lecture 13.
Introducing BLAST Software Verification John Gallagher CS4117.
Software Verification with BLAST Tom Henzinger Ranjit Jhala Rupak Majumdar.
1 Cover Algorithms and Their Combination Sumit Gulwani, Madan Musuvathi Microsoft Research, Redmond.
BLAST-A Model Checker for C Developed by Thomas A. Henzinger (EPFL) Rupak Majumdar (UC Los Angeles) Ranjit Jhala (UC San Diego) Dirk Beyer (Simon Fraser.
SAT and Model Checking. Bounded Model Checking (BMC) A.I. Planning problems: can we reach a desired state in k steps? Verification of safety properties:
The Software Model Checker BLAST by Dirk Beyer, Thomas A. Henzinger, Ranjit Jhala and Rupak Majumdar Presented by Yunho Kim Provable Software Lab, KAIST.
Thread-modular Abstraction Refinement Tom Henzinger Ranjit Jhala Rupak Majumdar Shaz Qadeer.
Termination Proofs for Systems Code Andrey Rybalchenko, EPFL/MPI joint work with Byron Cook, MSR and Andreas Podelski, MPI PLDI’2006, Ottawa.
Using Statically Computed Invariants Inside the Predicate Abstraction and Refinement Loop Himanshu Jain Franjo Ivančić Aarti Gupta Ilya Shlyakhter Chao.
Software Verification with Blast Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, George Necula, Grégoire Sutre, Wes Weimer UC Berkeley.
Scalable Program Verification by Lazy Abstraction Ranjit Jhala U.C. Berkeley.
Lazy Abstraction Thomas A. Henzinger Ranjit Jhala Rupak Majumdar Grégoire Sutre UC Berkeley.
Interpolants [Craig 1957] G(y,z) F(x,y)
Program Verification by Lazy Abstraction Ranjit Jhala UC San Diego Lecture 1 With: Tom Henzinger, Rupak Majumdar, Ken McMillan, Gregoire Sutre.
ESC Java. Static Analysis Spectrum Power Cost Type checking Data-flow analysis Model checking Program verification AutomatedManual ESC.
Abstractions. Outline Informal intuition Why do we need abstraction? What is an abstraction and what is not an abstraction A framework for abstractions.
Thread-modular Abstraction Refinement Tom Henzinger Ranjit Jhala Rupak Majumdar [UC Berkeley] Shaz Qadeer [Microsoft Research]
Lazy Predicate Abstraction in BLAST John Gallagher CS4117.
Synergy: A New Algorithm for Property Checking
Counter Example Guided Refinement CEGAR Mooly Sagiv.
Race Checking by Context Inference Tom Henzinger Ranjit Jhala Rupak Majumdar UC Berkeley.
Software Reliability Methods Sorin Lerner. Software reliability methods: issues What are the issues?
Formal Verification Group © Copyright IBM Corporation 2008 IBM Haifa Labs SAT-based unbounded model checking using interpolation Based on a paper “Interpolation.
Temporal-Safety Proofs for Systems Code Thomas A. Henzinger Ranjit Jhala Rupak Majumdar George Necula Westley Weimer Grégoire Sutre UC Berkeley.
ESC Java. Static Analysis Spectrum Power Cost Type checking Data-flow analysis Model checking Program verification AutomatedManual ESC.
Software Verification with BLAST Tom Henzinger Ranjit Jhala Rupak Majumdar.
Lazy Abstraction Lecture 2: Modular Analyses Ranjit Jhala UC San Diego With: Tom Henzinger, Rupak Majumdar, Ken McMillan, Gregoire Sutre.
Computing Over­Approximations with Bounded Model Checking Daniel Kroening ETH Zürich.
Lazy Abstraction Tom Henzinger Ranjit Jhala Rupak Majumdar Grégoire Sutre.
CS 267: Automated Verification Lecture 13: Bounded Model Checking Instructor: Tevfik Bultan.
Ranjit Jhala Rupak Majumdar Interprocedural Analysis of Asynchronous Programs.
Lazy Abstraction Lecture 3 : Partial Analysis Ranjit Jhala UC San Diego With: Tom Henzinger, Rupak Majumdar, Ken McMillan, Gregoire Sutre.
Program Verification by Lazy Abstraction Ranjit Jhala UC San Diego Lecture 1 With: Tom Henzinger, Rupak Majumdar, Ken McMillan, Gregoire Sutre, Adam Chlipala.
272: Software Engineering Fall 2012 Instructor: Tevfik Bultan Lecture 4: SMT-based Bounded Model Checking of Concurrent Software.
Thread-modular Abstraction Refinement Thomas A. Henzinger, et al. CAV 2003 Seonggun Kim KAIST CS750b.
Software Engineering Prof. Dr. Bertrand Meyer March 2007 – June 2007 Chair of Software Engineering Static program checking and verification Slides: Based.
Inferring Specifications to Detect Errors in Code Mana Taghdiri Presented by: Robert Seater MIT Computer Science & AI Lab.
Race Checking by Context Inference Tom Henzinger Ranjit Jhala Rupak Majumdar UC Berkeley.
Type Systems CS Definitions Program analysis Discovering facts about programs. Dynamic analysis Program analysis by using program executions.
Lazy Abstraction Jinseong Jeon ARCS, KAIST CS750b, KAIST2/26 References Lazy Abstraction –Thomas A. Henzinger et al., POPL ’02 Software verification.
Convergence of Model Checking & Program Analysis Philippe Giabbanelli CMPT 894 – Spring 2008.
Proving Non-Termination Gupta, Henzinger, Majumdar, Rybalchenko, Ru-Gang Xu presentation by erkan.
Localization and Register Sharing for Predicate Abstraction Himanshu Jain Franjo Ivančić Aarti Gupta Malay Ganai.
Symbolic and Concolic Execution of Programs Information Security, CS 526 Omar Chowdhury 10/7/2015Information Security, CS 5261.
Random Interpretation Sumit Gulwani UC-Berkeley. 1 Program Analysis Applications in all aspects of software development, e.g. Program correctness Compiler.
SMT and Its Application in Software Verification (Part II) Yu-Fang Chen IIS, Academia Sinica Based on the slides of Barrett, Sanjit, Kroening, Rummer,
Ranjit Jhala Rupak Majumdar Interprocedural Analysis of Asynchronous Programs.
CS357 Lecture 13: Symbolic model checking without BDDs Alex Aiken David Dill 1.
Counter Example Guided Refinement CEGAR Mooly Sagiv.
#1 Having a BLAST with SLAM. #2 Software Model Checking via Counter-Example Guided Abstraction Refinement Topic: Software Model Checking via Counter-Example.
© Anvesh Komuravelli Spacer Model Checking with Proofs and Counterexamples Anvesh Komuravelli Carnegie Mellon University Joint work with Arie Gurfinkel,
The software model checker BLAST Dirk Beyer, Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar Presented by Yunho Kim TexPoint fonts used in EMF. Read.
Presentation Title 2/4/2018 Software Verification using Predicate Abstraction and Iterative Refinement: Part Bug Catching: Automated Program Verification.
Having a BLAST with SLAM
Abstractions from Proofs
Program Verification by Lazy Abstraction
Abstraction, Verification & Refinement
Software Verification with BLAST
Predicate Abstraction
BLAST: A Software Verification Tool for C programs
Presentation transcript:

Proofs and Counterexamples Rupak Majumdar

In the good old days… Decision Procedure  yes no

Now… Sooh, short answer: yes with an if, long answer, no with a but. –Reverend Lovejoy Decision Procedure  yes no Interpolant  +,  - Model Predicate Abstraction Boolean SAT Pre- Process Proof Invariant Generation

It’s the Application! Validating large scale software w.r.t. partial specifications Rules in documentation –Order of operations & data access –Resource management –Incomplete, unenforced, wordy Violated rules ) bad behavior –System crash or deadlock –Unexpected exceptions –Failed runtime checks

Property: Double Locking “An attempt to re-acquire an acquired lock or release a released lock will cause a deadlock.” Calls to lock and unlock must alternate. lock unlock

Example Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } lock unlock Does program Example satisfy the locking spec?

Ok, but what does this have to do with Decision Procedure  yes no Decision Procedure  yes no Interpolant  +,  - Model Predicate Abstraction Boolean SAT Pre- Process Proof Invariant Generation vs

Automatic Verification of Programs Search Refine Program Safe Trace Yes No Property [POPL 02] [POPL 04] Automatic Tool Model and analyze program behavior in the language of logic, reduce reasoning about programs to reasoning about logic

What a program really is… State Transition 3: unlock(); new++; 4:} … 3: unlock(); new++; 4:} … pc lock old new q  3   5  0x133a pc lock old new q  4   5  6  0x133a Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return;} Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return;}

The Safety Verification Problem Initial Error Is there a path from an initial to an error state ? Problem: Infinite state graph Solution : Set of states ' logical formula Safe

Representing States as Formulas [F] states satisfying F {s | s ² F } F FO fmla over prog. vars [F 1 ] Å [F 2 ]F1 Æ F2F1 Æ F2 [F 1 ] [ [F 2 ]F 1 Ç F 2 [F][F] : F [F 1 ] µ [F 2 ]F 1 implies F 2 i.e. F 1 Æ: F 2 unsatisfiable

Idea 1: Symbolic Analysis Initial Error Program State Space 1.Given F, and a program operation, describe a formula for the set of successor states “Strongest postcondition” 2. Iterate the SP, checking at each stage if Error is hit, or if new states are added

Idea 1: Symbolic Analysis Iterate the SP, checking at each stage if Error is hit, or if new states are added CurrentReach Æ Error is unsat OldReach Ç SP(OldReach)  CurrentReach Problem: Too many details tracked Symbolic Iteration may not terminate

Idea 1’: Symbolic Analysis + Annotations Ask programmer for Invariant Check Invariant does not hit Error, and Invariant is closed under SP Invariant Æ Error is unsat Invariant Ç SP(Invariant)  Invariant No more iterations: Programmer provides an inductive assertion, just make two queries Problem: Increased work for the programmer

Idea 2: Predicate Abstraction Initial Error Program State Space Abstraction: Predicates on program state –Signs: x > 0 –Aliasing:&x  &y States satisfying the same preds are equivalent –Merged into single abstract state Abstraction [Graf-Saidi 97]

Example: Predicate Abstraction For the red region above, the predicate abstraction is the set of “boxes” that cover the red region For each small box, ask the theorem prover if Æ is satisfiable

Search finite state space symbolically Conservative – Abstraction safe ) System safe – Too coarse ) spurious counterexample Abstract Idea 2: Predicate Abstraction

BUG 1. Feasible ) BUG Abstract Counterexamples: Analyze counterexamples to check feasibility Idea 3: Counterexample Analysis Get satisfying assignment from decision procedure: test demonstrating bug

BUG 1. Feasible ) BUG 2. Infeasible ) What predicates distinguish states across cut? 3. Build refined abstraction Abstract Counterexamples: Analyze counterexamples to check feasibility Idea 3: Counterexample Analysis

Idea 3: Counterex.-Guided Refinement Abstract Refine Add predicates to rule out spurious trace Repeat reachability –Till safe or real trace is found –Or go on refining forever [Kurshan et al. 93] [Clarke et al. 00] [Ball-Rajamani 01]

safe Abstract Refine Idea 3: Counterex.-Guided Refinement Add predicates to rule out spurious trace Repeat reachability –Till safe or real trace is found –Or go on refining forever

Problem: Abstraction is Expensive Reachable Problem #abstract states = 2 #predicates Exponential Thm. Prover queries Observe Fraction of state space reachable #Preds ~ 100’s, #States ~ 2 100, #Reach ~ 1000’s

Solution1 : Only Abstract Reachable States Solution Build abstraction on-the-fly, during search Problem #abstract states = 2 #predicates Exponential Thm. Prover queries

Solution Build abstraction on-the-fly, during search Problem #abstract states = 2 #predicates Exponential Thm. Prover queries Solution1 : Only Abstract Reachable States

Solution Build abstraction on-the-fly, during search Problem #abstract states = 2 #predicates Exponential Thm. Prover queries Solution1 : Only Abstract Reachable States

Safe Solution Build abstraction on-the-fly, during search Problem #abstract states = 2 #predicates Exponential Thm. Prover queries Solution1 : Only Abstract Reachable States

Key Idea: Reachability Tree Unroll Abstraction 1. Pick tree-node (=abs. state) 2. Add children (=abs. successors) 3. On re-visiting abs. state, cut-off Counterexample Analysis -Learn new predicates locally -Rebuild subtree with new predicates “Lazy Abstraction” Initial [HenzingerJhalaM.Sutre02]

Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 Reachability Tree

Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 lock() old = new q=q->next LOCK 2 2 Reachability Tree

Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK 2 2 [q!=NULL] 3 3 Reachability Tree

Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK q ->data = new unlock() new : LOCK Reachability Tree

Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK : LOCK [new==old] 5 5 Reachability Tree

Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK : LOCK 5 5 unlock() : LOCK Reachability Tree

Analyze Counterexample Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK : LOCK 5 5 Reachability Tree lock() old = new q=q->next [q!=NULL] q ->data = new unlock() new++ [new==old] unlock()

Analyze Counterexample Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK : LOCK 5 5 [new==old] new++ old = new Inconsistent new == old Reachability Tree

Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 Reachability Tree

Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK, new==old 2 2 lock() old = new q=q->next Reachability Tree

Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK, new==old q ->data = new unlock() new++ : LOCK, : new = old Reachability Tree

Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK, new==old : LOCK, : new = old [new==old] Reachability Tree

Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK, new==old : LOCK, : new = old : LOCK, : new == old 1 [new!=old] 4 Reachability Tree

Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } LOCK, new=old 4 4 : LOCK, new==old 5 5 SAFE Reachability Tree LOCK, new==old : LOCK, : new = old : LOCK, : new == old

BLAST Abstract Refine C Program Safe Trace Yes No Property [HenzingerJhalaM.Sutre02]

Scaling BLAST Abstract Refine C Program Safe Trace Yes No Property To find: 1. Which predicates to track? 2.Where to track them? Key Problem: Find good predicates

Tracking lock not enough #Predicates grows with program size Problem: p 1,…,p n needed for verification Exponential reachable abstract states while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } TFTTFT

#Predicates grows with program size Problem: p 1,…,p n needed for verification Exponential reachable abstract states while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } : LOCK : LOCK, p 1 p1p2p1p2 p 1 : p 2 : p 1 p 2 : p 1 : p 2 LOCK, p 1 : LOCK, : p 1 2 n Abstract States : LOCK

Predicates useful locally while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } : LOCK : LOCK, p 1 LOCK, p 1 : LOCK, : p 1 2n Abstract States p1p1 p2p2 pnpn : LOCK LOCK, p 2 : LOCK, : p 2 : LOCK Solution: Use predicates only where needed

Predicates: LOCK, new==old Refinement Main Questions : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } LOCK, new=old 4 4 : LOCK, new==old 5 5 SAFE LOCK, new==old : LOCK, : new = old : LOCK, : new == old Q. How to find good predicates ? Where to track each predicate?

Counterexample Traces lock() old = new q=q->next [q!=NULL] q ->data = new unlock() new++ [new==old] unlock() TraceSSA Trace lock 1 = 1 old 1 = new 0 q 1 = q 0 - >next assume(q 1 != NULL) (q 1 -> data ) 1 = new 0 lock 2 = 0 new 1 = new 0 +1 assume(new 1 =old 1 ) assert(lock 2 =1) Trace Feasibility Formula Thm: Trace is feasible, TF is satisfiable lock 1 = 1 Æ old 1 = new 0 Æ q 1 = q 0 - >next Æ q 1 != NULL Æ (q 1 -> data ) 1 = new 0 Æ lock 2 = 0 Æ new 1 = new 0 +1 Æ new 1 =old 1

Predicates: old=new, new=new+1, new=old Proof of Unsatisfiability old 1 = new 0 new 1 = new new 1 = old 1 +1 new 1 = old 1 ; Proof of Unsatisfiability lock 1 = 1 Æ old 1 = new 0 Æ q 1 = q 0 - >next Æ q 1 != NULL Æ (q 1 -> data ) 1 = new 0 Æ lock 2 = 0 Æ new 1 = new 0 +1 Æ new 1 =old 1 Trace Feasibility Formula Add: old=new [HenzingerJhalaM.Sutre02]

Decision Procedure Must Produce Proofs Decision Procedure  yes no Boolean SAT Sat asgn Proof Proofs important in other applications as well: e.g., Proof carrying code [Necula97,HenzingerJhalaM.NeculaSutreWeimer02]

Counterexample Traces: Take 2 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) y = x +1 1: x = ctr; 2: ctr = ctr + 1; 3: y = ctr; 4: if (x = i-1){ 5: if (y != i){ ERROR: } } 1: x = ctr; 2: ctr = ctr + 1; 3: y = ctr; 4: if (x = i-1){ 5: if (y != i){ ERROR: } }

Trace Formulas 1: x = ctr 2: ctr = ctr+1 3: y = ctr 4: assume(x=i-1) 5: assume(y  i) Trace SSA Trace x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 1: x 1 = ctr 0 2: ctr 1 = ctr : y 1 = ctr 1 4: assume(x 1 =i 0 -1) 5: assume(y 1  i 0 ) Trace Feasibility Formula

Proof of Unsatisfiability Trace Formula x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 x 1 = ctr 0 x 1 = i 0 -1 ctr 0 = i 0 -1 ctr 1 = ctr 0 +1 ctr 1 = i 0 y 1 = ctr 1 y 1 = i 0 y 1  i 0 ; Proof of Unsatisfiability

The Present State… 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Trace … is all the information the executing program has here 1. … after executing trace past (prefix) 2. … knows present values of variables 3. … makes trace future (suffix) infeasible State… At pc 4, which predicate on present state shows infeasibility of future ?

What Predicate is needed ? Trace 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Trace Formula (TF) x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0

What Predicate is needed ? Trace 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Trace Formula (TF) x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 1. … after executing trace prefix Relevant Information … implied by TF prefix Predicate …

1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) 1. … after executing trace prefix 2. … has present values of variables What Predicate is needed ? Trace Trace Formula (TF) x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 … implied by TF prefix … on common variables Predicate … x1x1 x1x1 Relevant Information y1y1 y1y1

1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) What Predicate is needed ? Trace Trace Formula (TF) x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 Predicate … 1. … after executing trace prefix 2. … has present values of variables 3. … makes trace suffix infeasible … implied by TF prefix … on common variables … & TF suffix is unsatisfiable Relevant Information

Interpolant = Predicate ! -- ++ Interpolate  Trace Formula x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 y 1 = x Predicate … … implied by TF prefix … on common variables … & TF suffix is unsatisfiable Craig Interpolant [Craig 57] Computable from Proof of Unsat 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Trace Predicate at 4: y= x+1

Another interpretation … -- ++ Interpolate  Trace Formula x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 y 1 = x Unsat = Empty Intersection = Trace Infeasible Predicate at 4: y= x+1 -- ++ After exec prefix Can exec suffix  Interpolant  = Overapproximation of states after prefix that cannot execute suffix

Predicates: LOCK, new==old Refinement Main Questions : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } LOCK, new=old 4 4 : LOCK, new==old 5 5 SAFE LOCK, new==old : LOCK, : new = old : LOCK, : new == old Q. How to find good predicates ? Where to track each predicate? Q: How to compute interpolants? (And do they always exist?)

Another Proof of Unsatisfiability x 1 – ctr 0 =0 x 1 -i 0 +1=0 ctr 0 -i 0 +1=0 ctr 1 - ctr 0 -1=0 ctr 1 -i 0 =0 y 1 -ctr 1 =0 y 1 -i 0 =0 y 1 -i 0  0 0  0 Rewritten Proof £ 1 £ (-1) £ 1 x 1 = ctr 0 x 1 = i 0 -1 ctr 0 = i 0 -1 ctr 1 = ctr 0 +1 ctr 1 = i 0 y 1 = ctr 1 y 1 = i 0 y 1  i 0 ; Proof of Unsatisfiability

Interpolant from Rewritten Proof ? Trace Formula x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 x 1 – ctr 0 =0 x 1 -i 0 +1=0 ctr 0 -i 0 +1=0 ctr 1 - ctr 0 -1=0 ctr 1 -i 0 =0 y 1 -ctr 1 =0 y 1 -i 0 =0 y 1 -i 0  0 0  0 Rewritten Proof £ 1 £ (-1) £ 1 Interpolate

Interpolant from Rewritten Proof ? Trace Formula x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 x 1 – ctr 0 =0 ctr 1 - ctr 0 -1=0 y 1 -ctr 1 =0 Interpolant ! £ (-1) £ 1 Interpolate y 1 -x 1 -1=0 y 1 =x 1 +1

Building Predicate Maps TraceTrace Formula x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 Cut + Interpolate at each point Pred. Map: pc i  Interpolant from cut i -- ++ x 1 = ctr 0 Predicate Map 2: x= ctr 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Interpolate

Building Predicate Maps TraceTrace Formula x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 -- ++ 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Predicate Map 2: x = ctr 3: x= ctr-1 x 1 = ctr 1 -1 Interpolate Cut + Interpolate at each point Pred. Map: pc i  Interpolant from cut i

Building Predicate Maps TraceTrace Formula x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Predicate Map 2: x = ctr 3: x= ctr - 1 4: y= x + 1 y 1 = x 1 +1 -- ++ Interpolate Cut + Interpolate at each point Pred. Map: pc i  Interpolant from cut i

Building Predicate Maps TraceTrace Formula x 1 = ctr 0 Æ ctr 1 = ctr Æ y 1 = ctr 1 Æ x 1 = i Æ y 1  i 0 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Predicate Map 2: x = ctr 3: x= ctr - 1 4: y= x + 1 5: y = i y 1 = i 0 -- ++ Interpolate Cut + Interpolate at each point Pred. Map: pc i  Interpolant from cut i

Local Predicate Use Predicate Map 2: x = ctr 3: x= ctr - 1 4: y= x + 1 5: y = i Use predicates needed at location #Preds. grows with program size #Preds per location small Local Predicate use Ex: 2n states Global Predicate use Ex: 2 n states Verification scales …

pc 1 : x 1 = 3 pc 2 : assume (x 1 >0) pc 3 : x 3 = f 1 (x 1 ) pc 4 : y 2 = y 1 pc 5 : y 3 = f 2 (y 2 ) pc 6 : z 2 = z 1 +1 pc 7 : z 3 = 2*z 2 pc 8 : return z 3 pc 9 : return y 3 pc 10 : x 4 = x 3 +1 pc 11 : x 5 = f 3 (x 4 ) pc 12 : assume(w 1 <5) pc 13 : return w 1 pc 14 : assume x 4 >5 pc 15 : assume (x 1 =x 3 +2) Traces with Procedure Calls Trace Formula i Trace i pc 1 : x 1 = 3 pc 2 : assume (x 1 >0) pc 3 : x 3 = f 1 (x 1 ) pc 4 : y 2 = y 1 pc 5 : y 3 = f 2 (y 2 ) pc 6 : z 2 = z 1 +1 pc 7 : z 3 = 2*z 2 pc 8 : return z 3 pc 9 : return y 3 pc 10 : x 4 = x 3 +1 pc 11 : x 5 = f 3 (x 4 ) pc 12 : assume(w 1 <5) pc 13 : return w 1 pc 14 : assume x 4 >5 pc 15 : assume(x 1 =x 3 +2)

Interprocedural Analysis Trace Formula i Trace i Require at each point i : Scoped predicates YES: Variables visible at i NO: Caller’s local variables Find predicate needed at point i YES NO

Problems with Cutting Trace Formula i Trace i -- ++ Caller variables common to  - and  + Unsuitable interpolant: not well-scoped

Scoped Cuts Trace Formula i Call begins Trace i

Scoped Cuts -- ++ Trace Formula i Call begins Trace i Predicate at pc i = Interpolant from cut i

Common Variables Formals Current locals Trace Formula Predicate at pc i = Interpolant from i-cut i Trace i -- ++ Common Variables Formals Well-scoped

Question: When Do Interpolants Exist? Craig’s Theorem guarantees existence for first order logic But we are interpreting formulas over theories (arithmetic, theories of data structures)

The Good News Interpolants always exist for recursively enumerable theories –The proof is a simple application of compactness So: interpolants exist for Presburger arithmetic, sets with cardinality constraints, theory of lists, (quantifier-free) theory of arrays, multisets, …

The Bad News “The proof is a simple application of compactness” –May be algorithmically inefficient –Daunting engineering task to construct interpolating decision procedure for each individual theory

An Alternate Path: Reduction Want to compile formulas in a new theory to formulas in an old theory such that interpolation in the old theory imply interpolation in the new theory T reduces to R: can compile formulas in theory T to formulas in theory R –And use decision procedures for R to answer decision questions for T Technically: Given theories T and R, with R µ T, a reduction is a computable map  from T formulas to R formulas such that for any T-formula  : –  and  (  ) are T-equivalent –  is T-satisfiable iff  (  ) is R-satisfiable

Example: Theory of Sets Theory of sets reduces to theory of equality with uninterpreted functions x = y 8 e. e 2 x, e 2 y x= ;8 e.e  x x=U 8 e.e 2 x x={e}e 2 x Æ 8 e’.e’ 2 x ) e = e’ x=y [ z 8 e.e 2 x, e 2 y Ç e 2 z x=y Å z 8 e.e 2 x, e 2 y Æ e 2 z

Example: Theory of Multisets Theory of multisets reduces to the combination theory of equality with uninterpreted functions and linear arithmetic x = y 8 e. count(x,e) = count(y,e) x= ;8 e.count(x,e) = 0 x=[(e,n)]count(x,e)=max(0,n) Æ 8 e’.e’  e ) count(x,e’)=0 x=y ] z 8 e.count(x,e)= count(y,e)+count(z,e) x=y [ z 8 e. count(x,e) = max(count(y,e), count(z,e)) x=y Å z 8 e. count(x,e) = min(count(y,e), count(z,e))

Reduction and Interpolation  - and  + in Theory T  - and  + in Theory R Interpolant  in Theory R as well as T Quantifier-free interpolant Reduction from T to R Interpolate in R Eliminate quantifiers in T or R

Reduction Theorem Interpolants for the theory of arrays, sets, and multisets can be computed by reduction to the combination theory of linear arithmetic and equality with uninterpreted functions –We already have interpolating decision procedures for this latter theory [KapurM.Zarba06]

Lazy Abstraction Abstract Refine C Program Safe Trace Yes No Property Refine Trace Feas Formula Thm Pvr Proof of Unsat Pred. Map PC  Preds. Ctrex. Trace Interpolate Solution: Localize pred. use, find where preds. needed Problem: #Preds grows w/ Program Size

So far … Lazy Abstraction Predicates: –Abstract infinite program states Counterexample-guided Refinement: –Find predicates tailored to prog, property 1.Abstraction : Expensive Reachability Tree 2.Refinement : Find predicates, use locations Proof of unsat of TF + Interpolation

What the “decision procedure” must provide Decision Procedure  yes no Interpolant  +,  - Boolean SAT Sat asgn Proof

Predicates: LOCK, new==old Refinement : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } LOCK, new=old 4 4 : LOCK, new==old 5 5 SAFE LOCK, new==old : LOCK, : new = old : LOCK, : new == old So how well does all this work? Quite well, if the program and property are control-dominated Not so well when data is involved…

Refinement Failure: Unrolling Loops x = 0; y = 50; while ( x<100 ) { if ( x>=50 ) y = y+1; x = x+1; } assert( y==100 ); counterexample: x=0; y=50; x>=100; y==100 refinement: x==0 counterexample: x=0; y=50; x =100; y==100 refinement: x==1 counterexample: x=0; y=50; x<100; x=x+1; x<100; x=x+1; x>=100; y==100 refinement: x==2...

Refinement Failure: Unfolding Arrays for (i=0; i<n; i++) { a[i]=i; } for (j=0; j<n; j++) { assert( a[j]==j ); } counterexample: i=0; i =n; j=0; j<n; a[j]!=j refinement: a[0]==0 counterexample: i=0; i =n; j=0; j<n; a[j]==j; j++; j<n; a[j]!=j refinement: a[1]==1...

What went Wrong? Consider all unrolled counterexamples at once –Convergence of abstraction discovery Inspect families of counterexamples of unbounded length –Justification for unbounded universal quantification Looking at one counterexample path at a time is too weak [JhalaMcMillan05,JhalaMcMillan06]

Path Programs Treat counterexamples as programs –“Close” the loops A B C B D stmt1 stmt2 stmt3 stmt4 A B C D stmt1 stmt2 stmt3 stmt4 counterexample path path program, contains loops, etc [BeyerHenzingerM.Rybalchenko07]

Meaning of Path Programs Path program ´ (Possibly unbounded) sets of counterexamples: Unbounded counterexamples Property-determined fragment of original program –Can be analyzed independently to find good abstractions

Path Invariants Invariant for path programs ´ path invariant Abstraction refinement using path invariants –Elimination of all counterexamples within path program –Justification for unbounded quantification

Invariant Generation Given a path program, with a designated error location, find an invariant that demonstrates error is not reachable –Can scale: Reduced obligation to program fragment –Outer model checking loop integrates path invariants into program invariant Can use any technique We use constraint-based invariant generation [SankaranarayananSipmaManna04,BeyerHenzingerM.Rybalchenko07]

What we need Decision Procedure  yes no Interpolant  +,  - Model Boolean SAT Pre- Process Proof Invariant Generation

Summary Traditional distinctions between theorem proving, model checking, and dataflow analysis are getting blurred Is Blast a state space exploration engine with a theorem proving subroutine? A theorem prover with a model checking tactic? An abstract interpreter with a rich and variable domain? It is all of the above, and none of the above –The distinctions are mostly of historical interest –New algorithms are re-shaping the way we think about, and build, tools

More generally… What does it take to get verified software? –Blast is a small part in the whole process –How can we make the programmer’s job easier, through a combination of Language support Process support Tool support Debugging and visualization support? –How can we write software so that it is verifiable?

Acknowledgments Dirk Beyer Tom Henzinger Ranjit Jhala Deepak Kapur Ken McMillan Andrey Rybalchenko Gregoire Sutre Calogero Zarba

Trace Formula TraceTrace Formula m1: assume(flag!=0) m2: y=inc(x,flag) i1: assume(sign!=0) i2: rv = a+1 i4: return rv m3: assume(y<=x) Æ flag 0  0 Æ a 0 =x 0 Æ sign 0 = flag 0 Æ sign 0  0 Æ rv 0 =a 0 +1 Æ y 0 =rv 0 Æ y 0 <= x 0 i -- ++ i rv 0 a0a0 a0a0 rv 0 > a 0 Interpolate

partition(int *a, int n) { int i, gelen, ltlen; int ge[n], lt[n]; gelen = 0; ltlen = 0; L1: for(i = 0; i < n; i++) { if ( a[i] >= 0 ) { L2: ge[gelen] = a[i]; gelen++; } else { L3: lt[ltlen] = a[i]; ltlen++; } } L4: for(i = 0; i < gelen; i++) { assert( ge[i] >= 0 ); } L5: for(i = 0; i < ltlen; i++) { assert( lt[i] < 0 ); } } Example: quantified path invariant Counterexample L1,L2,L1,L4,Err Path invariant: L1 : 8 0 · k < gelen ! ge[k] ¸ 0 L4 : 8 i · k < gelen ! ge[k] ¸ 0 Counterexample L1,L3,L1,L4,L5,Err Path invariant: L1 : 8 0 · k < ltlen ! lt[k] < 0 L4 : 8 0 · k < ltlen ! lt[k] < 0 L5 : 8 i · k < ltlen ! lt[k] < 0

Verification by Theorem Proving 1. Loop Invariants 2. Logical formula 3. Check Validity Invariant: lock Æ new = old Ç : lock Æ new  old Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; }

Verification by Theorem Proving 1. Loop Invariants 2. Logical formula 3. Check Validity - Loop Invariants - Multithreaded Programs + Behaviors encoded in logic + Decision Procedures - Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Precise [ESC]

Verification by Program Analysis 1. Dataflow Facts 2. Constraint System 3. Solve constraints Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } - Imprecision due to fixed facts + Abstraction + Type/Flow Analyses Scalable [CQUAL, ESP, MC]

Verification by Model Checking 1. (Finite State) Program 2. State Transition Graph 3. Reachability Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } - Pgm ! Finite state model -State explosion + State Exploration + Counterexamples Precise [SPIN, SMV, Bandera,JPF ]

Combining Strengths Theorem Proving - loop invariants + Behaviors encoded in logic Refine + Theorem provers Computing Successors,Refine Program Analysis - Imprecise + Abstraction Shrink state space Model Checking - Finite-state model, state explosion + State Space Exploration Path Sensitive Analysis + Counterexamples Finding Relevant Facts Lazy Abstraction