1 Regression Verification: Proving the equivalence of similar programs Benny Godlin Ofer Strichman Technion, Haifa, Israel (This presentation is a subset of the invited cav’09 talk: ie.technion.ac.il/~ofers/presentations/rv1.ppt)

2 Functional Verification The main pillar of the grand challenge [H’03]. Suppose we ignore completeness. Still, there are two major problems:  Specification  Complexity

3 A more modest challenge: Regression Verification Develop a method for formally verifying the equivalence of two similar programs. Pros:  Default specification = earlier version.  Computationally easier than functional verification. Ideally, the complexity should depend on the semantic difference between the programs, and not on their size. Cons:  Defines a weaker notion of correctness.

4 Previous work In the theorem-proving world ACL2 community):  Not industrial programming languages  Not utilizing the similarity between the two programs Industrial / realistic programs:  Code free of: loops, recursion, dynamic-memory allocation Intel [AEFMMSSTVZ-05], embedded Feng & Hu [FH-05], symbolic Matsumoto et al. [TSF-06]

5 Our notion of equivalence Partial equivalence  Executions of P1 and P2 on equal inputs …which terminate, result in equal outputs. Undecidable

6 Partial equivalence Consider the call graphs:  … where A, B have: same prototype no loops Prove partial equivalence of A, B  How shall we handle the recursion ? A B Side 1Side 2

7 //in[A] A(... ) {... //in[call A] call A(...); //out[call A]... } //out[A] Proving partial equivalence A B //in[B] B(... ) {... // in[call B] call B(...); //out[call B]... } //out[B]

8 Rule 1: Proving partial equivalence Q: How can a verification condition for the premise look like? A: Replace the recursive calls with calls to functions that  over-approximate A, B, and  are partially equivalent by construction Natural candidates: Uninterpreted Functions

9 Proving partial equivalence Let A UF, B UF be A, B, after replacing the recursive call with a call to (the same) uninterpreted function. We can now rewrite the rule: The premise is decidable

10 unsigned gcd1 UF (unsigned a, unsigned b) { unsigned g; if (b == 0) g = a; else { a = a % b; g = gcd1(b, a); } return g; } unsigned gcd2 UF (unsigned x, unsigned y) { unsigned z; z = x; if (y > 0) z = gcd2(y, z % y); } return z; } Using (PART-EQ-1) : example ?=?= U U a,a,b)b) x,x, y)y) g;g; z;z; Transition functions Inputs Outputs T gcd1 T gcd2 a,b x,y g z

11 Rule 1: example Transition functions T gcd1 T gcd2 Inputs a,ba,bx,yx,y Outputs gz Equal inputs Equal outputs

12 Partial equivalence: Generalization Assume:  no loops;  1-1 mapping map between the recursive functions of both sides Mapped functions have the same prototype Define:  For a function f, UF( f ) is an uninterpreted function such that f and UF( f ) have the same prototype ( f, g ) 2 map, UF( f ) = UF( g ).

13 Partial equivalence: Generalization Definition: is called in A]

14 Partial equivalence: Example (1 / 3) Side 1 Side 2 f ’ g g’ f {(g,g’),(f,f’)} 2 map Need to prove: f ’ UF f g g’ UF = =

15 Partial equivalence: Example (2 / 3) An improvement:  Find a map that intersects all cycles, e.g., (g,g’)  Only when calling functions in this map replace with uninterpreted functions Side 1 Side 2 f ’ UF g g’ f UF

16 Partial equivalence: Example (3 / 3) Connected SCCs… Prove bottom-up Abstract partially-equivalent functions Inline Side 1 Side 2 f ’ gg’ f h h’ UF

17 RVT: Decomposition algorithm A: B: f1() f2() f5() f3()f4() f6() f1’() f2’() f3’()f4’() f5’() Equivalent pair Syntactically equivalent pair Equivalence undecided yet Could not prove equivalent Legend: check Unpaired function f7’() U UUU U U

18 RVT: Decomposition algorithm (with SCCs) A: B: f1() f2() f5() f3()f4() f6() f1’() f3’()f4’() f5’() f6’() Equivalent pair Syntactically equivalent pair Equivalence undecided yet Could not prove equivalent Legend: Equivalent if MSCC U UUU U U check U U U U f2’()

19 The Regression Verification Tool (RVT) Given two C programs:  loops  recursive functions.  Map functions, globals, etc. After that:  Decompose to the granularity of pairs of functions  Use a C verification engine (CBMC) to discharge

20 The Regression Verification Tool (RVT) CBMC: a C bounded model checker by Daniel Kroening Our use:  No loops or recursion to unroll...  Use “assume(…)” construct to enforce equal inputs.  Use assert() to demand equal outputs. Uninterpreted functions are implemented as C functions:  Return consistent nondeterminisitic values.

21 The Regression Verification Tool (RVT) The premise of ( PART-EQ ) requires comparing arguments. What if these arguments are pointers ? What our system does:  Dynamic structures: creates an unrolled nondeterministic structure  Arrays: attempts to find references to cells in the array.

22 RVT: User-defined equivalence specification The user can define pairs of ‘checkpoints’: side 1: side 2: In each side:  update an array with the value of exp each time it reaches label and condition holds. Assert that when executed on the same input…,  … these arrays are equivalent. exp 1 exp 2... P1: exp ’ 1 exp ’ 2... P2: = ===

23 RVT Version AVersion B CBMC  rename identical globals  enforce equality of inputs.  assert equality of outputs  add checkpoints  Supports:  Decomposition  Abstraction  some static analysis  … feedback  result  counterexample C program RVT

24

25 RVT: Experiments Automatically generated sizable programs with complex recursive structures and loops. up-to thousands of lines of code Limited-size industrial programs:  Parts of TCAS - Traffic Alert and Collision Avoidance System.  Core of MicroC/OS - real-time kernel for embedded systems.  Matlab examples: parts of engine-fuel-injection simulation. We tested the Regression Verification Tool (RVT) with:

26 Testing RVT on programs: Conclusions For equivalent programs, partial-equivalence checks were very fast:  proving equivalence in minutes. For non-equivalent programs:  RVT attempts to prove partial-equivalence but fails then RVT tries to prove k-equivalence

27 Summary Regression verification is an important problem  A solution to this problem has a better chance to succeed in the industry than functional verification  A grand challenge by its own right… Lots of future research...

28 More Challenges Q1: How can we generalize counterexamples ? Q2: What is the ideal gap between two versions of the same program, that makes Regression Verification most effective ? Q3: How can functional verification and equivalence verification benefit from each other ?

29 The end … Thank you