Termination Proofs from Tests Aditya Nori Rahul Sharma MSR India Stanford University

Goal Prove termination of a program Program terminates if all loops terminate Hard problem, undecidable in general Need to exploit all available information

Tests Previous techniques are static Tests have previously been used Tests are a neglected source of information Tests have previously been used Safety properties, empirical complexity, … This work, use tests for termination proofs

Example: GCD gcd(int x,int y) assume(x>0 && y>0); while( x!=y ) do if( y > x ) y = y–x; if( x > y) x = x-y; od return x; x=1, y=1 x=2, y=1 ⋮

Infer-and-Validate Approach (1,1) (2,1) ⋮ … while … … while … print x print y x=1, y=3 Data … while … assert … ML

Infer-and-Validate Approach (1,1) (2,1) ⋮ … while … … while … print x print y x=1, y=3 Data … while … assert … ML

Instrument the Program gcd(int x, int y) assume(x>0 && y>0); a := x; b := y; c := 0; while( x!=y ) do c := c + 1; if( y > x ) y := y–x; if( x > y) x := x-y; od print ( a, b, c ); New variables to capture initial values Introduce a loop counter Print values of input variables and counter

Infer-and-Validate Approach (1,1) (2,1) ⋮ … while … … while … print x print y x=1, y=3 Data … while … assert … ML

Generating Data For 𝑖∈ℕ, on inputs 𝐴 𝑖 , the loop iterates 𝐶 𝑖 times 𝐴≡ 1 𝑎 𝑏 1 1 1 1 2 1 1 1 3 𝐶≡ 𝑐 0 1 2 For 𝑖∈ℕ, on inputs 𝐴 𝑖 , the loop iterates 𝐶 𝑖 times Infer a bound using 𝐴 and 𝐶 gcd(int x, int y) assume(x>0 && y>0); a := x; b := y; c := 0; while( x!=y ) do c := c + 1; if( y > x ) y := y–x; if( x > y) x := x-y; od print( a, b, c)

Infer-and-Validate Approach (1,1) (2,1) ⋮ … while … … while … print x print y x=1, y=3 Data … while … assert … ML

Regression Predict number of iterations (final value of c) As a linear expression in a and b Find w 1 , w 2 , w 3 :𝑤 1 + 𝑤 2 𝑎 𝑖 + 𝑤 3 𝑏 𝑖 ≈ 𝑐 𝑖 Find w 1 , w 2 , w 3 : min 𝑖=1 𝑛 𝑤 1 + 𝑤 2 𝑎 𝑖 + 𝑤 3 𝑏 𝑖 − 𝑐 𝑖 2 But we want 𝑤 1 + 𝑤 2 𝑎+ 𝑤 3 𝑏≥𝑐 Add 𝑤 1 + 𝑤 2 𝑎 𝑖 + 𝑤 3 𝑏 𝑖 ≥ 𝑐 𝑖 as a constraint Solvable by quadratic programming

Quadratic Program (QP) The quadratic program is: min 1 2 𝑤 𝑇 𝐴 𝑇 𝐴𝑤− 𝑤 𝑇 𝐴 𝑇 𝐶 𝑠.𝑡. 𝐴𝑤≥𝐶 Solved in MATLAB quadprog(A’*A,-A’*C,-A,-C) For gcd example, 𝑤=[−2,1,1] Bound 𝑐≤𝑎+𝑏−2

Naïve Regression

Quadratic Program

Infer-and-Validate Approach (1,1) (2,1) ⋮ … while … … while … print x print y x=1, y=3 Data … while … assert … ML

Verification Burden Bound: 𝑐≤𝑎+𝑏−2 Difficult to validate assume(x>0 && y>0); a := x; b := y; c := 0; while( x!=y ) do c := c + 1; if( y > x ) y := y–x; if( x > y) x := x-y; assert(c <= a+b-2); od Bound: 𝑐≤𝑎+𝑏−2 Difficult to validate Infer invariants from tests

Regression for Invariant assume(x>0 && y>0); a := x; b := y; c := 0; while( x!=y ) do print(c, a, b, x, y); c := c + 1; if( y > x ) y := y–x; if( x > y) x := x-y; assert(c <= a+b-2); od Predict a bound on c Same tests, more data Solve same QP 𝐴 has five columns [1,a,b,x,y] 𝐶 has c at every iteration

Free Invariant Obtain 𝑐≤𝑎+𝑏−𝑥−𝑦 Add as a free invariant assume(x>0 && y>0); a:=x; b:=y; c := 0; free_inv(c<=a+b-x-y); while( x!=y ) do c := c + 1; if( y > x ) y := y – x; if( x > y) x := x-y; assert(c <= a+b-2 ); od Obtain 𝑐≤𝑎+𝑏−𝑥−𝑦 Add as a free invariant Use if checker can prove Otherwise discard

𝑐≤𝑎+𝑏−𝑥−𝑦∧𝑥>0∧𝑦>0 Validate Give program to assertion checker Inductive invariant for gcd example: 𝑐≤𝑎+𝑏−𝑥−𝑦∧𝑥>0∧𝑦>0 If check fails then return a cex as a new test

Non-linear Example u := x;v := y;w := z; while ( x >= y ) do if ( z > 0 ) z := z-1; x := x+z; else y := y+1; od Given degree 2, 𝐴≡[1,𝑢,𝑣,𝑤,𝑢𝑣,𝑣𝑤,𝑤𝑢, 𝑢 2 , 𝑣 2 , 𝑤 2 ] Bound: 𝑐≤1.9+𝑢−𝑣+0.95𝑤+0.24 𝑤 2 After rounding: 𝑐≤2+𝑢−𝑣+𝑤+ 𝑤 2

Assertion Checker Requirements from assertion checker: Handle non-linear arithmetic Consume free invariants Produce tests as counter-examples Micro-benchmarks: Use SGHAN’13 Handles non-linear arithmetic, no counter-examples Windows Device Drivers: Use Yogi (FSE’ 06) Cannot handle non-linear, produce counter-examples

Micro-benchmarks

Experiments with WDK

Related Work Regression: Goldsmith et al. ‘07 , Huang et al. ’10, … Mining specifications from tests: Dallmeier et al. `12,… Termination: Cousot `05, ResAna, Lee et al. ’12, … Bounds analysis: SPEED, WCET, Gulavani et al. `08, … Invariant inference: Daikon, InvGen, Nguyen et al.`12, …

Conclusion Use tests for termination proofs Infer bounds and invariants using QP Use off-the-shelf assertion checkers to validate Future work: disjunctions, non-termination

Disjunctions Example Partition using predicates 𝑎<𝑀∧𝑏≥𝑁⇒𝑐≤𝑀−𝑎 𝑎≥𝑀∧𝑏<𝑁⇒𝑐≤𝑁−𝑏 𝑎<𝑀∧𝑏<𝑁⇒ 𝑐≤𝑀+𝑁−𝑎−𝑏 Control flow refinement Sharma et al. ’11 a = i ; b = j ; while(i<M || j<N) i = i+1; j = j+1;