Rahul Sharma, Saurabh Gupta, Bharath Hariharan, Alex Aiken, and Aditya Nori (Stanford, UC Berkeley, Microsoft Research India) Verification as Learning Geometric Concepts

Invariants assume x<0; while ( x<0 ) { x = x+y; y = y+1; } assert y>0;

Disjunctive invariants assume n > 0; x = 0 ; while ( x < n ) { x = x+1; } assert x = n;

Classification positive examples negative examples

From invariants to classifiers Safety properties define bad states Invariants separate reachable states from bad states Possible to obtain some examples of states Invariants -> classifiers Examples of reachable/good states -> positive examples Examples of bad states -> negative examples Use a classifier to separate ALL good and bad states

Sample, guess, and check Generate examples of good and bad concrete states Guess an invariant using learner Check if verification succeeds If yes, then done If no, then guess again with more examples Use counter-examples to verification task

Sample good states assume x<0; while ( x<0 ) { print(x,y); x = x+y; y = y+1; } assert y>0; Reachable states Run the program

Sample bad states assume P; while ( B ) { S } assert Q;

From program to data assume x<0; while ( x<0 ) { x = x+y; y = y+1; } assert y>0; x = -1, y = 0 x>=0 && y<= x y

Learner Bshouty, Goldman, Mathias, Suri, Tamaki in STOC’96 Learn arbitrary boolean combinations of inequalities Create a large enough candidate set of planes Intelligently select from candidates Separate given examples of good and bad states Use only a few planes

Candidate planes x y

Example x y x y

Guarantees

From planes to predicates x y

Efficiency?

Small candidate sets

Guarantees on generalization Programs have unbounded behaviors Analyze some finite behaviors and generalize SLAM/BLAST: Ask for predicates to discard spurious cexs Impact: Unwind loops and interpolate Abstract interpretation: iterate and widen Need a formal definition of generalization Need generalization guarantees for useful tools

A step: PAC

Summary of results Given sufficient good and bad samples, with high probability, the learner generates a predicate, that has high accuracy for unseen samples The generated classifier is expressive Arbitrary boolean combinations of linear inequalities #Planes in classifier independent of samples Worst case only logarithmically more than invariant

Non-linear invariants

Implementation

Experiments

Related work Invariant inference Abstract interpretation – disjunctive completion Constraint based (Sting, InvGen, GSV’ 08) Use tests to help static analysis: Yogi, InvGen, … Guess and check: Daikon, SAN’ 12, SGHALN’ 13

Conclusion Connections between verification and learning Generalization is a fundamental problem for both Possible to obtain invariant generators with guarantees Handling disjunctions and non-linearities is easy Difficult for symbolic approaches Need data, which is available Future work, beyond numerical