Proving Programs Robust Swarat Chaudhuri 1,3 Sumit Gulwani 2 Roberto Lublinerman 1 Sara Navidpour 1 1 Pennsylvania State University 2 Microsoft Research 3 Rice University

Uncertainty Uncertainty: stale satellite data, erroneous sensor measurements, … Does your program handle uncertainty robustly? FSE’11: Szeged, Hungary. 2 Proving programs robust

Robustness FSE’11: Szeged, Hungary. 3 Proving programs robust  Correctness in settings without uncertainty does not imply correctness in uncertain environments.

Before we “measure” robustness Proving programs robust FSE’11: Szeged, Hungary

How would we “measure” robustness? No “jumps” due to loops and branches Analytical continuity P Chaudhuri, Gulwani, Lublinerman. Continuity Analysis of Programs. POPL f(x) = e x

Quantitative robustness properties k-robustness: Change in output is bounded by K times the change in inputs. P FSE’11: Szeged, Hungary. 6 Proving programs robust

Distance on arrays Proving programs robust FSE’11: Szeged, Hungary = = 3 max

Distance on graphs Proving programs robust FSE’11: Szeged, Hungary = 2max - =

 Robustness of shortest path Proving programs robust FSE’11: Szeged, Hungary. 9 You are here

 Robustness of shortest path Proving programs robust FSE’11: Szeged, Hungary You are here        

 Robustness of shortest path Proving programs robust FSE’11: Szeged, Hungary You are here   1  2   

 Robustness of shortest path Proving programs robust FSE’11: Szeged, Hungary You are here    

 Robustness of shortest path Proving programs robust FSE’11: Szeged, Hungary You are here   

 Robustness of shortest path Proving programs robust FSE’11: Szeged, Hungary You are here  

 Robustness of shortest path Proving programs robust FSE’11: Szeged, Hungary You are here 

 Robustness of shortest path Proving programs robust FSE’11: Szeged, Hungary You are here

 Robustness of shortest path Proving programs robust FSE’11: Szeged, Hungary You are here

 Robustness of shortest path Proving programs robust FSE’11: Szeged, Hungary You are here

k-robustness and classical problems ProblemInputOutputk-robustness Max of an array elements in the array maximum element of the array 1-robust Minimum spanning tree graphcostN-robust Sorting elements in the array sorted array1-robust FSE’11: Szeged, Hungary. 19 Proving programs robust

Robustness matrices R[y,x] = k  -variation in initial value of x produces at most a k  -variation in final value of y Proving programs robust FSE’11: Szeged, Hungary. 20

How do we prove k-robustness FSE’11: Szeged, Hungary. 21 Proving programs robust

How do we prove k-robustness FSE’11: Szeged, Hungary. 22 Proving programs robust

Proving k-robustness if B FT P1P1 P1P1 P2P2 P2P2 P 1. P 1 has robustness matrix R 1 2.P 2 has robustness matrix R 2 3.P is continuous *1 P has robustness matrix max(R 1, R 2 ) FSE’11: Szeged, Hungary. 23 Proving programs robust *1 Chaudhuri, Gulwani, Lublinerman. Continuity Analysis of Programs. POPL 2010.

Proving k-robustness P1P1 P1P1 P2P2 P2P2 P 1. P 1 has robustness matrix R 1 2.P 2 has robustness matrix R 2 P has robustness matrix R 2 R 1 FSE’11: Szeged, Hungary. 24 Proving programs robust Analogous to the “chain rule”

Proving piecewise k-robustness while B Q Q P 1. Q has robustness matrix R 2.N is the bound on loop iterations 3.P is continuous * 1 P has robustness matrix R N FSE’11: Szeged, Hungary. 25 Proving programs robust *1 Chaudhuri, Gulwani, Lublinerman. Continuity Analysis of Programs. POPL 2010.

Example: Dijsktra’s shortest path algorithm FSE’11: Szeged, Hungary. 26 Proving programs robust procedure Dijkstra(G: graph, src: node): for each node v in G: d[v] =  d[src] = 0; W := edges of G while W is not empty remove G[w,v] from W s.t. d[w] is minimal z = d[w] + G[w,v] if z < d[v] d[v] = z; prev[v] = w

Example: Dijsktra’s shortest path algorithm FSE’11: Szeged, Hungary. 27 Proving programs robust procedure Dijkstra(G: graph, src: node): for each node v in G: d[v] =  d[src] = 0; W := edges of G while W is not empty remove G[w,v] from W s.t. d[w] is minimal z = d[w] + G[w,v] if z < d[v] d[v] = z; prev[v] = w d[0] =  d[src] = 0

Example: Dijsktra’s shortest path algorithm FSE’11: Szeged, Hungary. 28 Proving programs robust procedure Dijkstra(G: graph, src: node): for each node v in G: d[v] =  d[src] = 0; W := edges of G while W is not empty remove G[w,v] from W s.t. d[w] is minimal z = d[w] + G[w,v] if z < d[v] d[v] = z; prev[v] = w if d[v] > d[w] + G[w,v] TF d[v]=d[w] + G[w,v]

Example: Dijsktra’s shortest path algorithm FSE’11: Szeged, Hungary. 29 Proving programs robust procedure Dijkstra(G: graph, src: node): for each node v in G: d[v] =  d[src] = 0; W := edges of G while W is not empty remove G[w,v] from W s.t. d[w] is minimal z = d[w] + G[w,v] if z < d[v] d[v] = z; prev[v] = w while W not empty Q Q d is N-robust in G

Applications: Robustness Proofs for Embedded Systems FSE’11: Szeged, Hungary. 30 Proving programs robust

for i:= …. Approximate Computation: Trade accuracy for reduced cost FSE’11: Szeged, Hungary. 31 Proving programs robust P’

Approximate loop rewriting: Our static analysis can guide this sort of approximation Accuracy vs. Performance trade-off of loop perforation [Sidiroglou, Misailovic, Hoffman, Rinard FSE11] Applications: Approximate Computation FSE’11: Szeged, Hungary. 32 Proving programs robust for i := 1 to n by 1 z = f(x[i]) sum = sum + z 2

Differential privacy [Dwork] FSE’11: Szeged, Hungary. 33 Proving programs robust Applications: Privacy in Statistical Databases

Results Implemented the analysis in a tool. – use Z3 to discharge continuity judgments. Proved robustness for many classical algorithms Proving programs robust FSE’11: Szeged, Hungary. 34 Bubble sortDijkstraKnapsack Insertion sortBellman-fordSmall controller Selection sortKruskall Merge sortPrim

Related Work Work on interval polyhedra [Chen et al 2009] [Chen et al 2010] Continuity analysis of programs [Chaudhuri et al 2010] Testing for robustness [Majmudar et al 2010] Calculus for differential privacy [Reed and Pierce 2010] Proving programs robust FSE’11: Szeged, Hungary. 35

Conclusion Robustness is an important correctness property for programs operating under uncertainty. We provide a automated program analysis for robustness. k-robustness can be proved mechanically for many interesting programs. Applications in different domains. FSE’11: Szeged, Hungary. 36 Proving programs robust

Sorting 1-robust? Proving programs robust FSE’11: Szeged, Hungary