From Verification to Synthesis Sumit Gulwani Microsoft Research, Redmond August 2013 Marktoberdorf Summer School Lectures: Part 1

1 Synthesis Goal: Synthesize a computational concept in some underlying language from user intent using some search technique. State of the art: We can synthesize programs of size

Language –Programs Straight-line programs –Automata –Queries User Intent –Logic, Natural Language –Examples, Demonstrations/Traces –Program Search Technique –SAT/SMT solvers (Formal Methods) –A*-style goal-directed search (AI) –Version space algebras (Machine Learning) 2 Dimensions in Synthesis PPDP 2010: “Dimensions in Program Synthesis”, Gulwani. (Application) (Ambiguity) (Algorithm)

3 Compilers vs. Synthesizers DimensionCompilersSynthesizers Concept Language Executable ProgramVariety of concepts: Program, Automata, Query, Sequence User IntentStructured languageVariety/mixed form of constraints: logic, examples, traces Search Technique Syntax-directed translation (No new algorithmic insights) Uses some kind of search (Discovers new algorithmic insights)

From verification to synthesis –Bitvector algorithms (PLDI 2011, ICSE 2012) –General loopy programs (POPL 2010) –SIMD algorithms (PPoPP 2013) –Program inverses (PLDI 2011) –Graph algorithms (OOPSLA 2010) End-user Programming (Examples & Natural Language) –Syntactic string transformations: Flash Fill (POPL 2011) –Semantic string transformations (VLDB 2012) –Table layout transformations (PLDI 2011) –Smartphone scripts (MobiSys 2013) Computer-aided Education –Problem Synthesis (AAAI 2012, CHI 2013) –Solution Synthesis (PLDI 2011, IJCAI 2013) –Feedback Synthesis (PLDI 2013, IJCAI 2013) –Content Authoring (CHI 2012) 4

ApplicationGenerating Synthesis Constraint Solving Synthesis Constraint BitvectorLocation variablesCEGIS + SMT Loopy Alg.Template-basedSMT SIMDRelational verificationCEGIS + Reachability value graph InversesTemplate-based + symbolic execution SMT Graph Alg. From Verification to Synthesis 5 Reference: Path-based Inductive Synthesis for Program Inversion, PLDI 2011, Srivastava, Gulwani, Chaudhuri, Foster

Language –Programs Straight-line programs –Automata –Queries User Intent –Logic, Natural Language –Examples, Demonstrations/Traces –Program Search Technique –SAT/SMT solvers (Formal Methods) –A*-style goal-directed search (AI) –Version space algebras (Machine Learning) 6 Dimensions in Synthesis

In-place run-length encoding: A = [1,1,1,0,0,2,2,2,2] Encoder A=[1,0,2] N=[3,2,4] Decoder A’=[1,1,1,0,0,2,2,2,2] Program Inversion: Example 7 IN(A,n); Assume (n >= 0) i, m := 0, 0; // parallel assignment while (i<n) r := 1; while (i+1<n && A[i]=A[i+1]) r, i := r+1, i+1; A[m], N[m], m, i := A[i], r, m+1, i+1; OUT(A,N,m); IN(A,N,m) i’, m’ := 0, 0; while (m’ < m) r’ := N[m’]; while (r’>0) r’,i’, A’[i’] := r’-1, i’+1, A[m’]; m’ := m’+1; OUT(A’,m’); assert(A’=A; m’=n);

In-place run-length encoding: A = [1,1,1,0,0,2,2,2,2] Encoder A=[1,0,2] N=[3,2,4] Decoder A’=[1,1,1,0,0,2,2,2,2] Program Inversion as Synthesis Problem 8 IN(A,n); Assume (n >= 0) i, m := 0, 0; // parallel assignment while (i<n) r := 1; while (i+1<n && A[i]=A[i+1]) r, i := r+1, i+1; A[m], N[m], m, i := A[i], r, m+1, i+1; OUT(A,N,m);

Synthesis Technique 9

Reference: Program Synthesis by Sketching, Phd Thesis 2008, Armando Solar-Lezama (Advisor: Ras UC-Berkeley) Key Ideas: –Write an arbitrary program with holes, where each hole takes values from a finite domain. –Use CEGIS to generate SAT constraints on holes. Cons: Not as efficient as domain-specific synthesizers. –(On bitvector benchmark, times out on 9/25 tasks, and on the remaining it is slower by 20x on average). Pros: –A very powerful formalism that can be used to model a variety of synthesis problems. –Sees synthesis as an interactive process. Related Work: Program Sketching 10