1. Deductive Techniques for synthesis from Inductive Specifications Dagstuhl Seminar Oct 2015 Sumit Gulwani

2. Vu Le Collaborators Dan Barowy Ted Hart Alex Polozov Dileep Kini Rishabh Singh Mikael Mayer Gustavo Soares Ben Zorn Bill Harris

3. 2 Reference “Programming by Examples (and its applications in Data Wrangling)”, Gulwani; 2016; In Verification and Synthesis of Correct and Secure Systems; IOS Press [based on Marktoberdorf Summer School 2015 Lecture Notes]

4. Deductive Synthesis Refers to synthesis using deductive methods. Has traditionally been applied to synthesis in the presence of logical specifications. Inductive Synthesis Refers to synthesis from inductive (example-based) specifications. Various kinds of techniques have been applied including constraint solving, stochastic, and enumerative search. This talk describes techniques for synthesis from inductive specifications using deductive methods! 3 Deductive Synthesis vs Inductive Synthesis

5. 4 PBE Architecture Example-based specification Search Algorithm Challenge 1: Ambiguous/under-specified intent may result in unintended programs. Challenge 2: Designing efficient search strategy. Ranking Function Program Ordered set of Programs

6. Key Ideas Restrict search to an appropriately designed domain- specific language (DSL) specified as a grammar. –Expressive enough to cover wide range of tasks –Restricted enough to enable efficient search 5 Challenge 2: Efficient search strategy “Spreadsheet Data Manipulation using Examples” [CACM 2012 Research Highlights] Gulwani, Harris, Singh

7. 6 FlashFill DSL Concatenate(A, C) SubStr(X, P, P) “Automating string processing in spreadsheets using input-output examples”; POPL 2011; Gulwani

8. 7 FlashExtract DSL substr expr S[z] := [z] [z]) “FlashExtract: A Framework for data extraction by examples”; PLDI 2014; Vu Le, Sumit Gulwani

9. 8 Challenge 2: Efficient search strategy “Spreadsheet Data Manipulation using Examples” [CACM 2012 Research Highlights] Gulwani, Harris, Singh

10. 9 Problem Reduction Spec for T Spec for L Spec for S FlashExtract DSL

11. substring expr E := SubStr(y, P 1, P 2 ) position expr P := K | Pos(y, R 1, R 2, K) 10 Problem Reduction Redmond, WA Spec for P 1 Spec for P 2 Spec for E SubStr grammar

12. 11 Programming by Examples Example-based specification Program Search Algorithm Challenge 1: Ambiguous/under-specified intent may result in unintended programs. Challenge 2: Designing efficient search strategy. Challenge 3: Lowering the barrier to design & development.

13. Developing a domain-specific robust search method is costly: Requires domain-specific algorithmic insights. Robust implementation requires good engineering. DSL extensions/modifications are not easy. Key Ideas: PBE algorithms employ a divide and conquer strategy, where synthesis problem for an expression F(e1,e2) is reduced to synthesis problems for sub-expressions e1 and e2. –The divide-and-conquer strategy can be refactored out. Reduction depends on the logical properties of operator F. –Operator properties can be captured in a modular manner for reuse inside other DSLs. 12 Challenge 3: Lowering the barrier “ FlashMeta: A Framework for Inductive Program Synthesis ” [OOPSLA 2015] Polozov, Gulwani

14. 13 Programming by Examples Example-based specification Program Search Algorithm Challenge 1: Ambiguous/under-specified intent may result in unintended programs. Challenge 2: Designing efficient search strategy. Challenge 3: Lowering the barrier to design & development. DSL

15. 14 Search Strategy

16. 15 Output properties Elements belonging to the output list Elements not belonging to the output list Contiguous subsequence of the output list Prefix of the output list Task

17. 16 Output properties Task Prefix of the output table (seq of records) We do not require explicit (magenta) record boundaries in which case the spec is: Prefixes of projections of the output table

18. 17 Search Strategy

19. 18 Search Strategy Key concepts in problem reduction: VSAs & Witness functions

20. 19 Version Space Algebra (VSA)

21. 20 VSA Operations

22. 21 Problem Reduction Rules

23. 22 Intersect Operation

24. 23 Problem Reduction Rules

25. 24 Cluster Operation

26. 25 Filter Operation

27. 26 Problem Reduction Rules

28. 27 Problem Reduction Rules

29. 28 Problem Reduction Rules

30. 29 Problem Reduction Rules

31. Provides efficient implementations of VSA operations Provides a library of witness functions Role of synthesis designer Can add new operators and witness functions. Can provide ranking strategies. Can specify tactics to resolve non-determinism in search –Which witness function to use? –How to order search branches? 30 FlashMeta Framework

32. Project FlashFill FlashExtractText FlashRelate FlashNormalize FlashExtractWeb 31 Comparison of FlashMeta with hand-tuned implementations OriginalFlashMeta 123 74 52 172 N/A2.5 OriginalFlashMeta 91 81 81 72 N/A1.5 Lines of Code (K) Development time (months) Running time of FlashMeta implementations vary between 0.5- 3x of the corresponding original implementation. Faster because of some free optimizations Slower because of larger feature sets & a generalized framework