Dimensions in Synthesis Part 3: Ambiguity (Synthesis from Examples & Keywords) Sumit Gulwani Microsoft Research, Redmond May 2012

Students and Teachers End-Users Algorithm Designers Software Developers Most Transformational Target Potential Users of Synthesis Technology 1 Most Useful Target Vision for End-users: Enable people to have (automated) personal assistants. Vision for Education: Enable every student to have access to free & high-quality education.

Examples –Bitvector Algorithms (ICSE ‘10) –Spreadsheet Macros (CACM ‘12) Syntactic String Transformations (POPL ‘11) Semantic String Transformations (VLDB ‘12) Number Transformations (CAV ‘12) Table Transformations (PLDI ‘11) Sketch –Drawings (CHI 2012) Keywords –SmartPhone Apps 2 Intent Specification

Examples  Bitvector Algorithms (ICSE ‘10) –Spreadsheet Macros (CACM ‘12) Syntactic String Transformations (POPL ‘11) Semantic String Transformations (VLDB ‘12) Number Transformations (CAV ‘12) Table Transformations (PLDI ‘11) Sketch –Drawings (CHI 2012) Keywords –SmartPhone Apps 3 Intent Specification ICSE 2010: Susmit Jha, Gulwani, Seshia, Tiwari.

4 Synthesis from Logical Specification Æ [ ( I[p]=1 Æ (I[j]=0) ) ) ( J[p]=0 Æ (J[j] = I[j]) ) ] p=1 b j=p+1 b jpjp Turn off rightmost 1-bit Functional Specification: Tool Output: J = I & (I-1) PLDI 2011: Gulwani, Jha, Tiwari, Venkatesan.

Turn-off rightmost contiguous string of 1’s User: I want a program that maps > Tool: There exist at least two programs that match the spec Program 1: (x+1) & (x-1) Program 2: (x+1) & x But they differ on (Distinguishing Input) What should be mapped to? User: >     5 Interactive Synthesis using Examples

Turn-off rightmost contiguous string of 1’s User: > Tool: ? User: Tool: ? User: Tool: ? User: Tool: ? User: Tool: ? User: Tool: Your program is x & (1 + ((x-1)|x)) 6 Interactive Synthesis using Examples

Examples –Bitvector Algorithms (ICSE ‘10)  Spreadsheet Macros (CACM ‘12) Syntactic String Transformations (POPL ‘11) Semantic String Transformations (VLDB ‘12) Number Transformations (CAV ‘12) Table Transformations (PLDI ‘11) Sketch –Drawings (CHI 2012) Keywords –SmartPhone Apps 7 Intent Specification

Examples –Bitvector Algorithms (ICSE ‘10) –Spreadsheet Macros (CACM ‘12)  Syntactic String Transformations (POPL ‘11) Semantic String Transformations (VLDB ‘12) Number Transformations (CAV ‘12) Table Transformations (PLDI ‘11) Sketch –Drawings (CHI 2012) Keywords –SmartPhone Apps 8 Intent Specification

Guarded Expression G := Switch((b 1,e 1 ), …, (b n,e n )) String Expression e := Concatenate(f 1, …, f n ) Base Expression f := s // Constant String | SubStr(v i, p 1, p 2 ) Index Expression p := k // Constant Integer | Pos(r 1, r 2, k) // k th position in string whose left/right side matches with r 1 /r 2 Notation: SubStr2(v i,r,k) ´ SubsStr(v i,Pos( ²,r,k),Pos(r, ²,k)) –Denotes k th occurrence of regular expression r in v i 9 Language for Constructing Output Strings

10 Example Switch((b 1, e 1 ), (b 2, e 2 )), where b 1 ´ Match(v 1,NumTok,3), b 2 ´ : Match(v 1,NumTok,3), e 1 ´ Concatenate(SubStr2(v 1,NumTok,1), ConstStr(“-”), SubStr2(v 1,NumTok,2), ConstStr(“-”), SubStr2(v 1,NumTok,3)) e 2 ´ Concatenate(ConstStr(“425-”),SubStr2(v 1,NumTok,1), ConstStr(“-”),SubStr2(v 1,NumTok,2)) Format phone numbers Input v 1 Output (425)

Reduction requires computing all solutions for each of the sub-problems: –This also allows to rank various solutions and select the highest ranked solution at the top-level. –A challenge here is to efficiently represent, compute, and manipulate huge number of such solutions. I will show three applications of this idea in the talk. –Read the paper for more tricks! 11 Key Synthesis Idea: Divide and Conquer Reduce the problem of synthesizing expressions into sub-problems of synthesizing sub-expressions.

12 Synthesizing Guarded Expression Goal: Given input-output pairs: (i 1,o 1 ), (i 2,o 2 ), (i 3,o 3 ), (i 4,o 4 ), find P such that P(i 1 )=o 1, P(i 2 )=o 2, P(i 3 )=o 3, P(i 4 )=o 4. Algorithm: 1. Learn set S 1 of string expressions s.t. 8 e in S 1, [[e]] i 1 = o 1. Similarly compute S 2, S 3, S 4. Let S = S 1 Å S 2 Å S 3 Å S 4. 2(a) If S ≠ ; then result is Switch((true,S)). Application #1: We reduce the problem of learning guarded expression P to the problem of learning string expressions for each input-output pair.

13 Example: Various choices for a String Expression Input Output Constant

Number of all possible string expressions (that can construct a given output string o 1 from a given input string i 1 ) is exponential in size of output string. –# of substrings is just quadratic in size of output string! –We use a DAG based data-structure, and it supports efficient intersection operation! 14 Synthesizing String Expressions Application #2: To represent/learn all string expressions, it suffices to represent/learn all base expressions for each substring of the output.

Various ways to extract “706” from “ ”: Chars after 1 st hyphen and before 2 nd hyphen. Substr(v 1, Pos(HyphenTok, ²,1), Pos( ²,HyphenTok,2)) Chars from 2 nd number and up to 2 nd number. Substr(v 1, Pos( ²,NumTok,2), Pos(NumTok, ²,2)) Chars from 2 nd number and before 2 nd hyphen. Substr(v 1, Pos( ²,NumTok,2), Pos( ²,HyphenTok,2)) Chars from 1 st hyphen and up to 2 nd number. Substr(v 1, Pos(HyphenTok, ²,1), Pos( ²,HyphenTok,2))  15 Example: Various choices for a SubStr Expression

The number of SubStr(v,p 1,p 2 ) expressions that can extract a given substring w from a given string v can be large! –This allows for representing and computing O(n 1 *n 2 ) choices for SubStr using size/time O(n 1 +n 2 ). 16 Synthesizing SubStr Expressions Application #3: To represent/learn all SubStr expressions, we can independently represent/learn all choices for each of the two index expressions.

17 Back to Synthesizing Guarded Expression Goal: Given input-output pairs: (i 1,o 1 ), (i 2,o 2 ), (i 3,o 3 ), (i 4,o 4 ), find P such that P(i 1 )=o 1, P(i 2 )=o 2, P(i 3 )=o 3, P(i 4 )=o 4. Algorithm: 1.Learn set S 1 of string expressions s.t. 8 e in S 1, [[e]] i 1 = o 1. Similarly compute S 2, S 3, S 4. Let S = S 1 Å S 2 Å S 3 Å S 4. 2(a). If S ≠ ; then result is Switch((true,S)). 2(b). Else find a smallest partition, say {S 1,S 2 }, {S 3,S 4 }, s.t. S 1 Å S 2 ≠ ; and S 3 Å S 4 ≠ ;. 3. Learn boolean formulas b 1, b 2 s.t. b 1 maps i 1, i 2 to true and i 3, i 4 to false. b 2 maps i 3, i 4 to true and i 1, i 2 to false. 4. Result is: Switch((b 1,S 1 Å S 2 ), (b 2,S 3 Å S 4 ))

Prefer shorter programs. –Fewer number of conditionals. –Shorter string expression, regular expressions. Prefer programs with less number of constants. 18 Ranking Strategy

Examples –Bitvector Algorithms (ICSE ‘10) –Spreadsheet Macros (CACM ‘12) Syntactic String Transformations (POPL ‘11)  Semantic String Transformations (VLDB ‘12)  Number Transformations (CAV ‘12) Table Transformations (PLDI ‘11) Sketch –Drawings (CHI 2012) Keywords –SmartPhone Apps 19 Intent Specification VLDB 2012/CAV 2012: Rishabh Singh, Gulwani

Examples –Bitvector Algorithms (ICSE ‘10) –Spreadsheet Macros (CACM ‘12) Syntactic String Transformations (POPL ‘11) Semantic String Transformations (VLDB ‘12) Number Transformations (CAV ‘12)  Table Transformations (PLDI ‘11) Sketch –Drawings (CHI 2012) Keywords –SmartPhone Apps 20 Intent Specification PLDI 2011: Bill Harris, Gulwani

Examples –Bitvector Algorithms (ICSE ‘10) –Spreadsheet Macros (CACM ‘12) Syntactic String Transformations (POPL ‘11) Semantic String Transformations (VLDB ‘12) Number Transformations (CAV ‘12) Table Transformations (PLDI ‘11) Sketch  Drawings (CHI 2012) Keywords –SmartPhone Apps 21 Intent Specification CHI 2012: Salman Cheema, Gulwani, LaViola

22 Architecture Sketch Recognition Engine [HCI] Model Synthesis/Beautification Engine [Theorem Proving] Pattern Synthesis Engine [Program Synthesis] (Partial) Sketch/Ink Strokes Circle/Line Objects Constraints between Objects (Partial) Drawing Suggestions for Drawing Completion Constraint Inference Engine [Machine Learning]

Examples –Bitvector Algorithms (ICSE ‘10) –Spreadsheet Macros (CACM ‘12) Syntactic String Transformations (POPL ‘11) Semantic String Transformations (VLDB ‘12) Number Transformations (CAV ‘12) Table Transformations (PLDI ‘11) Sketch –Drawings (CHI 2012) Keywords  SmartPhone Apps 23 Intent Specification Joint work with: Vu Le, Zhendong Su (UC-Davis)

Students and Teachers End-Users Algorithm Designers Software Developers Most Transformational Target Potential Users of Synthesis Technology 24 Most Useful Target Vision for End-users: Enable people to have (automated) personal assistants. Vision for Education: Enable every student to have access to free & high-quality education.

Concept Language –Programs Straight-line programs –Automata –Queries –Sequences User Intent –Logic, Natural Language –Examples, Demonstrations/Traces Search Technique –SAT/SMT solvers (Formal Methods) –A*-style goal-directed search (AI) –Version space algebras (Machine Learning) 25 Dimensions in Synthesis PPDP 2010: “Dimensions in Program Synthesis”, Gulwani. (Application) (Ambiguity) (Algorithm)