1. Program Synthesis for Automating Education Sumit Gulwani (sumitg@microsoft.com) Microsoft Research, Redmond

2. Program Synthesis = Synthesis of executable code from user intent expressed using some constraints. Enabling Technology is now available –Better search/logical reasoning techniques SAT/SMT solvers –Faster machines (Good application for multi-cores) A natural goal to enable people to easily leverage computational power. 1 Program Synthesis: Why Today?

3. Our techniques can synthesize a wide variety of algorithms/programs from logic and examples. Undergraduate book algorithms (e.g., sorting, dynamic prog) –[Srivastava/Gulwani/Foster, POPL 2010] Program Inverses (e.g, deserializers from serializers) –[Srivastava/Gulwani/Chaudhuri/Foster, MSR-TR-2010-34] Graph Algorithms (e.g., bi-partiteness check) –[Itzhaky/Gulwani/Immerman/Sagiv, OOPSLA 2010] String Manipulating Programs (e.g. ”Viktor Kuncak”-> “Kuncak, V.”) –[Gulwani, POPL 2011] Bit-vector algorithms (e.g., turn-off rightmost one bit) –[Jha/Gulwani/Seshia/Tiwari, ICSE 2010] 2 Program Synthesis: Recent Success

4. Straight-line programs that use –Arithmetic Operators: +,-,*,/ –Logical Operators: Bitwise and/or/not, Shift left/right Challenge: Combination of arithmetic + logical operators leads to unintuitive algorithms Application: Provides most-efficient way to accomplish a given task on a given architecture 3 Bitvector Algorithms

5. 1 0 1 0 1 1 0 0 Turn-off rightmost 1-bit 4 Examples of Bitvector Algorithms 1 0 1 0 1 1 0 0 1 0 1 0 1 0 0 0 Z Z & (Z-1) 1 0 1 0 1 0 1 1 Z Z-1 1 0 1 0 1 0 0 0 & Z & (Z-1)

6. 5 Examples of Bitvector Algorithms P25: Higher order half of product of x and y o1 := and(x,0xFFFF); o2 := shr(x,16); o3 := and(y,0xFFFF); o4 := shr(y,16); o5 := mul(o1,o3); o6 := mul(o2,o3); o7 := mul(o1,o4); o8 := mul(o2,o4); o9 := shr(o5,16); o10 := add(o6,o9); o11 := and(o10,0xFFFF); o12 := shr(o10,16); o13 := add(o7,o11); o14 := shr(o13,16); o15 := add(o14,o12); res := add(o15,o8); P24: Round up to next highest power of 2 o1 := sub(x,1); o2 := shr(o1,1); o3 := or(o1,o2); o4 := shr(o3,2); o5 := or(o3,o4); o6 := shr(o5,4); o7 := or(o5,o6); o8 := shr(o7,8); o9 := or(o7,o8); o10 := shr(o9,16); o11 := or(o9,o10); res := add(o10,1); [ICSE 2010] Joint work with Susmit Jha, Sanjit Seshia (UC-Berkeley), Ashish Tiwari (SRI) and Venkie (MSR Redmond)

7. Students and Teachers End-Users Algorithm Designers Software Developers Most Transformational Target Pyramid of Technology Users 6 Consumers of Program Synthesis Technology

8. Automating Education Make education interactive and fun Automated problem solving (for students) –Provide hints –Point out mistakes and suggest fixes Creation of teaching material (for teachers) –Authoring tools –Problem construction Group interaction (for teachers/students) –Ask questions –Share annotations Domains: Geometry, Algebra, Probability, Mechanics, Electrical Circuits, etc. 7

9. What is the role of PL + Logic + Synthesis? Programming Language for Geometry –Objects: Point, Line, Circle –Constructors Ruler(Point, Point) -> Line Compass(Point, Point) -> Circle Intersect(Circle, Circle) -> Pair of Points Intersect(Line, Circle) -> Pair of Points Intersect(Line, Line) -> Point Logic for Geometry –Inequality predicates over arithmetic expressions Distance(Point, Point), Angle(Line, Line), … Automated Problem Solving –Given pre/postcondition, synthesize a straight-line program 8 Geometry Constructions Domain

10. Automated Problem Solving Given pre/postcondition, synthesize a straight-line program Example: Draw a line L’ perpendicular to a given line L. Precondition: true Postcondition: Angle(L’,L) = 90 Program Step 1: P1, P2 = ChoosePoint(L); Step 2: C1 = Circle(P1,P2); Step 3: C2 = Circle(P2,P1); Step 4: = Intersect(C1,C2); Step 5: L’ = Line(P3,P4); 9 Geometry Domain: Automated Problem Solving

11. 10 Constructing line L’ perpendicular to given line L P1 P2 P3 P4 C1 C2 L L’ Step 1: P1, P2 = ChoosePoint(L); Step 2: C1 = Circle(P1,P2); Step 3: C2 = Circle(P2,P1); Step 4: = Intersect(C1,C2); Step 5: L’ = Line(P3,P4);

12. Bisect a given line. Bisect an angle. Copy an angle. Draw a line parallel to a given line. Draw an equilateral triangle given two points. Draw a regular hexagon given a side. Given 4 points, draw a square with each of the sides passing through a different point. Other Applications: New approximate geometric constructions 2D/3D planning problems 11 Examples of Geometry Constructions

13. Synthesis, in general, is harder than verification. –Synthesis Problem: Given pre/postcondition, synthesize a straight-line program –Verification Problem: Given pre/postcondition, and a straight- line program, determine whether the Hoare triple holds. Decision procedures for verification of geometry constructions are known, but are complex. –Because of symbolic reasoning. 12 Synthesis Algorithm for Geometry Constructions Precondition: True Postcondition: Angle(L,L’) = 90 Step 1: P1, P2 = ChoosePoint(L); Step 2: C1 = Circle(P1,P2); Step 3: C2 = Circle(P2,P1); Step 4: = Intersect(C1,C2); Step 5: L’ = Line(P3,P4);

14. Symbolic reasoning based decision procedures are complex. How about property testing? Theorem: A construction that works (i.e., satisfies the postcondition) for a randomly chosen model of precondition also works for all models (w.h.p.). Proof: Objects constructed using ruler/compass can be described using polynomial ops (+,-,*), square-root & division operator. The randomized polynomial identity testing algorithm lifts to square-root and division operators as well ! 13 A simpler strategy for verification of Constructions

15. Problem: Given two polynomials P1 and P2, determine whether they are equivalent. The naïve deterministic algorithm of expanding polynomials to compare them term-wise is exponential. A simple randomized test is probabilistically sufficient: –Choose random values r for polynomial variables x –If P1(r) ≠ P2(r), then P1 is not equivalent to P2. –Otherwise P1 is equivalent to P2 with high probability, 14 Randomized Polynomial Identity Testing

16. Problem: Symbolic reasoning is hard. Idea #1: Leverage Property Testing to reduce symbolic reasoning to concrete reasoning. Construct a random input-output example (I,O) for the problem and find a construction that can generate O from I. Example: Construct incenter of a triangle. –If I chose my input triangle to be an equilateral one, then the circumcenter construction also appears to work! Since incenter = circumcenter for an equilateral traingle. –But what are the chances of choosing an random triangle to be an equilateral one? 15 Synthesis Algorithm for Geometry Constructions

17. Exhuastive Search Strategy: Given input objects I and desired objects O, keep constructing new objects from I using ruler and compass until objects O are obtained. Problem: Search blows up, i.e., too many (useless) objects get constructed. –Example: n points lead to O(n^2) lines, which leads to O(n^4) points, and so on… 16 Synthesis Algorithm for Geometry Constructions

18. Problem: Search space is huge. Idea #2: Perform goal-directed reasoning. –Example: If an operation leads to construction of a line L that passes through a desired output point, it is worthwhile constructing line L. –Mimics human intelligence. –For this to be effective, we need solutions with small depth. Idea #3: Work with a richer library of primitives. –Common constructions picked up from chapters of text-books. –A search space of (small width, large depth) is converted into one of (large width, small depth). –Mimics human experience/knowledge. 17 Synthesis Algorithm for Geometry Constructions

19. 18 Search space Exploration: With/without goal-directness

20. 19 Problem Solving Engine with Natural Interfaces Natural Language Processing Paraphrasing Synthesis Engine Problem Description in English Problem Description as Logical Relation Solution as Functional Program Solution in English Joint work with: Kalika Bali, Monojit Chaudhuri (MSR Bangalore) Vijay Korthikanti (UIUC), Ashish Tiwari (SRI)

21. Useful modules powered by problem solving engine The next step is to architect several useful modules on top of the problem-solving architecture such as: Interactive feedback to students –Provide hints –Point out mistakes and suggest fixes Creation of teaching material (for teachers) –Problem construction –Authoring tools 20

22. What domains should we prioritize for automation? Mathematics –Algebra –Probability Physics –Mechanics –Electrical Circuits –Optics Chemistry –Quantitative Chemistry –Organic Chemistry 21 Other Domains

23. Consider the problem of computing effective resistance between two nodes in a graph of resistances. MATLAB implements Kirchoff’s law based decision procedure –Algebraic sum of the currents at any circuit junction = 0 –Sum of changes in potential in any complete loop = 0 22 Electrical Circuits: Concept-specific solutions Joint work with: Swarat Chaudhuri (Penn State University)

24. Consider the problem of computing effective resistance between two nodes in a graph of resistances. Kirchoff’s law based decision procedure is not useful for students who are expected to know only simpler concepts. Solutions need be parameterized by specific concepts such as: –Series/Parallel composition of resistances –Symmetry Reduction –Wheatstone Bridge 23 Electrical Circuits: Concept-specific solutions Joint work with: Swarat Chaudhuri (Penn State University)

25. 24 Resistance Reduction Concepts If R 3 /R 1 = R 4 /R 2, then V D = V B Parallel Combination Series Combination Wheat-stone Bridge

26. Students and Teachers End-Users Algorithm Designers Software Developers Most Transformational Target Pyramid of Technology Users 25 Consumers of Program Synthesis Technology

27. Automating Education: Long-term Goals Ultra-intelligent computer Model of human mind Inter-stellar travel 26

28. Tutorial at FMCAD 2010 (Oct 20, Wednesday, 9 am) –“Dimensions in Program Synthesis” –(More) applications and methodologies/techniques for synthesis –A cool demo Technical report –“Synthesizing Geometry Constructions” by Sumit Gulwani, Vijay Korthikanti, Ashish Tiwari –Send email to sumitg@microsoft for copy of the draft. 27 References