1. _Synthesis__________________ __Of_______________________ ___First-Order_____Dynamic___ _____________Programming___ _______________Algorithms___
Yewen (Evan) Pu
Rastislav Bodik
Saurabh Srivastava
University of California, Berkeley

2. Do you feel lucky?
Bob work, bob hand code, bob lucks out and gets R.E. which ken thompson did for him.
Other domain possible too, but compare to total amount of algorithms, they are island on a vast ocean
Let us build more islands! <natural transition> /*
In the ocean of algorithms / Most are solved by hand / but once in a while / we reach solid land / A group of algorithms we understand well / And a thing that compile / From high level specification / To low level detail / */

3. A new way to build an island?
Conventional Domain Specific Compiler:
Require deep domain theories
Takes a long time to implement
Constraint Based Synthesizer:
Write a template for desired algorithm
Constraint solver fills in the template
Use visual notation from the previous slide. Draw same island as before.
This compare domain-theory with sketch (a constraint-based synthesizer)
You write a template of the solution which is filled in by the c.b. synthesizer so that it functionally behaves like the spec. The template is a just a syntactic descripiton. Make templates, not theories.
Flow to next slide: to illustrate that point, here is an example where we build a template for a single problem, using sketch

4. Make Templates not Theories
Suppose we want to optimize x+x+x+x
With domain-specific rewrite rules:
With a template in SKETCH [Solar-Lezama]:
spec(x): return x+x+x+x
sketch(x): return x << ??
You need a lot of theory to do transform
But for sketch, just completes a template
Flow: but how does sketch finds the correct completion?
program equivalence found using bounded model checking

5. A Search for a Correct Program
Synthesizer finds in a space of candidate programs a correct one (it matches the specification)
Sketch uses constraint solving to implement a search.
Space of all possible algorithms, find one.
Admit that we use bounded model checking (check equivalence between spec and sketch algorithm over finite number of inputs)
This is justified because equivalence of two programs is undecidable in general.
And given enough input, the user can be sure the final algorithm

6. Research Question in This Talk
Can we write a synthesizer for an entire problem domain using a single template (sketch)?
Self explaintary… transition probably not needed even.

7. The Challenge
How do we write a template that covers all of our domain, yet the constraints it induces can be efficiently solved?
Transition to next slide via next slide’s title

8. Our Approach Define a general template that contains the entire domain
Optimize the search by reducing the search space
Make sure we say the general template CONTAINS the entire domain, it is not nessesarily the entire domain but possibly larger, but it is easier to define than the domain space.

9. Dynamic Programming Algorithms
A well-defined domain
We have no “DSL compiler” for it (taught as an art)
Difficulties:
inventing sub-problems
inventing recurrences
We focus on a first-order sub-class, FORDP, which captures many O(n) DP algorithms
Transition in: Let’s try our approach on this nice class of algorithms.

10. An Easy Problem Fibonacci Sequence: fib(n) = fib(n-1) + fib(n-2)
Here we get lucky as the recurrence is given to you, and they do indeed overlap.
In general through, we are not so lucky

11. A Harder Problem Maximal Independent Sum (MIS)
Input: Array of positive integers
Output: Maximum sum of a non-consecutive selections of its elements.
For this problem, you do not immediately know what is the overlapping sub-problem nor the recurrence.

12. What does the user do? mis(A): best = 0 forall selections:
if non_consec(selection):
best = max(best, value(A[selection]))
return best
Self explaintary… transition probably not needed even.

13. What does the template do?
Define a general template that contains the entire domain
Self explaintary… transition probably not needed even.

14. A General Template Covers every FORDP algorithm for_dpa(a):
p1 = array()
p2 = array()
p1[0] = init1()
p2[0] = init2()
for i from 1 to n:
p1[i] = update1(p1[i-1],p2[i-1],a[i])
p2[i] = update2(p1[i-1],p2[i-1],a[i])
return term(p1[n],p2[n])
Explain underlined functions are general templates of what they can be.
Init will cover all possible ways of initializing the base case for sub-problems
Update covers all the way to update the sub-problem values
Term wraps things up to yield a single output
Transition out: The update template is the crux as it defines the recurrence relation for the DP
Covers every FORDP algorithm

15. General Template for update
All possible compositions of user provided operators
To approach the update function, same idea, first define a general space
Transition out: This space is too large to be efficiently solved. We’ll now show how to prune it

16. Space Reduction: Optimality
All FORDP recurrences have this syntactic form
Take sub-problem at i-1 iteration
Extend them to candidate solutions at the ith iteration with the array input, here we extend each sub-problem in 2 possible ways. Mention what ext is, and it’s not too bad…
Only a subset of all the candidates can be used by the sub-problems, Since we don’t really know what these candidates are, we let synthesizer discover a wiring that correct select for each subproblem the correct candiddates
Remembering sub-problems are produced from optimal previous sub-problems, we optimize over these candidates

17. Space Reduction: Optimality
Recurrence for MIS:
For our example, MIS, the template will resolve the recurrence as follows:
There are two sub-problems, pick, and omit.
We can extend both as follows
If we were to pick at i, we see the candidate is only…
If we were to omit at i, we see the candidates are…

18. Space Reduction: Symmetry
Many operators are commutative
Pick a canonical representative syntactically

19. Space Reduction: Recap
All possible compositions
Syntactically Reduced
Which is about 2%

20. DEMO
We need to explain why a naïve sketch generates too large of a space for MIS

21. Benchmarks Here are some synthesized recurrences
We need to explain why a naïve sketch generates too large of a space for MIS

22. Experiments Synthesizer solving time, in seconds out of memory
Seconds in y axis
The shorter the better

23. Conclusion
It is possible to build a domain-specific synthesizer for FORDPA
Synthesizer developer only find a syntactic domain structure
The lessons learned in building the synthesizer may be general
If so, we can build more islands with constraint-based synthesis
We need to explain why a naïve sketch generates too large of a space for MIS