Rohit Singh, Armando Solar-Lezama Automatic Generation of Formula Simplifiers based on Conditional Rewrite Rules Research Qualifying Examination 9-Oct-15Automatic Generation of Formula Simplifiers1 Rohit Singh Adviser: Armando Solar-Lezama Massachusetts Institute of Technology, Cambridge, USA October 9, 2015

Rohit Singh, Armando Solar-Lezama SMT Solvers are great! Automatic Generation of Formula Simplifiers29-Oct-15 UCLID Boolector Spec# Can we do better?

Rohit Singh, Armando Solar-Lezama Domain specificity Automatic Generation of Formula Simplifiers39-Oct-15 UCLID Boolector Spec# solver generator, far from there

Rohit Singh, Armando Solar-Lezama Because… Automatic Generation of Formula Simplifiers49-Oct-15 SMT Solvers leverage the inherent structure in problems of practical interest To a limited extent Can find bugs Can’t crack RSA Only so much structure to use! Can we do better?

Rohit Singh, Armando Solar-Lezama Simplifier Automatic Generation of Formula Simplifiers59-Oct-15 Uses local term-rewriting to make formulas easier to solve Every solver has one of these

Rohit Singh, Armando Solar-Lezama Simplifier in Boolector [TACAS 09] Automatic Generation of Formula Simplifiers69-Oct-15 SMT-LIBParser Simplifier Formula Refinement SAT Solver Under- approximation Array Consistency Checker Model Generator Other Params SAT/UNSAT Model

Rohit Singh, Armando Solar-Lezama A part of a solver that can benefit Automatic Generation of Formula Simplifiers79-Oct-15 SMT-LIBParser Simplifier Formula Refinement SAT Solver Under- approximation Array Consistency Checker Model Generator Other Params Fairly common to have this phase: z3, Sketch Solver, Boolector etc

Rohit Singh, Armando Solar-Lezama In the context of Sketch Automatic Generation of Formula Simplifiers89-Oct-15 SketchParser Simplifier CEGIS Loop SAT Solver Optimizations Other Params

Rohit Singh, Armando Solar-Lezama Sketch Simplifier Automatic Generation of Formula Simplifiers99-Oct-15 Messy low-level C++ code Employs simple declarative Rewrite rules Huge impact on performance

Rohit Singh, Armando Solar-Lezama Simplifying constraints before sending them to the solver Adding an optimization to Sketch tool: For certain problems: +2x performance For others: way slower than before Domain specificity Automatic Generation of Formula Simplifiers109-Oct-15 Solver Simplifier-1 Formula Simplifier-2

Rohit Singh, Armando Solar-Lezama In the context of Sketch Automatic Generation of Formula Simplifiers119-Oct-15 Rewriting is at the core SketchParser Rewriter CEGIS Loop SAT Solver Optimizations Other Params Internal Representation(IR)

Rohit Singh, Armando Solar-Lezama Internal Representation Automatic Generation of Formula Simplifiers129-Oct-15 Internal language for constraints Directed Acyclic Graphs << OR a b d or(lt(a, b), lt(a, d))

Rohit Singh, Armando Solar-Lezama Internal Representation Automatic Generation of Formula Simplifiers139-Oct-15

Rohit Singh, Armando Solar-Lezama Conditional Rewrite Rules Automatic Generation of Formula Simplifiers149-Oct-15 Simple and declarative << OR a b d < da b<d abd

Rohit Singh, Armando Solar-Lezama Sketch Simplifier Automatic Generation of Formula Simplifiers159-Oct-15 Full-fledged code for implementing Rewrite Rules if(nfather->type == LT && nmother->type == LT){ // (a+e a+e<x when b<a if(nfather->mother->type == PLUS && nmother->mother- >type == PLUS){ bool_node* nfm = nfather->mother; bool_node* nmm = nmother->mother; bool_node* nmmConst = nmm->mother; bool_node* nmmExp = nmm->father; if(isConst(nmmExp)){ bool_node* tmp = nmmExp; nmmExp = nmmConst; nmmConst = tmp; } bool_node* nfmConst = nfm->mother; bool_node* nfmExp = nfm->father; if(isConst(nfmExp)){ bool_node* tmp = nfmExp; nfmExp = nfmConst; nfmConst = tmp; } if(isConst(nfmConst) && isConst(nmmConst) && nfmExp== nmmExp){ if(val(nfmConst) < val(nmmConst)){ return nmother; }else{ return nfather; } } } }

Rohit Singh, Armando Solar-Lezama Sketch Simplifier Automatic Generation of Formula Simplifiers169-Oct-15 Messy low-level C++ code Employs simple declarative Rewrite rules Huge impact on performance

Rohit Singh, Armando Solar-Lezama Problem Statement Automatic Generation of Formula Simplifiers179-Oct-15 Given a corpus of benchmark problems (formulas) from a domain: Learn recurrent sub-terms (patterns) Learn impactful conditional Rewrite Rules Generate a simplifier based on these rules

Rohit Singh, Armando Solar-Lezama Related Work Automatic Generation of Formula Simplifiers189-Oct-15

Rohit Singh, Armando Solar-Lezama Related Work Automatic Generation of Formula Simplifiers199-Oct-15 Motif discovery [ICMLA 07, IPDPS 04, ICIC 09]: complete algorithms are too slow, no labels on nodes Term/Graph Rewriting: Stratego/XT [ASF+SDF 97], GrGen: pattern matching and code generation is similar, absence of guards on rules LALR parser generation [Journal of Computer Languages 89]

Rohit Singh, Armando Solar-Lezama Problem Statement Automatic Generation of Formula Simplifiers209-Oct-15 Given a corpus of benchmark problems (formulas) from a domain: Learn recurrent sub-terms (patterns) Learn impactful conditional Rewrite Rules Generate a simplifier based on these rules Solution: S WAPPER framework

Rohit Singh, Armando Solar-Lezama Contribution Automatic Generation of Formula Simplifiers219-Oct-15 Automating generation of conditional Rewrite Rules Generating optimal simplifier using Auto- tuning Showing performance improvement on Sketch benchmarks Showing generality of the framework with SMT-LIB based rewriting

Rohit Singh, Armando Solar-Lezama S WAPPER framework Automatic Generation of Formula Simplifiers229-Oct-15 Rule Generation (Synthesis) Simplifier Generation (Compilation) Pattern Finding (Clustering) Auto-tuning (Machine Learning) Patterns Simplifier Subset of Rules Rules Optimal Simplifier Corpus of Benchmarks

Rohit Singh, Armando Solar-Lezama S WAPPER framework Automatic Generation of Formula Simplifiers239-Oct-15 Rule Generation (Synthesis) Simplifier Generation (Compilation) Pattern Finding (Clustering) Auto-tuning (Machine Learning) Patterns Simplifier Subset of Rules Rules Optimal Simplifier Corpus of Benchmarks

Rohit Singh, Armando Solar-Lezama Pattern Finding Automatic Generation of Formula Simplifiers249-Oct-15 Given a corpus of benchmark problems as DAGs, find common repeating patterns Different from motif discovery: Structure and semantics of DAGs: Labels on nodes (operation types), Static Analysis information Strict probabilistic significance not needed, approximations will work Tried multiple methods: Clustering based approach using parse feature vectors approximating similarity of two patterns Random sampling: Fast and approximate

Rohit Singh, Armando Solar-Lezama Pattern Finding: Random sampling Automatic Generation of Formula Simplifiers259-Oct-15 ANDSRC OR NOT Pick a node at random Randomly choose to pick a non-terminal parent or not Repeat until reached cutoff size Restart if can’t proceed Aggregate patterns with counts

Rohit Singh, Armando Solar-Lezama Pattern Finding: Random sampling Automatic Generation of Formula Simplifiers269-Oct-15 AND OR Similarity criterion for aggregation: DAG signature: incorporates symmetries Static Analysis information from benchmark DAGs AND OR [False] AND OR [False,True] [False]

Rohit Singh, Armando Solar-Lezama Pattern Finding: Random sampling Automatic Generation of Formula Simplifiers279-Oct-15 Fast and approximate Gives a sense of where to look for Rewrite Rules first Ergodic in nature: can cover all patterns eventually

Rohit Singh, Armando Solar-Lezama S WAPPER framework Automatic Generation of Formula Simplifiers289-Oct-15 Rule Generation (Synthesis) Simplifier Generation (Compilation) Pattern Finding (Clustering) Auto-tuning (Machine Learning) Patterns Simplifier Subset of Rules Rules Optimal Simplifier Corpus of Benchmarks

Rohit Singh, Armando Solar-Lezama Rule Generation: SyGuS problem Automatic Generation of Formula Simplifiers299-Oct-15

Rohit Singh, Armando Solar-Lezama Rule Generation: SyGuS problem Automatic Generation of Formula Simplifiers309-Oct-15

Rohit Singh, Armando Solar-Lezama Rule Generation: SyGuS problem Automatic Generation of Formula Simplifiers319-Oct-15

Rohit Singh, Armando Solar-Lezama Rule Generation: Techniques Automatic Generation of Formula Simplifiers329-Oct-15

Rohit Singh, Armando Solar-Lezama Rule Generation: Symbolic approach Automatic Generation of Formula Simplifiers339-Oct-15

Rohit Singh, Armando Solar-Lezama Rule Generation: Predicate Refinement Automatic Generation of Formula Simplifiers349-Oct-15

Rohit Singh, Armando Solar-Lezama Rule Generation: Enumerative approach Automatic Generation of Formula Simplifiers359-Oct-15

Rohit Singh, Armando Solar-Lezama Rule Generation: Example Automatic Generation of Formula Simplifiers369-Oct-15

Rohit Singh, Armando Solar-Lezama S WAPPER framework Automatic Generation of Formula Simplifiers379-Oct-15 Rule Generation (Synthesis) Simplifier Generation (Compilation) Pattern Finding (Clustering) Auto-tuning (Machine Learning) Patterns Simplifier Subset of Rules Rules Optimal Simplifier Corpus of Benchmarks

Rohit Singh, Armando Solar-Lezama Simplifier Generation Automatic Generation of Formula Simplifiers389-Oct-15 Given a set of conditional Rewrite Rules, we generate efficient C++ code for the simplifier Performs rule generalization to find the crux of each rule and avoid overheads Incorporates symmetries of the rules automatically Shares burden of pattern matching across rules

Rohit Singh, Armando Solar-Lezama Simplifier Generation: Rule Generalization Automatic Generation of Formula Simplifiers399-Oct-15

Rohit Singh, Armando Solar-Lezama Simplifier Generation: Pattern Matching Automatic Generation of Formula Simplifiers409-Oct-15

Rohit Singh, Armando Solar-Lezama S WAPPER framework Automatic Generation of Formula Simplifiers419-Oct-15 Rule Generation (Synthesis) Simplifier Generation (Compilation) Pattern Finding (Clustering) Auto-tuning (Machine Learning) Patterns Simplifier Subset of Rules Rules Optimal Simplifier Corpus of Benchmarks

Rohit Singh, Armando Solar-Lezama Auto tuning Automatic Generation of Formula Simplifiers429-Oct-15 Identifies the best subset of rules Problem Setup: Search space parameters: Permutation of rules Number of rules to be used Space reduction: consider permutation as different for only those rules which have conflicting patterns Optimization function Runs the solver and compares original performance to the new performance after applying the simplifier Ansel et al, PACT

Rohit Singh, Armando Solar-Lezama Auto tuning Automatic Generation of Formula Simplifiers439-Oct-15 Optimization function Runs the solver multiple times and compares original performance to the new performance after applying the simplifier Tradeoffs Capturing randomness inside the solver may lead to long runtimes More rules can increase the search space and search time by a lot

Rohit Singh, Armando Solar-Lezama Experiments: Questions Automatic Generation of Formula Simplifiers449-Oct-15 Can S WAPPER generate good simplifiers in reasonable time and low cost? How do S WAPPER generated simplifiers perform relative to hand written simplifier in Sketch? How domain specific are the simplifiers generated by S WAPPER ? How general is S WAPPER framework?

Rohit Singh, Armando Solar-Lezama Experiments: Setup & Terminology Automatic Generation of Formula Simplifiers459-Oct-15 We compare the following simplifiers: Hand-coded: default in Sketch. Comprises of: a)Rules that can be expressed in our framework b)Constant propagation c)Structural hashing d)Other complex rules Baseline: disables rules in a) from Hand-coded Auto-generated: incorporates generated rules over the Baseline Each simplifier is applied before the problem is handed to the solver Openstack: Two VMs (24 cores, 32GB RAM, 40 threads)

Rohit Singh, Armando Solar-Lezama Experiments: Domains & Benchmarks Automatic Generation of Formula Simplifiers469-Oct-15 Storyboard and QBS used for validation in initial stages Full evaluation was done on AutoGrader and Sygus Generated rewriter for SMTLIB benchmarks

Rohit Singh, Armando Solar-Lezama Experiments: Realistic Costs Automatic Generation of Formula Simplifiers479-Oct-15 Time and Cost Estimation (on AWS, parallelism of 40 threads) Costs less than an hour’s work of a good developer Can reduce time by increasing parallelism or smarter evaluations with timeouts

Rohit Singh, Armando Solar-Lezama Experiments: S WAPPER performance Automatic Generation of Formula Simplifiers489-Oct-15

Rohit Singh, Armando Solar-Lezama Experiments: S WAPPER performance Automatic Generation of Formula Simplifiers499-Oct-15 Impact on sizes AutoGrader: 33.2%, 6.8% reductions Sygus: 1.6%, 1.6% reductions

Rohit Singh, Armando Solar-Lezama Experiments: S WAPPER performance Automatic Generation of Formula Simplifiers509-Oct-15 Impact on times AutoGrader: 27.5s,20s,18s average times Sygus: 22s,21s,10s average times

Rohit Singh, Armando Solar-Lezama Experiments: Domain Specificity Automatic Generation of Formula Simplifiers519-Oct-15 Impact on times across domains

Rohit Singh, Armando Solar-Lezama Experiments: SMTLIB Translation Automatic Generation of Formula Simplifiers529-Oct-15 Size reduction by 19% on average, not much impact on time

Rohit Singh, Armando Solar-Lezama Conclusion Automatic Generation of Formula Simplifiers539-Oct-15 We demonstrated that S WAPPER can generate good simplifiers in reasonable time and low cost. We showed that S WAPPER generated simplifiers perform better than hand written simplifier in Sketch. We showed the domain specificity of simplifiers generated by S WAPPER We also showed generality of S WAPPER framework by extending it for SMTLIB benchmarks

Rohit Singh, Armando Solar-Lezama Automatic Generation of Formula Simplifiers549-Oct-15 Questions?

Rohit Singh, Armando Solar-Lezama Automatic Generation of Formula Simplifiers559-Oct-15 Thank You!