1. Exploiting Structure in Symmetry Detection for CNF Paul T. Darga, Mark H. Liffiton, Karem A. Sakallah, and Igor L. Markov The University of Michigan

2. Structure in SAT Human-designed artifacts possess considerable structure Human-designed artifacts possess considerable structure –Manifested in instances of Boolean satisfiability (SAT) Symmetry Symmetry –Some rearrangement of the components of the design that preserves its structure Sparsity Sparsity –Most design elements are directly related to only a few other elements in the whole design We can exploit this structure to speed up SAT solving! We can exploit this structure to speed up SAT solving!

3.  = (a'+b+c)(a+b'+c')(b'+c)(a') Symmetry Breaking Shatter (Aloul et. al) Shatter (Aloul et. al) –Converts CNF formula to a colored undirected graph (Crawford) –Uses nauty (McKay), a graph symmetry detection tool, to find the symmetries present in the graph –Converts the symmetries back into additional symmetry-breaking predicates (SBPs) –Appends the SBPs to the original formula –The resulting formula is often much faster to solve  = (a'+b+c)(a+b'+c')(b'+c) (a'+b+c) (a+b'+c') (b'+c) (a'+b+c) (a+b'+c') (b'+c) (a,a')(b,c')(b',c)

4. Symmetry Breaking On all but the synthetic Urquhart instances, symmetry detection with nauty dominates the run time of the Shatter flow On all but the synthetic Urquhart instances, symmetry detection with nauty dominates the run time of the Shatter flow Further improvements must come from improved symmetry detection Further improvements must come from improved symmetry detection 74.631.518.0123.502pipe 82.613.842.4111.43XOR 99.426.030.1725.86ChnlRoute 93.93.450.213.24FPGARoute 88.443.845.0838.76GRoute 39.41.931.170.76Urq 84.40.450.070.38Hole-n % Sym TotalSearchSymBenchmark

5. Outline Graph construction Graph construction nauty : "No Automorphisms, Yes?" nauty : "No Automorphisms, Yes?" –Problem description –Partition refinement –The search tree saucy : a new symmetry detection tool saucy : a new symmetry detection tool –Algorithmic improvements exploiting structure –Experimental results: saucy run time insignificant compared to SAT solver Conclusions and future work Conclusions and future work

6. Graph Construction for CNF Two vertices and one edge for each variable Two vertices and one edge for each variable One vertex for each clause One vertex for each clause One edge for each literal in each clause One edge for each literal in each clause Color literals and clauses differently Color literals and clauses differently b' c C3C3C3C3 c' C2C2C2C2 b C1C1C1C1 a a'  = C 1 C 2 C 3 = (a'+b+c)(a+b'+c')(b'+c)

7. G  = G c b' C3C3C3C3 b C1C1C1C1 c' C2C2C2C2 a' a Symmetry Detection Problem What precisely is a symmetry of a graph G? What precisely is a symmetry of a graph G? –A symmetry is a permutation  of the labels assigned to vertices of G such that G  = G –The set of all symmetries is denoted Aut(G) b' c C3C3C3C3 c' C2C2C2C2 b C1C1C1C1 a a'  = (a,a')(b,c')(b',c)(C 1,C 2 ) a b C3C3C3C3 b' C2C2C2C2 a' C1C1C1C1 c' c  = (a,b',c')(a',b,c) G  GG  GG  GG  G

8. Symmetry Detection Problem Problem: there are n! possible labelings! Problem: there are n! possible labelings! –Can we prune the search space?  (1,2,4) (1,2,4,3) (1,2,3,4) (1,2,3) (1,2)(3,4) (1,2) 2431 2413 2341 2314 2143 2134 (2,4) (2,4,3) (2,3,4) (2,3) (3,4)  1432 1423 1342 1324 1243 1234 (1,4)(2,3)4321(1,3,2,4)3421 (1,4,2,3)4312(1,3)(2,4)3412 (1,4)4231(1,3,4)3241 (1,4,3)4213(1,3)3214 (1,4,2)4132(1,3,4,2)3142 (1,4,3,2)4123(1,3,2)3124 3 1 4 2

9. Partition Refinement We can rule out many candidate labelings We can rule out many candidate labelings –Distinguish vertices that cannot possibly be mapped into each other by any symmetry –Fast distinguishing method: degree 35 4 67 1 2 0 Select a color in the graph Select a color in the graph Compute the number of connections every vertex has to that color Compute the number of connections every vertex has to that color Distinguish vertices within colors based on that count Distinguish vertices within colors based on that count Repeat until coloring stabilizes Repeat until coloring stabilizes Refinement distinguished all vertices! This graph has no symmetry besides the identity. Refinement distinguished all vertices! This graph has no symmetry besides the identity. 11 3 3 44 2 2 00 11 0 0 0 0 00 1 1 0 1 01 00 0 0 00 0 1 10

10. Partition Refinement In a stable coloring: In a stable coloring: –vertices in different colors definitely cannot map into each other in some symmetry of the graph –vertices in the same color may map into each other (i.e. refinement is only an approximation) b' c C3C3C3C3 c' C2C2C2C2 b C1C1C1C1 a a' 1 23 76 45

11. Select a non-singleton color T (for target) and generate |T| colorings, each with one element of T artificially distinguished from the remainder of T Select a non-singleton color T (for target) and generate |T| colorings, each with one element of T artificially distinguished from the remainder of T Discrete colorings (leaf nodes) yield likely symmetries Discrete colorings (leaf nodes) yield likely symmetries Out of 6! = 720 possible labelings, partition refinement pruned away all but the six symmetries of the graph Out of 6! = 720 possible labelings, partition refinement pruned away all but the six symmetries of the graph  1 = (2,4)(3,5)  2 = (0,3)(1,2)  3 = (0,3,5)(1,2,4) The Search Tree 2 0 4 1 35 012345 035124 035421 035241 035124 035241 305124 305142 503124 503124 053241 053241  = 035421  =  0 = 305412 305142 350142 350142 503214 503124 530124 530124  4 = (0,5,3)(1,4,2)  5 = (0,5)(1,4)  0 =   1 = (2,4)(3,5)  2 = (0,3)(1,2)  3 = (0,3,5)(1,2,4)  4 = (0,5,3)(1,4,2)  5 = (0,5)(1,4)

12. Pruning Using Generators Too many symmetries: |Aut(G)| is O(n!) Too many symmetries: |Aut(G)| is O(n!) Group theory provides the answer: generators Group theory provides the answer: generators –Irredundant set H  Aut(G) implicitly represents entire set of symmetries –Exponential compression: |H|  log 2 |Aut(G)| We prune away subtrees guaranteed to yield symmetries that we can already generate with previously discovered symmetries We prune away subtrees guaranteed to yield symmetries that we can already generate with previously discovered symmetries

13. saucy : Exploiting Structure nauty works very well on small graphs (and thus small formulas) but fails to scale nauty works very well on small graphs (and thus small formulas) but fails to scale –Takes considerably longer than the SAT solver after adding SBPs to the CNF formula –Runs out of memory on formulas with corresponding graphs having >50,000 vertices saucy improvement #1: sparse representation saucy improvement #1: sparse representation saucy improvement #2: use knowledge of graph construction saucy improvement #2: use knowledge of graph construction –Clause vertices only connected to their literals –Never connected to each other

14. 00 00 0 0 0 0 0 0 0 0 1111 1 1 1 11 1 1 1 00 00 saucy : Algorithmic Improvements Positive Negative Clauses nauty : nauty : –Iterate over all colors –For each vertex, count connections to refining color, and sort saucy improvement #3: saucy improvement #3: –Determine directly connected colors –For each vertex, count connections to refining color, and sort saucy improvement #4: saucy improvement #4: –For each vertex in refining color, count connections –For every color touched, sort the counts

15. saucy : Asymptotic Performance Partition refinement Partition refinement –nauty implementation: O(n 3 ) –saucy improvement #4: O(n 2 log n) Search tree size Search tree size –Worst case: exponential –No “bad leaves”: O(n 3 ) –In practice: O(n) Complete algorithm Complete algorithm –Worst case: exponential –No “bad leaves”: O(n 5 log n) –Much lower in practice

16. saucy : Empirical Performance Inst.nzChaffnauty % Sym saucy s4-4-3-110354218.5388.7428.90.110.05 s4-4-3-29974877.5979.678.30.100.01 s4-4-3-39970884.7875.987.90.090.01 s4-4-3-410714464.46155.3125.10.140.03 s4-4-3-511072134.09101.6343.10.110.08 s4-4-3-6962013.2476.4885.20.100.75 s4-4-3-71036218.2778.9681.20.100.54 s4-4-3-866080.6828.4297.70.068.11 2pipe35750.132.9395.80.0213.33 3pipe100486.4457.5389.90.131.98 4pipe21547153.50523.6477.30.490.32 5pipe38746122.853144.8596.21.651.33

17. saucy : Empirical Performance

18. Conclusions and Future Work CNF formulas from EDA applications exhibit considerable structure (symmetry and sparsity) CNF formulas from EDA applications exhibit considerable structure (symmetry and sparsity) saucy, a new implementation of the nauty symmetry-detection system saucy, a new implementation of the nauty symmetry-detection system –Exploits structure to improve symmetry detection performance by several orders of magnitude –Symmetry-detection time insignificant compared to SAT solver Future work Future work –Apply saucy to more sparse domains which may benefit from symmetry detection –Find other applications of partition refinement—a surprisingly general framework for distinguishing objects in a finite domain

20. saucy : Exploiting Structure Graphs from typical CNF formulas possess a particular structure Graphs from typical CNF formulas possess a particular structure –By construction, clauses are never connected to other clauses –Thus, when refining the partition with a color of clauses, we can ignore all colors containing clauses, since we know that the connection count for every vertex will be zero Such graphs are also very sparse Such graphs are also very sparse –Few literals are connected to most clauses –Few clauses are connected to most literals –Thus, we aggressively avoid work by maintaining data structures (like adjacency lists) which explicitly direct the search and refinement procedures

21. saucy : Exploiting Structure 1 1 1 1 1 1 111 111

22. saucy example: Hole-3

23. saucy : Exploiting Structure We can generalize this idea of avoiding obviously disconnected colors We can generalize this idea of avoiding obviously disconnected colors saucy improvement #3 saucy improvement #3 –Iterate over a color's adjacency lists to determine connected colors –Compute connection counts only for those colors

24. 00 00 0 0 0 0 0 0 0 0 00 00 1111 1 1 1 11 1 1 1 saucy example: Hole-3 nauty : nauty : –Iterate over all colors –For each vertex, count connections to refining color, and sort saucy improvement #3: saucy improvement #3: –Determine directly connected colors –For each vertex, count connections to refining color, and sort saucy improvement #4: saucy improvement #4: –For each vertex in refining color, count connections –For every color touched, sort the counts Positive Negative Clauses

25. saucy : Exploiting Structure nauty works very well on small graphs (and thus small formulas) but fails to scale nauty works very well on small graphs (and thus small formulas) but fails to scale –Takes considerably longer than the SAT solver after adding SBPs –Runs out of memory on formulas with corresponding graphs having >50,000 vertices saucy improvement #1: sparse representation saucy improvement #1: sparse representation –Input graph is represented in adjacency-list format

26. saucy : Exploiting Structure Graphs from CNF formulas possess a particular structure Graphs from CNF formulas possess a particular structure –Clause vertices only connected to their literals –Never connected to each other saucy improvement #2: ignore colors containing clauses when refining with clauses saucy improvement #2: ignore colors containing clauses when refining with clauses –We know that the connection count for every vertex will be zero

27. G  = G c b' C3C3C3C3 b C1C1C1C1 c' C2C2C2C2 a' a a b C3C3C3C3 b' C2C2C2C2 a' C1C1C1C1 c' c G  GG  GG  GG  G Symmetry Detection Problem What precisely is a symmetry of a graph G? What precisely is a symmetry of a graph G? –A symmetry is a permutation  of the labels assigned to vertices of G such that G  = G –The set of all symmetries is denoted Aut(G) b' c C3C3C3C3 c' C2C2C2C2 b C1C1C1C1 a a'  = (a,a')(b,c')(b',c)(C 1,C 2 )  = (a,c',b')(a',c,b)

28. The Search Tree We have a stable ordered partition  of the vertices of the graph. How can we extract Aut(G)? We have a stable ordered partition  of the vertices of the graph. How can we extract Aut(G)? Recall the naïve approach: we need labelings (i.e. discrete colorings) Recall the naïve approach: we need labelings (i.e. discrete colorings) We select a non-singleton color T (for target) and generate |T| colorings, each with one element of T individualized "in front of" the remainder of T We select a non-singleton color T (for target) and generate |T| colorings, each with one element of T individualized "in front of" the remainder of T –Partitions the set of all discrete colorings descendant from  We can then recursively apply partition refinement to further prune the search space! We can then recursively apply partition refinement to further prune the search space! Fix the first discrete coloring found as  ; the remaining discrete colorings yield likely candidates for Aut(G) Fix the first discrete coloring found as  ; the remaining discrete colorings yield likely candidates for Aut(G)

29.  1 = (2,4)(3,5)  2 = (0,3)(1,2)  3 = (0,3,5)(1,2,4) The Search Tree 2 0 4 1 35 012345 035124 035421 035241 035124 035241 305124 305142 503124 503124 053241 053241  = 035421  =  0 = 305412 305142 350142 350142 503214 503124 530124 530124  4 = (0,5,3)(1,4,2)  5 = (0,5)(1,4) Out of 6! = 720 possible labelings, partition refinement pruned away 714, leaving the six symmetries of the graph Out of 6! = 720 possible labelings, partition refinement pruned away 714, leaving the six symmetries of the graph Discrete colorings do not necessarily yield symmetries Discrete colorings do not necessarily yield symmetries –Refinement is only an approximation –Only occurs on highly regular graphs, which are uncommon in EDA applications

30. saucy : Empirical Performance InstanceVerticesnauty saucy #4 SpeedupzChaffw/SBPs Urq2990.020.012.001.730.02 Hole3010.0250.012.50523.390.02 Xor4640.060.016.00timeout1.75 Fpga6710.18250.012514.60timeout0.03 s4-4-3-11035488.740.11806.73441.18218.53 s4-4-3-2997479.670.1796.70204.29877.59 s4-4-3-3997075.980.09844.22timeout884.78 s4-4-3-410714155.310.141109.36timeout464.46 s4-4-3-511072101.630.11923.91timeout134.09 s4-4-3-6962076.480.1764.80679.1313.24 s4-4-3-71036278.960.1789.60831.0118.27 s4-4-3-8660828.420.06473.67123.820.68 s4-4-3-912920209.520.171232.4775.21timeout 2pipe35752.930.02146.500.180.13 3pipe1004857.530.13442.543.206.44 4pipe21547523.640.491068.65228.82153.50 5pipe387463144.851.651905.97347.92122.85

31. Inst.nzChaffSBPsnauty%Symsaucy%Sym Urq2991.730.020.0250.0%0.0133.3% Hole301523.390.020.02555.6%0.0133.3% Xor464timeout1.750.063.3%0.010.6% Fpga671timeout0.030.182585.9%0.0125

32. saucy : Empirical Performance Inst.nzChaffnauty % Sym saucy s4-4-3-110354218.5388.7428.90.110.05 s4-4-3-29974877.5979.678.30.100.01 s4-4-3-39970884.7875.987.90.090.01 s4-4-3-410714464.46155.3125.10.140.03 s4-4-3-511072134.09101.6343.10.110.08 s4-4-3-6962013.2476.4885.20.100.75 s4-4-3-71036218.2778.9681.20.100.54 s4-4-3-866080.6828.4297.70.068.11 2pipe35750.132.9395.80.0213.33 3pipe100486.4457.5389.90.131.98 4pipe21547153.50523.6477.30.490.32 5pipe38746122.853144.8596.21.651.33

33. saucy : Empirical Performance Inst.nzChaffnauty % Sym saucy 2pipe35750.132.9395.80.02 13.3 3pipe100486.4457.5389.90.13 1.98 4pipe21547153.50523.6477.30.49 0.32 5pipe38746122.853144.8596.21.65 1.33

34. saucy : Runtime Performance Speedup is roughly linear in the number of vertices Speedup is roughly linear in the number of vertices Primarily due to efficient use of sparsity within the partition refinement procedure Primarily due to efficient use of sparsity within the partition refinement procedure –Search tree maintenance has relatively low overhead We ran saucy and nauty on the complement graphs We ran saucy and nauty on the complement graphs –Isomorphic search trees! –Isolate performance difference in refinement Slowdown is roughly linear, which is expected given difference in representation Slowdown is roughly linear, which is expected given difference in representation

35. Ordered Partitions The partition refinement algorithm based on degree operates independently of the labeling of the graph The partition refinement algorithm based on degree operates independently of the labeling of the graph To guarantee identical partition representations for isomorphic graphs, we impose an ordering on the colors in the partition To guarantee identical partition representations for isomorphic graphs, we impose an ordering on the colors in the partition –When a color is split, the new colors are assigned in sorted order of degree with the refining color –Refining colors are chosen based on position within the partition ordering, not based on label The search algorithm absolutely the refinement procedure to be labeling-independent! The search algorithm absolutely the refinement procedure to be labeling-independent!

36. Partition Refinement In a stable coloring: In a stable coloring: –vertices in different colors definitely cannot map into each other in some symmetry of the graph –vertices in the same color may map into each other (i.e. refinement is only an approximation) b' c C3C3C3C3 c' C2C2C2C2 b C1C1C1C1 a a' 1 23 76 45 Refinement can't distinguish any vertices (all have degree two) Refinement can't distinguish any vertices (all have degree two) –Vertices in the triangle and square cannot map into each other –Fortunately, this rarely happens with EDA instances An exact, polynomial time partition refinement algorithm would prove that the graph isomorphism problem is in P An exact, polynomial time partition refinement algorithm would prove that the graph isomorphism problem is in P

37. Additional Pruning Methods Enumerating all symmetries is not an option Enumerating all symmetries is not an option –|Aut(G)| is O(n!) –Many EDA-related instances possess exponentially many symmetries Group theory provides the answer: generators Group theory provides the answer: generators –Find a set H  Aut(G) that implicitly represents the entire set of symmetries –Every element of Aut(G) is a product (composition) of integer powers of elements of H –Exponential compression: |H|  log 2 |Aut(G)| We prune away subtrees guaranteed to yield symmetries that we can already generate with previously discovered symmetries We prune away subtrees guaranteed to yield symmetries that we can already generate with previously discovered symmetries

38. Symmetry Breaking On all but the synthetic Urquhart instances, symmetry detection with nauty dominates the run time of the Shatter flow On all but the synthetic Urquhart instances, symmetry detection with nauty dominates the run time of the Shatter flow Further improvements must come from improved symmetry detection Further improvements must come from improved symmetry detection BenchmarkSymSearchTotal % Sym Hole-n0.380.070.4584.4 Urq0.761.171.9339.4 GRoute38.765.0843.8488.4 FPGARoute3.240.213.4593.9 ChnlRoute25.860.1726.0399.4 XOR11.432.4113.8482.6 2pipe23.508.0131.5174.6