1. New Ways of Generating Large Realistic Benchmarks for Testing Synthesis Tools Petr Fišer, Jan Schmidt Faculty of Information Technology Czech Technical University in Prague fiserp@fit.cvut.cz, schmidt@fit.cvut.cz

2. IWSBP 2010, Freiberg2 Outline Motivation New benchmark generation methods Experimental results Conclusions

3. IWSBP 2010, Freiberg3 Motivation … why another artificial benchmark generator? To test logic synthesis tools Capabilities of synthesis processes Immunity to “bad” structures Ability to discover “good” structures Iterative power Scalability …

4. IWSBP 2010, Freiberg4 Motivation J. Cong, K. Minkovich: Optimality study of logic synthesis for LUT-based FPGAs, IEEE Trans. on CAD, vol. 26, 2007 small They created artificially large circuits, functionally equivalent to their small origins (70 LUTs) Synthesis produced 10k – 30k LUTs

5. IWSBP 2010, Freiberg5 Motivation P. Fišer, J. Schmidt, J: Small but Nasty Logic Synthesis Examples, IWSBP'08 XOR tree is appended to the circuit outputs and the circuit is collapsed Synthesis produced >400 LUTs instead of 11

6. IWSBP 2010, Freiberg6 Motivation Will my synthesis tool produce the same result for different descriptions (versions) of one particular circuit? (a.k.a. iterative power) Most probably not! (if things go bad) What went wrong? What descriptions are bad for me? What structures caused my failure? What should I do to perform better?

7. IWSBP 2010, Freiberg7 Proposed Benchmarks Starting with seed circuit (could be small) Functionally equivalent “big” circuit is created The size of the benchmark circuit is adjustable Result Transformation 1 Transformation 2 Transformation 3 Bench circuit 1 Bench circuit 2 Bench circuit 3 Seed circuit Ideal case: Synthesis

8. IWSBP 2010, Freiberg8 Proposed Benchmarks Starting with seed circuit (could be small) Functionally equivalent “big” circuit is created The size of the benchmark circuit is adjustable Transformation 1 Transformation 2 Transformation 3 Bench circuit 1 Bench circuit 2 Bench circuit 3 Seed circuit Real case: Synthesis Result 1 Result 2 Result 3

9. IWSBP 2010, Freiberg9 Cong’s LEKU Benchmarks J. Cong, K. Minkovich: Optimality study of logic synthesis for LUT-based FPGAs, IEEE Trans. on CAD, vol. 26, 2007 LEKU = Logic Examples with Known Upper Bound Based on elimination of the original circuit structure … and bad decomposition

10. IWSBP 2010, Freiberg10 1. Realistic LEKU Benchmarks Any circuit may be used as a seed (instead of g25) Possible chance of success Global BDDs may be used instead of collapsing Upper bound = size of the original circuit

11. IWSBP 2010, Freiberg11 1. Realistic LEKU Benchmarks Size increase by collapsing 250 ISCAS and IWLS benchmarks Size increase in 61% of circuits

12. IWSBP 2010, Freiberg12 1. Realistic LEKU Benchmarks Experimental results Benchmark circuit Synthesis (# of 4-LUTs) BenchInp.Out.ProcessGatesABC#1#2 c432367original1458477118 c432367global BDD2,0171,0311,0231,333 c432367ABC collapse2,6581,2461,5481,648 c432367SIS collapse7,0753,3613,8724,738 c8806026original208113110122 c8806026global BDD407,09893,190174,983N/A c8806026ABC collapse13,7277,4378,1099,460 c8806026SIS collapse30,01519,78720,48728,017

13. IWSBP 2010, Freiberg13 2. Parity Benchmark Circuits XOR tree is appended to the circuit outputs, then the structure is destroyed (collapsing, BDD) No guarantee of circuit size increase Upper bound = size of the core circuit + XOR tree

14. IWSBP 2010, Freiberg14 2. Parity Benchmark Circuits Size increase by appending parity & collapsing 100 ISCAS and IWLS benchmarks Size increase in 25% of circuits

15. IWSBP 2010, Freiberg15 2. Parity Benchmark Circuits Experimental results Benchmark circuit Synthesis (# of 4-LUTs) BenchInp.Out.ProcessGatesABC#1#2 s1238321original493229241263 s1238321global BDD6,2823,8494,0553,839 s1238321ABC collapse31,83919,74121,87525,793 s1238321SIS collapse39,63626,31328,254N/A b4331original267110108116 b4331BDD16,9636,3476,0994,285 b4331ABC collapse1,405730841884 b4331SIS collapse4,0872,0362,4221,627

16. IWSBP 2010, Freiberg16 3. Tautology Benchmarks Large random SOP is generated When the number of terms exceeds some threshold, the SOP is a tautology Then, the big SOP is mapped into 2-input gates (SIS tech_decomp)   Big network Upper bound = 0 The benchmark size may be adjusted by 1. 1. Number of input variables 2. 2. Dimension of SOP terms

17. IWSBP 2010, Freiberg17 4. Partial Collapsing Only parts of the network are collapsed 1. 1. Choose one pivot gate 2. 2. Extract its transitive fan-in and fan-out to a given radius 3. 3. Collapse the extracted network part 4. 4. Decompose into 2-input gates 5. 5. Put it back 6. 6. Iterate several times Upper bound = size of the original circuit The benchmark size may be adjusted by 1. 1. Size of the extracted circuit 2. 2. Number of iterations

18. IWSBP 2010, Freiberg18 4. Partial Collapsing Example – c432

19. IWSBP 2010, Freiberg19 4. Partial Collapsing Example – big tautology

20. IWSBP 2010, Freiberg20 4. Partial Collapsing Example – big tautology

21. IWSBP 2010, Freiberg21 4. Partial Collapsing Experimental results Benchmark circuit Synthesis (4-LUTs) BenchInp.Out.ProcessGatesABC#1#2 c432367original1458477118 c432367Part. coll., size 981,247626782916 c432367Part. coll., size 1093,0771,4451,6992,422 c432367Part. coll., size 1385,0262,5982,7613,727 c432367Part. coll., size 14011,5316,6476,8449,255 c8806026original208113110122 c8806026Part. coll., size 1291,008485601597 c8806026Part. coll., size 1715,03429502,3943,769 c8806026Part. coll., size 20110,42362245,0107,887

22. IWSBP 2010, Freiberg22 5. Replicating Shared Logic Duplicate a part of the logic that is shared 1. 1. Find a branching signal 2. 2. Duplicate its transitive fan-in, to a given depth Upper bound = size of the original circuit The benchmark size may be adjusted by 1. 1. Number of duplicated branches 2. 2. Depth of duplication

23. IWSBP 2010, Freiberg23 5. Replicating Shared Logic Experimental results Benchmark circuitSynthesis (4-LUTs) BenchInp.Out.ProcessGatesABC#1#2 c432367original1458477118 c43236710k dup., depth 11,42884244333 c43236710k dup., depth 24,90584447586 c43236710k dup., depth 38,38984396637 c43236710k dup., depth 411,34984452739 c43236710k dup., depth 516,04084472771

24. IWSBP 2010, Freiberg24 6. Adding Inverters (special bonus – not included in the proceedings) Add pairs of inverters to random locations The network size may be arbitrarily expanded And all the synthesis tools… Are completely immune to this!

25. IWSBP 2010, Freiberg25 Summary Experiments

26. IWSBP 2010, Freiberg26 Summary Experiments

27. IWSBP 2010, Freiberg27 Conclusions Several new benchmark generation methods proposed Artificially “big” circuits are generated from seed circuits Benchmarks are functionally equivalent to the seed circuits  the complexity upper bound is known Tested on ABC and 2 commercial tools Unfortunate result – the bigger the circuit going to synthesis, the bigger the result