1. FPGA Intra-cluster Routing Crossbar Design Dr. Philip Brisk Department of Computer Science and Engineering University of California, Riverside CS 223

2. Generating Highly Routable Sparse Crossbars for PLDs Guy Lemieux, Paul Leventis, David Lewis International Symposium on FPGAs, 2000

3. Basic Notation

4. Fully Populated Crossbar Full capacity – can connect as many signals as the number of outputs Flexibility – Can connect any input to any output

5. Full-capacity Minimal Crossbars Full capacity Reduced Flexibility: you lose the ability to connect any input to any output p = m(m – n + 1) switches

6. Full-capacity Minimal Crossbars … Area savings is minimal if n >> m

7. Perfect and Sparse Crossbars Perfect crossbars – Can disjointly route any m-sized subset of the n inputs to the m outputs – Both full and full-capacity minimal crossbars are perfect Sparse crossbars – Has p < m(m – n + 1) switches – Cannot be perfect

8. Bipartite Graph Representation I1I1 I2I2 I3I3 I4I4 I5I5 I6I6 O1O1 O2O2 O3O3 O4O4 I1I1 I2I2 I3I3 I4I4 I5I5 I6I6 O1O1 O2O2 O3O3 O4O4

9. Evaluation Challenge How “routable” is a given crossbar? – Build an FPGA, map 20+ applications, observe results Slow, highly subject to the application mix – Monte Carlo Test Generate random test vectors Route each test vector on the crossbar (network flow) Report number of successes as a percentage A highly routable sparse crossbar has a >= 95% success rate

10. Hall’s Theorm Given a bipartite graph G = (V, E) – X, Y are the bipartite independent sets of G G has a matching of X onto Y if and only if N(v) is the set of neighbors of vertex v N(S) is the set of neighbors of all vertices in S Leverage Hall’s Theorem to generate routable sparse crossbars!

11. Practical Issues Cannot enumerate all subsets of m inputs N(x) should be approximately equal for all input vertices x in X – Otherwise, any subset containing a large number of low-degree vertices is unlikely to be routable N(y) should be approximately equal for all output vertices y in Y – Symmetric argument

12. Hamming Distance and Coding Theory Represent N(v) as a bitvector bv – bv[i] = 1 if v fans out to O i Hamming Distance – d(bv 1, bv 2 ) Strategy – Maximize d(bv i, bv j ) for every pair of distinct vertices v i and v j

13. Switch Placement Optimizer Start with initial switch placement Generate random swap of switch positions – Accept the swap if there is an improvement – Otherwise, reject the swap Stop after a fixed number of swap candidates (e.g., 10K) fails to find an improvement Objective is to minimize:

14. Example Identical Hamming costs before and after the swap Before: cannot route {1, 2, 3} After: reduces Hamming costs

15. 168x24 Crossbar, 10K Test Vectors

16. Altera Flex 8000 HP Plasma Hextant

17. # Switches vs. Routability

18. Using Sparse Crossbars within LUT Clusters Guy Lemieux, David Lewis International Symposium on FPGAs, 2001

19. Five Questions 1.Will depopulation save area, require greater routing area, or create unroutable architectures? 2.Will depopulation reduce or increase routing delays? 3.What amount of depopulation is reasonable? 4.How much area or delay reduction can be attained, if any? 5.What are the other effects of depopulating the cluster?

20. Architecture and Parameters

21. Results

22. Designing Efficient Input Interconnect Blocks for LUT Clusters Using Counting and Entropy Wenyi Feng and Sinan Kaptanoglu ACM Transactions on Reconfigurable Technology and Systems (TRETS), 1(1): article #6, March, 2008 Note: Paper is from Actel (now Microsemi)

23. Count Configurations (Details Omitted) 784 Configurations 312 Configurations256 Configurations

24. Routing Requirement Vector (RRV) An ordered list of N subsets containing K distinct signals The i th subset is K distinct signals to route to the i th K-LUT Total number of RRVs for the crossbar: M inputs KN outputs

25. Entropy of an Intra-cluster Routing Crossbar H = lg(# routable RRVs) – Accounts for equivalence of LUT inputs Why Entropy? – # routable RRVs is huge – Minimum number of configuration bits to program the crossbar – Inversely correlated with usage of global routing muxes (details omitted) If we reduce the routability of the crossbar, we will end up programming more global routing muxes to compensate for the entropy loss

26. Conceptual Idea intra-cluster crossbar global routing

27. Theorem Let P and L be the number of muxes and switches in a crossbar – The entropy is at most Plg(L/P) – The entropy per switch is at most log(L/P) / (L/P) – These bounds are achieved only when each mux has size L/P and each configuration realizes a unique RRV Proof omitted because I DO NOT HATE YOU!

28. What are we doing here? Lemieux and Lewis – Routability: Monte Carlo simulations – Area: Count switches Feng and Kaptanoglu – Routability: Crossbar entropy – Area: Entropy per switch – Caveat: Focus only on crossbars where we can count routable, non-redundant RRVs!

29. Type-1 Crossbar 1-level – L2 muxes are driven directly by crossbar input signals – #routable RRVs depends on L2 crossbar topology Not area-efficient due to big L2 muxes Xilinx Virtex-style

30. Type-2 Crossbar 2-level – L1 is sparsely populated – L2 is fully populated Fully populated L2 reduces area efficiency VPR – F c,in determines L1 population density

31. Type-3 Crossbar 2-level, Partitioned – L1 partition P i only drives L2 partition O i – From input m to LUT input n, all paths go through muxes in P i and O i exclusively – #Routable RRVs is the product of #Routable RRVs for each disjoint sub-crossbar

32. Proposed Type-3 Crossbar and Generation Algorithm Each sub-crossbar is Type-2 Can count #routable RRVs (Details omitted)

33. Entropy vs. # Switches

34. Entropy vs. Global Routing Mux Usage

35. The Bottom Line… Who cares… – Theoretical properties are cute – Actel/Microsemi did not use these crossbars in their FPGAs Practical observation… – The cheaper you make the intra-cluster routing crossbar, the more expensive the global routing…

36. A 65nm flash-based FPGA fabric optimized for low cost and power Jonathan W. Greene, et al. International Symposium on FPGAs, 2011 Note: Paper is from Microsemi (Feng and Kaptanoglu are co-authors)

37. Corporate Secrets Divulged They used a Clos Network – Three parameters: m, n, r

38. Clos Network Properties Used when the physical circuit switching needs to exceed the capacity of the largest feasible single crossbar Much cheaper than a fully populated nxn crossbar

39. Strict-sense Nonblocking Clos Network (m > 2n – 1) An unused input on an ingress switch can always be connected to an unused output on an egress switch, without reconfiguration!

40. Rearrangeably Nonblocking Clos Network (m > n) An unused input on an ingress switch can always be connected to an unused output on an egress switch, but reconfiguration may be necessary!

41. Recursive Clos Network Design Scalable to any ODD number of stages – Replace center crossbar with a 3-stage Clos Network