1. Configuring a Load-Balanced Switch in Hardware Srikanth Arekapudi, Shang-Tse (Da) Chuang, Isaac Keslassy, Nick McKeown Stanford University

2. 2 Outline  Load Balanced Switch  Scalability  Reconfiguration Algorithm  Hardware Implementation

3. 3 R R R R R R Typical Router Architecture Input Switch Fabric Scheduler Output 1 1 2 2 1 1 N x N

4. 4 Out R R R R/N In R R R R/N 1 1 2 2 3 3 Load-Balanced Switch Load-balancing mesh Forwarding mesh

5. 5 Out R R R R/N In R R R R/N 3 3 2 2 1 1 Load-Balanced Switch Load-balancing mesh Forwarding mesh

6. 6 Out R R R R/N In R R R R/N Load-Balanced Switch Load-balancing mesh Forwarding mesh 3 3  100% throughput for broad class of traffic  No scheduler needed  Scalable

7. 7 A Single Combined Mesh In Out In Out In Out In Out R In Out In Out In Out In Out R 2R/N N*2R/N = 2R = R +R

8. 8 A Single Combined Mesh In Out In Out In Out In Out R In Out In Out In Out In Out R 2R/N (N-1)*2R/N < R +R

9. 9 1 2 3 4 Scalability N =8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 2R/8

10. 10 When N is Too Large Decompose into groups (or racks) 4R/4 2R2R2R2R 1 2 3 4 5 6 7 8 2R2R 2R2R 1 2 3 4 5 6 7 8 4R

11. 11 When N is Too Large Decompose into groups (or racks) 12L 2R 12L Group/Rack 1 Group/Rack G 12L 2R Group/Rack 1 12L 2R Group/Rack G 2RL 2RL/G

12. 12 When Linecards are Missing Failures, Incremental Additions, and Removals… 12L 2R 12L Group/Rack 1 Group/Rack G 12L 2R Group/Rack 1 12L 2R Group/Rack G 2RL 2RL/G 2RL Solution: replace mesh with sum of permutations = + + 2RL/G = 2RL 2RL/G + +

13. 13 When Linecards Fail 12L 2R 12L Group/Rack 1 Group/Rack G 12L 2R Group/Rack 1 12L 2R Group/Rack G MEMS Switch MEMS Switch

14. 14 Questions  Number of MEMS Switches?  TDM Schedule?

15. 15 Example – 3 Linecards In Out In Out In Out R In Out In Out In Out R 2R/3 R R R R

16. 16 Group/Rack 1 1 2 2R 4R Group/Rack 2 1 2R Example 2 Groups 12 2R Group/Rack 1 1 2R Group/Rack 2 4R 2R 8R/3 4R/3 2R/3

17. 17 Group/Rack 1 1 2 2R 4R Group/Rack 2 1 2R Example 2 Groups 12 2R Group/Rack 1 1 2R Group/Rack 2 4R 2R 4R/3 2R/3 4R/3

18. 18 Number of MEMS Switches  MEMS switches between groups i and j  Total Number of MEMS switches: M ≤ L+G-1

19. 19 Questions  Number of MEMS Switches?  TDM Schedule?

20. 20 Group A 1 2 2R 4R Group B 12 2R 4R TDM Schedule 12 2R Group A 12 2R Group B 4R 2R Constraints on linecards at each time-slot Constraints on groups at each time-slot

21. 21 At each time-slot:  Each transmitting linecard sends one packet  Each receiving linecard receives one packet  (MEMS constraint) Each transmitting group i sends at most one packet to each receiving group j through each MEMS connecting them In a schedule of N time-slots:  Each transmitting linecard sends exactly one packet to each receiving linecard Rules for TDM Schedule

22. 22 TDM Schedule T+1T+2T+3T+4 Tx LC A1???? Tx LC A2???? Tx LC B1???? Tx LC B2???? Tx Group A Tx Group B

23. 23 TDM Schedule T+1T+2T+3T+4 Tx LC A1A1A2B1B2 Tx LC A2B2A1A2B1 Tx LC B1B1B2A1A2 Tx LC B2A2B1B2A1 Tx Group A Tx Group B

24. 24 Bad TDM Schedule T+1T+2T+3T+4 Tx LC A1A1A2B1B2 Tx LC A2B2A1A2B1 Tx LC B1B1B2A1A2 Tx LC B2A2B1B2A1 Tx Group A Tx Group B

25. 25 TDM Schedule Algorithm  The algorithm constructs three consecutive schedules. 1. Sending Groups to Receiving Groups Connection Assignment Problem 2. Sending Linecards to Receiving Groups. Matrix Decomposition Problem 3. Sending Linecards to Receiving Linecards Matrix Decomposition Problem

26. 26 TDM Schedule T+1T+2T+3T+4 Tx Group AAB Tx Group BAB

27. 27 Good TDM Schedule T+1T+2T+3T+4 Tx LC A1A1A2B1B2 Tx LC A2B2B1A2A1 Tx LC B1B1B2A1A2 Tx LC B2A2A1B2B1 Tx Group A Tx Group B

28. 28 Good TDM Schedule T+1T+2T+3T+4 Tx LC A1A1A2B1B2 Tx LC A2B2B1A2A1 Tx LC B1B1B2A1A2 Tx LC B2A2A1B2B1 Tx Group A Tx Group B

29. 29 Connection Assignment Problem G1G1 G2G2 G3G3 G1G1 G2G2 G3G3 2 1 2 1 1 1 G1G1 G2G2 G3G3 G1G1 G2G2 G3G3 1 1 1 0 0 0 0 0 0 Not ScheduledScheduled

30. 30 G1G1 G1G1 G2G2 G3G3 G2G2 G3G3 1 1 1 1 0 0 0 0 0 Connection Assignment Problem G1G1 G2G2 G3G3 G1G1 G2G2 G3G3 1 1 1 1 0 0 0 0 G1G1 G2G2 G3G3 G1G1 G2G2 G3G3 1 1 1 0 0 0 0 0 0 G1G1 G2G2 G3G3 G1G1 G2G2 G3G3 1 1 1 1 0 0 0 0 1 0 After GreedyBack Tracing 1

31. 31 Matrix Decomposition Problem 1 0 1 1 0 1 0 1 0 1 0 1 0 1 1 1 1 1 0 0 0 1 0 1 1 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0 0 1 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 1 0 0 0 = ++

32. 32 Matrix Decomposition Problem  Use of sparsity of matrices to represent the ones as a row-column pair  Consists of two stages  Greedy Algorithm  Slepian-Duguid Algorithm 1.Decomposes all the permutation matrices at once 2.Uses the row-column pair list structure

33. 33 Synthesis  40 Groups and 640 Linecards  0.13u process  Cycle time within 4ns  Connection Assignment Problem 1. 10K gates 2. 24Kbits memory  Matrix Decomposition Problem 1. 25K gates 2. 230Kbits of memory

34. 34 Reconfiguration Time

35. Thank you.