1. 1 Maintaining Packet Order in Two-Stage Switches Isaac Keslassy, Nick McKeown Stanford University

2. 2 Two Successive Scaling Problems OQ routers: + work-conserving - memory bandwidth = (N+1)R R R R R IQ routers: + memory bandwidth = 2R - arbitration complexity Bipartite Matching R R

3. 3 Today: 64 ports at 10Gbps, 64-byte cells. Arbitration Time = = 51.2ns Request/Grant Communication BW = 17.5Gbps 10Gbps 64bytes IQ Arbitration Complexity Two main alternatives for scaling: 1.Increase cell size (Kar et al., 2000) 2.Eliminate arbitration (Chang et al., 2001) Scaling to 160Gbps: Arbitration Time = 3.2ns Request/Grant Communication BW = 280Gbps

4. 4 Desirable Characteristics for Router Architecture Ideal: OQ 100% throughput Minimum delay Maintains packet order Necessary: able to regularly connect any input to any output What if the world was perfect? Assume Bernoulli iid uniform arrival traffic...

5. 5 Cyclic Shift? 1 N 1 N Uniform Bernoulli iid traffic: 100% throughput

6. 6 Cyclic Shift? 1 N 1 N b(t) (t) q(t) Problem: real traffic is non-uniform Long-term service opportunities exceed arrivals:

7. 7 Two-Stage Switch External Outputs Internal Inputs 1 N External Inputs Load-balancing cyclic shift Switching cyclic shift 1 N 1 N 1 1 2 2 100% throughput for broad range of traffic types (C.S. Chang et al., 2001)

8. 8 Two-Stage Switch External Outputs Internal Inputs 1 N External Inputs 1 N 1 N 1 (t) 2 (t) b(t) q(t) a(t) Traffic rate: First cyclic shift: Long-term service opportunities exceed arrivals: ( 2 = 1 possible)

9. 9 Eliminates arbitration 100% throughput Conventional router packaging Two-Stage Switch Characteristics

10. 10 1 2 3 Phase 2 Phase 1 Using a Single Stage Twice Lookup Buffer Lookup Buffer Lookup Buffer Linecards 1 N 1 N 1 N

11. 11 Racks of linecards Optical links Two-Stage Switch Characteristics

12. 12 Problem: unbounded mis-sequencing External Outputs Internal Inputs 1 N External Inputs Cyclic Shift 1 N 1 N 1 1 2 2 Two-Stage Switch Characteristics

13. 13 Full Frames First (FFF): Intuitive Idea External Outputs Internal Inputs 1 N External Inputs Cyclic Shift 1 N 1 N 1 1 2 2 Idea: 1. Spread cells evenly across all linecards t 12

14. 14 Full Frames First (FFF): Intuitive Idea Idea: 1. Spread cells evenly across all linecards 2. Read them in order External Outputs Internal Inputs 1 N External Inputs Cyclic Shift 1 N 1 N 1 1 2 3 2

15. 15 Full Frames First (FFF): Intuitive Idea Idea: 1. Spread cells evenly across all linecards 2. Read them in order External Outputs Internal Inputs 1 N External Inputs Cyclic Shift 1 N 1 N 1 2 2 3 1

16. 16 First Problem Problem: if two packets dont arrive consecutively, there may be a hole in the reading sequence External Outputs Internal Inputs 1 N External Inputs Cyclic Shift 1 N 1 N 1 1 2 2 3 t 123

17. 17 Coordination Buffer Solution: collect cells from a flow in a coordination buffer, and load-balance them among linecards 1 N Cyclic Shift 1 N 1 N 1 2 3 2 Flow Load Balancing Coordination Buffer (VOQ) 1 t 123

18. 18 Second Problem Problem: No access to cell 2 because of head-of- line blocking 1 N Cyclic Shift 1 N 1 N 1 2 2 Flow Load Balancing Coordination Buffer (VOQ) 1 3 a b t 1 3 2 t a b Input 1: Input 2:

19. 19 Expanding VOQ Structure Solution: expand VOQ structure by distinguishing among switch inputs 2 1 3 a b

20. 20 FFF: Guarantees Theorem 2: for any arrival process, D avg (FFF) D avg (OQ) + (4N 2 - 2) Theorem 1: for any arrival process for which OQ has 100% throughput, so does FFF Theorem 3: FFF maintains packet order

21. 21 Two-Stage Switch in Optics R/N Passive mesh 1 2 3 1 2 3 R/N Passive mesh 1 2 3 1 2 3 2R/N 1 2 3 1 2 3 Cyclic Shift RR

22. 22 Summary FFF: practical algorithm that solves mis-sequencing Same throughput as OQ, and average delay within a bound New approach to optical switching