1. Making Parallel Packet Switches Practical Sundar Iyer, Nick McKeown (sundaes,nickm)@stanford.edu Departments of Electrical Engineering & Computer Science, Stanford University http://klamath.stanford.edu/pps

2. Stanford University 2 Motivation To design and analyze: –an architecture of a very high capacity packet switch –in which the memories run slower than the line rate” [Ref: S. Iyer, A. Awadallah, N. McKeown, “Analysis of Packet Switch with Memories Running Slower than the Line Rate, Proc. Infocom, Tel Aviv, Mar 2000.]

3. Stanford University 3 What limits capacity of packet switches today? Memory bandwidth for packet buffers –Shared memory: B = 2NR –Input queued: B = 2R Switch Arbitration –At the line rate R Packet Processing –At the line rate R

4. Stanford University 4 How can we scale the capacity of switches? What we’d like: R RR R NxN The building blocks we’d like to use: R R R R Slower NxN Switches Large NxN Switch

5. Stanford University 5 Why this might be a good idea Larger Capacity Slower than the line rate –Buffering –Arbitration –Packet Processing Redundancy

6. Stanford University 6 Observations and Questions Random load-balancing: –It’s hard to predict system performance. Flow-by-flow load-balancing: –Worst-case performance is very poor. Can we do better? –What if we switch packet by packet? –Can we achieve 100% throughput –Can we give delay guarantees? 1 2 … … k R R R R/k R R R

7. Stanford University 7 Architecture of a PPS Definition: A PPS is comprised of multiple identical lower- speed packet-switches operating independently and in parallel. An incoming stream of packets is spread, packet-by- packet, by a demultiplexor across the slower packet-switches, then recombined by a multiplexor at the output. We call this “parallel packet switching”

8. Stanford University 8 Architecture of a PPS OQ Switch 1 2 3 N=4 R R R R 1 2 3 R R R R Multiplexor Demultiplexor Multiplexor (sR/k) k=3 1 2 (sR/k)

9. Stanford University 9 We will compare it to an OQ Switch 1 2 N 1 2 N Output Queued Switch R R R R R R R R Internal BW = 2NR Why? –There is no internal contention –No queueing at the inputs –They give the minimum delay –They can give QoS guarantees

10. Stanford University 10 Definition Relative Queueing Delay –This is defined as the increased queueing delay faced by a cell in the PPS relative to the delay it receives in a shadow output queued switch –It includes the time difference attributed only due to queueing –A switch is said to emulate an OQ switch if the relative queueing delay is zero

11. Stanford University 11 A PPS which a bounded relative delay Shadow OQ Switch R R R R R R R R PPS Yes No =? C C t C C t’ C C t” C t’ t” –t’ < Constant

12. Stanford University 12 Problem Statement Redefined Motivation: “To design and analyze an architecture of a very high capacity packet switch in which the memories run slower than the line rate, which preserves the good properties of an OQ switch” This talk: Expanding the capacity of a FIFO packet switch, with a bounded relative queueing delay, using the PPS architecture.

13. Stanford University 13 Layer 1 Layer 2 Layer 3 1 2 3 N=4 R R R R 1 2 3 R R R R 2 24153 1 2 1 3 2 1 4 2 3 1 4 21 3 4 1 23 4 5 1 23 5 1 234 1 234 5 1 2345 R/3 A Bad Scenario for the PPS

14. Stanford University 14 Parallel Packet Switch Result Theorem: If S >= 2 then a PPS can emulate a FIFO OQ switch for all traffic patterns.

15. Stanford University 15 Is this Practical? Load Balancing Algorithm –Is Centralized –Requires N 2 communication complexity –Ideally we want a distributed algorithm Speedup –A speedup of 2 is required –We would ideally like no speedup

16. Stanford University 16 Layer 1 Layer 2 Layer 3 1 2 3 N=4 R R R R 1 2 3 R R R R 2 R/3 Load Balancing in a PPS 1 3 1 1 1 2 3 2 3 2 2 3 123 123 123 123

17. Stanford University 17 Distribution of Cells from a Demultiplexor R 1243568 7 9101211131416 15 511151641289127 R/3 Demultiplexor: Input 1 FIFOs for all k=3 layers 9 12 1 2 7 85 11 15 3 6 10 14 1336 10144 16 Cells from every input to a every output are sent to the center stage switches in a round robin manner “No more than 4 consecutive cells can go to the same FIFO i.e. center stage switch”

18. Stanford University 18 Modification to the PPS Relax the relative queueing bound –Allows a distributed load balancing arbiter Run an independent load balancing algorithm on each demultiplexor –Eliminates N 2 Communication Complexity Keep small & bounded delay buffers at the demultiplexor –Eliminates speedup in the links between the demultiplexor and the center stage switches

19. Stanford University 19 Layer 1 Layer 2 Layer 3 1 2 3 N=4 R R R R 1 2 3 R R R R 2 1 2 3 3 2 1 R/3 Cells as seen by the Multiplexor 1 3 2 1 3 2 1 2 3 123 123 123 123

20. Stanford University 20 Solution Read –cells from the corresponding queues (which may be out of order) based on the arrival time from all center stage switches to maintain throughput Introduce –a small and bounded re-sequencing buffer at the multiplexor to re-order cells and send them in sequence Tolerate –a bounded delay relative to the shadow FIFO OQ switch

21. Stanford University 21 Properties of the PPS Demultiplexor Demultiplexors –Cells arrive at combined rate R over all k FIFOs –Each cell has a property: output –Cells to same output are inserted into the k FIFOs in RR. –Cells are written into each FIFO buffer at leaky bucket rate of less than R/Ck + N –Cells are read from each FIFOs at constant service rate R/k –Max delay faced by a cell is N internal time slots

22. Stanford University 22 Relative Queueing Delay faced by a Cell Demultiplexors –A maximum relative queueing delay of N internal time slots is encountered by a cell Multiplexors –A maximum relative queueing delay of N internal time slots is encountered by a cell Total Relative Queueing Delay –2N time slots

23. Stanford University 23 Buffered PPS Results A PPS with a completely distributed algorithm and no speedup with a buffer of size Nk, can emulate a FIFO output queued switch for all traffic patterns within a relative queueing delay bound of 2N internal time slots I.e. 2Nk time slots.

24. Stanford University 24 Conclusion –Its possible to expand the capacity of a FIFO packet switch using multiple slower speed packet switches. –There remain a couple of open questions Making QoS practical. Making multicasting practical.