Belgrade University Aleksandra Smiljanić: High-Capacity Switching High-Capacity Packet Switches

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Switches with Input Buffers (Cisco)

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Packet Switches with Input Buffers  Switching fabric  Electronic chips (Mindspeed, AMCC, Vitesse)  Space-wavelength selector (NEC, Alcatel)  Fast tunable lasers (Lucent)  Waveguide arrays (Chiaro)  Scheduler  Packets compete not only with the packets destined for the same output but also with the packets sourced by the same input. Scheduling might become a bottleneck in a switch with hundreds of ports and gigabit line bit- rates.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Optical Packet Cross-bar (NEC,Alcatel)  A 2.56 Tb/s multiwavelength and scalable switch-fabric for fast packet- switching network, PTL 1998,1999, NEC

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Optical Packet Cross-bar (Lucent)  A fast 100 channel wavelength tunable transmitter for optical packet switching, PTL 2001, Bell Labs

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Scheduling Algorithms for Packet Switches with Input Buffers  In parallel iterative matching (PIM), SLIP or dual round- robin (DRR) inputs send requests to outputs, outputs grant inputs, and inputs then grant outputs in one iteration. It was proven that PIM finds a maximal matching after log2N +4/3 steps on average.  Maximum weighted matching and maximum matching algorithm maximize the weight of the connected pairs, and achieve 100% for i.i.d. traffic but have complexities O(N 3 log 2 N) and O(N 2.5 ).  Sequential greedy scheduling is a maximal matching algorithm that is simple to implement. Maximal matching algorithm does not leave input-output pair unmatched.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching PIM, SLIP and DRR  In PIM and SLIP each input sends requests to all outputs for which it has packets, and in DRR only to one chosen output. SLIP and DRR use round-robin choices.  Theorem: PIM finds a maximal matching after log2N +4/3 steps on average.  Proof: Let n inputs request output Q, and let k of these inputs receive no grants. With probability k/n all requests are resolved, and with probability 1-k/n at most k requests are unresolved. The average number of requests is at most (1-k/n)·k≤n/4. So if there are N 2 requests at the beginning, the expected number of unresolved requests after I iterations is N 2 /4 i

Belgrade University Aleksandra Smiljanić: High-Capacity Switching PIM, SLIP and DRR  Proof (cont.): Let C be the last step on which the last request is resolved. Then:

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Typical Central Controllers (Cisco)

Belgrade University Aleksandra Smiljanić: High-Capacity Switching SGS Implementation  All inputs one after another choose outputs, SGS is a maximal matching algorithm

Belgrade University Aleksandra Smiljanić: High-Capacity Switching SGS Uses Pipelining I i -> T k Input i chooses output for time slot k

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Bandwidth Reservations Packet Switches with Input Buffers  Anderson et al.: Time is divided into frames of F time slots. Schedule is calculated in each frame; Statistical matching algorithm.  Stiliadis and Varma: Counters are loaded per frame. Queues with positive counters are served with priority according to parallel iterative matching (PIM), their counters are then decremented by 1. DRR proposed by Chao et al. could be used as well.  Kam et al.: Counter is incremented for the negotiated bandwidth and decremented by 1 when the queue is served. Maximal weighted matching algorithm is applied.  Smiljanić: Counters are loaded per frame. Queues with positive counters are served with priority according to the maximal matching algorithm preferrably sequential greedy scheduling algorithm (SGS), where inputs sequentially choose outputs to transmit packets to.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Weighted Sequential Greedy Scheduling  i=1;  Input i chooses output j from O k for which it has packet to send; Remove i from I k and j from O k ;  If i<N choose i=i+1 and go to the previous step;

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Weighted Sequential Greedy Scheduling  If k=1 mod F then c ij =a ij; I k ={1,...,N}; O k ={1,...,N}; i=1;  Input i chooses output j from O k for which it has packet to send such that c ij >0; Remove i from I k and j from O k ; c ij =c ij -1;  If i<N choose i=i+1 and go to the previous step;

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Performance of WSGS Theorem: The WSGS protocol ensures aij time slots per frame to input-output pair (i,j), if Proof: Note that where T i is the number of slots reserved for input i, and R j is the number of slots reserved for output j.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Analogy with Circuit Switches  Inputs ~ Switches in the first stage  Time slots in a frame ~ Switches in the middle stage  Outputs ~ Switches in the last stage Non-blocking condition: Strictly non-blocking condition :

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Admission Control for WSGS The WSGS protocol ensures a ij time slots per frame to input-output pair (i,j) if: I: II: III: F frame length T i the number of slots reserved for input i, R j the number of slots reserved for output j. t i, r j are normalized T i, R j.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Non-blocking Nature of WSGS  Maximal matching algorithm does not leave input or output unmatched if there is a packet to be transmitted from the input to the output in question.  It can be proven that all the traffic passes through the cross-bar with the speedup of two which is run by a maximal matching algorithm, as long as the outputs are not overloaded.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Rate and Delay Guranteed by WSGS  Assume a coarse synchronization on a frame by frame basis, where a frame is the policing interval comprising F cell time slots of duration T c.  Then, the delay of D=2·F·T c is provided for the utilization of 50%. Or, this delay and utilization of 100% are provided for the fabric with the speedup of 2.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Port Congestion Due to Multicasting bit-rate reserved for multicast session k of input i multicast group k sourced by input i Solution: Packets should be forwarded through the switch by multicast destination ports.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Forwarding Multicast Traffic

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Forwarding Multicast Traffic

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Forwarding Multicast Traffic

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Adding the Port to the Multicast Tree

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Adding the Port to the Multicast Tree

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Adding the Port to the Multicast Tree

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Removing the Port from the Multicast Tree

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Removing the Port from the Multicast Tree

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Removing the Port from the Multicast Tree

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Removing the Port from the Multicast Tree

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Removing the Port from the Multicast Tree

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Admission Control for Modified WSGS where E i is the number of forwarded packets per frame

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Admission Control for Modified WSGS for

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Admission Control for Modified WSGS I: II: Modified WSGS protocol ensures negotiated bandwidths to input-output pairs if for : T i the number of slots reserved for input i, R i the number of slots reserved for output i. t i, r i are normalized T i, R i. F frame length, P forwarding fan-out

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Rate and Delay Guaranteed by Modified WSGS  Assume again a coarse synchronization on a frame by frame basis.  Then, the delay of D= F·T c is provided for the utilization of 1/(P+2), where P is the forwarding fan-out. Or, this delay and utilization of 100% are provided for the fabric speedup of P+2.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Quality of Service, P=2, S=4, B=10Gb/s, T c =50ns N F10 4 5· ·10 4 C [Tb/s] G [Mb/s]10.21 D [ms]

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Clos Packet Switches

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Load Balancing in Packet Switches  J. Turner introduces load balancing of multicast sessions in Benes packet switches, INFOCOM 1993  C. Chang et al. propose load balancing in two-stage Birkhoff-von-Neumann switch, while Iyer et al. analyze the performance of the parallel plane switch (PPS) which applies load balancing.  Keslassy et al. propose the implementation of high- capacity PPS or Birkhoff-von-Neumann architecture.  Smiljanić examines rate and delay guarantees in three-stage Clos packet switches based on load balancing. These switches provide the larger number of lower speed ports.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Load Balancing Algorithms  Packets are split into cells, and cells are grouped into flows.  Cells of each flow are balanced over center SEs  Balancing of a flow can be implemented in the following way:  One counter is associated with each flow.  When a cell of the flow arrives, it is marked to be transmitted through the center SE whose designation equals the counter value, and then counter is incremented (decremented) modulo l, where l is the number of center SEs.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Load Balancing Algorithms  A flow comprises cells determined by different rules, but that have the same input port or the input switching element (SE), and have the same output port or the output SE. Examples:  SEs with input buffers  Cells sourced by the same input  Cells sourced by the same input bound for the same output  Cells sourced by the same input bound for the same output SE  SEs with shared buffers  Cells sourced by the same input SE bound for the same output  Cells sourced by the same input SE bound for the same output SE

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Non-Blocking Load Balancing Non-blocking if:, no speedup is needed l

Belgrade University Aleksandra Smiljanić: High-Capacity Switching  Let us assume the implementation with the coarse synchronization of the switching elements (SEs), i.e:  the switching elements are synchronized on a frame-by-frame basis  in each frame any SE passes cells that arrived to this SE in the previous frame  The delay through a three-stage Clos network with such coarse synchronization including packet reordering delay is:  Note that if multicasting is accomplished by the described packet forwarding, the utilization is decreased 3 times, and the delay is increased log P N times. Rate and Delay Guarantees c FTD4 

Belgrade University Aleksandra Smiljanić: High-Capacity Switching  Utilization under which the delay is guaranteed to be below D:  where S is the switching fabric speedup, N f is the number of flows whose cells pass the internal fabric link, and T c is the cell time slot duration. Utilization Formula

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Derivation of Utilization  The maximum number of cells transmitted over a given link from an input to a center SE, F c, fulfills:  where f ig is the number of cells per frame in flow g of cells from input SE i, and F u is the maximum number of cells assigned to some port  If N f -n flows have one cell per frame, and remaining n flows are assigned max(0,nF u -N f +n) cells per frame

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Derivation of Utilization  So:  The same expression holds for F c and U a over the links from center to output SEs  Since F=D/(4T c ):

Belgrade University Aleksandra Smiljanić: High-Capacity Switching  The delay of D is guaranteed for 100% utilization for the speedup of:  where N f is the maximum number of flows whose cells pass any internal fabric link, and T c is the cell time slot duration. Speedup Formula

Belgrade University Aleksandra Smiljanić: High-Capacity Switching  We put U a =1 in the formula for utilization, and readily obtain expression for the required speedup: Derivation of Speedup

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Counter Synchronization  The utilization was decreased because all flows may be balanced starting for the same center SE, so this SE will not be able to deliver all the passing cells within a frame.  Higher utilization can be achieved if the counters of different flows are synchronized.  Counter of flow g sourced by input SE 1i is reset at the beginning of each frame to c ig =( i+g ) mod l, where l is the number of center SEs. And, counter of flow g bound for output SE 3j is reset at the beginning of each frame to c jg =( j+g ) mod l.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching  Utilization under which the delay is guaranteed to be below D:  where S is the switching fabric speedup, N f is the maximum number of flows whose cells pass any internal fabric link, and T c is the cell time slot duration. Utilization Formula when Counters are Synchronized

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Derivation of Utilization when Counters are Synchronized  The maximum number of cells transmitted over a given link from an input to a center SE 2(l-1), F c, fulfills:  where f ig denotes the number of cells in flow g that are balanced starting from input SE 1i

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Derivation of Utilization when Counters are Synchronized  F c is maximized for:  where y ig > 0 are integers.  F c is maximized if:  And F c is then equal to:

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Derivation of Utilization when Counters are Synchronized  If F u <lN f /(2n), it holds that for some z<l:  In this case, F c is maximized for  and equal to:

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Derivation of Utilization when Counters are Synchronized  From F c nSF/l, it follows that:

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Derivation of Utilization when Counters are Synchronized  And:

Belgrade University Aleksandra Smiljanić: High-Capacity Switching  The delay of D is guaranteed for 100% utilization for the speedup of:  where N f is the maximum number of flows whose cells pass any internal fabric link, and T c is the cell time slot duration. Speedup Formula when Counters are Synchronized

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Derivation of Speedup when Counters are Synchronized  Speedup providing 100% utilization of the transmission capacity is derived when F u =F in inequality F c nSF/l:

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Derivation of Speedup when Counters are Synchronized  Because F N f >(10N f )/(8N), for N 2.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Utilization vs. Number of Ports, T c =50ns, n=m=l N f =nN N f =N COUNTERS OUT OF SYNC COUNTERS IN SYNC

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Speedup vs. Number of Ports, T c =50ns, n=m=l N f =nN N f =N COUNTERS OUT OF SYNC COUNTERS IN SYNC

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Utilization and Speedup vs. Number of Ports, D=3ms, N f =m=l=640 UTIL SPEED COUNTERS OUT OF SYNC COUNTERS IN SYNC 1ms

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Conclusions: Scalable Implementations  Switches with input buffers require simple implementation: pipelining relaxes processing of output selector, central controller has a linear structure  Clos packet switches based on load balancing is even more scalable: they require neither the synchronization on a cell by cell basis across the whole fabric nor the high-capacity fabric.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching Conclusions: Performance Advantages  Both examined architectures provide nonblocking with moderate fabric speedups, i.e. the fabric passes all the traffic as long as outputs are not overloaded.  Rate and delay are guaranteed even to the most sensitive applications.  Due to the nonblocking nature of the fabric, the admission control can be distributed, and therefore more agile.

Belgrade University Aleksandra Smiljanić: High-Capacity Switching References  T. E. Anderson, S. S. Owicki, J. B. Saxe, and C. P. Thacker, “Highspeed switch scheduling for local-area networks,” ACM Transactions on Computer Systems, vol. 11, no. 4, November 1993, pp  N. McKeown et al., “The Tiny Tera: A packet switch core,” IEEE Micro, vol. 17, no. 1, Jan.-Feb. 1997, pp  A. Smiljanić, “Flexible bandwidth allocation in high- capacity packet switches,” IEEE/ACM Transactions on Networking, April 2002, pp

Belgrade University Aleksandra Smiljanić: High-Capacity Switching References  A. Smiljanić, “Scheduling of multicast trafc in high-capacity packet switches,” IEEE Communication Magazine, November 2002, pp  A. Smiljanić, “Performance of load balancing algorithms in Clos packet switches,” Proceedings of IEEE HPSR, April 2004, pp  J. S. Turner, “An optimal nonblocking multicast virtual circuit switch,” Proceeding of INFOCOM 1994, vol. 1, pp