A Scalable, Cache-Based Queue Management Subsystem for Network Processors Sailesh Kumar, Patrick Crowley Dept. of Computer Science and Engineering

2 - Sailesh Kumar - 8/18/2015 Packet processing systems n ASIC based »High performance »Low configurability »May be expensive when volumes are low n Network processors (NP) based »Very high degree of configurability »High volumes can result in low cost »Challenges in matching the ASIC performance n In this paper we concentrate on queuing bottlenecks associated with NP based packet processors

3 - Sailesh Kumar - 8/18/2015 Basic NP architecture n Chip-Multiprocessor (CMP) architecture »A pool of relatively simple processors »Has dedicated hardware units for common cases »Provides high speed interconnection between processors »Integrates network interface and memory controllers n Group of processors collectively performs the packet processing task (queuing, scheduling, etc) n Best case performance is N times higher when each processor operates in parallel n Example, Intel’s IXP architecture

4 - Sailesh Kumar - 8/18/2015 Intel’s IXP2850 Introduction n CMP architecture n 16 RISC type processors called Microengines (ME) n 8 hardware thread contexts on each ME n SPI4.2 and CSIX interface cores n 3 Rambus DRAM and 4 QDR SRAM controllers n Various hardware units, like Hash, CAM, etc. n Typically MEs are arranged in pipeline and groups of MEs collectively perform the packet processing task.

5 - Sailesh Kumar - 8/18/2015 Why does PP need queues? n Routers and switch fabrics are packet processing systems which, »Receives packets at input ports »Classify and identifies the next hop for the packet »Transmit the packet at the appropriate output port n (Ingress rate > output link capacity) Due to, »traffic from many input ports destined to one output port »Bursty nature of internet traffic »Statistical oversubscription n Implications: »Unbounded delay for all flows »Packet loss across every active flow »A single misbehaving flow can affect all other flows

6 - Sailesh Kumar - 8/18/2015 Solution n Keep queues for every flow or group of flows »Put arriving packets into appropriate queue »Treat each queue such that resources are allocated fairly to all flows »Send packets from queues such that each will receive fair share of the aggregate link bandwidth n In fact, queues are the fundamental data structure in any packet processing system. It ensures »Fair allocation of resources (bandwidth, buffer space, etc) »Isolate misbehaving and high priority flows »Guaranteed traffic treatment, delay, bandwidth, QoS n Conclusion: any packet processing system must handle large number of queues at very high speed

7 - Sailesh Kumar - 8/18/2015 A simple queuing model n DRAM space is divided such that it can hold each arriving packets in an address space »Called as buffer n SRAM keeps the queue descriptor (QD) and next pointers »QD is the set of head, tail addresses and length of any queue »Next pointers are the address of next buffer in a queue n We need two categories of queues »A queue to keep all the free buffers available (i.e. free buffer queue) »Another set of queues to keep the buffers which holds the packets belonging to various flows (i.e. virtual queues) –These enables isolation of flows

8 - Sailesh Kumar - 8/18/2015 A simple queuing model (cont)

9 - Sailesh Kumar - 8/18/2015 Queuing Operation n For each arriving packet »A buffer is dequeued from the free buffer queue »Packet is written into it »Buffer is enqueued into the appropriate queue »Queue descriptors are updated n Thus enqueue operation into any queue involves »Update of free queue descriptor –Read followed by write »Update of virtual queue descriptor –Read followed by write n Free queue descriptor is kept on-chip, so updates fast n However, virtual queue descriptors are off-chip and hence their updates are slow

10 - Sailesh Kumar - 8/18/2015 Queuing Operation in a NP n To achieve high throughput, a group of processors and associated threads are used to collectively perform the queuing »Each thread handles a Packet at a time and enqueues/dequeues it into the appropriate queue n When arriving/departing packets all belong to different queues, such a scheme effectively speeds up the operation linearly with increasing number of threads n However, when packets belong to the same queue, the entire operation gets serialized, and threads start competing for the same queue descriptor »Multiple processors/threads doesn’t result in any benefit

11 - Sailesh Kumar - 8/18/2015 Operation Read QD A UpdateWrite QD A Thread 0 Read QD B UpdateWrite QD B Read QD C UpdateWrite QD C Thread 1 Read QD D Update Read QD E UpdateWrite QD E Thread 2 Read QD F Read QD G UpdateWrite QD G Thread x Read QD H Read QD A UpdateWrite QD A Thread 0 Wait Wait for thread 0 UpdateWrite QD A Thread 1 Wait Thread 2 Wait Thread x Read QD A n What if all threads access the same queue n If all threads access different queues

12 - Sailesh Kumar - 8/18/2015 Solution n Accelerate the serialized operations »Use mechanisms which will enable serialized operations run relatively faster n This can be done by putting a small on chip cache to hold the queue descriptors currently being accessed n Thus all threads but the first thread will be able to update the queue descriptor relatively much faster »In situations where threads access different queue descriptors, the operation will go as it is »When threads access the same queue descriptor, even if the operation gets serialized, each operation will be very fast

13 - Sailesh Kumar - 8/18/2015 Queuing cache n Thus, queuing cache will sit between the memory hierarchy and MEs. »Whenever queue descriptors are accessed, they will be put into the cache n Questions »Size of cache? »Eviction policy? n Intuitively the size of cache should be same as the maximum number of threads that are collectively performing the queuing operation »Because only so many QDs can be accessed at a time n The eviction policy can be Least Recently Used (LRU)

14 - Sailesh Kumar - 8/18/2015 Operation with queuing cache Read QD A UpdateWrite QD LRU Thread 0 Read QD B UpdateWrite QD LRU Read QD C UpdateWrite QD LRU Thread 1 Read QD D Update Read QD E UpdateWrite QD LRU Thread 2 Read QD F Read QD G UpdateWrite QD LRU Thread x Read QD H Read QD A Update Thread 0 Wait Wait for QD A Thread 1 Wait Thread 2 Wait Thread x n If all threads access the same queue Update Wait Update n If all threads access different queues

15 - Sailesh Kumar - 8/18/2015 Performance comparison n For a 200 MHz DDR SRAM with SRAM access latency of 80 ns and queuing cache access latency of 20 ns n And assuming that processor takes 10 ns to execute all the queuing related instructions associated with a single packet

16 - Sailesh Kumar - 8/18/2015 Our design approach n Since queuing is so common in NPs, it may be a very good idea to add the hardware level support for enqueue and dequeue operations n Queuing cache is the best place to put these functionalities, because then the queuing will be very fast in situation where it get serialized n Thus each NP will support standard instructions, like enqueue, dequeue, etc »These instructions will be sent to the queuing cache »Queuing cache will internally manage the pointers and also handle any contention when threads access the same queue n Also threads themselves are relived from the burden of synchronization and pointer management and can operate independently

17 - Sailesh Kumar - 8/18/2015 Implementation

18 - Sailesh Kumar - 8/18/2015 Intel’s Approach n Intel’s second-generation IXP network processors have support for queuing via, »SRAM controller, which holds queue descriptors and implements queuing operations, and »MEs, which support enqueue and dequeue instructions n Caching of queue descriptors is implemented using »A Q-array in memory controller »Any queuing operation precedes a transfer of queue descriptors from SRAM to Q-array »A CAM in kept in each ME –To keep track of which QD are cached and their position in the Q- array n CAM supports LRU which is used to evict entries from the Q-array

19 - Sailesh Kumar - 8/18/2015 Comparison n Reduced instruction count on each processor »If we move all the logic associated with the enqueues and dequeues to the queuing cache, software may become simple n Simple and modular software code for queuing tasks »No need for synchronization, etc n Queuing cache built near the memory controller results in significantly reduced on chip communication »Since queuing cache handles the pointer processing as well, the processors needn’t fetch the queue descriptors at all »Only communication between processors and queuing cache is instruction exchange n More scalable »Any number of MEs can participate in queuing »No local CAM per ME needed unlike Intel’s IXP approach

20 - Sailesh Kumar - 8/18/2015 Conclusion n Contributions »Brief qualitative and quantitative analysis of queuing cache »A proposal for efficient and scalable design n Future work »Comparison to other caching technique »Implementation to measure the real complexity n We believe that such a cache based centralized queuing hardware unit will make the future network processors more »Scalable and »Easy to program n Questions?