David M. Zar Applied Research Laboratory Computer Science and Engineering Department ONL Freelist Manager

2 - David M. Zar - 12/25/2015 SRAM ONL NP Router Rx (2 ME) HdrFmt (1 ME) Parse, Lookup, Copy (3 MEs) TCAM SRAM Mux (1 ME) Tx (1 ME) QM (1 ME) xScale Assoc. Data ZBT-SRAM Plugin0Plugin1 Plugin2 Plugin3Plugin4 NN FreeList Mgr (1 ME) Tx, QM Parse Plugin XScale Stats (1 ME) QM Copy Plugins SRAM NN SRAM Ring Scratch Ring NN Ring NN SRAM 64KW Each

3 - David M. Zar - 12/25/2015 MEs -> FM (Freelist Manager) FM Rsv (8b) Buffer Handle(24b)

4 - David M. Zar - 12/25/2015 ONL DRAM Buffer and SRAM Buffer Descriptor Buffer_Next (32b) EtherType (16b) Packet_Next (32b) Reserved (4b) Free_list 0000 (4b) Ref_Cnt (8b) MAC DAddr_47_32 (16b)Stats Index (16b) MAC DAddr_31_00 (32b) Reserved (32b) Buffer_Size (16b) Packet_Size (16b) Offset (16b) Reserved (16b) 0x000 IP Packet Empty Ethernet Hdr 0x180 0x800 0x18E Normal Unicast case: »One copy of packet being sent to one output port SRAM Buffer Descriptor Fields: »Buffer_Next: NULL »Buffer_Size: IP_Pkt_Length »Packet_Size: IP_Pkt_Length »Offset: 0x18E »Freelist: 0 »Ref_Cnt: 1 »MAC_DAddr: »Stats Index: »EtherType: 0x0800 (IP) »Packet_Next:

5 - David M. Zar - 12/25/2015 ONL DRAM Buffer and SRAM Buffer Descriptor Buffer_Next (32b) EtherType (16b) Packet_Next (32b) Reserved (4b) Free_list 0000 (4b) Ref_Cnt (8b) MAC DAddr_47_32 (16b)Stats Index (16b) MAC DAddr_31_00 (32b) Reserved (32b) Buffer_Size (16b) Packet_Size (16b) Offset (16b) Reserved (16b) 0x000 Empty 0x180 0x800 0x18E Multi-copy case: »>1 copy of packet in system »This copy going from Copy to QM to go out on an output port Buffer_Next (32b) EtherType (16b) Packet_Next (32b) Reserved (4b) Free_list 0000 (4b) Ref_Cnt (8b) MAC DAddr_47_32 (16b)Stats Index (16b) MAC DAddr_31_00 (32b) Reserved (32b) Buffer_Size (16b) Packet_Size (16b) Offset (16b) Reserved (16b) 0x000 IP Packet Empty Ethernet Hdr 0x180 0x800 0x18E Header Buf DescriptorPayload Buf Descriptor

6 - David M. Zar - 12/25/2015 ONL DRAM Buffer and SRAM Buffer Descriptor Buffer_Next (32b) EtherType (16b) Packet_Next (32b) Reserved (4b) Free_list 0000 (4b) Ref_Cnt (8b) MAC DAddr_47_32 (16b)Stats Index (16b) MAC DAddr_31_00 (32b) Reserved (32b) Buffer_Size (16b) Packet_Size (16b) Offset (16b) Reserved (16b) 0x000 Empty 0x180 0x800 0x18E Buffer_Next (32b) EtherType (16b) Packet_Next (32b) Reserved (4b) Free_list 0000 (4b) Ref_Cnt (8b) MAC DAddr_47_32 (16b)Stats Index (16b) MAC DAddr_31_00 (32b) Reserved (32b) Buffer_Size (16b) Packet_Size (16b) Offset (16b) Reserved (16b) 0x000 IP Packet Empty Ethernet Hdr 0x180 0x800 0x18E Header Buf DescriptorPayload Buf Descriptor Multi-copy case (continued): »>1 copy of packet in system »This copy going from Copy to QM to go out on an output port Header Buf Descriptor: »SRAM Buffer Descriptor Fields: Buffer_Next: ptr to payload buf desc Buffer_Size: 0 (Don’t Care) Packet_Size: IP_Pkt_Length Offset: 0 (Don’t Care) Freelist: 0 Ref_Cnt: 1 MAC_DAddr: Stats Index: Ø Different copies of the same packet may actually have different Stats Indices EtherType: 0x0800 (IP) Packet_Next:

7 - David M. Zar - 12/25/2015 ONL DRAM Buffer and SRAM Buffer Descriptor Buffer_Next (32b) EtherType (16b) Packet_Next (32b) Reserved (4b) Free_list 0000 (4b) Ref_Cnt (8b) MAC DAddr_47_32 (16b)Stats Index (16b) MAC DAddr_31_00 (32b) Reserved (32b) Buffer_Size (16b) Packet_Size (16b) Offset (16b) Reserved (16b) 0x000 Empty 0x180 0x800 0x18E Buffer_Next (32b) EtherType (16b) Packet_Next (32b) Reserved (4b) Free_list 0000 (4b) Ref_Cnt (8b) MAC DAddr_47_32 (16b)Stats Index (16b) MAC DAddr_31_00 (32b) Reserved (32b) Buffer_Size (16b) Packet_Size (16b) Offset (16b) Reserved (16b) 0x000 IP Packet Empty Ethernet Hdr 0x180 0x800 0x18E Header Buf DescriptorPayload Buf Descriptor Multi-copy case (continued): »>1 copy of packet in system »This copy going from Copy to QM to go out on an output port Payload Buf Descriptor: »SRAM Buffer Descriptor Fields: Buffer_Next: NULL Buffer_Size: IP_Pkt_Length Packet_Size: IP_Pkt_Length Offset: 0x18E Freelist: 0 Ref_Cnt: MAC_DAddr: Stats Index: EtherType: Packet_Next:

8 - David M. Zar - 12/25/2015 FM() While (true) { dl_source_scr_1word() if (BufHandleNextBuffer = UC_NULL) { if (BufHandleRefCnt-- = 1) { WU_dl_buf_free(BufHandle) } else { // do nothing else… this is the standard case for TX transmitting // all but the last copy of a copied packet } } else { DataBuffer = BufHandleNextBuffer; if (BufHandleRefCnt != 1) { ERROR // RefCnt !=1 but this is pointing to a data buffer } else { BufHandle->NextBuffer = UC_NULL WU_dl_buf_free(BufHandle) if (DataBuffer->RefCnt-- = 1) WU_dl_buf_free(DataBuffer) } WU_dl_buf_free does the actual cleanup and enqueing of the SRAM buffer dl_buf_free will be modified to send commands to FM (???)

9 - David M. Zar - 12/25/2015 Performance Targets To hit 5 Gb rate: »76B per min IPv4 packet (64 min Enet Frame + 12B IFS) »1.4Ghz clock rate »5 Gb/sec * 1B/8b * packet/76B = 8.23 Mp/sec »1.4Gcycle/sec * 1 sec/ 8.23 Mp = 170 cycles per packet »compute budget: 170 cycles »latency budget: (threads*170) 8 threads: 1360 cycles

10 - David M. Zar - 12/25/2015 FM Block Diagram (worst case) Read Scratch Ring Set BufNext = UC_NULL WU_dl_buf_free SRAM Write: 1W 150 cycles mem access Latency ctx_swap --RefCnt = 0 SRAM Test-and-decr 150 cycles ctx_swap WU_dl_buf_free SRAM Enqueue TOTAL (No optimization) 510 cycles SCR Read: 1W 60 cycles Check BufHandle  NextBuffer SRAM Read: 1W 150 cycles

11 - David M. Zar - 12/25/2015 Lookup File locations Code »src/applications/ONL_Router/src/freelistMgr/freelistMgr.uc Include Paths »src/applications/ONL_Router/src/dispatch_loop/ONL/ dl_source.h and dl_source.uc Ø dl_source() and dl_sink() functions »Other, standard, include paths (Intel SDK provided)