Network Processor and Its Applications Yan Luo yluo@cs.ucr.edu Chris Baron cbaron@cs.ucr.edu 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Outline Overview of the network processors The NP market IXP2400/2800 High Performance Networking with IXP Initial Design To-do list 9/19/2018 Network Processor & Its Applications

What the Internet Needs? ASIC (large, expensive to develop, not flexible) Increasing Huge Amount of Packets & Routing, Packet Classification, Encryption, QoS, New Applications and Protocols, etc….. High processing power Support wire speed Programmable Scalable Specially for network applications … General Purpose RISC (not capable enough) 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications NP market Net Processor Market Share by Revenues 2003 share 2002 share AMCC 39% 38% Intel 16 11 IBM 13 19 Motorola 12 15 Vitesse 8 Agere 6 9 EZchip 2 < 1 Source: The Linley Group 9/19/2018 Network Processor & Its Applications

Typical NP Architecture SDRAM (Packet buffer) SRAM (control structures) multi-threaded processing elements Co-processor Input ports Output ports Network Processor Bus 9/19/2018 Network Processor & Its Applications

Some commercial NP products Wait, see Tilman Wolf’s slides 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Intel IXP2400 See another file 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Cluster Architecture Myrinet/Quadrics/etc. provide their own protocol (much simpler than TCP/IP) Limited to LAN Cost (Ethernet is cheap and standard) Future networks require high-speeds 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications System Architecture Pentium DRAM 850E PCI ` I21555 PCI bridge PCI Unit ME XScale ME SDRAM 256MB SRAM 8MB ENP2611 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Memory Spaces Virtual memory Physical memory PCI space memory Primary side Secondary side Bridge translates from one side to the other 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Memory Mapping 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications PCI Configuration 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Bridge Configuration Host->IXP2400 setup at boot time 00000000-0009ffff : System RAM 000a0000-000bffff : Video RAM area 000c0000-000c7fff : Video ROM 000f0000-000fffff : System ROM 00100000-1ff76fff : System RAM 00100000-0027588a : Kernel code 0027588b-002d90ff : Kernel data 1ff77000-1ff78fff : ACPI Non-volatile Storage 1ff79000-1fffffff : reserved 81882000-82882000 : Bigphysarea d0000000-efffffff : PCI Bus #02 d0000000-dfffffff : PCI device 8086:b555 (Intel Corp.) eff00000-efffffff : PCI device 8086:b555 (Intel Corp.) f0000000-f7ffffff : Intel Corp. 82850 850 (Tehama) Chipset Host Bridge (MCH) f8000000-fbffffff : PCI Bus #01 f8000000-fbffffff : ATI Technologies Inc Rage 128 Pro TF fec00000-fec0ffff : reserved fee00000-fee0ffff : reserved ff600000-ff7fffff : PCI Bus #02 ff6fec00-ff6fec7f : 3Com Corporation 3c905C-TX [Fast Etherlink] ff6ff000-ff6fffff : PCI device 8086:b555 (Intel Corp.) ff800000-ff9fffff : PCI Bus #01 ff8fc000-ff8fffff : ATI Technologies Inc Rage 128 Pro TF ffb00000-ffffffff : reserved 9/19/2018 Network Processor & Its Applications

What does the device driver(s) need to do? Identify Device Pci_table[] contains vendorid, deviceid, etc. Identify Device Resources pci_resource_start(dev, i) pci_resource_end(dev, i) Primary side maps secondary registers and SDRAM Secondary side maps host’s SDRAM (mempool) Request Device Resources request_mem_region(start, end) 9/19/2018 Network Processor & Its Applications

What does the device driver(s) need to do? Allocate Device Resources ioremap(start_phys, size) make_proc_entry(start_virt, size) Allocate Host Resources bigphysarea_alloc_pages(size) 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Bridge Configuration Direct Address Translation (DAT) Stick in figure here Alert secondary side off required offset (since the mempool may not start on a perfect boundary) Lookup Table (LUT) Bridge has 64 entries in the LUT Each entry is configurable up to 32MB 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Bridge Configuration Input address divided into three parts: Bit 31: base address Bits 30-25: lookup table index (2^6 = 64) Bits 24-0: offset (2^25 = 32MB) Highest 7 bits of LUT entry indexed by bits 30-25 of input address, concatenated with bits 24-0 of input address This is where we are having problems… 9/19/2018 Network Processor & Its Applications

Host->IXP2400 Transfer DMA or PIO? PIO provides zero-copy through remap_page_range() DMA requires an ioctl() call Copying data Standard memcpy() (8-bits) Memcpy32() (32-bits) memcpy(64() (64-bits using MMX registers) Utilizing write-combining 9/19/2018 Network Processor & Its Applications

IXP2400->Host Transfer DMA or PIO? DMA is required Give both channels to microengines Prefers 64-byte data transfers 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications DMA Descriptor 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications M-VIA 9/19/2018 Network Processor & Its Applications

Basic Theory of Operation Polling Host application places data into secondary SDRAM region Host application alerts secondary of data being placed and it’s location through mailbox register Host application polls receive mailbox register until it is updated by secondary side 9/19/2018 Network Processor & Its Applications

Basic Theory of Operation Interrupt Secondary must poll Secondary triggers interrupt on host side to alert host of new data When do you interrupt (how often)? Who provides the network interface? Speed Security 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Host->IXP1200: Host places packed into IXP1200 SDRAM, updates tail of buffer, host ingress polls for tail change, then creates entry in freelist, and descriptor in sram queue with pointers to packet and freelist entry. Once ether Egree is done, updates freelist entry. Freelist manager constantly polls freelist for free spaces, updates host ingree With available space, and host head and tail indexes. IXP1200->Host: ether egress places packet into single queue, host egress grab packet and use dma to send to host, along with the tail index. 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications END 9/19/2018 Network Processor & Its Applications

Network Architecture and Applications Introduction to networking IP packet structure Control and data planes Traditional applications IPv4 routing, classification etc. New applications URL-based switching,transcoding, etc. 9/19/2018 Network Processor & Its Applications

OSI Network Architecture DATA Application Pre. Session Transport Network Data Link Physical 7 6 5 4 3 2 1 AH PH SH TH NH DH A B Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications TCP/IP Model Application Pre. Session Transport Network Data Link Physical 7 6 5 4 3 2 1 TCP IP Host-to-Net OSI TCP/IP Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik ISO OSI (Open Systems Interconnection) not fully implemented Presentation and Session layers not present in TCP/IP 9/19/2018 Network Processor & Its Applications

TCP/IP packet TCP IP MAC APP. DATA Source Port Destination Port Sequence Number Acknowledgement Number Header Length and Options Window Size Checksum Urgent Pointer Options (0 or more 32-bit words) Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik 9/19/2018 Network Processor & Its Applications

TCP/IP packet TCP IP MAC APP. DATA Ver. Total Length Identification Time to Live Source Address IHL Service Type Options and Fragment Offset Protocol Header Checksum Options (0 or more 32-bit words) Destination Address Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications TCP/IP packet MAC IP TCP APP. DATA MAC Preamble D. Add. S. Add. CRC Leng. Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik 9/19/2018 Network Processor & Its Applications

Application Categorization Control-Plane tasks Less time-critical Control and management of device operation Table maintenance, port states, etc. Data-Plane tasks Operations occurring real-time on “packet path” Core device operations Receive, process and transmit packets Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Processing Tasks Policy Applications Network Management Signaling Topology Management Queuing / Scheduling Data Transformation Classification Data Parsing Media Access Control Physical Layer Data Plane Control Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Data Plane Tasks Media Access Control Low-level protocol implementation Ethernet, SONET framing, ATM cell processing, etc. Data Parsing Parsing cell or packet headers for address or protocol information Classification Identify packet against a criteria (filtering / forwarding decision, QoS, accounting, etc.) Data Transformation Transformation of packet data between protocols Traffic Management Queuing, scheduling and policing packet data Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik 9/19/2018 Network Processor & Its Applications

Network Processor Applications Routing table lookup Determine the next hop for incoming packets Packet Classification classify packets using header fields against a set of rules URL-based Switching Distribute HTTP requests based on URLs. Transcoding Encryption/Decryption, intrusion detection, firewall, access control checking, denial-of-service 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications IPv4 Routing P P P B A Router C Routers determine next hop and forward packets 9/19/2018 Network Processor & Its Applications

Trie-based Routing Table Lookup rt_ptr trie_ptr Trie block Next hop 3 Next hop 4 15 Next hop 1 Next hop 2 Prefix Netmask Next hop 0010 0xF 1 0001,0011 0xFF 2 1110,1110 0001,0011,0000 0xFFF 3 0001,0011,1111,111 0xFFFE 4 Trie block keeps pointers to route entry and other trie blocks Destination IP address bits are examined group by group (4-bit) 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Example rt_ptr trie_ptr Next hop 3 Next hop 4 15 Next hop 1 Next hop 2 Prefix Netmask Next hop 0010 0xF 1 0001,0011 0xFF 2 1110,1110 0001,0011,0000 0xFFF 3 0001,0011,1111,111 0xFFFE 4 Packet destination IP address =0x13fe2233 (0001,0011,1111,1110,…) 9/19/2018 Network Processor & Its Applications

IP Lookups using Multi-way Multi-column Search Encoding prefixes as ranges 1* -> [100000,111111] 101* -> [101000,101111] 10101* -> [101010,101011] 1 0 0 0 0 0 1 0 1 0 0 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 1 1 1 1 1 1 1 101011 101110 111110 Illustration of the idea with 6-bit address Prefixes: 1* 101 * 10101* 1 0 0 0 0 0 1 0 1 0 0 0 1 0 1 0 1 0 101011 101110 111110 L H L,H Narrowest enclosing Range containing A A Binary search does not work with variable length strings. end up far away from the matching prefix Multiple addresses that match to different prefix, end up in the same region 9/19/2018 Network Processor & Its Applications

Multi-way multi-column search k3 k6 k1 k2 k4 k5 k7 k8 info 3-way tree for 8 keys P1: 1* -> [100000,111111] P2: 101* -> [101000,101111] P3: 10101* -> [101010,101011] > = p1) 1 0 0 0 0 0 p1 p1 p2) 1 0 1 0 0 0 p2 p2 p3) 1 0 1 0 1 0 p3 p3 1 0 1 0 1 1 p2 p3 1 0 1 1 1 1 p1 p2 1 1 1 1 1 1 - p1 Any region in the binary search between two consecutive numbers corresponds to a unique prefix Multi-column search is used for wide address such as IPv6 A C E A D C A M W B M W B N X B N Y W/M words Probe 1 2 3 9/19/2018 Network Processor & Its Applications

Packet Classification Routers are required to distinguish packets for Flow identification Fair sharing of bandwidth QoS Security Accounting, billing etc Packets are classified by rules Src IP, Dest IP, src port #, dest port # etc Classification Algorithm Metrics Search speed Storage cost Scalability Updates Etc. 9/19/2018 Network Processor & Its Applications

Classification: Hierarchical tries rule F1 F2 R1 00* R2 0* 01* R3 1* R4 R5 R6 * Pankaj Gupta and Nick McKeown,"Algorithms for Packet Classification", IEEE Network Special Issue, March/April 2001, vol. 15, no. 2, pp 24-32. Extension of the one dimensional radix trie Construct trie recursively: Contruct F1-trie on the set of prefix {Rj1} For each prefix p in F1-trie, we recursively construct (d-1) dimensional hierarchical trie on rules where {Rj:Rj1=p} 9/19/2018 Network Processor & Its Applications

Classification: Bitmap-intersection The set of rules S that a packet matches is the intersection of d sets, Si Where Si is the set of rules that match the packet in the i-th dimension alone. Pankaj Gupta and Nick McKeown,"Algorithms for Packet Classification", IEEE Network Special Issue, March/April 2001, vol. 15, no. 2, pp 24-32. 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications URL-based switching www.yahoo.com Internet Image Server IP TCP APP. DATA Application Server GET /cgi-bin/form HTTP/1.1 Host: www.yahoo.com… Switch Increase efficiency Tasks Traverse the packet data (request) for each arriving packet and classify it: Contains ‘.jpg’ -> to image server Contains ‘cgi-bin/’ -> to application server HTML Server Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik 9/19/2018 Network Processor & Its Applications

Transcoders Two important requirements If the receiver is not capable of interpreting the stored data (multimedia transcoders) wireless receivers, hand-held devices, etc. Compression for bandwidth and storage efficiency Internet Transcoder Video-on-demand server Mpeg encoder Corporate Network Media Player Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Summary NP is developing very fast and is a hot research area Multithreaded NP Architectures provide tremendous packet processing capability NP can be applied in various network layers and applications Traditional apps – forwarding, classification Advanced apps – transcoding, URL-based switching, security etc. New apps 9/19/2018 Network Processor & Its Applications

Network Processor & Its Applications Reference Pankaj Gupta and Nick McKeown,"Algorithms for Packet Classification", IEEE Network Special Issue, March/April 2001, vol. 15, no. 2, pp 24-32. Bill Mangione-Smith, Gokhan Memik, Network Processing: Applications, Architectures and Examples – Tutorial at Micro 34, Dec. 2001 B. Lampson, V. Srinivasan and G. Varghese, IP Lookups using Multiway and Multicolumn Search, IEEE/ACM Transactions on Networking, August 1, 1997 Intel IXP1200 Network Processor Family - Development Tools User's Guide, Intel 9/19/2018 Network Processor & Its Applications

Why are Network Processors useful? Flexible but fast Heat dissipation Opteron 248 2.2Ghz CPU with 1GB RAM and 10Gb/s Ethernet Card CPU maxed out at ~5Gb/s With TOE, achieved 7.4 Gb/s (PCI-X limit) with < 50% of CPU utilization 9/19/2018 Network Processor & Its Applications