1. Network Processor and Its Applications
Yan Luo
Chris Baron
9/19/2018
Network Processor & Its Applications

2. Network Processor & Its Applications
Outline
Overview of the network processors
The NP market
IXP2400/2800
High Performance Networking with IXP
Initial Design
To-do list
9/19/2018
Network Processor & Its Applications

3. What the Internet Needs?
ASIC
(large,
expensive to develop,
not flexible)
Increasing Huge
Amount of Packets
&
Routing,
Packet Classification,
Encryption, QoS,
New Applications
and Protocols, etc…..
High processing power
Support wire speed
Programmable
Scalable
Specially for network applications …
General Purpose RISC
(not capable enough)
9/19/2018
Network Processor & Its Applications

4. Network Processor & Its Applications
NP market
Net Processor Market Share by Revenues
2003 share
2002 share
AMCC
39%
38%
Intel 16 11
IBM 13 19
Motorola 12 15
Vitesse 8
Agere 6 9
EZchip 2
< 1
Source: The Linley Group
9/19/2018
Network Processor & Its Applications

5. Typical NP Architecture
SDRAM
(Packet buffer)
SRAM
(control structures)
multi-threaded processing elements
Co-processor
Input ports
Output ports
Network Processor
Bus
9/19/2018
Network Processor & Its Applications

6. Some commercial NP products
Wait, see Tilman Wolf’s slides
9/19/2018
Network Processor & Its Applications

7. Network Processor & Its Applications
Intel IXP2400
See another file
9/19/2018
Network Processor & Its Applications

8. Network Processor & Its Applications
Cluster Architecture
Myrinet/Quadrics/etc. provide their own protocol (much simpler than TCP/IP)
Limited to LAN
Cost (Ethernet is cheap and standard)
Future networks require high-speeds
9/19/2018
Network Processor & Its Applications

9. Network Processor & Its Applications
System Architecture
Pentium
DRAM
850E
PCI `
I21555
PCI bridge
PCI Unit ME
XScale ME
SDRAM
256MB
SRAM
8MB
ENP2611
9/19/2018
Network Processor & Its Applications

10. Network Processor & Its Applications
Memory Spaces
Virtual memory
Physical memory
PCI space memory
Primary side
Secondary side
Bridge translates from one side to the other
9/19/2018
Network Processor & Its Applications

11. Network Processor & Its Applications
Memory Mapping
9/19/2018
Network Processor & Its Applications

12. Network Processor & Its Applications
PCI Configuration
9/19/2018
Network Processor & Its Applications

13. Network Processor & Its Applications
Bridge Configuration
Host->IXP2400 setup at boot time
ffff : System RAM
000a bffff : Video RAM area
000c c7fff : Video ROM
000f fffff : System ROM
ff76fff : System RAM
a : Kernel code
b-002d90ff : Kernel data
1ff ff78fff : ACPI Non-volatile Storage
1ff fffffff : reserved
: Bigphysarea
d efffffff : PCI Bus #02
d dfffffff : PCI device 8086:b555 (Intel Corp.)
eff00000-efffffff : PCI device 8086:b555 (Intel Corp.)
f f7ffffff : Intel Corp (Tehama) Chipset Host Bridge (MCH)
f fbffffff : PCI Bus #01
f fbffffff : ATI Technologies Inc Rage 128 Pro TF
fec00000-fec0ffff : reserved
fee00000-fee0ffff : reserved
ff ff7fffff : PCI Bus #02
ff6fec00-ff6fec7f : 3Com Corporation 3c905C-TX [Fast Etherlink]
ff6ff000-ff6fffff : PCI device 8086:b555 (Intel Corp.)
ff ff9fffff : PCI Bus #01
ff8fc000-ff8fffff : ATI Technologies Inc Rage 128 Pro TF
ffb00000-ffffffff : reserved
9/19/2018
Network Processor & Its Applications

14. What does the device driver(s) need to do?
Identify Device
Pci_table[] contains vendorid, deviceid, etc.
Identify Device Resources
pci_resource_start(dev, i)
pci_resource_end(dev, i)
Primary side maps secondary registers and SDRAM
Secondary side maps host’s SDRAM (mempool)
Request Device Resources
request_mem_region(start, end)
9/19/2018
Network Processor & Its Applications

15. What does the device driver(s) need to do?
Allocate Device Resources
ioremap(start_phys, size)
make_proc_entry(start_virt, size)
Allocate Host Resources
bigphysarea_alloc_pages(size)
9/19/2018
Network Processor & Its Applications

16. Network Processor & Its Applications
Bridge Configuration
Direct Address Translation (DAT)
Stick in figure here
Alert secondary side off required offset (since the mempool may not start on a perfect boundary)
Lookup Table (LUT)
Bridge has 64 entries in the LUT
Each entry is configurable up to 32MB
9/19/2018
Network Processor & Its Applications

17. Network Processor & Its Applications
Bridge Configuration
Input address divided into three parts:
Bit 31: base address
Bits 30-25: lookup table index (2^6 = 64)
Bits 24-0: offset (2^25 = 32MB)
Highest 7 bits of LUT entry indexed by bits of input address, concatenated with bits 24-0 of input address
This is where we are having problems…
9/19/2018
Network Processor & Its Applications

18. Host->IXP2400 Transfer
DMA or PIO?
PIO provides zero-copy through remap_page_range()
DMA requires an ioctl() call
Copying data
Standard memcpy() (8-bits)
Memcpy32() (32-bits)
memcpy(64() (64-bits using MMX registers)
Utilizing write-combining
9/19/2018
Network Processor & Its Applications

19. IXP2400->Host Transfer
DMA or PIO?
DMA is required
Give both channels to microengines
Prefers 64-byte data transfers
9/19/2018
Network Processor & Its Applications

20. Network Processor & Its Applications
DMA Descriptor
9/19/2018
Network Processor & Its Applications

21. Network Processor & Its Applications
M-VIA
9/19/2018
Network Processor & Its Applications

22. Basic Theory of Operation
Polling
Host application places data into secondary SDRAM region
Host application alerts secondary of data being placed and it’s location through mailbox register
Host application polls receive mailbox register until it is updated by secondary side
9/19/2018
Network Processor & Its Applications

23. Basic Theory of Operation
Interrupt
Secondary must poll
Secondary triggers interrupt on host side to alert host of new data
When do you interrupt (how often)?
Who provides the network interface?
Speed
Security
9/19/2018
Network Processor & Its Applications

24. Network Processor & Its Applications
Host->IXP1200: Host places packed into IXP1200 SDRAM, updates tail of buffer, host ingress polls for tail change, then creates entry in freelist, and descriptor in sram queue with pointers to packet and freelist entry. Once ether
Egree is done, updates freelist entry. Freelist manager constantly polls freelist for free spaces, updates host ingree
With available space, and host head and tail indexes.
IXP1200->Host: ether egress places packet into single queue, host egress grab packet and use dma to send to host, along with the tail index.
9/19/2018
Network Processor & Its Applications

25. Network Processor & Its Applications
END
9/19/2018
Network Processor & Its Applications

26. Network Architecture and Applications
Introduction to networking
IP packet structure
Control and data planes
Traditional applications
IPv4 routing, classification etc.
New applications
URL-based switching,transcoding, etc.
9/19/2018
Network Processor & Its Applications

27. OSI Network Architecture
DATA
Application
Pre.
Session
Transport
Network
Data Link
Physical 7 6 5 4 3 2 1 AH PH SH TH NH DH A B
Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik
9/19/2018
Network Processor & Its Applications

28. Network Processor & Its Applications
TCP/IP Model
Application
Pre.
Session
Transport
Network
Data Link
Physical 7 6 5 4 3 2 1
TCP IP
Host-to-Net
OSI
TCP/IP
Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik
ISO OSI (Open Systems Interconnection) not fully implemented
Presentation and Session layers not present in TCP/IP
9/19/2018
Network Processor & Its Applications

29. TCP/IP packet TCP IP MAC APP. DATA Source Port Destination Port
Sequence Number
Acknowledgement Number
Header Length and Options
Window Size
Checksum
Urgent Pointer
Options (0 or more 32-bit words)
Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik
9/19/2018
Network Processor & Its Applications

30. TCP/IP packet TCP IP MAC APP. DATA Ver. Total Length Identification
Time to Live
Source Address
IHL
Service Type
Options and Fragment Offset
Protocol
Header Checksum
Options (0 or more 32-bit words)
Destination Address
Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik
9/19/2018
Network Processor & Its Applications

31. Network Processor & Its Applications
TCP/IP packet
MAC IP
TCP
APP. DATA
MAC
Preamble
D. Add.
S. Add.
CRC
Leng.
Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik
9/19/2018
Network Processor & Its Applications

32. Application Categorization
Control-Plane tasks
Less time-critical
Control and management of device operation
Table maintenance, port states, etc.
Data-Plane tasks
Operations occurring real-time on “packet path”
Core device operations
Receive, process and transmit packets
Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik
9/19/2018
Network Processor & Its Applications

33. Network Processor & Its Applications
Processing Tasks
Policy Applications
Network Management
Signaling
Topology Management
Queuing / Scheduling
Data Transformation
Classification
Data Parsing
Media Access Control
Physical Layer
Data
Plane
Control
Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik
9/19/2018
Network Processor & Its Applications

34. Network Processor & Its Applications
Data Plane Tasks
Media Access Control
Low-level protocol implementation
Ethernet, SONET framing, ATM cell processing, etc.
Data Parsing
Parsing cell or packet headers for address or protocol information
Classification
Identify packet against a criteria (filtering / forwarding decision, QoS, accounting, etc.)
Data Transformation
Transformation of packet data between protocols
Traffic Management
Queuing, scheduling and policing packet data
Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik
9/19/2018
Network Processor & Its Applications

35. Network Processor Applications
Routing table lookup
Determine the next hop for incoming packets
Packet Classification
classify packets using header fields against a set of rules
URL-based Switching
Distribute HTTP requests based on URLs.
Transcoding
Encryption/Decryption, intrusion detection, firewall, access control checking, denial-of-service
9/19/2018
Network Processor & Its Applications

36. Network Processor & Its Applications
IPv4 Routing P P P B A
Router C
Routers determine next hop and forward packets
9/19/2018
Network Processor & Its Applications

37. Trie-based Routing Table Lookup
rt_ptr
trie_ptr
Trie block
Next hop 3
Next hop 4 15
Next hop 1
Next hop 2
Prefix
Netmask
Next hop
0010
0xF 1
0001,0011
0xFF 2
1110,1110
0001,0011,0000
0xFFF 3
0001,0011,1111,111
0xFFFE 4
Trie block keeps pointers to route entry and other trie blocks
Destination IP address bits are examined group by group (4-bit)
9/19/2018
Network Processor & Its Applications

38. Network Processor & Its Applications
Example
rt_ptr
trie_ptr
Next hop 3
Next hop 4 15
Next hop 1
Next hop 2
Prefix
Netmask
Next hop
0010
0xF 1
0001,0011
0xFF 2
1110,1110
0001,0011,0000
0xFFF 3
0001,0011,1111,111
0xFFFE 4
Packet destination IP address =0x13fe2233
(0001,0011,1111,1110,…)
9/19/2018
Network Processor & Its Applications

39. IP Lookups using Multi-way Multi-column Search
Encoding prefixes as ranges
1* -> [100000,111111]
101* -> [101000,101111]
10101* -> [101010,101011]
101011
101110
111110
Illustration of the idea with 6-bit address
Prefixes: 1*
101 *
10101*
101011
101110
111110 L H
L,H
Narrowest enclosing
Range containing A A
Binary search does not work with variable length strings.
end up far away from the matching prefix
Multiple addresses that match to different prefix, end up in the same region
9/19/2018
Network Processor & Its Applications

40. Multi-way multi-column search
k3 k6
k1 k2
k4 k5
k7 k8
info
3-way tree for 8 keys
P1: 1* -> [100000,111111]
P2: 101* -> [101000,101111]
P3: 10101* -> [101010,101011]
> =
p1) p1 p1
p2) p2 p2
p3) p3 p3
p2 p3
p1 p2 p1
Any region in the binary search
between two consecutive numbers
corresponds to a unique prefix
Multi-column search
is used for wide address such as IPv6
A C E
A D C
A M W
B M W
B N X
B N Y
W/M words
Probe 1 2 3
9/19/2018
Network Processor & Its Applications

41. Packet Classification
Routers are required to distinguish packets for
Flow identification
Fair sharing of bandwidth
QoS
Security
Accounting, billing
etc
Packets are classified by rules
Src IP, Dest IP, src port #, dest port # etc
Classification Algorithm Metrics
Search speed
Storage cost
Scalability
Updates
Etc.
9/19/2018
Network Processor & Its Applications

42. Classification: Hierarchical tries
rule F1 F2 R1
00* R2 0*
01* R3 1* R4 R5 R6 *
Pankaj Gupta and Nick McKeown,"Algorithms for Packet Classification",
IEEE Network Special Issue, March/April 2001, vol. 15, no. 2, pp
Extension of the one dimensional radix trie
Construct trie recursively:
Contruct F1-trie on the set of prefix {Rj1}
For each prefix p in F1-trie, we recursively construct (d-1) dimensional hierarchical trie on rules where {Rj:Rj1=p}
9/19/2018
Network Processor & Its Applications

43. Classification: Bitmap-intersection
The set of rules S that a packet matches is the intersection of d sets, Si
Where Si is the set of rules that match the packet in the i-th dimension alone.
Pankaj Gupta and Nick McKeown,"Algorithms for Packet Classification",
IEEE Network Special Issue, March/April 2001, vol. 15, no. 2, pp
9/19/2018
Network Processor & Its Applications

44. Network Processor & Its Applications
URL-based switching
Internet
Image Server IP
TCP
APP. DATA
Application Server
GET /cgi-bin/form HTTP/1.1
Host:
Switch
Increase efficiency
Tasks
Traverse the packet data (request) for each arriving packet and classify it:
Contains ‘.jpg’ -> to image server
Contains ‘cgi-bin/’ -> to application server
HTML Server
Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik
9/19/2018
Network Processor & Its Applications

45. Transcoders Two important requirements
If the receiver is not capable of interpreting the stored data (multimedia transcoders)
wireless receivers, hand-held devices, etc.
Compression for bandwidth and storage efficiency
Internet
Transcoder
Video-on-demand server
Mpeg encoder
Corporate Network
Media Player
Source: Network Processor Tutorial in Micro 34 - Mangione-Smith & Memik
9/19/2018
Network Processor & Its Applications

46. Network Processor & Its Applications
Summary
NP is developing very fast and is a hot research area
Multithreaded NP Architectures provide tremendous packet processing capability
NP can be applied in various network layers and applications
Traditional apps – forwarding, classification
Advanced apps – transcoding, URL-based switching, security etc.
New apps
9/19/2018
Network Processor & Its Applications

47. Network Processor & Its Applications
Reference
Pankaj Gupta and Nick McKeown,"Algorithms for Packet Classification", IEEE Network Special Issue, March/April 2001, vol. 15, no. 2, pp
Bill Mangione-Smith, Gokhan Memik, Network Processing: Applications, Architectures and Examples – Tutorial at Micro 34, Dec. 2001
B. Lampson, V. Srinivasan and G. Varghese, IP Lookups using Multiway and Multicolumn Search, IEEE/ACM Transactions on Networking, August 1, 1997
Intel IXP1200 Network Processor Family - Development Tools User's Guide, Intel
9/19/2018
Network Processor & Its Applications

48. Why are Network Processors useful?
Flexible but fast
Heat dissipation
Opteron Ghz CPU with 1GB RAM and 10Gb/s Ethernet Card
CPU maxed out at ~5Gb/s
With TOE, achieved 7.4 Gb/s (PCI-X limit) with < 50% of CPU utilization
9/19/2018
Network Processor & Its Applications