Download presentation
Presentation is loading. Please wait.
1
CS 838: NetFPGA Tutorial Theophilus Benson
2
Outline Background: What is the NetFPGA?
Life cycle of a packet through a NetFPGA Demo
3
What is the NetFPGA? Networking Software running on a standard PC
CPU Memory PCI A hardware accelerator built with Field Programmable Gate Array driving Gigabit network links FPGA Memory 1GE
4
NetFPGA Router Open-source FPGA hardware --
Function 4 Gigabit Ethernet ports Fully programmable FPGA hardware Open-source FPGA hardware -- Verilog base design Open-source Software -- Linux user Level Drivers in C and C++
5
NetFPGA Platform Major Components Interfaces Memories FPGA Resources
4 Gigabit Ethernet Ports PCI Host Interface Memories 36Mbits Static RAM 512Mbits DDR2 Dynamic RAM FPGA Resources Block RAMs Configurable Logic Block (CLBs) Memory Mapped Registers
6
NetFGPA: Router Design
Pipeline of modules FIFO queues between each module Inter module communication CTRL: Send on ctrl bus (8 bits) Metadata about the data being send DATA: Send on data bus (64 bits) RDY: Signifies ready to receive packet (1 bit) WR: Signifies packet being send(1bit)
7
NetFPGA Linux user-level Software processes Verilog on
Hardware Linux user-level processes Linux Processes Verilog on NetFPGA PCI board FGPA Modules 1 FGPA Modules 2
8
Example: An IP Router on NetFPGA
Software Hardware Management & CLI Linux user-level processes Routing Protocols Exception Processing Routing Table Verilog on NetFPGA PCI board Forwarding Table Switching
9
Life of a Packet through the hardware
port0 port2 y x IP packet
10
Router Stages MAC RxQ CPU Input Arbiter Output Port Lookup TxQ
Output Queues
11
Inter-module Communication
Using “Module Headers”: Ctrl Word (8 bits) Data Word (64 bits) x Module Hdr Contain information such as packet length, input port, output port, … … … y Last Module Hdr Eth Hdr IP Hdr … 0x10 Last word of packet
12
Inter-module Communication
Module i Module i+1 data ctrl wr rdy
13
Dst MAC = port 0, Ethertype = IP
MAC Rx Queue MAC Rx Queue IP Hdr: IP Dst: , TTL: 64, Csum:0x3ab4 Eth Hdr: Dst MAC = port 0, Ethertype = IP Data
14
Dst MAC = port 0, Ethertype = IP
Rx Queue Rx Queue IP Hdr: IP Dst: , TTL: 64, Csum:0x3ab4 Eth Hdr: Dst MAC = port 0, Ethertype = IP Data Pkt length, input port = 0 0xff
15
Input Arbiter Rx Q 7 Input Arbiter Pkt … Rx Q 1 Pkt Rx Q 0 Pkt
16
Output Port Lookup Data Pkt length, Output Port Lookup 0xff
IP Hdr: IP Dst: , TTL: 64, Csum:0x3ab4 EthHdr: Dst MAC = 0 Src MAC = x, Ethertype = IP Data Pkt length, input port = 0 0xff
17
Output Port Lookup 5- Add output port module
1- Check input port matches Dst MAC Output Port Lookup 0x04 output port = 4 6- Modify MAC Dst and Src addresses 2- Check TTL, checksum 0xff Pkt length, input port = 0 EthHdr: Dst MAC = nextHop Src MAC = port 4, Ethertype = IP EthHdr: Dst MAC = 0 Src MAC = x, Ethertype = IP 3- Lookup next hop IP & output port (LPM) IP Hdr: IP Dst: , TTL: 63, Csum:0x3ac2 IP Hdr: IP Dst: , TTL: 64, Csum:0x3ab4 7-Decrement TTL and update checksum 4- Lookup next hop MAC address (ARP) Data
18
Output Queues Output Queues OQ0 OQ4 Pkt OQ7
19
EthHdr: Dst MAC = nextHop Src MAC = port 4,
MAC Tx Queue MAC Tx Queue IP Hdr: IP Dst: , TTL: 64, Csum:0x3ab4 IP Dst: , TTL: 63, Csum:0x3ac2 EthHdr: Dst MAC = nextHop Src MAC = port 4, Ethertype = IP Data Pkt length, input port = 0 0xff output port = 4 0x04
20
EthHdr: Dst MAC = nextHop Src MAC = port 4,
MAC Tx Queue MAC Tx Queue 0x04 output port = 4 0xff Pkt length, input port = 0 EthHdr: Dst MAC = nextHop Src MAC = port 4, Ethertype = IP IP Hdr: IP Dst: , TTL: 64, Csum:0x3ab4 IP Hdr: IP Dst: , TTL: 63, Csum:0x3ac2 Data
21
NetFPGA-Host Interaction
Linux driver interfaces with hardware Packet interface via standard Linux network stack Register reads/writes via ioctl system call (with convenience wrapper functions) readReg(nf2device *dev, int address, unsigned *rd_data) writeReg(nf2device *dev, int address, unsigned *wr_data) eg: readReg(&nf2, OQ_NUM_PKTS_STORED_0, &val);
22
NetFPGA-Host Interaction
Register access 2. Driver performs PCI memory read/write PCI Bus 1. Software makes ioctl call on network socket. ioctl passed to driver.
23
NetFPGA-Host Interaction
Packet transfers shown using DMA interface Alternative: use programmed IO to transfer packets via register reads/writes slower but eliminates the need to deal with network sockets
24
DEMO: Life of a Packet through the hardware
port0 port2 y x IP packet
25
Programming the FPGA with your code Mirror linux arp Helpful tool
nf2_download NF2/bitfiles/reference_router.bit Mirror linux arp ./NF2/projects/router_kit/sw/rkd Helpful tool ./NFlib/C/router/cli Shows forwarding tables {arp table, ip table} Allows to modify tables
26
Useful Links NetFPGA Website NetFPGA Wiki NetFPGA Guide
Walkthrough the Reference Designs The Verilog Golden Reference Guide
27
Questions
28
Verilog
29
Hardware Description Languages
Concurrent By Default, Verilog statements evaluated concurrently Express fine grain parallelism Allows gate-level parallelism Provides Precise Description Eliminates ambiguity about operation Synthesizable Generates hardware from description
30
Verilog Data Types reg [7:0] A; // 8-bit register, MSB to LSB // (Preferred bit order for NetFPGA) reg [0:15] B; // 16-bit register, LSB to MSB B = {A[7:0],A[0:7]}; // Assignment of bits reg [31:0] Mem [0:1023]; // 1K Word Memory integer Count; // simple signed 32-bit integer integer K[1:64]; // an array of 64 integers time Start, Stop; // Two 64-bit time variables From: CSCI 320 Computer Architecture Handbook on Verilog HDL, by Dr. Daniel C. Hyde :
![Verilog Data Types reg [7:0] A; // 8-bit register, MSB to LSB // (Preferred bit order for NetFPGA)](http://slideplayer.com./slide/5850364/19/images/30/Verilog+Data+Types+reg+%5B7%3A0%5D+A%3B+%2F%2F+8-bit+register%2C+MSB+to+LSB+%2F%2F+%28Preferred+bit+order+for+NetFPGA%29.jpg "reg [0:15] B; // 16-bit register, LSB to MSB. B = {A[7:0],A[0:7]}; // Assignment of bits. reg [31:0] Mem [0:1023]; // 1K Word Memory. integer Count; // simple signed 32-bit integer. integer K[1:64]; // an array of 64 integers. time Start, Stop; // Two 64-bit time variables. From: CSCI 320 Computer Architecture. Handbook on Verilog HDL, by Dr. Daniel C. Hyde :")
31
Signal Multiplexers Two input multiplexer (using if / else) reg y;
if (select) y = a; else y = b; Two input multiplexer (using ternary operator ?:) wire t = (select ? a : b); From:
32
Larger Multiplexers Three input multiplexer reg s;
begin case (select2) 2'b00: s = a; 2'b01: s = b; default: s = c; endcase end
33
Synchronous Storage Elements
Values change at times governed by clock Clock Din Dout Q D Clock Transition t=0 t=1 t=2 1 Clock time Clock Input to circuit Clock Event Example: Rising edge Din A B C t=0 Flip/Flop Transfers Value From Din to Dout on Clock event Clock Transition Dout A B S0 t=0
34
Finite State Machines
35
Synthesizable Verilog : Delay Flip/Flops
D-type flip flop reg q; (posedge clk) q <= d; D type flip flop with data enable reg q; (posedge clk) if (enable) q <= d; From:
36
More on NetFPGA System
37
NetFPGA System User Space Linux Kernel NIC (nf2c0 .. 3) (eth1 .. 2)
CAD Tools Monitor Software Web & Video Server Browser & Video Client Packet Forwarding Table PCI PCI-e VI VI VI VI NIC NetFPGA Router Hardware GE GE GE GE GE GE (nf2c0 .. 3) (eth1 .. 2)
38
NetFPGA System Implementation
NetFPGA Blocks Virtex-2 Pro FPGA 4.5MB ZBT SRAM 64MB DDR2 DRAM PCI Host Interface 4 Gigabit Ethernet ports Intranet Test Ports Dual or Quad Gigabit Etherents on PCI-e Internet Gigabit Ethernet on Motherboard Processor Dual-Core CPU Operating System Linux CentOS 4.4
39
NetFPGA Lab Setup Dual NIC CPU x2 Net-FPGA (eth1 .. 2) Client Server
Eth2 : Server PCI-e GE (eth1 .. 2) Eth1 : Local host Server GE Net-FPGA GE Nf2c3 : Adj. Server NetFPGA Control SW PCI Internet Router Hardware Nf2c2 : Local Host GE GE Nf2c1 : Adjacent Nf2c0 : Adjacent CAD Tools GE
40
Exception Path
41
Exception Packet Example: TTL = 0 or TTL = 1
Packet has to be sent to the CPU which will generate an ICMP packet as a response Difference starts at the Output Port lookup stage
42
Exception Packet Path Software PCI Bus PW-OSPF Java GUI Driver DMA
NetFPGA PW-OSPF Java GUI Driver CPU RxQ TxQ nf2_reg_grp user data path DMA Registers nf2c0 nf2c1 nf2c2 nf2c3 ioctl MAC Ethernet
43
Output Port Lookup 1- Check input port matches Dst MAC
0x04 output port = 1 2- Check TTL, checksum – EXCEPTION! 0xff Pkt length, input port = 0 EthHdr: Dst MAC = 0, Src MAC = x, Ethertype = IP IP Hdr: IP Dst: , TTL: 1, Csum:0x3ab4 3- Add output port module Data
44
Output Queues Output Queues OQ0 OQ1 OQ2 Pkt OQ7
45
CPU Tx Queue Data CPU Tx Queue IP Hdr:
IP Dst: , TTL: 64, Csum:0x3ab4 IP Dst: , TTL: 1, Csum:0x3ab4 EthHdr: Dst MAC = 0, Src MAC = x, Ethertype = IP Data Pkt length, input port = 0 0xff output port = 1 0x04
46
CPU Tx Queue Data CPU Tx Queue 0x04 output port = 1 0xff Pkt length,
input port = 0 EthHdr: Dst MAC = 0, Src MAC = x, Ethertype = IP IP Hdr: IP Dst: , TTL: 1, Csum:0x3ab4 Data
47
ICMP Packet For the ICMP packet, the packet arrives at the CPU Rx Queue from the PCI Bus Follows the same path as a packet from the MAC until the Output Port Lookup. The OPL module seeing the packet is from the CPU Rx Queue 1, sets the output port directly to 0. The packet then continues on the same path as the non-exception packet to the Output Queues and then MAC Tx queue 0.
48
ICMP Packet Path Software PCI Bus PW-OSPF Java GUI Driver DMA NetFPGA
CPU RxQ TxQ nf2_reg_grp user data path DMA Registers nf2c0 nf2c1 nf2c2 nf2c3 ioctl MAC Ethernet
49
NetFPGA-Host Interaction
NetFPGA to host packet transfer 1. Packet arrives – forwarding table sends to CPU queue 2. Interrupt notifies driver of packet arrival 3. Driver sets up and initiates DMA transfer PCI Bus
50
NetFPGA-Host Interaction
NetFPGA to host packet transfer (cont) 4. NetFPGA transfers packet via DMA 5. Interrupt signals completion of DMA PCI Bus 6. Driver passes packet to network stack
51
NetFPGA-Host Interaction
Host to NetFPGA packet transfers 3. Interrupt signals completion of DMA 2. Driver sets up and initiates DMA transfer PCI Bus 1. Software sends packet via network sockets. Packet delivered to driver.
Similar presentations
