Presentation is loading. Please wait.

Presentation is loading. Please wait.

EuroSys 2008 - NetFPGA Tutorial 1 S T A N F O R D U N I V E R S I T Y Presented by: John W. Lockwood & G. Adam Covington (Stanford University) Andrew Moore.

Similar presentations


Presentation on theme: "EuroSys 2008 - NetFPGA Tutorial 1 S T A N F O R D U N I V E R S I T Y Presented by: John W. Lockwood & G. Adam Covington (Stanford University) Andrew Moore."— Presentation transcript:

1 EuroSys 2008 - NetFPGA Tutorial 1 S T A N F O R D U N I V E R S I T Y Presented by: John W. Lockwood & G. Adam Covington (Stanford University) Andrew Moore (University of Cambridge) Monday March 31, 2008 9am-5pm Glasgow, Scotland http://NetFPGA.org Building Gigabit-rate Routers with the NetFPGA: EuroSys Tutorial at University of Glasgow

2 EuroSys 2008 - NetFPGA Tutorial 2 S T A N F O R D U N I V E R S I T Y FPGA Memory 1GE What is the NetFPGA? PCI CPU Memory NetFPGA Board PC with NetFPGA Networking Software running on a standard PC A hardware accelerator built with Field Programmable Gate Array driving Gigabit network links

3 EuroSys 2008 - NetFPGA Tutorial 3 S T A N F O R D U N I V E R S I T Y Introduction Who uses the NetFPGA Teachers Students Researchers How they use the NetFPGA 1.To run the Router Kit 2.To build modular reference designs IPv4 router 4-port NIC Ethernet switch, … 3.To create new networking systems

4 EuroSys 2008 - NetFPGA Tutorial 4 S T A N F O R D U N I V E R S I T Y FPGA Memory 1GE Running the Router Kit User-space development, 4x1GE line-rate forwarding PCI CPU Memory OSPFBGP My Protocol user kernel Routing Table Usage #1 IPv4 Router 1GE Fwding Table Packet Buffer “Mirror”

5 EuroSys 2008 - NetFPGA Tutorial 5 S T A N F O R D U N I V E R S I T Y FPGA Memory 1GE Building Modular Router Modules PCI CPU Memory Usage #2 NetFPGA Driver Java GUI Front Panel (Extensible) PW-OSPF In Q Mgmt IP Lookup L2 Parse L3 Parse Out Q Mgmt 1GE Verilog modules interconnected by FIFO interfaces My Block Verilog EDA Tools (Xilinx, Mentor, etc.) 1.Design 2.Simulate 3.Synthesize 4.Download 1.Design 2.Simulate 3.Synthesize 4.Download

6 EuroSys 2008 - NetFPGA Tutorial 6 S T A N F O R D U N I V E R S I T Y FPGA Memory 1GE Creating new systems PCI CPU Memory Usage #3 NetFPGA Driver 1GE My Design (1GE MAC is soft/replaceable) Verilog EDA Tools (Xilinx, Mentor, etc.) 1.Design 2.Simulate 3.Synthesize 4.Download 1.Design 2.Simulate 3.Synthesize 4.Download

7 EuroSys 2008 - NetFPGA Tutorial 7 S T A N F O R D U N I V E R S I T Y Tutorial Outline Background –Basics of an IP Router (Andrew) –The NetFPGA Platform (John) The Stanford Base Reference Router –Demo1 : Reference Router running on the NetFPGA (Adam) –Inside the NetFPGA hardware (John) –Breakneck introduction to Verilog (John) –Exercise 1: Build your own Reference Router (Adam) The Enhanced Reference Router –Motivation: Understanding buffer size requirements in a router (Andrew) –Demo 2: Observing and controlling the queue size (Adam) –Exercise 2: Enhancing the Reference Router (Adam) The Life of a Packet Through the NetFPGA –Hardware Datapath (Adam) –Interface to software: Exceptions and Host I/O (John) –Exercise 3: Drop 1 in N Packets (Adam) Concluding Remarks –Using NetFPGA for research and teaching (John) –Group Discussion (All) –Survey (You)

8 EuroSys 2008 - NetFPGA Tutorial 8 S T A N F O R D U N I V E R S I T Y Basic Operation of an IP Router R3 A B C R1 R2 R4D E F R5 F R3E D Next HopDestination D

9 EuroSys 2008 - NetFPGA Tutorial 9 S T A N F O R D U N I V E R S I T Y What does a router do? R3 A B C R1 R2 R4D E F R5 F R3E D Next HopDestination D 16 32 41 Data Options (if any) Destination Address Source Address Header ChecksumProtocolTTL Fragment Offset Flags Fragment ID Total Packet LengthT.ServiceHLenVer 20 bytes

10 EuroSys 2008 - NetFPGA Tutorial 10 S T A N F O R D U N I V E R S I T Y What does a router do? A B C R1 R2 R3 R4D E F R5

11 EuroSys 2008 - NetFPGA Tutorial 11 S T A N F O R D U N I V E R S I T Y Basic Components of an IP Router Control Plane Datapath per-packet processing Switching Forwarding Table Routing Table Routing Protocols Management & CLI Software Hardware

12 EuroSys 2008 - NetFPGA Tutorial 12 S T A N F O R D U N I V E R S I T Y Per-packet processing in an IP Router 1. Accept packet arriving on an incoming link. 2. Lookup packet destination address in the forwarding table, to identify outgoing port(s). 3. Manipulate IP header: e.g., decrement TTL, update header checksum. 5. Buffer packet in the output queue. 6. Transmit packet onto outgoing link.

13 EuroSys 2008 - NetFPGA Tutorial 13 S T A N F O R D U N I V E R S I T Y Generic Datapath Architecture Lookup IP Address Update Header Header Processing DataHdrDataHdr Forwarding Table Forwarding Table IP AddressNext Hop Queue Packet Buffer Memory Buffer Memory

14 EuroSys 2008 - NetFPGA Tutorial 14 S T A N F O R D U N I V E R S I T Y CIDR and Longest Prefix Matches  The IP address space is broken into line segments.  Each line segment is described by a prefix.  A prefix is of the form x/y where x indicates the prefix of all addresses in the line segment, and y indicates the length of the segment.  e.g. The prefix 128.9/16 represents the line segment containing addresses in the range: 128.9.0.0 … 128.9.255.255. 0 2 32 -1 128.9/16 128.9.0.0 2 16 142.12/19 65/8 128.9.16.14

15 EuroSys 2008 - NetFPGA Tutorial 15 S T A N F O R D U N I V E R S I T Y Classless Interdomain Routing (CIDR) 0 2 32 -1 128.9/16 128.9.16.14 128.9.16/20128.9.176/20 128.9.19/24 128.9.25/24 Most specific route = “longest matching prefix”

16 EuroSys 2008 - NetFPGA Tutorial 16 S T A N F O R D U N I V E R S I T Y Techniques for LPM in hardware Linear search Direct lookup –Currently requires too much memory –Updating a prefix leads to many changes Tries –Deterministic lookup time –Easily pipelined –But requires multiple memories/references TCAM (Ternary CAM) –Simple and widely used –But low-density, high-power –Gradually being replaced by new algorithms

17 EuroSys 2008 - NetFPGA Tutorial 17 S T A N F O R D U N I V E R S I T Y An IP Router on NetFPGA Switching Forwarding Table Routing Table Routing Protocols Management & CLI Software Hardware Linux user-level processes Verilog on NetFPGA PCI board Exception Processing

18 EuroSys 2008 - NetFPGA Tutorial 18 S T A N F O R D U N I V E R S I T Y NetFPGA Router Function –4 Gigabit Ethernet ports Fully programmable –FPGA hardware Low cost Open-source FPGA hardware –Verilog base design Open-souce Software –Drivers in C and C++

19 EuroSys 2008 - NetFPGA Tutorial 19 S T A N F O R D U N I V E R S I T Y NetFPGA Platform Major Components –Interfaces 4 Gigabit Ethernet Ports PCI Host Interface –Memories 36Mbits Static RAM 512Mbits DDR2 Dynamic RAM –FPGA Resources Block RAMs Configurable Logic Block (CLBs) Memory Mapped Registers

20 EuroSys 2008 - NetFPGA Tutorial 20 S T A N F O R D U N I V E R S I T Y NetFPGA System User Space Linux Kernel NIC GE PCI-e PCI Browser & Video Client Monitor Software GE CAD Tools NetFPGA Router Hardware VI Packet Forwarding Table (eth1.. 2)(nf2c0.. 3) Web & Video Server

21 EuroSys 2008 - NetFPGA Tutorial 21 S T A N F O R D U N I V E R S I T Y NetFPGA Hardware

22 EuroSys 2008 - NetFPGA Tutorial 22 S T A N F O R D U N I V E R S I T Y NetFPGA System Implementation NetFPGA Blocks –Virtex-2 Pro FPGA –4.5MB ZBT SRAM –64MB DDR2 DRAM –PCI Host Interface –4 Gigabit Ethernet ports Intranet Test Ports –Dual Gigabit Ethernets on PCI-e Internet –Gigabit Ethernet on Motherboard Processor –Quad-Core CPU Operating System –Linux CentOS 4.4

23 EuroSys 2008 - NetFPGA Tutorial 23 S T A N F O R D U N I V E R S I T Y Dell 2950: Another support NetFPGA host NetFPGA properly Inserts in PCI / PCI-X slot Dell 2950 with PCI-X and PCI-Express Slots Thanks: Brian Cashman for providing machine

24 EuroSys 2008 - NetFPGA Tutorial 24 S T A N F O R D U N I V E R S I T Y NetFPGA Lab Setup (eth1.. 2) Nf2c3 : Adj. Server CPU x2 Net-FPGA Dual NIC GE PCI-e PCI Client NetFPGA Control SW GE CAD Tools Internet Router Hardware Eth2 : Server Server Eth1 : Local host Nf2c1 : Adjacent Nf2c2 : Local Host Nf2c0 : Adjacent

25 EuroSys 2008 - NetFPGA Tutorial 25 S T A N F O R D U N I V E R S I T Y Net-FPGA NetFPGA Hardware Set for Demo #1 Net-FPGA GE Internet Router Hardware CPU x2 Net-FPGA NIC GE PCI-e PCI Video Display GE CAD Tools GE Internet Router Hardware Internet Router Hardware CPU x2 PCI-e PCI Video Server NIC GE PCI-e GE … Server delivers streaming HD video through a chain of NetFPGA Routers

26 EuroSys 2008 - NetFPGA Tutorial 26 S T A N F O R D U N I V E R S I T Y Tutorial Outline Background –Basics of an IP Router (Andrew) –The NetFPGA Platform (John) The Stanford Base Reference Router –Demo1 : Reference Router running on the NetFPGA (Adam) –Inside the NetFPGA hardware (John) –Breakneck introduction to Verilog (John) –Exercise 1: Build your own Reference Router (Adam) The Enhanced Reference Router –Motivation: Understanding buffer size requirements in a router (Andrew) –Demo 2: Observing and controlling the queue size (Adam) –Exercise 2: Enhancing the Reference Router (Adam) The Life of a Packet Through the NetFPGA –Hardware Datapath –Interface to software –Exercise 3: Drop 1 in N Packets Concluding Remarks –Using NetFPGA for research and teaching (John) –Group Discussion –Survey

27 EuroSys 2008 - NetFPGA Tutorial 27 S T A N F O R D U N I V E R S I T Y HD Display Topology of NetFPGA Routers Video Server Demo 1

28 EuroSys 2008 - NetFPGA Tutorial 28 S T A N F O R D U N I V E R S I T Y Setup for the Reference Router Each NetFPGA card has four ports Port 2 connected to Client / Server Ports 0 and 3 connected to adjacent NetFPGA cards NetFPGA Video Client Demo 1 NetFPGA Video Server

29 EuroSys 2008 - NetFPGA Tutorial 29 S T A N F O R D U N I V E R S I T Y Subnet Configuration.1.1.1.2.3.1.30.2.4.1.4.2.6.1.3.2.7.1.7.2.9.1.6.2.10.1.10.2.12.1.9.2.13.1.13.2.15.1.12.2.16.1.16.2.15.2.28.1.28.2.27.1.30.1.25.1.25.2.24.1.27.2.22.1.22.2.21.1.24.2.19.1.19.2.17.1.21.2.18.2.5.1.8.1.11.1.14.1.18.1.20.1.23.1.26.1.29.1.2.1 Video Client Shortest Path Video Server Demo 1

30 EuroSys 2008 - NetFPGA Tutorial 30 S T A N F O R D U N I V E R S I T Y Physical Configuration Qnty = N * –NetFPGA Mrg Part#: NETFPGA from DigilentInc.com –Ethernet Adapter Intel Pro/1000 Dual-port Gigabit PCI-Express x4 NIC Mfg Part#: EXPI9402PT from Buy.com –Category 5E Gigabit Ethernet Cables Loopback (Red, 1’ = 30cm) Network (Green, 3’ = 1m) Adjacent Host (Blue, 3’ = 1m) Configuration (Yellow, 9’ = 3m) Additional Parts –RJ45 Cable connectors, Qnty=2

31 EuroSys 2008 - NetFPGA Tutorial 31 S T A N F O R D U N I V E R S I T Y 3 2 1 0 1 2 nf2c eth 12 3 2 1 0 1 2 nf2c eth 11 3 2 1 0 1 2 nf2c eth 9 3 2 1 0 1 2 nf2c eth 8 3 2 1 0 1 2 nf2c eth 7 3 2 1 0 1 2 nf2c eth 6 3 2 1 0 1 2 nf2c eth 5 3 2 1 0 1 2 nf2c eth 4 3 2 1 0 1 2 nf2c eth 3 3 2 1 0 1 2 nf2c eth 2 Cable Configuration NetFPGA Gigabit Ethernet Interfaces –nf2c3 : Left neighbor in network –nf2c2 : Local host interface –nf2c1 : Direct connection to adjacent server –nf2c0 : Right neighbor in network Host Ethernet Interfaces –eth1 : Local host interface –eth2 : Direct connection for adjacent client

32 EuroSys 2008 - NetFPGA Tutorial 32 S T A N F O R D U N I V E R S I T Y Photos of Machines During Setup

33 EuroSys 2008 - NetFPGA Tutorial 33 S T A N F O R D U N I V E R S I T Y Working IP Router Objectives –Become familiar with Stanford Reference Router –Observe PW-OSPF re-routing traffic around a failure Demo 1

34 EuroSys 2008 - NetFPGA Tutorial 34 S T A N F O R D U N I V E R S I T Y Streaming Video through the NetFPGA Video server –Source files /var/www/html/video –Network URL : http://192.168. Net. Host /video Video client –Windows Media Player –Linux mplayer Video traffic –MPEG2 HDTV (35 Mbps) –MPEG2 TV (9 Mbps) –DVI (3 Mbps) –WMF (1.7 Mbps) Demo 1

35 EuroSys 2008 - NetFPGA Tutorial 35 S T A N F O R D U N I V E R S I T Y Step 1 – Observe the Routing Tables The router is already configured and running on your machines The routing table has converged to the routing decisions with minimum number of hops Next, break a link … Demo 1

36 EuroSys 2008 - NetFPGA Tutorial 36 S T A N F O R D U N I V E R S I T Y Step 2 - Dynamic Re-routing Break the link between video server and video client Routers re-route traffic around the broken link and video continues playing.1.1.1.2.3.1.30.2.4.1.4.2.6.1.3.2.7.1.7.2.9.1.6.2.10.1.10.2.12.1.9.2.13.1.13.2.15.1.12.2.16.1.16.2.15.2.28.1.28.2.27.1.30.1.25.1.25.2.24.1.27.2.22.1.22.2.21.1.24.2.19.1.19.2.17.1.21.2.18.2.5.1.8.1.11.1.14. 1.18.1.20.1.23.1.26.1.29.1.2.1 Demo 1

37 EuroSys 2008 - NetFPGA Tutorial 37 S T A N F O R D U N I V E R S I T Y Tutorial Outline Background –Basics of an IP Router (Andrew) –The NetFPGA Platform (John) The Stanford Base Reference Router –Demo1 : Reference Router running on the NetFPGA (Adam) –Inside the NetFPGA hardware (John) –Breakneck introduction to Verilog (John) –Exercise 1: Build your own Reference Router (Adam) The Enhanced Reference Router –Motivation: Understanding buffer size requirements in a router (Andrew) –Demo 2: Observing and controlling the queue size (Adam) –Exercise 2: Enhancing the Reference Router (Adam) The Life of a Packet Through the NetFPGA –Hardware Datapath (Adam) –Interface to software: Exceptions and Host I/O (John) –Exercise 3: Drop 1 in N Packets (Adam) Concluding Remarks –Using NetFPGA for research and teaching (John) –Group Discussion (All) –Survey (You)

38 EuroSys 2008 - NetFPGA Tutorial 38 S T A N F O R D U N I V E R S I T Y Integrated Circuit Technology Full-custom Design –Complementary Metal Oxide Semiconductor (CMOS) Semi-custom ASIC Design –Gate array –Standard cell Programmable Logic Device –Programmable Array Logic –Field Programmable Gate Arrays Processors

39 EuroSys 2008 - NetFPGA Tutorial 39 S T A N F O R D U N I V E R S I T Y Combinatorial Logic A B C D Z Look-Up Tables Combinatorial logic is stored in Look-Up Tables (LUTs) –Also called Function Generators (FGs) –Capacity is limited only by number of inputs, not complexity –Delay through the LUT is constant ABCDZ 00000 00010 00100 00111 01001 01011... 11000 11010 11100 11111 Diagram From: Xilinx, Inc

40 EuroSys 2008 - NetFPGA Tutorial 40 S T A N F O R D U N I V E R S I T Y Slice 0 LUT Carry LUT Carry DQ CE PRE CLR D Q CE PRE CLR Xilinx CLB Structure Each slice has four outputs –Two registered outputs, two non-registered outputs –Two BUFTs associated with each CLB, accessible by all 16 CLB outputs Carry logic run vertically –Signals run upwards –Two independent carry chains per CLB Diagram From: Xilinx, Inc (Courtesy Jeff Weintraub)

41 EuroSys 2008 - NetFPGA Tutorial 41 S T A N F O R D U N I V E R S I T Y Field Programmable Gate Arrays CLB –Primitive element of FPGA Routing Module –Global routing –Local interconnect Macro Blocks –Block Memories –Microprocessor I/O Block

42 EuroSys 2008 - NetFPGA Tutorial 42 S T A N F O R D U N I V E R S I T Y NetFPGA Block Diagram

43 EuroSys 2008 - NetFPGA Tutorial 43 S T A N F O R D U N I V E R S I T Y Details of NetFPGA Fits into Standard PCI slot –Standard Bus : 32 bits, 33 MHz Provides Interfaces for processing network packets –4 Gigabit Ethernet Ports Allows hardware-accelerated processing –Implemented with Field Programmable Gate Array (FPGA) Logic

44 EuroSys 2008 - NetFPGA Tutorial 44 S T A N F O R D U N I V E R S I T Y Tutorial Outline Background –Basics of an IP Router (Andrew) –The NetFPGA Platform (John) The Stanford Base Reference Router –Demo1 : Reference Router running on the NetFPGA (Adam) –Inside the NetFPGA hardware (John) –Breakneck introduction to Verilog (John) –Exercise 1: Build your own Reference Router (Adam) The Enhanced Reference Router –Motivation: Understanding buffer size requirements in a router (Andrew) –Demo 2: Observing and controlling the queue size (Adam) –Exercise 2: Enhancing the Reference Router (Adam) The Life of a Packet Through the NetFPGA –Hardware Datapath (Adam) –Interface to software: Exceptions and Host I/O (John) –Exercise 3: Drop 1 in N Packets (Adam) Concluding Remarks –Using NetFPGA for research and teaching (John) –Group Discussion (All) –Survey (You)

45 EuroSys 2008 - NetFPGA Tutorial 45 S T A N F O R D U N I V E R S I T Y Hardware Description Languages Concurrent –By Default, Verilog statements evaluated concurrently Express fine grain parallelism –Allows gate-level parallelism Provides Precise Description –Eliminates ambiguity about operation Synthesizable –Generates hardware from description

46 EuroSys 2008 - NetFPGA Tutorial 46 S T A N F O R D U N I V E R S I T Y Verilog Data Types reg [7:0] A; // 8-bit register, MSB to LSB // (Preferred bit order for NetFPGA) reg [0:15] B; // 16-bit register, LSB to MSB B = {A[7:0],A[0:7]}; // Assignment of bits reg [31:0] Mem [0:1023]; // 1K Word Memory integer Count; // simple signed 32-bit integer integer K[1:64]; // an array of 64 integers time Start, Stop; // Two 64-bit time variables From: CSCI 320 Computer Architecture Handbook on Verilog HDL, by Dr. Daniel C. Hyde : http://eesun.free.fr/DOC/VERILOG/verilog-manual.html

47 EuroSys 2008 - NetFPGA Tutorial 47 S T A N F O R D U N I V E R S I T Y Signal Multiplexers From: http://eesun.free.fr/DOC/VERILOG/synvlg.html Two input multiplexer (using if / else) reg y; always @* if (select) y = a; else y = b; Two input multiplexer (using ternary operator ?:) wire t = (select ? a : b);

48 EuroSys 2008 - NetFPGA Tutorial 48 S T A N F O R D U N I V E R S I T Y Larger Multiplexers Three input multiplexer reg s; always @* begin case (select2) 2'b00: s = a; 2'b01: s = b; default: s = c; endcase end

49 EuroSys 2008 - NetFPGA Tutorial 49 S T A N F O R D U N I V E R S I T Y Synchronous Storage Elements Values change at times governed by clock Clock Transition t=0t=1t=2 0 1 Clock time Clock Transition S0 Dout t=0 AB ABC Din t=0 Clock DinDoutQD –Clock Input to circuit –Clock Event Example: Rising edge –Flip/Flop Transfers Value From D in to D out on Clock event

50 EuroSys 2008 - NetFPGA Tutorial 50 S T A N F O R D U N I V E R S I T Y Finite State Machines

51 EuroSys 2008 - NetFPGA Tutorial 51 S T A N F O R D U N I V E R S I T Y Synthesizable Verilog : Delay Flip/Flops From: http://eesun.free.fr/DOC/VERILOG/synvlg.html D type flip flop with data enable reg q; always @ (posedge clk) if (enable) q <= d; D-type flip flop reg q; always @ (posedge clk) q <= d;

52 EuroSys 2008 - NetFPGA Tutorial 52 S T A N F O R D U N I V E R S I T Y Tutorial Outline Background –Basics of an IP Router (Andrew) –The NetFPGA Platform (John) The Stanford Base Reference Router –Demo1 : Reference Router running on the NetFPGA (Adam) –Inside the NetFPGA hardware (John) –Breakneck introduction to Verilog (John) –Exercise 1: Build your own Reference Router (Adam) The Enhanced Reference Router –Motivation: Understanding buffer size requirements in a router (Andrew) –Demo 2: Observing and controlling the queue size (Adam) –Exercise 2: Enhancing the Reference Router (Adam) The Life of a Packet Through the NetFPGA –Hardware Datapath (Adam) –Interface to software: Exceptions and Host I/O (John) –Exercise 3: Drop 1 in N Packets (Adam) Concluding Remarks –Using NetFPGA for research and teaching (John) –Group Discussion (All) –Survey (You)

53 Eurosys 2008 - NetFPGA Tutorial 53 S T A N F O R D U N I V E R S I T Y Reference Router Pipeline Five stages –Input –Input Arbitration –Routing Decision and packet modification –Output Queuing –Output Packet-based module interface Pluggable design Exercise 1 MAC RxQ MAC RxQ CPU RxQ CPU RxQ MAC RxQ MAC RxQ CPU RxQ CPU RxQ MAC RxQ MAC RxQ CPU RxQ CPU RxQ MAC RxQ MAC RxQ CPU RxQ CPU RxQ Input Arbiter Output Port Lookup MAC TxQ MAC TxQ CPU TxQ CPU TxQ MAC TxQ MAC TxQ CPU TxQ CPU TxQ MAC TxQ MAC TxQ CPU TxQ CPU TxQ MAC TxQ MAC TxQ CPU TxQ CPU TxQ Output Queues

54 EuroSys 2008 - NetFPGA Tutorial 54 S T A N F O R D U N I V E R S I T Y Make your own router Objectives: –Learn how to build hardware –Run the software –Explore router architecture Execution –Start synthesis –Rerun the GUI with the new hardware –Test connectivity and statistics with pings –Explore pipeline in the details page –Explore detailed statistics in the details page Exercise 1

55 Eurosys 2008 - NetFPGA Tutorial 55 S T A N F O R D U N I V E R S I T Y Step 1 - Build the hardware Start terminal, cd to “NF2/projects/tutorial_router/synth” Run “make clean” Start synthesis with “make” Exercise 1

56 EuroSys 2008 - NetFPGA Tutorial 56 S T A N F O R D U N I V E R S I T Y First Break on 5 th Floor (while hardware compiles)

57 Eurosys 2008 - NetFPGA Tutorial 57 S T A N F O R D U N I V E R S I T Y Step 2 - Run Homemade Router cd to “NF2/projects/tutorial_router/sw” Type: “tutorial_router_gui.pl” to use the just built router hardware The same interface should start again Exercise 1

58 Eurosys 2008 - NetFPGA Tutorial 58 S T A N F O R D U N I V E R S I T Y Step 4 - Connectivity and Statistics Ping any addresses 192.168.x.y where x is from 1-20 and y is 1 or 2 Open the statistics tab in the Quickstart window to see some statistics Explore more statistics in modules under the details tab Exercise 1

59 EuroSys 2008 - NetFPGA Tutorial 59 S T A N F O R D U N I V E R S I T Y Step 5 - Explore Router Architecture Click the Details tab of the Quickstart window This is the reference router pipeline – a canonical, simple to understand, modular router pipeline Exercise 1

60 Eurosys 2008 - NetFPGA Tutorial 60 S T A N F O R D U N I V E R S I T Y Step 6 - Explore Output Queues Click on the Output Queues module in the Details tab The page gives configuration details …and statistics Exercise 1

61 EuroSys 2008 - NetFPGA Tutorial 61 S T A N F O R D U N I V E R S I T Y Tutorial Outline Background –Basics of an IP Router (Andrew) –The NetFPGA Platform (John) The Stanford Base Reference Router –Demo1 : Reference Router running on the NetFPGA (Adam) –Inside the NetFPGA hardware (John) –Breakneck introduction to Verilog (John) –Exercise 1: Build your own Reference Router (Adam) The Enhanced Reference Router –Motivation: Understanding buffer size requirements in a router (Andrew) –Demo 2: Observing and controlling the queue size (Adam) –Exercise 2: Enhancing the Reference Router (Adam) The Life of a Packet Through the NetFPGA –Hardware Datapath (Adam) –Interface to software: Exceptions and Host I/O (John) –Exercise 3: Drop 1 in N Packets (Adam) Concluding Remarks –Using NetFPGA for research and teaching (John) –Group Discussion (All) –Survey (You)

62 Eurosys 2008 - NetFPGA Tutorial 62 S T A N F O R D U N I V E R S I T Y Buffer Requirements in a Router Buffer size matters: –Small queues reduce delay –Large buffers are expensive Theoretical tools predict requirements –Queuing theory –Large deviation theory –Mean field theory Yet, there is no direct answer. –Flows have a closed-loop nature –Question arises on whether focus should be on equilibrium state or transient state..

63 Eurosys 2008 - NetFPGA Tutorial 63 S T A N F O R D U N I V E R S I T Y Universally applied rule-of-thumb: –A router needs a buffer size: –2T is the two-way propagation delay (or just 250ms) –C is capacity of bottleneck link Context –Mandated in backbone and edge routers. –Appears in RFPs and IETF architectural guidelines. –Already known by inventors of TCP [Van Jacobson, 1988] –Has major consequences for router design C Router Source Destination 2T Rule-of-thumb

64 Eurosys 2008 - NetFPGA Tutorial 64 S T A N F O R D U N I V E R S I T Y The Story So Far 10,000 20 # packets at 10Gb/s 1,000,000 (1)Assume: Large number of desynchronized flows; 100% utilization (2)Assume: Large number of desynchronized flows; <100% utilization

65 Eurosys 2008 - NetFPGA Tutorial 65 S T A N F O R D U N I V E R S I T Y Using NetFPGA to explore buffer size Need to reduce buffer size and measure occupancy Alas, not possible in commercial routers So, we will use NetFPGA instead Objective: –Use NetFPGA to understand how large a buffer we need for a single TCP flow.

66 Eurosys 2008 - NetFPGA Tutorial 66 S T A N F O R D U N I V E R S I T Y Rule for adjusting W –If an ACK is received:W ← W+1/W –If a packet is lost:W ← W/2 Why 2TxC for a single TCP Flow? Only W packets may be outstanding

67 Eurosys 2008 - NetFPGA Tutorial 67 S T A N F O R D U N I V E R S I T Y Time evolution of a single TCP flow through a router. Buffer is < 2T*C Time Evolution of a Single TCP Flow Time evolution of a single TCP flow through a router. Buffer is 2T*C

68 EuroSys 2008 - NetFPGA Tutorial 68 S T A N F O R D U N I V E R S I T Y NetFPGA Hardware Set for Demo #2 CPU x2 PCI-e Video Server NIC GE PCI-e GE Net-FPGA CPU x2 NIC GE PCI-e PCI Video Client GE Internet Router Hardware … Server delivers streaming HD video to adjacent client

69 EuroSys 2008 - NetFPGA Tutorial 69 S T A N F O R D U N I V E R S I T Y Tutorial Outline Background –Basics of an IP Router (Andrew) –The NetFPGA Platform (John) The Stanford Base Reference Router –Demo1 : Reference Router running on the NetFPGA (Adam) –Inside the NetFPGA hardware (John) –Breakneck introduction to Verilog (John) –Exercise 1: Build your own Reference Router (Adam) The Enhanced Reference Router –Motivation: Understanding buffer size requirements in a router (Andrew) –Demo 2: Observing and controlling the queue size (Adam) –Exercise 2: Enhancing the Reference Router (Adam) The Life of a Packet Through the NetFPGA –Hardware Datapath (Adam) –Interface to software: Exceptions and Host I/O (John) –Exercise 3: Drop 1 in N Packets (Adam) Concluding Remarks –Using NetFPGA for research and teaching (John) –Group Discussion (All) –Survey (You)

70 EuroSys 2008 - NetFPGA Tutorial 70 S T A N F O R D U N I V E R S I T Y Setup for the Demo 2 Each NetFPGA card has four ports Port 2 connected to Local Host Port 3 connected to adjacent Server Local Host Demo 2 NetFPGA Adjacent Server

71 Eurosys 2008 - NetFPGA Tutorial 71 S T A N F O R D U N I V E R S I T Y Topology for Second Demonstration Routers connected point-to-point topology Port 3 connects to local host Port 1 connects to adjacent neighbor Ports 0 and 2 unused.1.1.1.2.4.1.4.2.7.1.7.2.10.1.10.2.13.1.13.2.16.1.16.2.28.1.28.2.25.1.25.2.22.1.22.2.19.1.19.2 Demo 2.2.2.2.1.5.2.5.1.8.2.8.1.11.2.11.1.14.2.14.1.17.2.17.1.20.2.20.1.23.2.23.1.26.2.26.1.29.2.29.1

72 EuroSys 2008 - NetFPGA Tutorial 72 S T A N F O R D U N I V E R S I T Y Enhanced Router Objectives –Observe router with new modules –New modules: rate limiting, delay, event capture Execution –Run event capture router –Look at routing tables –Explore details pane –Start tcp transfer, look at queue occupancy –Change rate/delay, look at queue occupancy Demo 2

73 Eurosys 2008 - NetFPGA Tutorial 73 S T A N F O R D U N I V E R S I T Y Step 1 - Run Pre-made Enhanced Router Start terminal and cd to “NF2/projects/tutorial_ro uter/sw/” Type “./tut_adv_router_gui.pl” A familiar GUI should start Demo 2

74 Eurosys 2008 - NetFPGA Tutorial 74 S T A N F O R D U N I V E R S I T Y Step 3 - Explore Enhanced Router Click on the Details tab A similar Pipeline to the one seen previously shows with some additions Demo 2

75 Eurosys 2008 - NetFPGA Tutorial 75 S T A N F O R D U N I V E R S I T Y Enhanced Router Pipeline Two modules added 1.Event Capture to capture output queue events (writes, reads, drops) 2.Rate Limiter to create a bottleneck Demo 2 MAC RxQ MAC RxQ CPU RxQ CPU RxQ MAC RxQ MAC RxQ CPU RxQ CPU RxQ MAC RxQ MAC RxQ CPU RxQ CPU RxQ MAC RxQ MAC RxQ CPU RxQ CPU RxQ Input Arbiter Output Port Lookup MAC TxQ MAC TxQ CPU TxQ CPU TxQ MAC TxQ MAC TxQ CPU TxQ CPU TxQ MAC TxQ MAC TxQ CPU TxQ CPU TxQ MAC TxQ MAC TxQ CPU TxQ CPU TxQ Output Queues Rate Limiter Rate Limiter Event Capture

76 Eurosys 2008 - NetFPGA Tutorial 76 S T A N F O R D U N I V E R S I T Y Step 4 - Decrease the Link Rate To create bottleneck and show the TCP “sawtooth”, link-rate is decreased. In the Details tab click the “Rate Limit” module Check Enable Set link rate to 1.953Mbps Demo 2

77 Eurosys 2008 - NetFPGA Tutorial 77 S T A N F O R D U N I V E R S I T Y Step 5 – Decrease Queue Size Go back to the Details Panel and click on “Output Queues”. Select the “Output Queue 2” tab. Change the output queues size in packets slider to 16 Demo 2

78 Eurosys 2008 - NetFPGA Tutorial 78 S T A N F O R D U N I V E R S I T Y Step 6 - Start Event Capture Click on the Event Capture module under the Details tab This should start the configuration page Demo 2

79 Eurosys 2008 - NetFPGA Tutorial 79 S T A N F O R D U N I V E R S I T Y Step 7 - Configure Event Capture Check Send to local host to receive events on the local host Check Monitor Queue 2 to monitor output queue of MAC port1 Check “Enable Capture” to start Event capture Demo 2

80 Eurosys 2008 - NetFPGA Tutorial 80 S T A N F O R D U N I V E R S I T Y Step 8 - Start TCP Transfer We will use iperf to run a large TCP transfer and look at queue evolution Start a terminal and cd to “NF2/projects/tutorial_router/sw” type “iperf.sh” Demo 2

81 Eurosys 2008 - NetFPGA Tutorial 81 S T A N F O R D U N I V E R S I T Y Step 9 - Look at Event Capture Results Click on the Event Capture module under the details tab. The sawtooth pattern should now be visible. Demo 2

82 Eurosys 2008 - NetFPGA Tutorial 82 S T A N F O R D U N I V E R S I T Y Queue Occupancy Charts

83 EuroSys 2008 - NetFPGA Tutorial 83 S T A N F O R D U N I V E R S I T Y Tutorial Outline Background –Basics of an IP Router (Andrew) –The NetFPGA Platform (John) The Stanford Base Reference Router –Demo1 : Reference Router running on the NetFPGA (Adam) –Inside the NetFPGA hardware (John) –Breakneck introduction to Verilog (John) –Exercise 1: Build your own Reference Router (Adam) The Enhanced Reference Router –Motivation: Understanding buffer size requirements in a router (Andrew) –Demo 2: Observing and controlling the queue size (Adam) –Exercise 2: Enhancing the Reference Router (Adam) The Life of a Packet Through the NetFPGA –Hardware Datapath (Adam) –Interface to software: Exceptions and Host I/O (John) –Exercise 3: Drop 1 in N Packets (Adam) Concluding Remarks –Using NetFPGA for research and teaching (John) –Group Discussion (All) –Survey (You)

84 EuroSys 2008 - NetFPGA Tutorial 84 S T A N F O R D U N I V E R S I T Y Enhance Your Router Objectives –Add new modules to datapath –Synthesize and test router Execution –Open user_datapath.v, uncomment delay/rate/event capture modules –Synthesize –After synthesis, test the new system. Exercise 2

85 EuroSys 2008 - NetFPGA Tutorial 85 S T A N F O R D U N I V E R S I T Y An aside: xemacs Tips We will be modifying the Verilog source code Slides show xemacs, but vim also available. xemacs: –To undo, use ctrl+shift+'-' –To cancel a multi-keystroke command, just type ctrl+g –To select lines, hold shift and press the arrow keys. –To comment some selected lines, type ctrl+c+c –To uncomment a commented block, move the cursor to one of the lines inside the commented block and type ctrl+c+u –To save type ctrl+x+s –To search, type ctrl+s search_pattern

86 Eurosys 2008 - NetFPGA Tutorial 86 S T A N F O R D U N I V E R S I T Y Step 1 - Open the Source We will modify the Verilog source code to add event capture, rate limiter, and delay modules We will simply comment and uncomment existing code Open terminal Type “xemacs NF2/projects/tutorial_router/sr c/user_data_path.v Exercise 2

87 Eurosys 2008 - NetFPGA Tutorial 87 S T A N F O R D U N I V E R S I T Y Step 2 - Add wires Now we need to add wires to connect the new modules Search for “new wires” (ctrl+s new wires) then press Enter Uncomment the wires (ctrl+c+u) Exercise 2

88 Eurosys 2008 - NetFPGA Tutorial 88 S T A N F O R D U N I V E R S I T Y Step 3a - Connect Event Capture Search for opl_output (ctrl+s opl_output) then press Enter Comment the four lines above (up, shift + up + up + up + up, ctrl+c+c) Uncomment the block below to connect the outputs (ctrl+s opl_out, ctrl+c+u) Exercise 2

89 Eurosys 2008 - NetFPGA Tutorial 89 S T A N F O R D U N I V E R S I T Y Step 3b - Connect the Output Queue Registers Search for opl_output (ctrl+s opl_output, Enter) Comment the 6 lines (select the six lines by using shift+arrow keys, then type ctrl+c+c) Uncomment the commented block by scrolling down into the block and typing ctrl+c+u Exercise 2

90 Eurosys 2008 - NetFPGA Tutorial 90 S T A N F O R D U N I V E R S I T Y Step 4 - Add the Event Capture Module Search for evt_capture_top (ctrl+s evt_capture_top) then press Enter Uncomment the block (ctrl+c+u) Exercise 2

91 Eurosys 2008 - NetFPGA Tutorial 91 S T A N F O R D U N I V E R S I T Y Step 5 - Add the Drop Nth Module Search for drop_nth_packet (ctrl+s drop_nth_packet) then press Enter Uncomment the block (ctrl+c+u) Exercise 2

92 Eurosys 2008 - NetFPGA Tutorial 92 S T A N F O R D U N I V E R S I T Y Step 6 - Connect the Output Queue to the Rate Limiter Search for port_outputs (ctrl+s port_outputs, Enter) Comment the 4 lines above (select the four lines by using shift+arrow keys, then type ctrl+c+c) Uncomment the commented block by scrolling down into the block and typing ctrl+c+u Exercise 2

93 Eurosys 2008 - NetFPGA Tutorial 93 S T A N F O R D U N I V E R S I T Y Step 7 - Connect the Registers Search for port_outputs (ctrl+s port_outputs, Enter) Comment the 6 lines (select the six lines by using shift+arrow keys, then type ctrl+c+c) Uncomment the commented block by scrolling down into the block and typing ctrl+c+u Exercise 2 six

94 Eurosys 2008 - NetFPGA Tutorial 94 S T A N F O R D U N I V E R S I T Y Step 8 - Add Rate Limiter Scroll down until you reach the next “Excluded” block Uncomment the block containing the rate limiter instantiations. (scroll into the block and type ctrl+c+u) Save (ctrl+x+s) Exercise 2

95 Eurosys 2008 - NetFPGA Tutorial 95 S T A N F O R D U N I V E R S I T Y Step 9 - Build the hardware Start terminal, cd to “NF2/projects/tutorial_rou ter/synth” Run “make clean” Start synthesis with “make” Exercise 2

96 EuroSys 2008 - NetFPGA Tutorial 96 S T A N F O R D U N I V E R S I T Y Second Break Lunch on 5 th Floor (while hardware compiles)

97 EuroSys 2008 - NetFPGA Tutorial 97 S T A N F O R D U N I V E R S I T Y Tutorial Outline Background –Basics of an IP Router (Andrew) –The NetFPGA Platform (John) The Stanford Base Reference Router –Demo1 : Reference Router running on the NetFPGA (Adam) –Inside the NetFPGA hardware (John) –Breakneck introduction to Verilog (John) –Exercise 1: Build your own Reference Router (Adam) The Enhanced Reference Router –Motivation: Understanding buffer size requirements in a router (Andrew) –Demo 2: Observing and controlling the queue size (Adam) –Exercise 2: Enhancing the Reference Router (Adam) The Life of a Packet Through the NetFPGA –Hardware Datapath (Adam) –Interface to software: Exceptions and Host I/O (John) –Exercise 3: Drop 1 in N Packets (Adam) Concluding Remarks –Using NetFPGA for research and teaching (John) –Group Discussion (All) –Survey (You)

98 EuroSys 2008 - NetFPGA Tutorial 98 S T A N F O R D U N I V E R S I T Y Full System Components Software PCI Bus NetFPGA CPU RxQ CPU RxQ CPU TxQ CPU TxQ nf2_reg_grp user data path nf2c0 nf2c1 nf2c2 nf2c3 ioctl MAC TxQ MAC TxQ MAC RxQ MAC RxQ Ethernet CPU RxQ CPU RxQ CPU TxQ CPU TxQ CPU RxQ CPU RxQ CPU TxQ CPU TxQ CPU RxQ CPU RxQ CPU TxQ CPU TxQ MAC TxQ MAC TxQ MAC RxQ MAC RxQ MAC TxQ MAC TxQ MAC RxQ MAC RxQ MAC TxQ MAC TxQ MAC RxQ MAC RxQ

99 EuroSys 2008 - NetFPGA Tutorial 99 S T A N F O R D U N I V E R S I T Y port0port2 192.168.2.y 192.168.1.x Life of a Packet through the hardware IP packet

100 EuroSys 2008 - NetFPGA Tutorial 100 S T A N F O R D U N I V E R S I T Y Router Stages Again MAC RxQ MAC RxQ CPU RxQ CPU RxQ MAC RxQ MAC RxQ CPU RxQ CPU RxQ MAC RxQ MAC RxQ CPU RxQ CPU RxQ MAC RxQ MAC RxQ CPU RxQ CPU RxQ Input Arbiter Output Port Lookup MAC TxQ MAC TxQ CPU TxQ CPU TxQ MAC TxQ MAC TxQ CPU TxQ CPU TxQ MAC TxQ MAC TxQ CPU TxQ CPU TxQ MAC TxQ MAC TxQ CPU TxQ CPU TxQ Output Queues

101 EuroSys 2008 - NetFPGA Tutorial 101 S T A N F O R D U N I V E R S I T Y Inter-module Communication Using “Module Headers”: IP Hdr Eth Hdr … 0 0 0 Last word of packet0x10 Last Module Hdry …… Module Hdrx Contain information such as packet length, input port, output port, … Data Word (64 bits) Ctrl Word (8 bits)

102 EuroSys 2008 - NetFPGA Tutorial 102 S T A N F O R D U N I V E R S I T Y data Inter-module Communication ctrl wr rdy

103 EuroSys 2008 - NetFPGA Tutorial 103 S T A N F O R D U N I V E R S I T Y MAC Rx Queue IP Hdr: IP Dst: 192.168.2.3, TTL: 64, Csum:0x3ab4 Eth Hdr: Dst MAC = port 0, Ethertype = IP Data

104 EuroSys 2008 - NetFPGA Tutorial 104 S T A N F O R D U N I V E R S I T Y Rx Queue IP Hdr: IP Dst: 192.168.2.3, TTL: 64, Csum:0x3ab4 Eth Hdr: Dst MAC = port 0, Ethertype = IP Data 0 0 0 Pkt length, input port = 0 0xff

105 EuroSys 2008 - NetFPGA Tutorial 105 S T A N F O R D U N I V E R S I T Y Input Arbiter Pkt

106 EuroSys 2008 - NetFPGA Tutorial 106 S T A N F O R D U N I V E R S I T Y Output Port Lookup IP Hdr: IP Dst: 192.168.2.3, TTL: 64, Csum:0x3ab4 EthHdr: Dst MAC = 0 Src MAC = x, Ethertype = IP Data 0 0 0 Pkt length, input port = 0 0xff

107 EuroSys 2008 - NetFPGA Tutorial 107 S T A N F O R D U N I V E R S I T Y IP Hdr: IP Dst: 192.168.2.3, TTL: 64, Csum:0x3ab4 IP Hdr: IP Dst: 192.168.2.3, TTL: 63, Csum:0x3ac2 Output Port Lookup EthHdr: Dst MAC = 0 Src MAC = x, Ethertype = IP Data 0 0 0 Pkt length, input port = 0 0xff output port = 40x04 1- Check input port matches Dst MAC 2- Check TTL, checksum 3- Lookup next hop IP & output port (LPM) 4- Lookup next hop MAC address (ARP) 5- Add output port module 6- Modify MAC Dst and Src addresses 7-Decrement TTL and update checksum EthHdr: Dst MAC = nextHop Src MAC = port 4, Ethertype = IP

108 EuroSys 2008 - NetFPGA Tutorial 108 S T A N F O R D U N I V E R S I T Y Output Queues Pkt OQ0 OQ4 OQ7

109 EuroSys 2008 - NetFPGA Tutorial 109 S T A N F O R D U N I V E R S I T Y MAC Tx Queue IP Hdr: IP Dst: 192.168.2.3, TTL: 64, Csum:0x3ab4 IP Hdr: IP Dst: 192.168.2.3, TTL: 63, Csum:0x3ac2 EthHdr: Dst MAC = nextHop Src MAC = port 4, Ethertype = IP Data 0 0 0 Pkt length, input port = 0 0xff output port = 40x04

110 EuroSys 2008 - NetFPGA Tutorial 110 S T A N F O R D U N I V E R S I T Y MAC Tx Queue IP Hdr: IP Dst: 192.168.2.3, TTL: 64, Csum:0x3ab4 IP Hdr: IP Dst: 192.168.2.3, TTL: 63, Csum:0x3ac2 EthHdr: Dst MAC = nextHop Src MAC = port 4, Ethertype = IP Data 0 0 0 Pkt length, input port = 0 0xff output port = 40x04

111 EuroSys 2008 - NetFPGA Tutorial 111 S T A N F O R D U N I V E R S I T Y Exception Packet Example: TTL = 0 or TTL = 1 Packet has to be sent to the CPU which will generate an ICMP packet as a response Difference starts at the Output Port lookup stage

112 EuroSys 2008 - NetFPGA Tutorial 112 S T A N F O R D U N I V E R S I T Y Exception Packet Path Software PCI Bus NetFPGA CPU RxQ CPU RxQ CPU TxQ CPU TxQ CPU RxQ CPU RxQ CPU TxQ CPU TxQ CPU RxQ CPU RxQ CPU TxQ CPU TxQ CPU RxQ CPU RxQ CPU TxQ CPU TxQ nf2_reg_grp user data path nf2c0 nf2c1 nf2c2 nf2c3 ioctl MAC TxQ MAC TxQ MAC RxQ MAC RxQ MAC TxQ MAC TxQ MAC RxQ MAC RxQ MAC TxQ MAC TxQ MAC RxQ MAC RxQ MAC TxQ MAC TxQ MAC RxQ MAC RxQ Ethernet

113 EuroSys 2008 - NetFPGA Tutorial 113 S T A N F O R D U N I V E R S I T Y IP Hdr: IP Dst: 192.168.2.3, TTL: 1, Csum:0x3ab4 Output Port Lookup EthHdr: Dst MAC = 0, Src MAC = x, Ethertype = IP Data 0 0 0 Pkt length, input port = 0 0xff output port = 10x04 1- Check input port matches Dst MAC 2- Check TTL, checksum – EXCEPTION! 3- Add output port module

114 EuroSys 2008 - NetFPGA Tutorial 114 S T A N F O R D U N I V E R S I T Y Output Queues Pkt OQ0 OQ1 OQ2 OQ7

115 EuroSys 2008 - NetFPGA Tutorial 115 S T A N F O R D U N I V E R S I T Y CPU Tx Queue IP Hdr: IP Dst: 192.168.2.3, TTL: 64, Csum:0x3ab4 IP Hdr: IP Dst: 192.168.2.3, TTL: 1, Csum:0x3ab4 EthHdr: Dst MAC = 0, Src MAC = x, Ethertype = IP Data 0 0 0 Pkt length, input port = 0 0xff output port = 10x04

116 EuroSys 2008 - NetFPGA Tutorial 116 S T A N F O R D U N I V E R S I T Y CPU Tx Queue IP Hdr: IP Dst: 192.168.2.3, TTL: 1, Csum:0x3ab4 EthHdr: Dst MAC = 0, Src MAC = x, Ethertype = IP Data 0 0 0 Pkt length, input port = 0 0xff output port = 10x04

117 EuroSys 2008 - NetFPGA Tutorial 117 S T A N F O R D U N I V E R S I T Y ICMP Packet For the ICMP packet, the packet arrives at the CPU Rx Queue from the PCI Bus Follows the same path as a packet from the MAC until the Output Port Lookup. The OPL module seeing the packet is from the CPU Rx Queue 1, sets the output port directly to 0. The packet then continues on the same path as the non-exception packet to the Output Queues and then MAC Tx queue 0.

118 EuroSys 2008 - NetFPGA Tutorial 118 S T A N F O R D U N I V E R S I T Y ICMP Packet Path Software PCI Bus NetFPGA CPU RxQ CPU RxQ CPU TxQ CPU TxQ CPU RxQ CPU RxQ CPU TxQ CPU TxQ CPU RxQ CPU RxQ CPU TxQ CPU TxQ CPU RxQ CPU RxQ CPU TxQ CPU TxQ nf2_reg_grp user data path nf2c0 nf2c1 nf2c2 nf2c3 ioctl MAC TxQ MAC TxQ MAC RxQ MAC RxQ MAC TxQ MAC TxQ MAC RxQ MAC RxQ MAC TxQ MAC TxQ MAC RxQ MAC RxQ MAC TxQ MAC TxQ MAC RxQ MAC RxQ Ethernet

119 EuroSys 2008 - NetFPGA Tutorial 119 S T A N F O R D U N I V E R S I T Y NetFPGA-Host Interaction Linux driver interfaces with hardware –Packet interface via standard Linux network stack –Register reads/writes via ioctl system call (with convenience wrapper functions) readReg(nf2device *dev, int address, unsigned *rd_data) writeReg(nf2device *dev, int address, unsigned *wr_data) eg: readReg(&nf2, OQ_NUM_PKTS_STORED_0, &val);

120 EuroSys 2008 - NetFPGA Tutorial 120 S T A N F O R D U N I V E R S I T Y NetFPGA-Host Interaction NetFPGA to host packet transfer PCI Bus 2. Interrupt notifies driver of packet arrival 3. Driver sets up and initiates DMA transfer 1. Packet arrives – forwarding table sends to CPU queue

121 EuroSys 2008 - NetFPGA Tutorial 121 S T A N F O R D U N I V E R S I T Y NetFPGA-Host Interaction NetFPGA to host packet transfer (cont) PCI Bus 4. NetFPGA transfers packet via DMA 5. Interrupt signals completion of DMA 6. Driver passes packet to network stack

122 EuroSys 2008 - NetFPGA Tutorial 122 S T A N F O R D U N I V E R S I T Y NetFPGA-Host Interaction Host to NetFPGA packet transfers PCI Bus 3. Interrupt signals completion of DMA 1. Software sends packet via network sockets. Packet delivered to driver. 2. Driver sets up and initiates DMA transfer

123 EuroSys 2008 - NetFPGA Tutorial 123 S T A N F O R D U N I V E R S I T Y NetFPGA-Host Interaction Register access PCI Bus 1. Software makes ioctl call on network socket. ioctl passed to driver. 2. Driver performs PCI memory read/write

124 EuroSys 2008 - NetFPGA Tutorial 124 S T A N F O R D U N I V E R S I T Y NetFPGA-Host Interaction Packet transfers shown using DMA interface Alternative: use programmed IO to transfer packets via register reads/writes –slower but eliminates the need to deal with network sockets

125 EuroSys 2008 - NetFPGA Tutorial 125 S T A N F O R D U N I V E R S I T Y Step 10 – Perfect the Router If interested, go back to “Demo 2: Step 1” after synthesis is done and redo the steps with your own router. You can also change the bandwidth and queue size settings to see how that effects the queue occupancy evolution. To run your router: 1- cd NF2/projects/tutorial_router/sw 2- type “./tut_adv_router_gui.pl --use_bin../../../bitfiles/tutorial_router.bit” Exercise 2

126 EuroSys 2008 - NetFPGA Tutorial 126 S T A N F O R D U N I V E R S I T Y Drop 1 in N Packets Objectives –Add counter and FSM to the code –Synthesize and test router Execution –Open drop_nth_packet.v –Insert counter code –Synthesize –After synthesis, test the new system. Exercise 3

127 EuroSys 2008 - NetFPGA Tutorial 127 S T A N F O R D U N I V E R S I T Y Step 1 - Open the Source We will modify the Verilog source code to add a counter to the drop_nth_packet module Open terminal Type “xemacs NF2/projects/tutorial_router/src/drop_nth_packet.v Exercise 3

128 EuroSys 2008 - NetFPGA Tutorial 128 S T A N F O R D U N I V E R S I T Y Step 2 - Add Counter to Module Add Counter using the following signals: counter – (16 bit signal) rst_counter – reset inc_counter – increment Search for insert counter (ctrl+s insert counter, Enter) Insert Counter and Save (ctrl+x+s) Exercise 3

129 EuroSys 2008 - NetFPGA Tutorial 129 S T A N F O R D U N I V E R S I T Y Step 3 - Build the hardware Start terminal, cd to “NF2/projects/ tutorial_router/synth” Run “make clean” Start synthesis with “make” Exercise 3

130 EuroSys 2008 - NetFPGA Tutorial 130 S T A N F O R D U N I V E R S I T Y Third Break on 5 th Floor (while hardware compiles)

131 EuroSys 2008 - NetFPGA Tutorial 131 S T A N F O R D U N I V E R S I T Y Step 4 – Test the Router You can watch the number of received and sent packets to watch the module drop every Nth packet. Ping a local machine (i.e. 192.168.7.1) and watch for missing pings. To run your router: 1- cd NF2/projects/tutorial_router/sw 2- type “./tut_adv_router_gui.pl --use_bin../../../bitfiles/tutorial_router.bit” To set the value of N (which packet to drop) type “regwrite 0x2000704 0xN” – replace N with a number To enable packet dropping: To disable packet dropping: type “regwrite 0x2000700 0x1” type “regwrite 0x2000700 0x0” Exercise 3

132 EuroSys 2008 - NetFPGA Tutorial 132 S T A N F O R D U N I V E R S I T Y Using the NetFPGA in the Classroom

133 EuroSys 2008 - NetFPGA Tutorial 133 S T A N F O R D U N I V E R S I T Y NetFPGA in the Classroom Stanford University –EE109 “Build an Ethernet Switch” Undergraduate Course for all EE students http://www.stanford.edu/class/ee109/ –CS344 “Building an Internet Router” Quarter-Long Course Taught Spr’05, Spr’07, Spr’08 http://cs344.stanford.edu Rice University –Nework Systems Architecture Spr’08 http://comp519.cs.rice.edu/

134 EuroSys 2008 - NetFPGA Tutorial 134 S T A N F O R D U N I V E R S I T Y Components of NetFPGA Course Documentation –System Design –Implementation Plan Deliverables –Hardware Circuits –System Software –Milestones Testing –Proof of Correctness –Integrated Testing –Interoperabilty Post Mortem –Lessons Learned

135 EuroSys 2008 - NetFPGA Tutorial 135 S T A N F O R D U N I V E R S I T Y NetFPGA in the Classroom Stanford CS344: “Build an Internet Router” –Courseware available on-line –Students work in teams of three 1-2 software 1-2 hardware –Design and implement router in 8 weeks –Write software for CLI and PW-OSPF –Show interoperability with other groups –Add new features in remaining two weeks Firewall, NAT, DRR, Packet capture, Data generator, …

136 EuroSys 2008 - NetFPGA Tutorial 136 S T A N F O R D U N I V E R S I T Y software hardware Switching Forwarding Table Routing Table Routing Protocols Management & CLI Exception Processing Interoperability Build basic routerRouting Protocol (PWOSPF) Integrate with H/W Emulated h/w in VNS Routing Table Routing Protocols Management & CLI Exception Processing Emulated h/w in VNS Routing Table Routing Protocols Management & CLI Exception Processing Emulated h/w in VNS Routing Table Routing Protocols Management & CLI Exception Processing Command Line Interface 123456 Innovate and add! Presentations Judges 4-port non-learning switch 4-port learning switch IPv4 router forwarding path Integrate with S/W Interoperability Switching Forwarding Table Learning Environment Modular design Testing CS344 Milestones Final Project

137 EuroSys 2008 - NetFPGA Tutorial 137 S T A N F O R D U N I V E R S I T Y Typical NetFPGA Course Plan WeekSoftwareHardwareDeliver 1Verify Software ToolsVerify CAD ToolsWrite Design Document 2Build Software RouterBuild non-learning switchRun Software Router 3Cmd. Line InterfaceBuild learning switchRun basic Switch 4Router ProtocolsOutput QueuesRun Learning Switch 5Implement ProtocolForwarding pathInterface SW & HW 6Control HardwareHardware RegistersHW/SW Test 7Interoperate Software & HardwareRouter Submission 8Plan new Advanced FeatureProject Design Plan 9Show new Advanced FeatureDemonstartion

138 EuroSys 2008 - NetFPGA Tutorial 138 S T A N F O R D U N I V E R S I T Y CS344 Project Presentations http://cs344.stanford.edu

139 EuroSys 2008 - NetFPGA Tutorial 139 S T A N F O R D U N I V E R S I T Y Network Systems Architecture at Rice http://comp519.cs.rice.edu/

140 EuroSys 2008 - NetFPGA Tutorial 140 S T A N F O R D U N I V E R S I T Y Networked FPGAs in Research 1.RCP: Congestion control New module for parsing and overwriting new packet New software to calculate explicit rates 2.Packet Monitoring (ICSI) Network Shunt 3.Deep Packet Inspection (FPX) TCP/IP Flow Reconstruction Regular Expression Matching Bloom Filters 4.Ethane: Network security New switch (“managed flow-table”) deployed 5.Buffer Sizing Reduce buffer size and measure effect on network performance. Need a way to set buffer size, and measure buffer occupancy.

141 EuroSys 2008 - NetFPGA Tutorial 141 S T A N F O R D U N I V E R S I T Y The NetFPGA.org Community http://netfpga.org/

142 EuroSys 2008 - NetFPGA Tutorial 142 S T A N F O R D U N I V E R S I T Y Group Discussion Your plans for using the NetFPGA –Teaching –Research –Other Resources needed for your class –Source code –Courseware –Examples Your plans to contribute –Expertise –Capabilities –Collaboration Opportunities

143 EuroSys 2008 - NetFPGA Tutorial 143 S T A N F O R D U N I V E R S I T Y Survey How did you like this this tutorial? –What did you find useful? –What should be improved? –What should be removed? –What should be added? Can we post the video from this event? –If not, please let us know. Complete On-line survey –http://netfpga.org/tutorial_survey.html

144 EuroSys 2008 - NetFPGA Tutorial 144 S T A N F O R D U N I V E R S I T Y Acknowledgements Jianying Luo, Glen Gibb, Nick McKeown, Greg Watson, Jim Weaver, Jad Naous, Ramanan Raghuraman, Paul Hartke, John Lockwood Also (not shown): Adam Covington, David Erickson, Brandon Heller, Neda Beheshti, Sara Bolouki David Miller, University of Cambridge NetFPGA Team at Stanford University: January 2007

145 EuroSys 2008 - NetFPGA Tutorial 145 S T A N F O R D U N I V E R S I T Y Major support for this Tutorial Previous Tutorial Hosted By: SIGMETRICS Special thanks to Joe Sventek

146 EuroSys 2008 - NetFPGA Tutorial 146 S T A N F O R D U N I V E R S I T Y Acknowledgements Support for the NetFPGA project has been provided by the following companies and institutions Disclaimer: Any opinions, findings, conclusions, or recommendations expressed in this material do not necessarily reflect the views of the National Science Foundation or of any other sponsors supporting this project.

147 EuroSys 2008 - NetFPGA Tutorial 147 S T A N F O R D U N I V E R S I T Y Reference on the Web NetFPGA homepage http://NetFPGA.org


Download ppt "EuroSys 2008 - NetFPGA Tutorial 1 S T A N F O R D U N I V E R S I T Y Presented by: John W. Lockwood & G. Adam Covington (Stanford University) Andrew Moore."

Similar presentations


Ads by Google