Customizing Virtual Networks with Partial FPGA Reconfiguration Dong Yin, Deepak Unnikrishnan, Yong Liao Lixin Gao and Russell Tessier Funded by National Science Foundation Grant CNS-0831940 Electrical and Computer Engineering University of Massachusetts, Amherst USA

Outline Network virtualization FPGA as a network virtualization platform Customizing virtual routers with Partial FPGA reconfiguration Dynamic virtual router allocation Results Conclusions

Network Virtualization Many logical networks share a physical network infrastructure Reduces costs Independent routing policies Key Challenges Isolation Performance Scalability Flexibility Usability

Network Virtualization Techniques Software Full/Container virtualization ASIC/Network Processors Supercharging PlanetLab Juniper E series S/W ASIC Scalability High Limited Flexibility Performance Low Isolation Moderate Usability Good

FPGA as a Virtualization Platform High performance Flexible hardware through reconfiguration Full reconfiguration Reprogram entire chip Chip shut down during reconfiguration Partial reconfiguration Reprogram ‘part’ of the chip Non-reconfigured regions can still operate during reconfiguration

FPGA as a Virtualization Platform Key Challenges Resource isolation between virtual routers Scalability - Limited on-chip logic In this paper, Use partial reconfiguration for better resource isolation between virtual routers Combine software virtual routers with hardware routers to create a scalable system

System Overview NIC NIC Linux NetFPGA OpenVZ VRouter OpenVZ VRouter Software bridge Linux Kernel driver PCI 1G Ethernet I/F SDRAM SRAM PHY HW VRouter SDRAM HW VRouter SRAM NetFPGA

Reconfigurable Region Architecture Fwd Logic Fwd Table PRR1 PRR2 CPU Transceiver MAC RXQ CPU RXQ MAC TXQ CPU TXQ Input Arbiter Design Select Output Queues PCI I/F Bridge OpenVZ Virtual Router VIP TYPE x.x.x.x HW y.y.y.y SW VID 1 CONTROL FPGA Static Reconfigurable Region Linux

Reconfigurable Region Architecture Fwd Logic Fwd Table PRR1 PRR2 CPU Transceiver MAC RXQ CPU RXQ MAC TXQ CPU TXQ Input Arbiter Design Select Output Queues PCI I/F Bridge OpenVZ Virtual Router VIP TYPE x.x.x.x HW y.y.y.y SW VID 1 CONTROL FPGA Static Reconfigurable Region Linux 9

Dynamic Virtual Router Management Virtual router requirements change over time Varying bandwidth/latency requirements Different routing policies -> Different h/w resources Solution - Exploit heterogeneity High throughput routers -> Hardware Low throughput routers -> Software Dynamically migrate virtual routers between s/w and h/w

Dynamic Virtual Router Management Allocation Try allocation in software (if b/w permits) Else try allocation in hardware (if b/w and resources permit) Removal S/W - Destroy OpenVZ container H/W - Program blank bitstream to reconfigurable region Upgrades Greedily migrate virtual routers between FPGA and Software based on bandwidth requirement

Evaluation Source Virtual Router Sink 2 Partially reconfigurable virtual routers on Virtex II Software virtual routers - 3Ghz AMD X2 2GB RAM Metrics Throughput Reconfiguration time Packet generation NetFPGA packet generator Source NetFPGA Pktgen Virtual Router Sink NetFPGA Pktcap 1Gbps 1Gbps

Partial Bitstream Generation Virtual router 2 (Verilog) Virtual router 1 (Verilog) Xilinx Early Access Partial Reconfiguration Partial bitstream repository PlanAhead JTAG

Throughput of Partially Reconfigurable Virtual Router Consistent line rate (1Gbps) across packet sizes

Traffic Isolation and Reconfiguration Time Full Reconfiguration Approach Fully Reconfigure FPGA eth OpenVZ H/W Virtual Router A Virtual Router B Source Sink MAC Qs Design Select Output S/W Bridge FPGA H/W Virtual router B’ Linux

Traffic Isolation and Reconfiguration time Full reconfiguration 12 seconds Partial reconfiguration 20x reduction in downtime

Throughput per virtual router Partial reconfiguration benefits at higher reconfiguration frequencies

Dynamic Virtual Network Allocation Evaluation Model 1000 virtual networks Bandwidth distribution from PlanetLab nodes. Poisson arrivals and Poisson lifetimes Mean arrival period = 2hrs Mean lifetime = ~2.5 days (64 hrs) Bandwidth of live network changes according to Uniform distribution between 0%-X% from initial allocation b/w All networks can be either allocated in FPGA or Software virtual routers

Benefit of Virtual Network Migration Upto 20% more upgrade requests serviced by dynamic assignment Reallocation benefits over wide fluctuations in bandwidth

Resource Usage and Power Consumption FPGA # Partially Reconfigurable Virtual Routers Virtex II 2 Virtex 5 32 Power management via Clock gating Shutdown virtual routers when not in use Saves 10% overall power

Conclusion A novel heterogeneous network virtualization platform Enhanced resource isolation and reconfiguration time With partial FPGA reconfiguration Scalable Combines fast FPGA virtual routers with software virtual routers Dynamic network allocation and migration Towards a green virtualization platform Integrates power management techniques

Future Work Usability Higher level abstractions for creating and managing partial bitstreams Explore non-JTAG interfaces for reconfiguration (PCI Express/ICAP)

Thank you Questions?