LiNK: An Operating System Architecture for Network Processors Steve Muir, Jonathan Smith Princeton University, University of Pennsylvania

The Network Processor environment Many types of network device include a processor High performance host NICs e.g., gigabit cards Remote management (ILO) devices Router line cards Virtual network devices (more on this later...)

Example applications High-speed packet processing Traffic shaping Firewall/intrusion detection Monitoring/logging Remote management

Why a network processor OS? Device-independent portability layer –run the same application on diverse network processors Hardware abstraction –hide details of network processor environment from app. Isolation between components –protect services and drives from bugs Multitasking –simplifies the programming task Library of common functionality –memory management, profiling, logging, etc.

LiNK: The Lightweight Network Kernel Simple, event-driven main loop –interrupt handlers scheduled as events –synchronous processing reduces complexity of kernel Network service components –take advantage of simple, uniform application structure Posted service requests –efficient communication between LiNK and clients

Network Service Components Fundamental structure shared by all applications Three elements: 1.Transmit filter – add packet headers, packet scheduling 2.Receive handler – generate response packets 3.Timeout callback – flush caches, packet retransmit Service functions scheduled by kernel –functions run to completion or explicit yield Many services fit into this model –ARP, ICMP, traffic shaping

Posted Service Request object

Posted Service Request processing Client posts to shared object - asynchrony Kernel polls and processes - concurrency Chaining required for efficiency Speculation and reference arguments Events for completion and failure notification Alloc_Mem Cap_Ref_New 12 Alloc_Mem Cap_Ref_New 12 Make_Frameset Cap_Ref_New Set_Peer Speculation Data Dependency

Virtual Network Processors Multi-core processors are cheaply available –multi-threaded and/or multi-core CPUs Virtualisation technology has matured too –low overhead, enhanced by hardware support Use a processing element as a network processor A perfect prototyping and development environment –testing and debugging on real hardware is hard But maybe something more...

Current Implementation Hybrid Linux/LiNK system –Linux as host operating system, LiNK as kernel module –LiNK accessible to Linux as standard ethernet device LiNK provides user-space network subsystem –Linux provides filesystem, processes, scheduling, VM, etc. Simple way to prototype new NP feature –e.g., a direct user-space interface to the (virtual) NP

Evaluating the Virtual NIC Ported the Flash webserver to Linux+LiNK Provided TCP/IP protocol stack as user-space library –webserver used unmodified WebStone 2.5 HTTP benchmark –simulates realistic workload with multiple clients –small number of files, representative size and distribution Compare performance of Linux and Linux+LiNK

Evaluation: Communication Overhead

Evaluation: Flash performance

Conclusions Network processors need operating systems The Lightweight Network Kernel is one alternative –Simple, specialised structure –Asynchronous, high performance communication The future: virtual network processors

Questions/comments

Evaluation: Network Throughput

Evaluation: Polling Performance

Evaluation: Polling Scalability

Related Work Network device polling –Click modular router [Kohler] - 4x performance increase –Scout/IXP1200 router [Peterson] - similar Parallel network protocol stacks [Bjornberg, Naburn] –processor-per-packet scales well for simple protocols –complex protocols => severe lock contention Network appliance optimisations –I/O Lite - unified buffer management [Pai] –Soft timers - polling in an interrupt-driver kernel [Aron]

Future Work Responsiveness of polling –dynamic specialisation e.g., run-time code generation Scalability –how many processors can Piglet support? –how many applications/devices? Alternative applications for Piglet –network processor e.g., IXP1200, operating system