An Active Networking Testbed for Storage Presenter Mel Tsai People Mel Tsai Anshi Liang Paul Huang Perry Dong and Tal Lavian

Outline Motivation The Testbed The Alteon-iSD Application Driver: Storage Networks Early Experiments with iSCSI Discussion & Results Next Steps

Motivation Faster silicon enables increased processing and strong filtering in the network fabric Emergence of high-speed programmable network processors L5+ processing ability Per-flow classification and routing Traditionally, routers excel at L2-L3 classification and forwarding, servers must do L5-L7 computation Neither can do both with high performance

A New Platform Router++ L2-L7 classification, QoS, load balancing, & more… L5-L7 Compute Processor(s) User Flows Servers

Platform Characteristics A new approach towards network programmability Programming model: separate forwarding and compute planes High performance can be achieved when Most packets take the fast path: # of packets sent to compute processor << total packet rate Average latency of computation is low enough

The Testbed Goals Develop a useful research platform based on this approach Understand the performance, programming model, and versatility of this approach

Enabling device: The Alteon-iSD A combination “router/server” Alteon 184 Web Switch 9-port load-balancing switch with L2-HTTP filtering Integrated Services Director (iSD) A dual-processor 1U PC with GbE interfaces Low-level control/data protocol: NAAP

Testbed Setup 8 PCs 4 Alteon Web Switches 2 iSDs 2 Passport 8600 Routers 10 Bay Networks L2/L3 switches GbE connections between many points

Basic Alteon-iSD Operation Client(s) Server(s)

1 Client(s) Server(s) Write TCP/IP or HTTP filters for redirection to iSD Current filter 2: sip dip proto tcp vlan any action redir, group 1, rport 23 ack_or_reset disabled BW Contract 1024 Basic Alteon-iSD Operation

Client(s) 2 On the iSD, open control & data tunnels to the Alteon Server(s) Basic Alteon-iSD Operation

Client(s) 3 Begin forwarding matching packets to the iSD via a datagram tunnel Server(s) Basic Alteon-iSD Operation

Client(s) 4 Received packets are examined and possibly modified by software on the iSD Server(s) Basic Alteon-iSD Operation

Client(s) 5 Packets are returned to Alteon. Status and table/filter updates are sent via a control tunnel. Server(s) Basic Alteon-iSD Operation

Client(s) 6 Alteon forwards packet to (possibly new) destination. Alteon’s configuration, filters, and routing tables are updated if instructed by the iSD. Server(s) Basic Alteon-iSD Operation

Application Driver We want to evaluate the performance, versatility, and programming model of our testbed Where are the performance bottlenecks? What is the right programming model? We have chosen storage networks as a new application driver Recent convergence of storage networks with IP networks: iFCP, FCIP, iSCSI New crop of intelligent “storage directors” with Alteon-iSD-like properties

The Alteon-iSD as a Storage Building Block Clients Large Storage Array or FC SAN (EMC, IBM, etc.) Clients Commodity storage devices (PCs, RAIDs, disks)  versatile, low cost, IP-based, redundant, wide-area

An IP Storage Protocol: iSCSI iSCSI Initiator (“client”) SCSI-over-TCP/IP connection A standard point-to-point iSCSI session: Ethernet IP TCP iSCSI SCSI Cmd Data iSCSI Target (“server”)

An IP Storage Protocol: iSCSI A standard point-to-point iSCSI session: /dev/sdb iSCSI Target (“server”) iSCSI Initiator (“client”)

Alteon-iSD Overhead (1) No iSD involvement iSD intercepts pkts, no computation iSD intercepts pkts, recomputes IP cksum MB/sec when copying a 650MB file For a single point-to-point iSCSI session:

Alteon-iSD Overhead (2) No iSD involvement iSD intercepts pkts, no computation iSD intercepts pkts, recomputes IP cksum MB/sec when copying 10,000 small 18KB files For a single point-to-point iSCSI session:

A Work in Progress: iSCSI Synchronous Mirroring iSCSI Client Backup Disk iSCSI Target Disk Objective: Synchronously and transparently mirror operations sent to the iSCSI Target on a backup iSCSI disk : : 3260 “The Alteon-iSD Storage Director”

Implementation on the Alteon-iSD Alteon’s role: just a basic TCP/IP redirection filter TCP dest port 3260 IP addresses matching client or target iSD’s role: inspects incoming packets, copies iSCSI requests to the backup disk, merges responses

Testbed Discussion Need access to Alteon firmware to perform useful filtering on iSCSI packets The Alteon cannot do simple packet modification beyond simple TCP/IP header updates Desirable to perform some computation in the fast path Breaks the clean separation between forwarding vs. computation

Testbed “Wish List” Arbitrary L2-L7 pattern matching with fine- grained tagging Fast-path L2-L7 packet modification Hardware acceleration in both planes FPGAs or some other hardware mechanism is required when simple operations must be performed on every packet What is the right abstraction for the programmer?

Some Next Steps Understand the fundamental trade-offs when separating forwarding vs. computation Understand where the system-level bottlenecks exist Implement some more complex benchmarks Striping, clustering and virtualization, disk-disk backup, asynchronous mirroring, FC-to-iSCSI gateway Compare & contrast our testbed to existing storage directors

Conclusion This testbed is general enough to implement IP storage applications, but… The Alteon-iSD was not designed for storage, and in some respects it shows Need better filtering & packet modification There is probably no “ideal” programming model for applications that benefit from L2-L7 filtering and increased computation in the network Application-specific, high-level approaches may be the best