High Speed Interconnect Project May08-06

At this time, the maximum real-world throughput of 10 Gbps network configurations is unknown.

Lockheed Martin (LM) needs a test plan designed and executed to measure the maximum real-world throughput of a 10 Gbps network composed of COTS components.

Project Goals Create and test a network capable of reaching 10 Gbps with Commercial Off The Shelf (COTS) components Topology has to use fiber optics Remain within approx. $3500 budget

Possible Strategies Use Advanced TCA or µTCA Architectures Use PCI Express Network Cards with a XFP Switch Use PCI-X Network Cards with a XFP Switch

µTCA Node Strategy

µTCA Configuration Single Node composed of the following Three 10 Gbps Network Interface Cards MicroTCA Carrier Hub Power module Nodes can be connected in any way possible Testing would be done with a single node due to the cost of components

PCI Express Node Strategy Node to Node Testing Configuration Final Testing Configuration

PCI Express Implementations Encoded Data Rate Unencoded Data Rate x1 5 Gbps 4 Gbps (500 MB/sec) x4 20 Gbps 16 Gbps (2 GB/sec) x8 40 Gbps 32 Gbps (4 GB/sec) x16 80 Gbps 64 Gbps (8 GB/sec) Source: http://www.dell.com/content/topics/global.aspx/vectors/en/2004_pciexpress?c=us&l=en&s=corp System PCI-E slots must be at least x4 to allow for 10 Gbps testing

PCI Express Configuration Composed of three systems and a Ethernet switch Initial testing will be completed with two systems directly connected Final testing will be done with whole system

Comparison Advanced TCA and µTCA PCI Express Advantages Disadvantages Modular design allows for expansion 12.5 Gbps native system bus Disadvantages Network Interface Card Technology for Advanced Mezzanine Cards hasn’t reached 10 Gbps Costly components Advantages Readily available optical 10 Gbps NICs Variety of 10 Gbps XFP Switches Relatively low cost components Disadvantages PCI-E bus bandwidth does not reach 10 Gbps Lack of systems that support PCI-E NICs Advanced TCA and µTCA PCI Express

PCI-X Configuration Same node strategy as PCI Express Bus speed max of approx. 4.25 Gbps Rarity of cards and slots achieving this bandwidth This configuration would be used only as a backup plan

Presented Hardware Setup NXB-10GXxR Intelligent NIC® 10 Gigabit Ethernet PCIe Adapter with pluggable XFP optical interface (http://www.netxen.com/products/boardsolutions/NXB-10GXxR.html) Node 1 Node 2 Node 3 TigerSwitch 10G 8-Port Standalone XFP 10Gigabit Ethernet Managed Layer 2 Switch SMC Networks, Inc.

Network Interface Card NXB-10GXxR Intelligent NIC® PCI-E x1/x4/x8 Connection

Testing Specifications Bandwidth Measurement Channel Capacity Bandwidth Efficiency (Throughput) Switching Time Measurement Latency Measurement Quality of Service Measurement Load Balancing Measurement

Bandwidth Measurement Node 1 Node 2 What is the maximum Channel Capacity between two nodes? What is the effective bandwidth efficiency or amount of information transmitted versus overhead (frame formatting, coding, etc.)?

Switching Time Measurement Node 1 Node 2 Node 3 How quickly can the source for information to Node 3 be switched?

Latency Measurement Node 3 Node 1 Node 2 What is the time delay between output on Node 1 or Node 2 to input on Node 3?

Quality of Service Measurement Node 1 Node 2 Node 3 If two Nodes are transmitting to the same Node how is bandwidth allocation or QOS handled?

Load Balancing Measurement Node 1 Node 2 Node 3 If two Nodes are transmitting to the same Node how is bandwidth balanced between the two sources?

Questions? ?