An Efficient Gigabit Ethernet Switch Model for Large-Scale Simulation Dong (Kevin) Jin

Outline Overview and Motivation Our Approach Measurement Current Results Ongoing Work

Overview and Motivation Gigabit Ethernet Widely used in large-scale network e.g. network backbone, power grid, data center… High bandwidth, low latency Packet delay and packet loss is now mainly caused by switch

Overview and Motivation Use Simulation to Study Large-scale Gigabit Ethernet RINSE Simulator Expand the network Explore different architectures An efficient switch model in RINSE is needed.

Existing Switch Models Detailed models (OPNET, OMNet++) Different models for different type of switches High computational cost, not scalable Require constantly update and validation Simple Queuing Model (Ns-2, DETER) Simple FIFO queue One model for everything Queuing model based on data collected from real switch [Roman2008] [Nohn2004] Device-independent Parameter based on experimental observations

Model Requirements Fast simulation speed highly abstract, no internal details Accurate packet delay and loss Device-independent parameters derived without knowing device internals Same model derivation process Our Model Ns-2 OMNet++ OPENNet Queue model based on experiments Ns-2 OMNet++ OPENNet Queue model based on experiments Our Model Accurate packet delay and packet loss Less accurateMore Accurate Simulation Speed Slow Fast

Our Approach Black-Box Model Focus on packet delay and packet loss No detailed architecture, no queues and no forwarding algorithm Explore the statistical relation between data- in and data-out Paramters derived from data collected on real swtiches

Our Approach Perform Experiments on real switch Build Analytical model Build RINSE model Evaluate Simulation Speed and Accuracy

Experiment Gigabit Ethernet Environment High bit rate - 1Gb/s Low latency in switch - order of microsecond Experiment Sender -> swtich -> Receiver Constant Bit Rate (CBR) UDP flows Different sending rate, packet size, number of flows Timestamp each packet and collect trace Obtain one-way delay of each packet packet loss sequence

Experiment Difficulties Clock synchronization Sender and receiver on the same computer Accurate timestamp for one way delay One Way Delay = transmission delay + wire propagation delay + delay in switch + delay in end host Software Timestamp at NIC driver, microsecond resolution Large delay at end hosts at high bit rate (>500Mb/s) Have to use hardware timestamp (NetFPGA) 4 on-board Gigabit Ethernet ports 10 nanosecond resolution No end-host delay, processing on the card

Experiment Setup CBR UDP flows packet size sending rate, #background flows T_4 - T_2 = delay per packet packet capture problem: 2000 packets without missing at 1Gb/s

Results - Packet Delay (Low Load) Single flow Fixed packet size, delay is constant for any sending rate Sufficient processing power to handle 1Gb/s single flow Model packet delay with a constant

Results - Packet Delay (High Load) 3 extra 950Mb/s background UDP flows NetGear constant delay with small variance 3COM processor-sharing scheduling weight to a flow - bit rate Mean Delay Vs Sending Rate (packet size = 100 Bytes)

Results - Packet Delay (Beginning) Packet Delay at Beginning with differenet sending rate (Mb/s) Beginning No idea about bit rate Assign maximum weight Sufficient packets passed Weight decreased packet delay increased Reach Stable mean delay Weight is fixed

Results - Packet Loss A sample portion of entire 40,000 packets Loss rate NetGear 0.4% 3COM 0.6% Strong autocorrelation exists 0 - received 1 - lost Packet Loss Sample Pattern

Results - Packet Loss Kth order Markov Chain Large K, large state space L01S0 State Space: State of i th packet Our Model, next state depends on the current state sum of previous K packets in the same state

RINSE - Architecture Scalable, parallel and distributed simulations Incorporates hosts, routers, links, interfaces, protocols, etc Domain Modeling Language (DML) A range of implemented network protocols Emulation support DML Configuration SSFNet configure SSF [Simulation Kernel] enhance SSF Standard/API implements Protocol Graph Interface 1 MAC PHY Interface N MAC PHY IPV4 ICMP Emulation Socket TCPUDPDNP3 MODBUS BGPOSPF …

RINSE - Switch Model Ethernet MAC and PHY Switch Layer black-box model Simple output queue model Flip-coin model - random delay and packet loss Simulation Time: complex queuing model > simple output queuing model > our black-box model > coin model Switch Ethernet MAC Ethernet PHY Switch IP Ethernet MAC Ethernet PHY Host A UDP APP IP Ethernet MAC Ethernet PHY Host B UDP APP

Ongoing work Experiment Collect long trace with Endace DAG card Cross-interface traffic Model Study correlation between packet loss and delay Generate loss-delay correlated traffic Evaluate Simulation speed in large-scale network Accuracy of the black-box model

Thank You

Experiment Setup I Host NIC 1NIC 2traffic sendertraffic receiver timestamp Packet capture Switch send to self timestamp at NIC driver NIC to NIC overhead