An Efficient Gigabit Ethernet Switch Model for Large-Scale Simulation Dong (Kevin) Jin

2 Overview and Motivation Gigabit Ethernet High bandwidth (1Gb/s) Widely used to build large-scale networks with many applications Packet delay and packet loss is mainly caused by switch

3 Overview and Motivation Switch Model Fast simulation speed Accurate delay and loss Use simulation to study applications on large-scale Gigabit Ethernet Ethernet Switch … Relay Data Aggregator … …......

4 Switch Models Model Simulation Speed Accuracy of delay/loss Detailed Models SlowVery High E.g. OPNET, OMNet++ Model internal details Different models for different types of switches High computational cost

5 Switch Models Model Simulation Speed Accuracy of delay/loss Detailed Models SlowVery High Simple Queuing Model Very FastLow E.g. Ns-2, DETER Simple FIFO queue One model for everything

6 Switch Models Model Simulation Speed Accuracy of delay/loss Detailed Models SlowVery High Simple Queuing Model Very FastLow [Roman2008] [Nohn2004] FastHigh One model Multiple queues Based on data collected from real switches

7 Switch Models Model Simulation Speed Accuracy of delay/loss Detailed Models SlowVery High Simple Queuing Model Very FastLow [Roman2008] [Nohn2004] FastHigh Our ModelFastestHigh Black-box Switch Model Not model internal details Explore relations between data-in and data-out Based on data from real switches

8 Model Design Approach 1. Perform Experiments on real switch 2. Build Analytical model 3. Build RINSE model 4. Evaluate Simulation Speed and Accuracy

9 Model Design Approach 1. Perform Experiments on real switch 2. Build Analytical model 3. Build RINSE model 4. Evaluate Simulation Speed and Accuracy

10 Experiment Data sequence to collect One-way delay in switch Packet loss pattern in switch Challenges in Gigabit Environment High bit rate - 1Gb/s Small delay in switch -  s

11 Experiment Difficulty Accurate timestamp for one-way delay (  s resolution) Software Timestamp At NIC driver Large delay generated at end hosts at high bit rate (>500Mb/s) Hardware timestamp (NetFPGA) 10 ns resolution Eliminating end-host delay

12 NetFPGA Card 1234 Experiment Setup Constant Bit Rate UDP flows Time_2 - Time_4 = delay per packet inside switch Problem: capture only about 2000 packets without a miss at 1Gb/s Input pcap Time 2Time 4 switch

13 Preliminary Experimental Results - Packet Delay (Low Load) Delay NOT dependent on sending rate Sufficient processing power to handle single flow up to 1Gb/s Model packet delay as a constant Delay Vs Sending Rate (packet size = 100 Bytes)

14 Preliminary Experimental Results - Packet Delay (High Load) Flow1: Mean Delay Vs Sending Rate (packet size = 100 Bytes) Performance varies about 100 times One model not enough for all switches Have to build model based on experimental data flow 1 Background traffic {

15 Preliminary Experimental Results - Packet Loss 0 - received 1 - lost A Packet Loss Sample Pattern 3COM Loss rate NetGear 0.4% 3COM 0.6% Strong autocorrelation exists among neighboring packets

16 Model Design Approach 1. Perform Experiments on real switch 2. Build Analytical model 3. Build RINSE model 4. Evaluate Simulation Speed and Accuracy

17 Packet Loss Model Goal Fast simulation speed Accurate average loss rate Accurate autocorrelation Existing Model - Kth Order Markov Chain Next state depends on previous K packets 2^K states e.g. K=2

received 1 - lost state 1 state 2 state 3 Packet Loss Model Our Markov Chain Model state type 1 - long burst of 0s state type 2 - short burst of 0s state type 3 - burst of 1s Next state depends on Current state #successive packets already in the current state

19 Delay Model - Copula Delay model needs to capture Marginal distribution of packet delay Autocorrelation among neighboring packets Copula Model Sklar’s theorem: Joint Distribution is characterized by Marginal distributions of each component Copula

20 Gaussian Copula - Result Gaussian Copula Given , we can generate the output series {Z t } e.g. for n=3, (Z 1, Z 2, Z 3 )  Z 4, then (Z 2, Z 3, Z 4 )  Z 5,… Same autocorrelation specific by  Each Z t has the marginal distribution ~ N(0,1)  - Standard Gaussian pdf N~(0,1),  - correlation coefficient matrix

21 Model autocorrelation very well Computational efficiency conditional distribution for multivariate Gaussian where Why Gaussian Copula?

22 Gaussian Copula - Apply to Empirical Distribution If the empirical delay distribution X(t) is not Gaussian? (1)Transform X(t) standard Gaussian Y(t), through CDF F(X) = G(Y) (2)Gaussian Copula: Y(t)  Z(t) (3)Transform Z(t) back to W(t), similar to (1) Delay (  s) CDF X W

23 Gaussian Copula - Apply to Empirical Distribution If the empirical delay distribution X(t) is not Gaussian? (1)Transform X(t) standard Gaussian Y(t), through CDF F(X) = G(Y) (2)Gaussian Copula: Y(t)  Z(t) (3)Transform Z(t) back to W(t), similar to (1) Autocorrelation Table L ag

24 Summary Need an efficient switch model to study applications on large-scale gigabit Ethernet A black-box model with focus on Fast simulation speed Accurate delay and loss Experimental results justified our approach Experiment  analytical model  Simulation model  Evaluation

25 Ongoing Work Experiment Collect long data traces with Endace DAG cards Test with cross-interface traffic Model Model for more complicated traffic (TCP, mixing traffic) Correlation between delay and loss Evaluation Compare simulation speed with the other models Compare simulation-generated trace with real data traces

26 Thank You

27 Gaussian Copula Details Autocorrelation,  matrix Conditional distribution for multivariate Gaussian

28 RINSE - Switch Model 3 Models in Switch Layer Our black-box model Simple output queue model Flip-coin model Expected Simulation Time on a chain of switches (about routers): complex queuing model > simple output queuing model > our black-box model ≥ coin model Switch Ethernet MAC Ethernet PHY Switch IP Ethernet MAC Ethernet PHY Host A UDP APP IP Ethernet MAC Ethernet PHY Host B UDP APP

29 Expanding Our Packet Loss Model 1,13,1 1,23,2 Lost 2,1 3,3 3,N 2,2 Received (long burst) Received (short burst) 1,32, ,M ,K......

30 Experimental Results - Packet Delay (High Load) Packet Delay at Beginning of experiment under differenet sending rate (Mb/s) 3COM - Processor Sharing No idea about bit rate until sufficient packets passed Assign max weight at beginning Passed packets   bit rate dertermined  weight   delay 

31 Experimental Results - Packet Delay (Low Load) Single flow Delay NOT depends on sending rate Sufficient processing power to handle 1Gb/s single flow Model packet delay as a constant

32 Experiment Setup with Software Timestamp Host NIC 1NIC 2Sender PortReceiver Port Timestamp process Packet capture process Switch Send to self Timestamp at NIC driver NIC to NIC overhead

33 RINSE - Architecture Large scale network simulation Incorporates hosts, routers, links, interfaces, protocols, etc Domain Modeling Language (DML) A range of implemented network protocols Emulation support DML Configuration SSFNet configure SSF [Simulation Kernel] enhance SSF Standard/API implements Protocol Graph Interface 1 MAC PHY Interface N MAC PHY IPV4 ICMP Emulation Socket TCPUDPDNP3 MODBUS BGPOSPF …

34 Black-Box Testing RFCs 2544 and Guidelines Describing the steps to determine the capabilities of a router No discussing on how to create model from measurements [Hohn 2004] Bridging router performance and queuing theory Simple queuing model, no loss events, and no interactions among ports 12 DAG cards synchronized by GPS [Roman 2007] A black-box router profiler Software testbed (ns2, Click modular router) Focus on single UDP flow and multiple TCP flows

35 [Hohn 2004] “Bridging router performance and queuing theory” Simple output FIFO queue Adding delay based on empirical data before entering the queue No packet loss No interactions among ports

36 [Roman 2008] [Roman 2008] A Device-Independent Router Model Multiple queue model Device-specific parameters derived from experimental data Focus on accurate queue size, number of servers Input ports Output ports

37 Kth order Markov Chain [Yajnik 1999] 2^k states, 2^(k+1) conditional probability Special case: bernoulli model (k=0), two-state markov Chain (k=1) Two-state Markov Chain (Gilbert Model) Extended Gilbert Model Existing Packet Loss Model

38 Autocorrelation of Packet Loss Pattern

39 Latency expectations on wired Ethernet End-to-end latency in switched Ethernet a function of the scheduling and call admission procedure in use not specific to Ethernet hard guarantee the Guaranteed Service specs (RFC 2212) statistical guarantees earliest-deadline-first virtual clock service curve based schedulers (INFOCOM 2000 paper by Liebeherr)