A Hybrid Systems Modeling Framework for Communication Network Junsoo Lee (USC) Prof. Stephan Bohacek Prof. Joao Hespanha Prof. Katia Obraczka Sep 8, 2004

2 Motivation Problems in network research: –Design, test and analyze protocols (throughput, fairness, security). –Validate designs (scalability, performance). –Tune network/protocol parameters (Queue size, bandwidth, timers, etc.). Study of networks and network protocols have used: –Analytical Fast Significant accuracy loss Only applicable to limited application –Simulation Accurate Long simulation time Large memory overhead –Combination

3 Publications A Hybrid Systems Modeling Framework for Fast and Accurate Simulation of Data Communication Networks. In Proc. SIGMETRICS, June 2003 Analysis of a TCP hybrid model. In Proc. of the 39th Annual Allerton Conf. on Comm., Contr., and Computing, Oct Hybrid Modeling of TCP Congestion Control. In Maria Domenica Di Benedetto, Alberto Sangiovanni-Vincentelli, Hybrid Systems: Computation and Control, number 2034 in Lect. Notes in Comput. Science, pages , Mar. 2001

4 Outline Related work Simple case studies (e.g., UDP, TCP variations). Generalized Hybrid Model Framework Validation Scalability of Hybrid Systems Software Tools Conclusion

5 Related Work: Packet Model Track individual data packets. Computationally intensive. Complexity depends on the number of events. Does not scale to high bandwidths and complex topologies. NS-2 (NS00) Pdns (Riley99) QualNet Opnet (Desbrandes93) SSFNET Dummynet, Nistnet

6 Related Work: Fluid Model Track time/ensemble-average packet rates. Computationally efficient. Complexity depends on the rate changes. Only suitable to model many flows. Does not explicitly model individual events. ATM (Kesidis96) Time driven (Yan99) Stochastic Differential Equation (Misra99,20) Time-Stepped Hybrid Simulation (Guo00) Fluid-Simulation using SSF (Nicol98) More efficient and larger scale (Liu03)

7 Our Approach: Hybrid System Track packet rates for each flow averaged over small time scales. Capture dynamics of each flow Explicitly models some discrete events (drops, queues becoming empty, etc.). Time accuracy of a few milliseconds (round-trip time).

8 Related Work: Other Models Hybrid models: –Discrete event + analytical technique (Schetman78). –Packet (foreground) + fluid model (back-ground) (Melamed01). –Packet (edge) + fluid model (backbone) (Riley02, Yu04)). –Hybrid buffer model (Cameron03). Simulation, Sampling & Analysis: –SHRiNK (Psounis03) Abstraction: –Abstract Technique (Huang99) –Packet Train (Ahn96)

9 Talk Outline Related work Simple case studies (e.g., UDP, TCP variations). Generalized Hybrid Model Framework Validation Scalability of Hybrid Systems Software Tools Conclusion

10 Simple Hybrid Model Example State 1 State 2 transition enabling condition state reset [Shaft00]

11 Cwnd of TCP Slow Start Fast Recovery Congestion Avoidance

12 Cwnd of TCP Slow Start Fast Recovery Congestion Avoidance

13 Queue Size Queue Empty Queue Full Queue Not Full

14 Queue Size Queue Empty Queue Not Full Queue Full

15 Dumbbell topology When  i r i exceeds B the queue fills and data is lost (drops) drop (discrete event) r 1 bps r 2 bps r 3 bps q( t ) = queue size queue (temporary storage for data) f1 f1 f2 f2 f3 f3 f1 f1 f2 f2 f3 f3 rate = B bps

16 Window-based rate adjustment w f (window size) = number of packets that can remain unacknowledged for by the destination 1 st packet sent e.g., w f = 3 t 2 nd packet sent 3 rd packet sent 1 st packet received & ack. sent 2 nd packet received & ack. sent 3 rd packet received & ack. sent 1 st ack. received ) 4 th packet can be sent t source fdestination f w f effectively determines the sending rate r f : t0t0 t1t1 t2t2 t3t3 00 11 22 propagation delay time in queue until transmission round-trip time

17 TCP Sack Congestion Control 1.w f (window size)= number of packets that can remain unacknowledged for by the destination 2.While there are no drops, increase w f by 1 on each RTT 3.When a drop occurs, divide w i by 2 Queuing model TCP controllers  drop  RTT rfrf Consider only CA for now for the simplicity

18 On-Off UDP model On: Off: transition enabling condition state reset

19 Talk Outline Related work Simple case studies (e.g., UDP, TCP variations). Generalized Hybrid Model Framework Validation Scalability of Hybrid Systems Software Tools Conclusion

20 General Topology f1 f1 f2 f2 f1 f1 f2 f2 F := { f 1, f 2, … } : set of end2end flows N := { n 1, n 2, … } : set of nodes L := { 1, 2, … } : set of links n1 n1 n2 n2 n3 n3 n4 n4 n5 n5 n6 n B = bandwidth of link T = prop. delay of link

21 Congestion Control routing queue dynamics (Droptail) sending rates drops out-queue rates in-queue rates TCP RTTs UDP HSTCP RED Wireless-Droptail

22 Queue Dynamics total queue size queue size due to flow f the packets of each flow are assumed uniformly distributed in the queue   in-queue rates out-queue rates … drop rates Queue dynamics:

23 Queue Dynamics queue not empty/full queue full queue empty same in and out- queue rates out-queue rates proportional to fraction of packets in the queue no drops drops proportional to fraction in-queue rates   in-queue rates out-queue rates … drop rates

24 Hybrid Queue Model transition enabling condition -queue-not-full -queue-full exported discrete event -queue-empty Discrete State

25 TCP: AIMD congestion- avoidance set of links transversed by flow f propagation delays 1.While there are no drops, increase w f by 1 on each RTT (additive increase) 2.When a drop occurs, divide w f by 2 (multiplicative decrease) (congestion controller constantly probe the network for more bandwidth) imported discrete event

26 TCP: Slow Start 3.Until a drop occurs (or a threshold ssth f is reached), double w f on each RTT 4.When a drop occurs, divide w f and the threshold ssth f by 2 cong.-avoid. slow-start

27 TCP: Timeout, Fast Recovery 6. w f is initialized to 1 and ssthresh becomes w f /2 7. During fast recovery, data is sent at a rate consistent with a window size of w f /2 8. Duration of fast recovery (RTT) for Tcp-sack 5. Timeout occurs when

28 Full TCP: Sack Timeout, Delay between drop occurrence and detection are considered

29 Other Models TCP-FASTWireless-Droptail TCP-NewReno, TCP-Reno, TCP-Tahoe, HSTCP, STCP, RED, Drop-rotation, UDP,etc Reacts to congestion by using queuing delay in addition to packet loss Packet loss occurs in queue-empty or queue- not-full because of wireless medium

30 Talk Outline Related work Simple case studies (e.g., UDP, TCP variations). Generalized Hybrid Model Framework Validation Scalability of Hybrid Systems Software Tools Conclusion

31 Validation Methodology Compared simulation results from ns-2 packet-level simulator. Hybrid models implemented in Modelica and Shift. Plots in the following slides refer to test topologies below: 10ms propagation delay drop-tail queuing 5-500Mbps bottleneck throughput 45,90,135,180ms propagation delays drop-tail queuing 5-500Mbps bottleneck throughput 0-10% UDP on/off background traffic Y-shape dumbbell

32 Validation Methodology Parking-lot 45,90,135,180ms propagation delays drop-tail queuing 5-500Mbps bottleneck throughput 0-10% UDP on/off background traffic from R7 to R8 0-10% UDP on/off background traffic from R9 to R10

33 Slow Start : Dumbbell Single TCP flow. 5Mbps bottleneck throughput. No background traffic.

34 4 Flows : Dumbbell Four competing TCP flows. 5Mbps bottleneck throughput. No background traffic. hybrid modelns-2 Hybrid model accurately captures flow synchronization

35 4 Flows with BG:Y-shape hybrid model Four competing TCP flows. 5 Mbps bottleneck throughput. 10% UDP background traffic (exponentially distributed on-off times). ns-2

36 Average Throughput and RTT Four competing TCP flows. 5Mbps bottleneck throughput. 20 trials with 10 minutes simulation time. 10% UDP background traffic (exponentially distributed on-off times). Hybrid model accurately captures TCP unfairness in 10% relative error for different propagation delays 45,90,135,180ms propagation delays drop-tail queuing Thru. 1Thru. 2Thru. 3Thru. 4RTT1RTT2RTT3RTT4 ns hybrid model relative error5.0%2.4%6.7%1.0%2.1%1.5%0.5%1.3%

37 Average throughput and RTT Thru. 1Thru. 2Thru. 3Thru. 4RTT1RTT2RTT3RTT4 relative error (5M)5.0%2.4%6.7%1.0%2.1%1.5%0.5%1.3% relate error (500M)1.2%4.1%9.4%4.1%0.5%1.6%.1% Four competing TCP flows 500Mbps bottleneck throughput. 5 trials with 133 minutes simulation time. 10% UDP background traffic (exponentially distributed on-off times). Hybrid model accurately captures TCP unfairness in 10% relative error with 500 Mbps for different propagation delays 45,90,135,180ms propagation delays drop-tail queuing

38 Empirical Distribution (5M Y-shape) hybrid modelns-2 Hybrid model captures distribution of congestion windows and queue size L1-distance/2cwnd1cwnd2cwnd3cwnd4bottleneck queue Y-shape (5M) Y-shape (500M)

39 Four competing TCP flows. 500Mbps bottleneck throughput. 10% UDP background traffic (exponentially distributed on-off times). hybrid model ns-2 Hybrid model can reproduce similar probability densities compared to ns-2 for y-shape. Empirical Distribution (500M Y-shape)

40 Four competing flows. 500Mbps bottleneck. 10% UDP background. hybrid model ns-2 Hybrid model can reproduce similar probability densities compared to ns-2 for parking-lot. Empirical Distribution (500M Parking-lot)

41 Talk Outline Related work Simple case studies (e.g., UDP, TCP variations). Generalized Hybrid Model Framework Validation Scalability of Hybrid Systems Software Tools Conclusion

42 Execution Time Speedup: Dumbbell ns-2 complexity approximately scales with hybrid simulator complexity approximately scales with number of flows per-flow throughput (# packets) hybrid models are particularly suitable for large, high- bandwidth simulations (satellite, fiber optics, backbone) Execution time of ns-2 / hybrid model

43 Execution Time Speed-up Y-shape and Parking-Lot Execution speed-ups of parking lot is less than that of y-shape because of frequent discrete transition. 200 times faster with parking-lot and 2.5 Gbps. Execution time of ns-2 / hybrid model

44 Memory Usage: Y-shape Hybrid model needs memory for each variable and state whereas ns- 2 needs memory for each packet.

45 Talk Outline Related work Simple case studies (e.g., UDP, TCP variations). Generalized Hybrid Model Framework Validation Scalability of Hybrid Systems Software Tools Conclusion

46 Network Description Script (NDS) Script similar to OTCL in ns-2. Easy to reconfigure network. Example NDS primitives: –Define Define qsize 2000 –Node Node node1 node2 –Link with various drop policies Link node0 node1 bandwidth q-type q-size q-opt –Tcp-Sack TcpSack count start-time finish-time time-opt opt-para src path –UDP UDP count start-time finish-time time-opt dist rate on-time off-time src path

47 Example of NDS: Internet2.nds # NDS script fot internet-2 define qsize define qtype Droptail define band 10000M node STTL DNVR KSCY IPLS CHIN NYCM WASH ATLA HSTN LOSA SNVA link STTL DNVR band qtype qsize link SNVA DNVR band qtype qsize link DNVR KSCY band qtype qsize  link HSTN KSCY band qtype qsize link ATLA IPLS band qtype qsize TcpSack ATLA IPLS CHIN TcpSack HSTN KSCY IPLS CHIN TcpSack SNVA DNVR KSCY IPLS CHIN NYCM

48 NDS Translator Parses and translates Network Description Script into Modelica. Written in Perl. Reads NDS input file (*.nds) and generates Modelica output file (*.mo). Extracts only necessary modules from netlib.mo. Creates new module which connects extracted modules. Netlib.mo is written in Modelica (TCP variations, RED, Droptail, Drop-rotation, Wireless, Fast-Tcp, HSTCP, Round-robin-combine, functions, connectors, etc).

49 Case Study: Abilene Backbone SrcDestdelay SeattleDenver SunnyvaleDenver25.01 DenverKansas City Kansas CityIndianapolis9.34 IndianapolisChicago3.99 ChicagoNew York SunnyvaleLos Angeles7.772 Los AngelesHouston HoustonAtlanta AtlantaWashington WashingtonNew York4.412 SunnyvaleSeattle HoustonKansas City AtlantaIndianapolis10.95 Two-way propagation delay setNum flowsMin RTTPath One1015 msATLA to CHIN Two msHSTN to CHIN Three msSNVA to NYCM TCP traffic on Internet

50 Example of NDS: Internet2.nds # NDS script fot internet-2 define qsize define qtype Droptail define band 10000M node STTL DNVR KSCY IPLS CHIN NYCM WASH ATLA HSTN LOSA SNVA link STTL DNVR band qtype qsize link SNVA DNVR band qtype qsize link DNVR KSCY band qtype qsize  link HSTN KSCY band qtype qsize link ATLA IPLS band qtype qsize TcpSack ATLA IPLS CHIN TcpSack HSTN KSCY IPLS CHIN TcpSack SNVA DNVR KSCY IPLS CHIN NYCM

51 Results: Abilene Backbone Buffer size between 25,000 and 150,000 packets. Throughputs computed by averaging 10 flows in each set. Fairness Ratio is large when queue size is small. Average round trip time shows queuing delay. Further analysis on the fairness ratio can be found in thesis (Section 5.4).

52 TCP for High-Speed Network FAST-TCP, HSTCP, STCP. LA to New York with 71.7 ms RTT. 10 Gbps bandwidth with buffer size of 5,000 packets. TCP-SACK increase window too slow in congestion avoidance. HSTCP,STCP use aggressive increase in the congestion avoidance. Fast TCP can be in steady state without packet loss.

53 Average Throughput Average throughput for the first 3,600 seconds TCP-Sack shows 76% of throughput when buffer size is 2,500 TCP-Sack fully utilize when queue size is 90,000 FAST-TCP’s utilization is 100% when queue is 2,500

54 Talk Outline Related work Simple case studies (e.g., UDP, TCP variations). Generalized Hybrid Model Framework Validation Scalability of Hybrid Systems Software Tools Conclusion

55 Conclusion Hybrid Systems provide a promising approach to model network traffic –Retain the low-dimensionality of continuous approximations to traffic flow –Represent event based control mechanisms with high accuracy, even at small time-scales –Complexity scales inversely with throughput and RTT –Amenable to formal analysis

56 Future Work Verify and Improve FAST-TCP protocols for high-speed network using hybrid systems Study on the impact of bursty background traffic to the TCP fairness in the high-speed network Develop new models –Predicting voice-over-IP performance –Network planning to accommodate voice-over-IP –Modeling data communication over mobile phone networks –Effect of congestion to the routing –Modeling the impact of routing anomalies such as route flap

57 Thank You!

58 Comparison of Hybrid Model Simulation Environments SimulatorSHIFTDYMOLA LanguageSHIFTMODELICA InstitutionUC BerkeleyDynasim SolverFixed time-stepFixed/Variable time-step Analysis ToolNoYes Object OrientedYes SpeedSlowFast PlatformLinux/win32Redhat/win32 PublicYesNo Dymola has variety of solvers and efficient methods for determining when discrete events occur

59 Drop probability vs. fraction of arrival rate Blue flow gets most of the drops, in spite of using a smaller fraction of bandwidth when synchronization occurs, with sufficient randomness drop probability

60 TCP Sack congestion control (Slow-Start) 1. While there are no drops, increase w i exponentially, being multiplied by 2 on each RTT 2. For an appropriately define constant m. If was constant we get 3. Since w f packets are sent each round-trip time, sending rate is

61 Hybrid Queue Model (RED) Random Early Drop active queuing stochastic counter -queue-not-full -queue-full discrete modes

62 Publications Hybrid Systems Modeling –A Hybrid Systems Modeling Framework for Fast and Accurate Simulation of Data Communication Networks. In Proc. SIGMETRICS, June 2003 –Analysis of a TCP hybrid model. In Proc. of the 39th Annual Allerton Conf. on Comm., Contr., and Computing, Oct –Hybrid Modeling of TCP Congestion Control. In Maria Domenica Di Benedetto, Alberto Sangiovanni-Vincentelli, Hybrid Systems: Computation and Control, number 2034 in Lect. Notes in Comput. Science, pages , Mar. 2001

63 Publications Stochastic Multipath Routing –Enhancing security via stochastic routing. In Proc. of the 11th IEEE Int. Conf. on Comput. Communications and Networks TCP-PR: –A New TCP for Persistent Packet Reordering- TCP-PR, Accepted for publication, Transaction on Networking –TCP-PR: TCP for Persistent Packet Reordering. In Proc. of the IEEE 23rd Int. Conf. on Distributed Computing Systems, pages , May 200

64 High Speed TCP (2) The whole point is that a(w) increases and b(w) decreases as cwnd becomes larger. Example: behavior when cwnd = 80,000 packets: TCPHSTCP a(w)172 b(w)0.50.1