1. Some Internet Dynamics and Alternatives to TCP Friendly Stephan Bohacek University of Southern California Joint work with: Boris Rozovskii, Joao Hespanha, Katia Obraczka, Junsoo Lee, Khusbboo Shah, Vinay Sridhara, and Raghava Sivaramu

2. Developing stochastic models of the Internet and investigating what can be determined about the state of the Internet by observing round-trip times and drops. Stephan Bohacek University of Southern California Joint work with: Boris Rozovskii, Joao Hespanha, Katia Obraczka, Junsoo Lee, Khusbboo Shah, Vinay Sridhara, and Raghava Sivaramu preliminary work

3. Outline TCP friendly basics Problems with TCP – fixes Modeling RTT and estimating queue parameters Observed dynamics of queue model parameters Drop Model Observed dynamics of the drop model parameters An alternative to TCP friendly congestion control

4. TCP Linear increase when no drop is detected Multiplicative decrease when a drop is detected TCP: router queue router queue sourcedestination 100Mb 10Mb Single bottleneck cwnd (congestion window) time drop detected

5. Derivation of the TCP Friendly Rule cwnd (congestion window) (number of packets sent / RTT) time w*w* w * /2 RTT – Roundtrip Time Total number of packets sent between drops = w * /2 + w * /2+1+…+ w * /2 + w * /2  3/8 w *2 p := probability of a packet being dropped = 8/3 · 1/ w *2 (assumes one drop) cwnd = ¾ w * T := Sending Rate (pkts/sec) = cwnd (pkts/rtt) / RTT (sec/rtt) T · RTT = cwnd = ¾ w * =(8/3) 1/2 1/p 1/2 T · RTT = C · 1/p 1/2 not true

6. Similar formulas T · RTT = C · 1/p 1/2 C = 1.31 Ott, Kemperman and Mathis (1996) C = (8/3) 1/2 Mathis, Semke, Mahdavi and Ott (1997) C = (3/2) 1/2 Altman (2000) and others. C = 1.27 Hespanha, Bohacek, Obraczka and Lee 2001

7. RTT varies Hybrid systems model yields: Hespanha, Bohacek, Obraczka and Lee In Maria Domenica Di Benedetto and Alberto Sangiovanni-Vincentelli, Mar. 2001. time cwnd

8. Dumbbell topology router S1 S2 S3 Sn D1 D2 D3 Dn Single bottleneck link 100Mbs 10Mbs

9. Validation of the TCP Friendly Rule RTT ·T = 1.02/p 1/2

10. TCP performs poorly over multi-hop connections router sourcedestination S1S1 D1D1 S2S2 D n-1 DnDn SnSn Competing TCP flows p 1 = drop probability p n = drop probability Each competing flow satisfies T i · RTT i = C 1/p i 1/2 Drop probability for multi-hop flow = 1 -  i (1 - p i ) >> p j

11. End-to-end model with many queues Objective: Observe RTT and estimate the per-link drop probability and adjust TCP accordingly. q1tq1t q2tq2t q n-1 t qntqnt sourcedestination queues queuing delay at time t = D 1 t queuing delay at time t = D n-1 t The k th is sent at time T k

12. Probability Density Fitting Minimum Distance Fit Invariant density of the delay experienced in M/M/1 queue (ignore increments) p(D i = d) = p(d) =  {d=0} (  i - i ) /  i + (  i - i ) i /  i exp(-(  i - i )d)  i - departure rate at i th queue i - arrival rate at i th queue Heavy-traffic assumption Account for most of the delay p  (d)   i exp(-  i d) for  i  i i

13. Minimum Distance Fit non - convex optimization Theoretical: Actual: where convolution

14. RTT and Fitted Model 3/27/01 LA – San Jose

15. Basic Queue Model constant sending rate random packet size Poisson process with intensity indicator so 0  q i  q i max – maximum queue size Delay = D i t = q i t /  i

16. Distribution of Packet Sizes Cumulative distribution of packet size. A packet of size one has 1500 bytes. The data was collected by the NLANR project. Density function –  (z)=0.2/z for 0.011  z  1

17. Conditional Probability Density of Queue Size Define p(q t |q 0, , )=p(q,t|q o ) 0510152025303540 0 0.02 0.04 0.06 0.08 0.1 0.12 0510152025303540 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 p(q,10ms|15) qq p(q,1000ms|15) p p

18. Diffusion Approximation Diffusion approximation assumes m  0, that is,  . This is true for queues with significant delay.

19. Modification to the Diffusion Approximation time work remaining to be processed 0 10 sec 1 sec  9 sec P(d|t, d o ) = 0 for d< d o - t Two options 1.Reflection 2.Accumulation

20. Reflection diffusion reflection sum Probability density function f(d | d 0 =2, t=1) d f

21. Accumulation diffusion Accumulate at d = d o - t Probability density function f(d | d 0 =2, t=1) d o - t d

22. Modified Diffusion Approximation

23. Histogram of Simulated RTT Increments One Queue m  0,  <1 t=10 Gaussian

24. One Queue m >1 t=10 Histogram of Simulated RTT Increments

25. Two Queues: Queue 1 – m 1 > 1 Queue 2 – m 2  0,  1 < 1

26. Histogram of Observed RTT Increments Gaussian (RTT 0 =28) Queue empties slowly Queue empties quickly

27. Estimating Parameters of the Diffusion Model

29. Estimating Delay From RTT Assume the k th drop occurs at t = tk P( q 1 tk > q j tk, j  1 | RTT K )  P( Queue 1 had the drop ) ( Assumes that all queues are the same size ) P 1 k := P( D 1 tk > D j tk, j  1 | RTT K )  P( Queue 1 had the drop ) ( Assumes that all routers have the same speed ) Objective: Observe RTT and estimate the per-link drop probability and adjust TCP accordingly.

30. Adjusting TCP p 1  probability of queue 1 dropping a packet p 1 =  k P 1 k / Total Number of Packets Sent cwnd = T · RTT = C · min i (1/p i 1/2 )

31. Computing per Queue Drop Probabilities Vector of delays experienced by the k th packet

32. Computing per Queue Drop Probabilities Transition probabilities (we can compute these)

33. Computing per Queue Drop Probabilities Monte-Carlo Markov Chain works, but is very slow. proposal density: where j is random where l is random

34. 051015202530354045 0 5 10 15 20 25 30 35 Q 1 (k) Time Size of Queue 1 050100150200250300350400450 0 5 10 15 20 25 Time (ms) RTT Round Trip Time

36. MCMC Results

38. 050100150200250300 0 2 4 6 8 10 12 14 16 18 Delay in Queue 1 Time (ms) D1D1

39. 050100150200250300 0 2 4 6 8 10 12 14 16 18 20 Delay in Queue 2 D2D2 actual expected

44. Adjusting the TCP Friendly Rule Given the queue parameters  i and drop probability p, the drop probability at each queue is (the case with 3 queues) end-to-end drop probability link drop probability Assume that a drop occurs on the queue with the longest delay. - Delay is related to the occupancy of queue (all delays are not equal). - All queues are the same size (all queues are not the same length). cwnd = T · RTT = C min i (1/p i 1/2 ) Per Link TCP Friendly

45. Observed Dynamics of the Queue Parameters RTT t =  i D t i p( D t i = d ) =  i exp( -  i d ) for d  0 E( D t i ) = 1/  i - The smaller the  i the more significant the queue.

48. The autocorrelation decays very quickly, hence a low order model should suffice (maybe even a Markov chain). Autocorrelation of the increments of the smallest queue parameter. smallest parameter next to smallest parameter

49. Transition Probabilities Matrix Smallest Queue ParameterNext to Smallest Queue Parameter P i,j  P |i-j| The transition probability are nearly spatially homogeneous Partition the parameter space and estimate transition probabilities

50. -3-20123 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 Distribution of the Next to Smallest Queue Parameter Increments -3-20123 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 Distribution of the Smallest Queue Parameter Increments Transition Probability Size of Increment probability Smallest Queue Parameter Next to Smallest Queue Parameter

51. TCP Friendly router S D S D S D S D Source Under Test Destination Under Test Backbone link Input link Variables: 1.Number of routers 2.Backbone link speed 3.Input link speed 4.Types of paths (all possible, downstream, single hop) 5.Number of flows per path

52. The TCP Friendly Rule It is possible that topologies that never exists were considered. It is possible that worst topologies exists that have yet to be considered. Further study is under way.

53. Time-Varying TCP Friendly cwnd = T · RTT = C min i (1/p i 1/2 ) Even if the TCP rule is accurate, it is not clear how to compute RTT, p, T.

54. Alternatives to the TCP Friendly Rule (TCP Mimicking) Very simple continuous time model of TCP X t := window size N t := drop process RTT t := roundtrip time Approach: 1.Observe RTT and drops. 2.Determine the pdf of TCP’s window size. At what rate would a TCP flow send data if it was in the same situation?

55. Drop Models Ott (1997) considered deterministic drops. Padhye (1998) assumed drops to be highly correlated over short time scales, but independent over longer time scales. Altman (2000) assumes drops are bursty. Altman (2000) drop events are modeled as renewal processes with particular examples, deterministic, Poisson, i.i.d., and Markovian. Savari (1999) drop events are modeled as Poisson where the intensity depends on the window size of the TCP protocol.

56. Drop Model N t – Poisson process with intensity n t (RTT, X/RTT,  ) n(RTT, X/RTT,  ) = X/RTT g(RTT, X/RTT,  ) g(RTT,X/RTT  ) – drop probability time-varying random variable sending rate g(RTT,  ) =  0 (  ) +  1 (  ) T 1 (RTT) +  2 (  ) T 2 (RTT) + … Chebyshev polynomials For simplicity, we neglect the effect of the sending rate on the drop probability and model the drop probability as

57. Drop Dependence 051015 0 0.2 0.4 0.6 0.8 1 probability of the next packet being dropped given the past N packets have been dropped probability N probability of N packets being dropped given the last packet was dropped N probability g(RTT,  ) is the probability of the the first drop – drop event.

58. Drop – Event Probability

59. Drop Model Parameter Variation g(RTT,  ) =  0 (  ) +  1 (  ) T 1 (RTT) +  2 (  ) T 2 (RTT) + …

60. Autocorrelation of the Drop Models Parameters There appears to be some correlation, but …

61. Transition Probability for Drop Model Parameter

62. At what rate would a TCP flow send data if it was in the same situation? Alternatives to the TCP Friendly Rule (TCP Mimicking)

63. Alternatives to the TCP Friendly Rule (single queue)

64. TCP Mimicking Every time an ack arrives and every time a packet is about to be sent, the pdf is evaluated. The decision whether to send a packet is based on the pdf and on the application, e.g. multimedia v.s. file transfer. If acks fail to arrive, the drop probability quickly adjusts to indicate that a TCP flow would send at a slow rate (unlike other rate based congestion control). It is not clear if TCP mimicking leads to TCP friendly.

65. TCP Mimicking v.s. TCP Friendly If these are equal, then TCP mimicking is also TCP friendly. Large network assumption -  is the constant function, i.e. the sending rate does not effect the network. In this case, TCP mimicking is TCP friendly. But so is everything else!

66. Future Work More of the same – more data, diversity paths. More data for drops during large RTT. Large network assumption – I/O dynamics (in progress) Determine q max from data – relating RTT and queue model parameters to drop model parameters.

67. Conclusions It appears possible to develop stochastic models of the internet. It appears possible the estimate parameters and the state of the Internet from the observations, RTT and drops. These models and estimation techniques may be useful in developing congestion control algorithms.

68. The end