1. Measurement, Modeling, and Analysis of the Internet: Part II

2. Overview  Traffic Modeling  TCP Modeling and Congestion Control  Topology Modeling

3. Part II.a: Traffic modeling

4. Traffic Modeling  Early modeling efforts: legacy of telephony  Packet arrivals: Call arrivals (Poisson)  Exponential holding times  Big Bang in 1993  “On the Self-Similar Nature of Ethernet Traffic” Will E. Leland, Walter Willinger, Daniel V. Wilson, Murad S. Taqqu

5. Self-Similarity in Traffic Measurement ( Ⅱ ) Network Traffic

6. That Changed Everything….. Extract from abstract “ We demonstrate that Ethernet local area network (LAN) traffic is statistically self- similar, that none of the commonly used traffic models is able to capture this fractal behavior, that such behavior has serious implications for the design, control, and analysis of high-speed…”

7. Properties of Self-Similarity oVar(X (m) ) (=  2 m -β ) decreases more slowly (than m – 1 ) or(k) decreases hyperbolically (not exponentially) so that  k r(k) =  (long range dependence) oThe spectral density [discrete time Fourier Transform of r(k)] f(λ)  cλ -(1- β), as λ  0 (not bounded)

8. What went wrong? What next?  Modelers realized Calls->Packets mapping inherently wrong  Self-similarity, or more accurately LRD evidenced by Burstiness of traffic  Explanations for LRD were sought and modeled  [LWWT] postulated heavy tails somewhere as likely cause of LRD

9. Explanations of LRD  Open loop models  Closed loop models  Mixed or structural models

10. Open loop models

11. Cox’s construction  Aggregate traffic is made up of many connections  Connections arrive at random  Each connection has a “size” (number of packets)  Each connection transmits packets at some “rate”  Heavy tailed distribution of size can cause LRD traffic

12. M/G/  traffic model  M/G/  traffic model  Poisson customer arrivals  Heavy tailed service times  Paretotypical distribution  Traffic  number of busy servers

13. Where are the heavy tails though…  Construction provided generative model for traffic  Still didn’t explain where the heavy tails were coming from..  …until 1997  “Self-similarity in World Wide Web traffic. Evidence and possible causes.” Mark E. Crovella and Azer Bestavros.  Postulated that web file sizes follow Pareto distribution

14. Crovella dataset

15. Picture seemed complete..  Generative model existed  Heavy tails were found  Performance analysts got to work  Simulations based on generative model  Analysis of multiplexers fed with traffic model  Grave predictions on buffer overflow sprung  Conservative buffer dimensioning was advocated  …but real world systems performed much better

16. Problems with open loop models  Upwards of 90% network traffic closed loop  Transmission of future packets depends on what happened to prior packets  Buffer overflows cause senders to back off/reduce rate, thereby affecting generation of packets  Open loop models ignored the network effects  Simulation/Analysis results misleading with open loop models

17. Closed loop models

18. Why is closed loop important?  Recall.. “Transmission of future packets depends on what happened to prior packets”  Suggests closed loop behavior induces correlations independently of file size distribution

19. Chaos?  “The chaotic nature of TCP congestion control” A. Veres and M. Boda, Infocom 2000 (winner best paper award)  Paper simulated TCP sources sharing a link and observed chaotic dynamics

20. Chaotic dynamics Onset of “chaos” depended on B/N ratio (B = Buffer size, N = number of flows)

21. Chaos continued..  Paper generated traffic, and preliminary analysis demonstrated presence of LRD  LRD completely determined by TCP, no role of variability of filesizes  Do the claims hold up?

22. Verification of TCP induced LRD

23. Another TCP based model  “On the Propagation of Long-Range Dependence in the Internet ” A. Veres, Zs. Kenesi, S. Molnár, G. Vattay Sigcomm 2000  Proposed the theory that TCP can get “infected” by long range dependence and then “spread” the infection

24. Model  Let F * be an LRD flow, sharing a link C 1 with a TCP flow T 1  Since TCP adapts to available capacity  T 1 = C 1 - F *  Implies T 1 becomes LRD (linearity and C 1 is a constant)  Now T 1 shares link C 2 with TCP flow T 2  T 2 = C 2 - T 1  Since T 1 has been established LRD, T 2 now becomes LRD  And so on…  Model has too many technical flaws to point out..

25. Combined (structural) models

26. Recent (and not so) thoughts on traffic modeling  Observation: Internet protocol hierarchy is layered  Different layers act at different timescales  Layering can lead to multiple timescale (and hence LRD) behavior  Short time scale(multi-fractal) behavior can be quite different from long time scale (mono-fractal)

27. From traces to traffic models  Implicit assumptions behind application modeling techniques:  Identify the application corresponding to a given flow recorded during a measurement period  Identify traffic generated by (instances) of the same application  Operation of the application-level protocol

28. Example of web traffic modeling  Primary random variables:  Request sizes/Reply sizes  User think time  Persistent connection usage  Nbr of objects per persistent connection  Number of embedded images/page  Number of parallel connections  Consecutive documents per server  Number of servers per page

29. Consider independent Markov on-off processes

30. Wavelet plot (PSD) of LRD vs Markovian LRD Product Of 3 Mark. On-Off Product of 2 Mark. On-Off Markovian On-Off Spectrum Indistinguishable!

31. Relating layers to traffic generation Session layer behavior Transport layer behavior application layer behavior Packet generated when all layers are “on”, i.e resultant process is product of component layers

32. The thousand word picture

33. Part II.b: Fluid modeling of TCP

34. Outline  Background  Stochastic Fluid Model  Deterministic Fluid Models  Control theoretic analysis  Delay, stability  Some limiting fluid models

35. TCP Congestion Control: window algorithm Window: can send W packets at a time increase window by one per RTT if no loss, W <- W+1 each RTT decrease window by half on detection of loss W  W/2

36. TCP Congestion Control: window algorithm Window: can send W packets  increase window by one per RTT if no loss, W <- W+1 each RTT  decrease window by half on detection of loss W  W/2 sender receiver W

37. TCP Congestion Control: window algorithm Window: can send W packets increase window by one per RTT if no loss, W <- W+1 each RTT decrease window by half on detection of loss W  W/2 sender receiver W

38. Background:  TCP throughput modeling: hot research topic in the late 90s  Earliest work by Teunis Ott (Bellcore)  Steady state analysis of TCP throughput using time rescaling  Padhye et al. (UMass, Sigcomm98) obtained accurate throughput formula for TCP  Formula validated with real Internet traces  Traces contained loss events

39. Loss modeling  What do losses in a wide area experiment look like?  First guess: is the loss process Poisson?  Analyze traces: several independent experiments, duration 100 seconds each.

40. Trace analysis Loss inter arrival events tested for  Independence  Lewis and Robinson test for renewal hypothesis  Exponentiality  Anderson-Darling test

41. Scatter plot of statistic

42. Experiment 1

43. Experiment 2

44. Experiment 3

45. Experiment 4

46. SDE based model Sender Loss Probability p i Traditional, Source centric loss model Sender Loss Indications arrival rate New, Network centric loss model New loss model proposed in “Stochastic Differential Equation Modeling and Analysis of TCP Window size behavior”, Misra et. al. Performance 99. Loss model enabled casting of TCP behavior as a Stochastic Differential Equation, roughly

47. Refinement of SDE model W(t) = f(,R) Window Size is a function of loss rate (  and round trip time (R) R Network Network is a (blackbox) source of R and Solution: Express R and as functions of W (and N, number of flows) R

48. Active Queue Management:RED  RED: Random Early Detect proposed in 1993  Proactively mark/drop packets in a router queue probabilistically to  Prevent onset of congestion by reacting early  Remove synchronization between flows

49. The RED mechanism RED: Marking/dropping based on average queue length x (t) (EWMA algorithm used for averaging) t min t max p max 1 2t max Marking probability p Average queue length x t -> - q (t) - x (t) x (t): smoothed, time averaged q (t)

50. Loss Model Sender AQM Router Packet Drop/Mark Receiver Loss Rate as seen by Sender: B(t-  p(t-  (t) Round Trip Delay (  ) B(t) p(t) (t)dt=E[dN(t)] -> deterministic fluid model

51. Deterministic System of Differential Equations Window Size: All quantities are average values. Additive increase Loss arrival rate Mult. decrease Queue length: Outgoing traffic Incoming traffic

52. System of Differential Equations (cont.) Average queue length: Where  = averaging parameter of RED(w q )  = sampling interval ~ 1/C Loss probability: Where is obtained from the marking profile p x

53. Closed loop W=Window size, R = RTT, q = queue length, p = marking probability

54. Verification of deterministic fluid model  Network simulated using ns  Differential equations setup for equivalent network  No. of flows changes at t=75 and t=100  DE solver captures transient performance  Observation: Sample path (simulation) matches deterministic fluid model: Fluid limit? DE method ns simulation Inst. queue length Time Inst. queue length at a router

55. Control theoretic analysis  Deterministic fluid model yields convenient control theoretic formulation  Non-linear system linearized about operating point  Frequency domain analysis reveals many interesting insights for the first time

56. Block diagram view N 1 __ Time Delay R tt TCP window control TCP load factor congested queue Control law (e.g. RED)

57. Small Signal model AQM Control Law

58.  Control theoretic analysis predicts stability of the system  Goes down as link capacity (C) increases  Goes down as number of flows (N) decreases  Goes down as feedback delay  increases  Analysis also reveals characteristics of controller  Stability decreases by increasing slope (or gain) of the RED drop profile ( ) Immediate insights p x

59. (Control) Theory based parameter tuning  Non-linear simulation with 60 ftp + 180 http flows  Design rules developed for RED parameter tuning given network conditions Default ns parameters for RED RED parameters tuned Queue length Time

60. PI Controller performance  RED and PI compared  Number of flows gradually increased between t=50 and t=100  PI faster to converge, react  PI controls queue length independent of number of flows - RED - PI controller Time Queue length

61. UNC Testbed

62. Plot of CDF of response time of requests (80% load) Cumulative probability Response time (ms)

63. Plot of CDF of response time of requests (100% load) Cumulative probability Response time (ms) PI, qref=20 FIFO, RED PI, qref=200

64. Recent fluid limits  Continuous setting  A Mean-Field Model for Multiple TCP Connections through a Buffer Implementing RED. [Baccelli, McDonald, Reynier]  Discrete setting  Limit Behavior of ECN/RED Gateways Under a Large Number of TCP Flows. [Tinnakornsrisuphap, Makowski]

65. Continuous setting Start with similar stochastic model, Scaling Fluid limit obtained: Where : is the loss rate Final fluid equations very similar to our mean value model

66. Discrete setting Start with discrete model for Windowsize behavior, obtain (with similar scaling, C->NC ), Similar conclusion as ours regarding role of gain of RED drop profile Demonstrate RED removes synchronization in the limit Also obtain

67. Srikant et al.  Studied different scalings for limiting fluid models  Obtained limits similar to Makowski et al., in a continuous setting  Interesting observations regarding choice of models (rate based vs queue based) for REM  If queue lengths have to be negligible compared to RTTs, use rate- based models.  If virtual queues are to be used, then either scaling doesn’t matter (using variance calculations).  Parameter choices for stability would be different, depending upon the model

68. Scaling 1 N.... 2 Nc versus

69. Intuition scaling leads to rate-based models N scaling leads to queue-based models Why? Queue length becomes either or N, depending on the scaling. Thus, the queue length hits zero often in the former case, leading to an averaging effect.

70. Other applications of fluid models  Design and analysis of DiffServ networks  Modeling and analysis of short-lived flows  Analysis of other mechanisms, e.g. Stochastic Fair dropping  Groups at Caltech and UIUC using similar models for design/analysis

71. Part II.c: Topology modeling

72. Why study topology?  Correctness of network protocols typically independent of topology  Performance of networks critically dependent on topology  e.g., convergence of route information  Internet impossible to replicate  Modeling of topology needed to generate test topologies

73. Internet topologies AT&T SPRINT MCI AT&T MCISPRINT Router level Autonomous System (AS) level

74. More on topologies..  Router level topologies reflect physical connectivity between nodes  Inferred from tools like traceroute or well known public measurement projects like Mercator and Skitter  AS graph reflects a peering relationship between two providers/clients  Inferred from inter-domain routers that run BGP and publlic projects like Oregon Route Views  Inferring both is difficult, and often inaccurate

75. Early work  Early models of topology used variants of Erdos-Renyi random graphs  Nodes randomly distributed on 2-dimensional plane  Nodes connected to each other w/ probability inversely proportional to distance  Soon researchers observed that random graphs did not represent real world networks

76. Real world topologies  Real networks exhibit  Hierarchical structure  Specialized nodes (transit, stub..)  Connectivity requirements  Redundancy  Characteristics incorporated into the Georgia Tech Internetwork Topology Models (GT-ITM) simulator (E. Zegura, K.Calvert and M.J. Donahoo, 1995)

77. So…are we done?  No!  In 1999, Faloutsos, Faloutsos and Faloutsos published a paper, demonstrating power law relationships in Internet graphs  Specifically, the node degree distribution exhibited power laws That Changed Everything…..

78. Power laws in AS level topology

79.  Faloutsos 3 (Sigcomm’99)  frequency vs. degree Power Laws topology from BGP tables of 18 routers

80.  Faloutsos 3 (Sigcomm’99)  frequency vs. degree Power Laws topology from BGP tables of 18 routers

81.  Faloutsos 3 (Sigcomm’99)  frequency vs. degree Power Laws topology from BGP tables of 18 routers

82.  Faloutsos 3 (Sigcomm’99)  frequency vs. degree  empirical ccdf P(d>x) ~ x - a Power Laws

83.  Faloutsos 3 (Sigcomm’99)  frequency vs. degree  empirical ccdf P(d>x) ~ x - a α ≈1.15

84. GT-ITM abandoned..  GT-ITM did not give power law degree graphs  New topology generators and explanation for power law degrees were sought  Focus of generators to match degree distribution of observed graph

85. Generating power law graphs Goal: construct network of size N with degree power law, P(d>x) ~ x - a  power law random graph (PLRG) (Aiello et al)  Inet (Chen et al)  incremental growth (BA) (Barabasi et al)  general linear preference (GLP) (Bu et al)

86. Power law random graph (PLRG) (Aiello et al)  operations 2 11 may be disconnected, contain multiple edges, self-loops  contains unique giant component for right choice of parameters  assign degrees to nodes drawn from power law distribution  create k v copies of node v; k v degree of v.  aggregate edges  randomly match nodes in pool

87. Inet (Chen et al)  assumption  max degree, size grow exponentially over time  algorithm  pick date, calculate maximum degree/size  compute degrees of other nodes  form spanning tree with degree 2 +  attach other nodes according to linear preference  match remaining nodes  remove self loops, multi-edges

88. Barabasi model: fixed exponent  incremental growth  initially, m 0 nodes  step: add new node i with m edges  linear preferential attachment  connect to node i with probability ∏(k i ) = k i / ∑ k j 0.5 0.25 0.50.25 new nodeexisting node may contain multi-edges, self-loops

89.  motivation  greater flexibility in assigning preference  removes need for rewiring  new preferential function ∏(k i ) = (k i -  ) / ∑ (k j -  ),  in (- ,1)  operations  prob. p: add m new links  prob. 1-p: add a new node with m new links  can achieve any  in (1,  ) General linear preference

90. “Scale-free” graphs  Preferential attachment leads to “scale free” structure in connectivity  Implications of “scale free” structure  Few centrally located and highly connected hubs  Network robust to random attack/node removal (probability of targeting hub very low)  Network susceptible to catastrophic failure by targeted attacks (“Achilles heel of the Internet” Albert, Jeong, Barabasi, Nature 2000)

91. Is the router-level Internet graph scale-free?  No…(There is no Memphis!)  Emphasis on degree distribution - structure ignored  Real Internet very structured  Evolution of graph is highly constrained

92. Topology constraints  Technology  Router out degree is constrained by processing speed  Routers can either have a large number of low bandwidth connections, or..  A small number of high bandwidth connections  Geography  Router connectivity highly driven by geographical proximity  Economy  Capacity of links constrained by the technology that nodes can afford, redundancy/performance they desire etc.

93. Optimization based models for topology  HOT-1 Highly Optimized Tolerances  Doyle et. al., Caltech, USC, ISI, AT&T..  HOT-2 Heuristically Optimized Tradeoffs  Fabrikant, Koutsoupias, Papadimitriou, Berkeley  HOT-3: variant of HOT-2  Chang, Jamin, Willinger, Michigan, AT&T

94. Fabrikant HOT  Each new node solves the local optimization problem to find a target node to connect to.  Each new node i connects to an existing node j that minimizes the weighted sum of two objectives: min (  d ij + h j )  d ij (last mile cost) = Euclidean distance from i to j  h j (transmission delay cost) = average hop distance from j to all other nodes

95. Modified Fabrikant HOT  Univariate HOT model.  Criteria: (i) AS geography.  Bivariate HOT model.  Criteria: (i) AS geography, (ii) AS business model.  Various extensions