Copyright, 1996 © Dale Carnegie & Associates, In Equilibrium & Dynamics of TCP/AQM Steven Low CS & EE, Caltech netlab.caltech.edu Sigcomm August 2001.

Copyright, 1996 © Dale Carnegie & Associates, In Equilibrium & Dynamics of TCP/AQM Steven Low CS & EE, Caltech netlab.caltech.edu Sigcomm August 2001

Fall 2002Advanced Internet Acknowledgments S. Athuraliya, D. Lapsley, V. Li, Q. Yin (UMelb) S. Adlakha (UCLA), J. Doyle (Caltech), F. Paganini (UCLA), J. Wang (Caltech) L. Peterson, L. Wang (Princeton) Matthew Roughan (AT&T Labs)

Fall 2002Advanced Internet Outline Introduction TCP/AQM algorithms Duality model Equilibrium of TCP/AQM Linear dynamic model Dynamics of TCP/AQM A scalable control

Fall 2002Advanced Internet Schedule 1:30 – 2:30TCP/AQM algorithms 2:30 – 3:00Duality model (equilibrium) 2:30 – 3:00Break 3:30 – 4:00Duality model (equilibrium) 4:00 – 4:30 Linear model (dynamic) 4:30 – 5:00Scalable control

Part 0 Introduction

Fall 2002Advanced Internet TCP/IP Protocol Stack Applications (e.g. Telnet, HTTP) TCPUDP ICMP ARPIP Link Layer (e.g. Ethernet, ATM) Physical Layer (e.g. Ethernet, SONET)

Fall 2002Advanced Internet Packet Terminology Application Message TCP dataTCP hdr MSS TCP Segment IP dataIP hdr IP Packet Ethernet dataEthernet Ethernet Frame 20 bytes 14 bytes 4 bytes MTU 1500 bytes

Fall 2002Advanced Internet Success of IP Applications IP Transmission Simple/Robust Robustness against failure Robustness against technological evolutions Provides a service to applications Doesn’t tell applications what to do Quality of Service Can we provide QoS with simplicity? Not with current TCP… … but we can fix it! WWW, Email, Napster, FTP, … Ethernet, ATM, POS, WDM, …

Fall 2002Advanced Internet IETF Internet Engineering Task Force Standards organisation for Internet Publishes RFCs - Requests For Comment standards track: proposed, draft, Internet non-standards track: experimental, informational best current practice poetry/humour (RFC 1149: Standard for the transmission of IP datagrams on avian carriers) TCP should obey RFC no means of enforcement some versions have not followed RFC http://www.ietf.org/index.html

Fall 2002Advanced Internet Simulation ns-2: http://www.isi.edu/nsnam/ns/index.html http://www.isi.edu/nsnam/ns/index.html Wide variety of protocols Widely used/tested SSFNET: http://www.ssfnet.org/homePage.html http://www.ssfnet.org/homePage.html Scalable to very large networks Care should be taken in simulations! Multiple independent simulations confidence intervals transient analysis – make sure simulations are long enough Wide parameter ranges All simulations involve approximation

Fall 2002Advanced Internet Other Tools tcpdump Get packet headers from real network traffic tcpanaly (V.Paxson, 1997) Analysis of TCP sessions from tcpdump traceroute Find routing of packets RFC 2398 http://www.caida.org/tools/

Part I Algorithms

Fall 2002Advanced Internet Outline Introduction TCP/AQM algorithms Window flow control Source algorithm: Tahoe, Reno, Vegas Link algorithm: RED, REM Duality model Equilibrium of TCP/AQM Linear dynamic model Dynamics of TCP/AQM A scalable control

Fall 2002Advanced Internet Early TCP Pre-1988 Go-back-N ARQ Detects loss from timeout Retransmits from lost packet onward Receiver window flow control Prevent overflows at receive buffer Flow control: self-clocking

Fall 2002Advanced Internet Why Flow Control? October 1986, Internet had its first congestion collapse Link LBL to UC Berkeley 400 yards, 3 hops, 32 Kbps throughput dropped to 40 bps factor of ~1000 drop! 1988, Van Jacobson proposed TCP flow control

Fall 2002Advanced Internet Window Flow Control ~ W packets per RTT Lost packet detected by missing ACK RTT time Source Destination 12W12W12W data ACKs 12W

Fall 2002Advanced Internet Source Rate Limit the number of packets in the network to window W Source rate = bps If W too small then rate « capacity If W too big then rate > capacity => congestion

Fall 2002Advanced Internet Effect of Congestion Packet loss Retransmission Reduced throughput Congestion collapse due to Unnecessarily retransmitted packets Undelivered or unusable packets Congestion may continue after the overload! throughput load

Fall 2002Advanced Internet Congestion Control TCP seeks to Achieve high utilization Avoid congestion Share bandwidth Window flow control Source rate = packets/sec Adapt W to network (and conditions) W = BW x RTT

Fall 2002Advanced Internet TCP Window Flow Controls Receiver flow control Avoid overloading receiver Set by receiver awnd:receiver (advertised) window Network flow control Avoid overloading network Set by sender Infer available network capacity cwnd: congestion window Set W = min (cwnd, awnd)

Fall 2002Advanced Internet Receiver Flow Control Receiver advertises awnd with each ACK Window awnd closed when data is received and ack’d opened when data is read Size of awnd can be the performance limit (e.g. on a LAN) sensible default ~16kB

Fall 2002Advanced Internet Network Flow Control Source calculates cwnd from indication of network congestion Congestion indications Losses Delay Marks Algorithms to calculate cwnd Tahoe, Reno, Vegas, RED, REM …

Fall 2002Advanced Internet TCP Congestion Control Tahoe (Jacobson 1988) Slow Start Congestion Avoidance Fast Retransmit Reno (Jacobson 1990) Fast Recovery Vegas (Brakmo & Peterson 1994) New Congestion Avoidance RED (Floyd & Jacobson 1993) Probabilistic marking REM (Athuraliya & Low 2000) Clear buffer, match rate Others…

Fall 2002Advanced Internet Variants Tahoe & Reno NewReno SACK Rate-halving Mod.s for high performance AQM RED, ARED, FRED, SRED BLUE, SFB REM, PI

Fall 2002Advanced Internet TCP Tahoe (Jacobson 1988) SS time window CA SS: Slow Start CA: Congestion Avoidance

Fall 2002Advanced Internet Slow Start Start with cwnd = 1 (slow start) On each successful ACK increment cwnd cwnd  cnwd + 1 Exponential growth of cwnd each RTT: cwnd  2 x cwnd Enter CA when cwnd >= ssthresh

Fall 2002Advanced Internet Slow Start data packet ACK receiversender 1 RTT cwnd 1 2 3 4 5 6 7 8 cwnd  cwnd + 1 (for each ACK)

Fall 2002Advanced Internet Congestion Avoidance Starts when cwnd  ssthresh On each successful ACK: cwnd  cwnd + 1/cwnd Linear growth of cwnd each RTT: cwnd  cwnd + 1

Fall 2002Advanced Internet Congestion Avoidance cwnd 1 2 3 1 RTT 4 data packet ACK cwnd  cwnd + 1 (for each cwnd ACKS) receiversender

Fall 2002Advanced Internet Packet Loss Assumption: loss indicates congestion Packet loss detected by Retransmission TimeOuts (RTO timer) Duplicate ACKs (at least 3) 1 23456 123 Packets Acknowledgements 3 3 7 3

Fall 2002Advanced Internet Fast Retransmit Wait for a timeout is quite long Immediately retransmits after 3 dupACKs without waiting for timeout Adjusts ssthresh flightsize = min(awnd, cwnd) ssthresh  max(flightsize/2, 2) Enter Slow Start (cwnd = 1)

Fall 2002Advanced Internet Successive Timeouts When there is a timeout, double the RTO Keep doing so for each lost retransmission Exponential back-off Max 64 seconds 1 Max 12 restransmits 1 1 - Net/3 BSD

Fall 2002Advanced Internet Summary: Tahoe Basic ideas Gently probe network for spare capacity Drastically reduce rate on congestion Windowing: self-clocking Other functions: round trip time estimation, error recovery for every ACK { if (W < ssthresh) then W++ (SS) else W += 1/W (CA) } for every loss { ssthresh = W/2 W = 1 }

Fall 2002Advanced Internet TCP Tahoe (Jacobson 1988) SS time window CA SS: Slow Start CA: Congestion Avoidance

Fall 2002Advanced Internet TCP Reno (Jacobson 1990) SS time window CA SS: Slow Start CA: Congestion Avoidance Fast retransmission/fast recovery

Fall 2002Advanced Internet Fast recovery Motivation: prevent `pipe’ from emptying after fast retransmit Idea: each dupACK represents a packet having left the pipe (successfully received) Enter FR/FR after 3 dupACKs Set ssthresh  max(flightsize/2, 2) Retransmit lost packet Set cwnd  ssthresh + ndup (window inflation) Wait till W=min(awnd, cwnd) is large enough; transmit new packet(s) On non-dup ACK (1 RTT later), set cwnd  ssthresh (window deflation) Enter CA

Fall 2002Advanced Internet 9 9 4 00 Example: FR/FR Fast retransmit Retransmit on 3 dupACKs Fast recovery Inflate window while repairing loss to fill pipe time S R 12345687 8 cwnd8 ssthresh 1 7 4 000 Exit FR/FR 4 4 4 11 00 1011

Fall 2002Advanced Internet Summary: Reno Basic ideas Fast recovery avoids slow start dupACKs: fast retransmit + fast recovery Timeout: fast retransmit + slow start slow start retransmit congestion avoidance FR/FR dupACKs timeout

Fall 2002Advanced Internet NewReno: Motivation On 3 dupACKs, receiver has packets 2, 4, 6, 8, cwnd=8, retransmits pkt 1, enter FR/FR Next dupACK increment cwnd to 9 After a RTT, ACK arrives for pkts 1 & 2, exit FR/FR, cwnd=5, 8 unack’ed pkts No more ACK, sender must wait for timeout 1 2 time S D 345687 1 8 FR/FR 09 000 0 0 9 8 unack’d pkts 2 5 3 timeout

Fall 2002Advanced Internet NewReno Fall & Floyd ‘96, (RFC 2583) Motivation: multiple losses within a window Partial ACK acknowledges some but not all packets outstanding at start of FR Partial ACK takes Reno out of FR, deflates window Sender may have to wait for timeout before proceeding Idea: partial ACK indicates lost packets Stays in FR/FR and retransmits immediately Retransmits 1 lost packet per RTT until all lost packets from that window are retransmitted Eliminates timeout

Fall 2002Advanced Internet SACK Mathis, Mahdavi, Floyd, Romanow ’96 (RFC 2018, RFC 2883) Motivation: Reno & NewReno retransmit at most 1 lost packet per RTT Pipe can be emptied during FR/FR with multiple losses Idea: SACK provides better estimate of packets in pipe SACK TCP option describes received packets On 3 dupACKs: retransmits, halves window, enters FR Updates pipe = packets in pipe Increment when lost or new packets sent Decrement when dupACK received Transmits a (lost or new) packet when pipe < cwnd Exit FR when all packets outstanding when FR was entered are acknowledged

Fall 2002Advanced Internet TCP Reno (Jacobson 1990) SS time window CA SS: Slow Start CA: Congestion Avoidance Fast retransmission/fast recovery

Fall 2002Advanced Internet TCP Vegas (Brakmo & Peterson 1994) SS time window CA Converges, no retransmission … provided buffer is large enough

Fall 2002Advanced Internet queue size for every RTT { if W/RTT min – W/RTT <  then W ++ if W/RTT min – W/RTT >  then W -- } for every loss W := W/2 Vegas CA algorithm

Fall 2002Advanced Internet Implications Congestion measure = end-to-end queueing delay At equilibrium Zero loss Stable window at full utilization Approximately weighted proportional fairness Nonzero queue, larger for more sources Convergence to equilibrium Converges if sufficient network buffer Oscillates like Reno otherwise

Fall 2002Advanced Internet RED (Floyd & Jacobson 1993) Idea: warn sources of incipient congestion by probabilistically marking/dropping packets Link algorithm to work with source algorithm (Reno) Bonus: desynchronization Prevent bursty loss with buffer overflows time window B Avg queue marking 1 router host

Fall 2002Advanced Internet RED Implementation Probabilistically drop packets Probabilistically mark packets Marking requires ECN bit (RFC 2481) Performance Desynchronization works well Extremely sensitive to parameter setting Fail to prevent buffer overflow as #sources increases

Fall 2002Advanced Internet Variant: ARED (Feng, Kandlur, Saha, Shin 1999) Motivation: RED extremely sensitive to #sources Idea: adapt max p to load If avg. queue < min th, decrease max p If avg. queue > max th, increase max p No per-flow information needed

Fall 2002Advanced Internet Variant: FRED (Ling & Morris 1997) Motivation: marking packets in proportion to flow rate is unfair (e.g., adaptive vs unadaptive flows) Idea A flow can buffer up to min q packets without being marked A flow that frequently buffers more than max q packets gets penalized All flows with backlogs in between are marked according to RED No flow can buffer more than avgcq packets persistently Need per-active-flow accounting

Fall 2002Advanced Internet Variant: SRED (Ott, Lakshman & Wong 1999) Motivation: wild oscillation of queue in RED when load changes Idea: Estimate number N of active flows An arrival packet is compared with a randomly chosen active flows N ~ prob(Hit) -1 cwnd~ p -1/2 and Np -1/2 = Q 0 implies p = (N/Q 0 ) 2 Marking prob = m(q) min( 1, p ) No per-flow information needed

Fall 2002Advanced Internet Variant: BLUE (Feng, Kandlur, Saha, Shin 1999) Motivation: wild oscillation of RED leads to cyclic overflow & underutilization Algorithm On buffer overflow, increment marking prob On link idle, decrement marking prob

Fall 2002Advanced Internet Variant: SFB Motivation: protection against nonadaptive flows Algorithm L hash functions map a packet to L bins (out of NxL ) Marking probability associated with each bin is Incremented if bin occupancy exceeds threshold Decremented if bin occupancy is 0 Packets marked with min { p 1, …, p L } 1 111 nonadaptive adaptive h1h1 hLhL h L-1 h2h2

Fall 2002Advanced Internet Variant: SFB Idea A nonadaptive flow drives marking prob to 1 at all L bins it is mapped to An adaptive flow may share some of its L bins with nonadaptive flows Nonadaptive flows can be identified and penalized

Fall 2002Advanced Internet REM Athuraliya & Low 2000 Main ideas Decouple congestion & performance measure Price adjusted to match rate and clear buffer Marking probability exponential in `price’ REM RED Avg queue 1

Part II Models

Fall 2002Advanced Internet Congestion Control Heavy tail  Mice-elephants Elephant Internet Mice Congestion control efficient & fair sharing small delay queueing + propagation CDN

Fall 2002Advanced Internet TCP x i (t)

Fall 2002Advanced Internet TCP & AQM x i (t) p l (t) Example congestion measure p l (t) Loss (DropTail) Queue length (RED) Queueing delay (Vegas) Price (REM)

Fall 2002Advanced Internet TCP & AQM x i (t) p l (t) Duality theory  equilibrium Source rates x i (t) are primal variables Congestion measures p l (t) are dual variables Congestion control is optimization process over Internet

Fall 2002Advanced Internet TCP & AQM x i (t) p l (t) Control theory  stability & robustness Internet is a gigantic feedback system Distributed & delayed

Fall 2002Advanced Internet Outline Introduction TCP/AQM algorithms Duality model Equilibrium of TCP/AQM Linear dynamic model Dynamics of TCP/AQM A scalable control

Fall 2002Advanced Internet Overview: equilibrium Interaction of source rates x s (t) and congestion measures p l (t) Duality theory They are primal and dual variables Flow control is optimization process Example congestion measure Loss (Reno) Queueing delay (Vegas)

Fall 2002Advanced Internet Overview: equilibrium Congestion control problem TCP/AQM protocols (F, G) Maximize aggregate source utility With different utility functions U s ( x s ) Primal-dual algorithm Reno, Vegas DropTail, RED, REM

Fall 2002Advanced Internet Overview: Reno Equilibrium characterization Duality Congestion measure p = loss Implications Reno equalizes window w i =  i x i inversely proportional to delay  i dependence for small p DropTail fills queue, regardless of queue capacity

Fall 2002Advanced Internet Overview: AQM DropTail queue = 94% RED min_th = 10 pkts max_th = 40 pkts max_p = 0.1 REM queue = 1.5 pkts utilization = 92%  = 0.05,  = 0.4,  = 1.15 Decouple congestion & performance measure

Fall 2002Advanced Internet Overview: equilibrium DropTail High utilization Fills buffer, maximize queueing delay RED Couples queue length with congestion measure High utilization or low delay REM Decouple performance & congestion measure Match rate, clear buffer Sum prices

Fall 2002Advanced Internet Overview: Vegas Delay Congestion measures: end to end queueing delay Sets rate Equilibrium condition: Little’s Law Loss No loss if converge (with sufficient buffer) Otherwise: revert to Reno (loss unavoidable) Fairness Utility function Proportional fairness

Fall 2002Advanced Internet Model c1c1 c2c2 Sources s L(s) - links used by source s U s (x s ) - utility if source rate = x s Network Links l of capacities c l x1x1 x2x2 x3x3

Fall 2002Advanced Internet Primal problem Assumptions Strictly concave increasing U s Unique optimal rates x s exist Direct solution impractical

Fall 2002Advanced Internet Prior Work Formulation Kelly 1997 Penalty function approach Kelly, Maulloo and Tan 1998 Kunniyur and Srikant 2000 Duality approach Low and Lapsley 1999 Athuraliya and Low 2000, Low 2000 Extensions Mo & Walrand 1998 La & Anantharam 2000

Fall 2002Advanced Internet Example Lagrange multiplier: p 1 = p 2 = 3/2 1 1 x1x1 x2x2 x3x3 Optimal: x 1 = 1/3 x 2 = x 3 = 2/3

Fall 2002Advanced Internet x s : proportionally fair p l : Lagrange multiplier, (shadow) price, congestion measure How to compute (x, p)? Relevance to TCP/AQM ?? Example 1 1 x1x1 x2x2 x3x3

Fall 2002Advanced Internet 1 1 x1x1 x2x2 x3x3 x s : proportionally fair (Vegas) p l : Lagrange multiplier, (shadow) price, congestion measure How to compute (x, p) ? Gradient algorithms, Newton algorithm, Primal-dual algorithms… Relevance to TCP/AQM ?? TCP/AQM protocols implement primal-dual algorithms over Internet Example Aggregate rate

Fall 2002Advanced Internet x s : proportionally fair (Vegas) p l : Lagrange multiplier, (shadow) price, congestion measure How to compute (x, p) ? Gradient algorithms, Newton algorithm, Primal-dual algorithms… Relevance to TCP/AQM ?? TCP/AQM protocols implement primal-dual algorithms over Internet Example 1 1 x1x1 x2x2 x3x3

Fall 2002Advanced Internet Duality Approach Primal-dual algorithm:

Fall 2002Advanced Internet Duality Model of TCP Primal-dual algorithm: Reno, VegasDropTail, RED, REM TCP iterates on rates (windows) AQM iterates on congestion measures With different utility functions

Fall 2002Advanced Internet Summary Congestion control problem TCP/AQM protocols (F, G) Maximize aggregate source utility With different utility functions U s ( x s ) Primal-dual algorithm Reno, Vegas DropTail, RED, REM

Fall 2002Advanced Internet ( F, G, U ) model Derivation Derive ( F, G ) from protocol description Fix point ( x, p ) = ( F, G ) gives equilibrium Derive U regard fixed point as Kuhn-Tucker condition Application: equilibrium properties Performance Throughput, loss, delay, queue length Fairness, friendliness

Fall 2002Advanced Internet Outline Introduction TCP/AQM algorithms Duality model (F, G, U) Queue management G : RED, REM TCP F and U : Reno, Vegas Linear dynamic model Dynamics of TCP/AQM A scalable control

Fall 2002Advanced Internet Active queue management Idea: provide congestion information by probabilistically marking packets Issues How to measure congestion ( p and G )? How to embed congestion measure? How to feed back congestion info? x(t+1) = F( p(t), x(t) ) p(t+1) = G( p(t), x(t) ) Reno, Vegas DropTail, RED, REM

Fall 2002Advanced Internet RED (Floyd & Jacobson 1993) Congestion measure: average queue length p l (t+1) = [p l (t) + x l (t) - c l ] + Embedding: p-linear probability function Feedback: dropping or ECN marking Avg queue marking 1

Fall 2002Advanced Internet REM (Athuraliya & Low 2000) Congestion measure: price p l (t+1) = [p l (t) +  (  l b l (t)+ x l (t) - c l )] + Embedding: Feedback: dropping or ECN marking

Fall 2002Advanced Internet REM (Athuraliya & Low 2000) Congestion measure: price p l (t+1) = [p l (t) +  (  l b l (t)+ x l (t) - c l )] + Embedding: exponential probability function Feedback: dropping or ECN marking

Fall 2002Advanced Internet Match rate Clear buffer and match rate Clear buffer Key features Sum prices Theorem (Paganini 2000) Global asymptotic stability for general utility function (in the absence of delay)

Fall 2002Advanced Internet Active Queue Management p l (t) G(p(t), x(t)) DropTailloss [1 - c l / x l (t) ] + (?) REDqueue [ p l (t) + x l (t) - c l ] + Vegasdelay [ p l (t) + x l (t)/c l - 1 ] + REMprice [ p l (t) +  l  b l (t)+ x l (t) - c l ) ] + x(t+1) = F( p(t), x(t) ) p(t+1) = G( p(t), x(t) ) Reno, Vegas DropTail, RED, REM

Fall 2002Advanced Internet Congestion & performance p l (t) G(p(t), x(t)) DropTailloss [1 - c l / x l (t) ] + (?) REDqueue [ p l (t) + x l (t) - c l ] + Vegasdelay [ p l (t) + x l (t)/c l - 1 ] + REMprice [ p l (t) +  l  b l (t)+ x l (t) - c l ) ] + Decouple congestion & performance measure RED: `congestion’ = `bad performance’ REM: `congestion’ = `demand exceeds supply’ But performance remains good!

Fall 2002Advanced Internet Utility functions Reno Vegas

Fall 2002Advanced Internet x(t+1) = F( p(t), x(t) ) p(t+1) = G( p(t), x(t) ) Reno: F Primal-dual algorithm: Reno, VegasDropTail, RED, REM for every ack (ca) {W += 1/W } for every loss {W := W/2 }

Fall 2002Advanced Internet Implications Equilibrium characterization Duality Congestion measure p = loss Implications Reno equalizes window w i =  i x i inversely proportional to delay  i dependence for small p DropTail fills queue, regardless of queue capacity

Fall 2002Advanced Internet Validation - Reno 30 sources, 3 groups with RTT = 3, 5, 7ms + 6ms (queueing delay) Link capacity = 64 Mbps, buffer = 50 kB Measured windows equalized, match well with theory (black line)

Fall 2002Advanced Internet Validation – Reno/RED 30 sources, 3 groups with RTT = 3, 5, 7 ms + 6 ms (queueing delay) Link capacity = 64 Mbps, buffer = 50 kB

Fall 2002Advanced Internet Validation – Reno/REM 30 sources, 3 groups with RTT = 3, 5, 7 ms Link capacity = 64 Mbps, buffer = 50 kB Smaller window due to small RTT (~0 queueing delay)

Fall 2002Advanced Internet Queue DropTail queue = 94% RED min_th = 10 pkts max_th = 40 pkts max_p = 0.1 p = Lagrange multiplier! p increasing in queue! REM queue = 1.5 pkts utilization = 92%  = 0.05,  = 0.4,  = 1.15 p decoupled from queue

Fall 2002Advanced Internet Summary DropTail High utilization Fills buffer, maximize queueing delay RED Couples queue length with congestion measure High utilization or low delay REM Decouple performance & congestion measure Match rate, clear buffer Sum prices

Fall 2002Advanced Internet Gradient algorithm Theorem (ToN’99) Converge to optimal rates in asynchronous environment

Fall 2002Advanced Internet Reno & gradient algorithm Gradient algorithm TCP approximate version of gradient algorithm

Fall 2002Advanced Internet queue size for every RTT { if W/RTT min – W/RTT <  then W ++ if W/RTT min – W/RTT >  then W -- } for every loss W := W/2 Vegas F: p l (t+1) = [ p l (t) + x l (t)/c l - 1 ] + G:

Fall 2002Advanced Internet Implications Performance Rate, delay, queue, loss Interaction Fairness TCP-friendliness Utility function

Fall 2002Advanced Internet Performance Delay Congestion measures: end to end queueing delay Sets rate Equilibrium condition: Little’s Law Loss No loss if converge (with sufficient buffer) Otherwise: revert to Reno (loss unavoidable)

Fall 2002Advanced Internet Vegas Utility Equilibrium (x, p) = (F, G) Proportional fairness

Fall 2002Advanced Internet Vegas & Basic Algorithm Basic algorithm TCP smoothed version of Basic Algorithm …

Fall 2002Advanced Internet Vegas & Gradient Algorithm Basic algorithm TCP smoothed version of Basic Algorithm … Vegas

Fall 2002Advanced Internet Validation Source 1Source 3 Source 5 RTT (ms) 17.1 (17) 21.9 (22) 41.9 (42) Rate (pkts/s)1205 (1200) 1228 (1200) 1161 (1200) Window (pkts) 20.5 (20.4) 27 (26.4) 49.8 (50.4) Avg backlog (pkts) 9.8 (10) Single link, capacity = 6 pkts/ms 5 sources with different propagation delays,  s = 2 pkts/RTT meausred theory

Fall 2002Advanced Internet Persistent congestion Vegas exploits buffer process to compute prices (queueing delays) Persistent congestion due to Coupling of buffer & price Error in propagation delay estimation Consequences Excessive backlog Unfairness to older sources Theorem A relative error of  s in propagation delay estimation distorts the utility function to

Fall 2002Advanced Internet Evidence Single link, capacity = 6 pkt/ms,  s = 2 pkts/ms, d s = 10 ms With finite buffer: Vegas reverts to Reno Without estimation errorWith estimation error

Fall 2002Advanced Internet Evidence Source rates (pkts/ms) #src1 src2 src3 src4 src5 15.98 (6) 22.05 (2) 3.92 (4) 30.96 (0.94) 1.46 (1.49) 3.54 (3.57) 40.51 (0.50) 0.72 (0.73) 1.34 (1.35) 3.38 (3.39) 50.29 (0.29) 0.40 (0.40) 0.68 (0.67) 1.30 (1.30) 3.28 (3.34) #queue (pkts)baseRTT (ms) 1 19.8 (20) 10.18 (10.18) 2 59.0 (60)13.36 (13.51) 3127.3 (127)20.17 (20.28) 4237.5 (238)31.50 (31.50) 5416.3 (416)49.86 (49.80)

Fall 2002Advanced Internet end2end queueing delay Vegas/REM To preserve Vegas utility function & rates REM Clear buffer : estimate of d s Sum prices : estimate of p s Vegas/REM

Fall 2002Advanced Internet Vegas/REM To preserve Vegas utility function & rates REM Clear buffer : estimate of d s Sum prices : estimate of p s Vegas/REM end2end price

Fall 2002Advanced Internet Performance Vegas/REM Vegas peak = 43 pkts utilization : 90% - 96%

Fall 2002Advanced Internet Conclusion Duality model of TCP: (F, G, U) Reno, Vegas Maximize aggregate utility With different utility functions DropTail, RED, REM Decouple congestion & performance Match rate, clear buffer Sum prices

Fall 2002Advanced Internet Food for thought How to tailor utility to application? Choosing congestion control automatically fixes utility function Can use utility function to determine congestion control

Fall 2002Advanced Internet Outline Introduction TCP/AQM algorithms Duality model (F, G, U) Equilibrium of TCP/AQM Linear dynamic model Stability of TCP/AQM A scalable control

Fall 2002Advanced Internet Model assumptions Small marking probabilities End to end marking probability Congestion avoidance dominates Receiver not limiting Decentralized TCP algorithm depends only on end-to-end measure of congestion AQM algorithm depends only on local & aggregate rate or queue Constant (equilibrium) RTT

Fall 2002Advanced Internet Dynamics Small effect on queue AIMD Mice traffic Heterogeneity Big effect on queue Stability!

Fall 2002Advanced Internet Stable: 20ms delay Window Ns-2 simulations, 50 identical FTP sources, single link 9 pkts/ms, RED marking

Fall 2002Advanced Internet Queue Stable: 20ms delay Window Ns-2 simulations, 50 identical FTP sources, single link 9 pkts/ms, RED marking

Fall 2002Advanced Internet Unstable: 200ms delay Window Ns-2 simulations, 50 identical FTP sources, single link 9 pkts/ms, RED marking

Fall 2002Advanced Internet Unstable: 200ms delay Queue Window Ns-2 simulations, 50 identical FTP sources, single link 9 pkts/ms, RED marking

Fall 2002Advanced Internet Other effects on queue 20ms 200ms 50% noise avg delay 16ms avg delay 208ms

Fall 2002Advanced Internet Effect of instability Larger jitters in throughput Lower utilization No noise

Fall 2002Advanced Internet Linearized system F1F1 FNFN G1G1 GLGL R f (s) R b ’ (s) TCP Network AQM x y q p

Fall 2002Advanced Internet Linearized system F1F1 FNFN G1G1 GLGL R f (s) R b ’ (s) x y q p TCPREDqueuedelay

Fall 2002Advanced Internet Stability condition Theorem TCP/RED stable if 

Fall 2002Advanced Internet Stability condition Theorem TCP/RED stable if Small  Slow response Large delay

Fall 2002Advanced Internet Stability condition Theorem TCP/RED stable if

Fall 2002Advanced Internet Validation

Fall 2002Advanced Internet Stability region Unstable for Large delay Large capacity Small load

Fall 2002Advanced Internet Role of AQM (Sufficient) stability condition

Fall 2002Advanced Internet Role of AQM (Sufficient) stability condition TCP: AQM: scale down K & shape H

Fall 2002Advanced Internet Role of AQM TCP: RED:REM/PI: Queue dynamics (windowing) Problem!!

Fall 2002Advanced Internet Scalable Control REM Scalable control?? x i (t) p l (t) Stability Utilization Delay

Fall 2002Advanced Internet Outline Introduction TCP/AQM algorithms Duality model (F, G, U) Equilibrium of TCP/AQM Linear dynamic model Stability of TCP/AQM A scalable control

Fall 2002Advanced Internet Delay compensation Equilibrium High utilization: itegrator at links Low loss & delay: VQ, REM/PI Stability Integrator+network delay always unstable for high  ! SourceNetworkLink

Fall 2002Advanced Internet Delay compensation Equilibrium High utilization: itegrator at links Low loss & delay: REM, PI (HMTG01a) Stability Integrator+network delay always unstable for high  ! Delay invariance Scale down gain by  (known at sources) This is self-clocking! SourceNetworkLink

Fall 2002Advanced Internet Compensation for capacity Speed of adaptation increases with Link capacity #bottleneck links in path Capacity invariance Scale down by capacity c l at links Scale up by rate x i (t) at sources

Fall 2002Advanced Internet Scalable control (Paganini, Doyle, Low 01) Theorem (Paganini, Doyle, Low 2001) Provided R is full rank, feedback loop is locally stable for arbitrary delay, capacity, load and topology AQM: TCP:

Fall 2002Advanced Internet Scalable control (Paganini, Doyle, Low 01) Theorem (Paganini, Doyle, Low 2001) Provided R is full rank, feedback loop is locally stable for arbitrary delay, capacity, load and topology AQM: TCP: Utility function

Fall 2002Advanced Internet Stability 40ms, window queue 54% noise 200ms, window queue 56% noise

Fall 2002Advanced Internet Papers Scalable laws for stable network congestion control (CDC2001) A duality model of TCP flow controls (ITC, Sept 2000) Optimization flow control, I: basic algorithm & convergence (ToN, 7(6), Dec 1999) Understanding Vegas: a duality model (ACM Sigmetrics, June 2000) REM: active queue management (Network, May/June 2001) netlab.caltech.edu

The End

Fall 2002Advanced Internet Backup slides

Fall 2002Advanced Internet Law Equilibrium window size Equilibrium rate Empirically constant a ~ 1 Verified extensively through simulations and on Internet References T.J.Ott, J.H.B. Kemperman and M.Mathis (1996) M.Mathis, J.Semke, J.Mahdavi, T.Ott (1997) T.V.Lakshman and U.Mahdow (1997) J.Padhye, V.Firoiu, D.Towsley, J.Kurose (1998) J.Padhye, V.Firoiu, D.Towsley (1999)

Fall 2002Advanced Internet Implications Applicability Additive increase multiplicative decrease (Reno) Congestion avoidance dominates No timeouts, e.g., SACK+RH Small losses Persistent, greedy sources Receiver not bottleneck Implications Reno equalizes window Reno discriminates against long connections

Fall 2002Advanced Internet Area = 2w 2 /3 Derivation (I) Each cycle delivers 2w 2 /3 packets Assume: each cycle delivers 1/p packets Delivers 1/p packets followed by a drop Loss probability = p/(1+p) ~ p if p is small. Hence t window 2w/3 w = (4w/3+2w/3)/2 4w/3 2w/3

Fall 2002Advanced Internet Derivation (II) Assume: loss occurs as Bernoulli process rate p Assume: spend most time in CA Assume: p is small w n is the window size after nth RTT

Fall 2002Advanced Internet Simulations

Fall 2002Advanced Internet Renewal model including FR/FR with Delayed ACKs (b packets per ACK) Timeouts Receiver awnd limitation Source rate When p is small and W r is large, reduces to Refinement (Padhye, Firoin, Towsley & Kurose 1998)

Fall 2002Advanced Internet Further Refinements Further refinements of previous formula Padhye, Firoiu, Towsley and Kurose (1999) Other loss models E.Altman, K.Avrachenkov and C.Barakat (Sigcomm 2000) Square root p still appears! Dynamic models of TCP e.g. RTT evolves as window increases

Fall 2002Advanced Internet Link layer protocols Interference suppression Reduces link error rate Power control, spreading gain control Forward error correction (FEC) Improves link reliability Link layer retransmission Hides loss from transport layer Source may timeout while BS retransmits

Fall 2002Advanced Internet Split TCP Each TCP connection is split into two Between source and BS Between BS and mobile Disadvantages TCP not suitable for lossy link Overhead: packets TCP-processed twice at BS (vs. 0) Violates end-to-end semantics Per-flow information at BS complicates handover TCP

Fall 2002Advanced Internet Snoop protocol Snoop agent Monitors packets in both directions Detects loss by dupACKs or local timeout Retransmits lost packet Suppresses dupACKs Disadvantages Cannot shield all wireless losses One agent per TCP connection Source may timeout while BS retransmits TCP snooper

Fall 2002Advanced Internet Explicit Loss Notification Noncongestion losses are marked in ACKs Source retransmits but do not reduce window Effective in improving throughput Disadvantages Overhead (TCP option) May not be able to distinguish types of losses, e.g., corrupted headers

Fall 2002Advanced Internet REM (Athuraliya & Low 2000) Congestion measure: price p l (t+1) = [p l (t) +  (  l b l (t)+ x l (t) - c l )] + Embedding: exponential probability function Feedback: dropping or ECN marking

Fall 2002Advanced Internet Performance Comparison RED B 1 High utilization B 1 Low delay/loss OR REM match rate High utilization clear buffer Low delay/loss AND

Fall 2002Advanced Internet Comparison with RED Goodput Loss Queue REM RED

Fall 2002Advanced Internet Application: Wireless TCP Reno uses loss as congestion measure In wireless, significant losses due to Fading Interference Handover Not buffer overflow (congestion) Halving window too drastic Small throughput, low utilization

Fall 2002Advanced Internet Proposed solutions Ideas Hide from source noncongestion losses Inform source of noncongestion losses Approaches Link layer error control Split TCP Snoop agent SACK+ELN (Explicit Loss Notification)

Fall 2002Advanced Internet Third approach Problem Reno uses loss as congestion measure Two types of losses Congestion loss: retransmit + reduce window Noncongestion loss: retransmit Previous approaches Hide noncongestion losses Indicate noncongestion losses Our approach Eliminates congestion losses (buffer overflows)

Fall 2002Advanced Internet Third approach Idea REM clears buffer Only noncongestion losses Retransmits lost packets without reducing window Router REM capable Host Do not use loss as congestion measure Vegas REM

Fall 2002Advanced Internet Performance Goodput

Fall 2002Advanced Internet Food for thought How to tailor utility to application? Choosing congestion control automatically fixes utility function Can use utility function to determine congestion control Incremental deployment strategy? What if some, but not all, routers are ECN- capable

Fall 2002Advanced Internet Acronyms ACKAcknowledgement AQMActive Queue Management ARPAddress Resolution Protocol ARQAutomatic Repeat reQuest ATMAsynchronous Transfer Mode BSDBerkeley Software Distribution BByte (or octet) = 8 bits bpsbits per second CACongestion Avoidance ECNExplicit Congestion Notification FIFOFirst In First Out FTPFile Transfer Protocol HTTPHyper Text Transfer Protocol IABInternet Architecture Board ICMPInternet Control Message Protocol IETFInternet Engineering Task Force IPInternet Protocol ISOCInternet Society MSSMaximum Segment Size MTUMaximum Transmission Unit POSPacket Over SONET QoSQuality of Service REDRandom Early Detection/Discard RFCRequest for Comment RTTRound Trip Time RTORetransmission TimeOut SACKSelective ACKnowledgement SONETSynchronous Optical NETwork SSSlow Start SYNSynchronization Packet TCP Transmission Control Protocol UDP User Datagram Protocol VQVirtual Queue WWWWorld Wide Web

Copyright, 1996 © Dale Carnegie & Associates, In Equilibrium & Dynamics of TCP/AQM Steven Low CS & EE, Caltech netlab.caltech.edu Sigcomm August 2001.

Similar presentations

Presentation on theme: "Copyright, 1996 © Dale Carnegie & Associates, In Equilibrium & Dynamics of TCP/AQM Steven Low CS & EE, Caltech netlab.caltech.edu Sigcomm August 2001."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Copyright, 1996 © Dale Carnegie & Associates, In Equilibrium & Dynamics of TCP/AQM Steven Low CS & EE, Caltech netlab.caltech.edu Sigcomm August 2001.

Similar presentations

Presentation on theme: "Copyright, 1996 © Dale Carnegie & Associates, In Equilibrium & Dynamics of TCP/AQM Steven Low CS & EE, Caltech netlab.caltech.edu Sigcomm August 2001."— Presentation transcript:

Similar presentations

About project

Feedback