1. Internet congestion control
Damon Wischik, UCL
TexPoint fonts used in EMF.
Read the TexPoint manual before you delete this box.: AAAAAAAAA

2. What is queueing theory for?
DJW /24
Given the arrival process, what is
We often seek limit theorems to describe the arrival process, and to approximate this probability
Typical motivation: “this lets us choose buffer size etc. to ensure that queue overflow is rare”
This does not make sense for Internet queues, since TCP (which governs most Internet traffic) deliberately causes the queue to overflow
arrival process
queue
Example: for a queue with N independent flows, each self-similar with Hurst parameter H, then for some ,  >0

3. Some Internet history
DJW /24
1974: First draft of TCP/IP [“A protocol for packet network interconnection”, Vint Cerf and Robert Kahn]
1983: ARPANET switches on TCP/IP
1986: Congestion collapse
1988: Congestion control for TCP [“Congestion avoidance and control”, Van Jacobson]

4. TCP transmission control protocol
DJW /24
Internet congestion is controlled by the end-systems
The TCP algorithm, running on the server, decides how fast to send data
It sends data packets, and waits for ACKnowledgement packets
It sets a target window size wt the number of packets that have been sent but not yet acknowledged (here 5+3)
The average transmission rate is xt ≈ wt/RTT where RTT is the round trip time
It adapts wt based on the ACKs it receives
acknowledgements
End-to-end argument:
Generally, _some_ end2end control is necessary. It’s probably also sufficient. So it’s simplest/most elegant to use only end2end control.
(It may not be most efficient.)
Server
(TCP)
User
data

5. TCP transmission rate [0–100 kB/sec] time [0–8 sec]
if (seqno > _last_acked) {
if (!_in_fast_recovery) {
_last_acked = seqno;
_dupacks = 0;
inflate_window();
send_packets(now);
_last_sent_time = now;
return; }
if (seqno < _recover) {
uint32_t new_data = seqno - _last_acked;
if (new_data < _cwnd) _cwnd -= new_data; else _cwnd=0;
_cwnd += _mss;
retransmit_packet(now);
uint32_t flightsize = _highest_sent - seqno;
_cwnd = min(_ssthresh, flightsize + _mss);
_in_fast_recovery = false;
if (_in_fast_recovery) {
_dupacks++;
if (_dupacks!=3) {
_ssthresh = max(_cwnd/2, (uint32_t)(2 * _mss));
_cwnd = _ssthresh + 3 * _mss;
_in_fast_recovery = true;
_recover = _highest_sent;
DJW /24
transmission rate [0–100 kB/sec]
Multiplicative decrease -- when network is congested, load grows exponentially (transmit, retransmit, reretransmit...)
Additive increase -- helps with fairness (of relatively greater benefit to low-bandwidth users)
Devised by Jain+Ramakrishnan (1988) DECbit
time [0–8 sec]

6. How TCP shares capacity
DJW /24
individual flow rates
available capacity
aggregate flow rate
time

7. How to describe a network
DJW /24
Some networks admit models at three different levels:
Microscopic rules of behaviour for the individual elements
e.g. rules of motion of electrons
Macroscopic laws that describe the behaviour of aggregates
e.g. Kirchhoff’s law, Ohm’s law
Teleological principles that describe the operating point to which the system settles down
e.g. Thomson’s principle

8. TCP model I: the sawtooth
DJW /24
Consider a single link, and a single flow with round trip time RTT
When the link is not congested, transmission rate xt increases by 1/RTT i2 pkt/sec every sec
When the link becomes congested, transmission rate is cut by 50%
service capacity
transmission rate x(t) * * *
buffer size
queue size
time
queue becomes full and packets are dropped
TCP realizes there is congestion, and cuts its rate

9. TCP model II: billiards
DJW /24
Consider a single link, and many flows with round trip times RTT(1), RTT(2),...
When the link is not saturated, flow r increases its transmission rate xt(r) by 1/RTT(r) pkt/sec2
When the link becomes saturated, some flows experience packet drops and they cut their rates
Assume some probabilistic model that says which flows experience drops
[Baccelli+Hong 2002] + + = * * * * * * *

10. TCP model III: fluid + + + + = =
DJW /24
Individual TCP flows follow a sawtooth + +
individual flow rates + + = =
aggregate flow rate
time

11. TCP model III: fluid + + + + = =
DJW /24
Individual TCP flows follow a sawtooth
... but many TCP flows added together are smoother
the average rate xt varies according to a time-delayed differential equation
eqn involves pt, the packet drop probability experienced by packets sent at time t
[Misra+Gong+Towsley 2000, Baccelli+McDonald+Reynier 2002, after Kelly et al. 1998] + +
individual flow rates + + = =
aggregate flow rate
time

12. TCP model III: fluid
DJW /24
There are three possible limit models for describing the behaviour of the queue and calculating the drop probability
TCP selects a limiting model, depending on the size of the buffer
Round trip time RTT
N TCP flows
Service rate NC
Buffer size B(N)

13. Three buffer-sizing regimes
DJW /24
small buffers
B(N)=B
medium buffers
B(N)=B√N
large buffers
B(N)=BN
drop probability [0–.3]
queue size [0–B pkt]
traffic intensity [0–1]
time [0–5s]
queue size [(B-25)–B pkt]
time [50ms]
These three regimes are qualitatively different; they are described by different dynamical systems, operating over different timescales
The relationship between buffer size and timescales was first observed by [Deb+Srikant, 2004]

14. Instability / synchronization
DJW /24
Sometimes the dynamical system for the network is unstable, and the queue size flip-flops
This means there are short periods of concentrated packet loss
This might manifest itself as a burst of static in an Internet telephone call
In fact, this corresponds to the billiards model for TCP
Engineers can use the dynamical system description of TCP to help them design the network so that it is stable
queue size [0–619pkt]
TCP window sizes [0-25pkt]
time [0–5s]

15. Equilibrium analysis
DJW /24
A well-designed network should be stable, i.e. it should reach some equilibrium state
How can we characterize the equilibrium state?

16. Teleological description
DJW /24
Round trip time RTT
N TCP flows; flow r has rate xr
Service rate C
Medium buffer size
The equilibrium state is the solution to the optimization problem
U(x) = const - 2 / RTT^2 x^2 x
U(x)

17. Teleological description
DJW /24
Round trip time RTT
N TCP flows; flow r has rate xr
Service rate C
Buffer size medium
little extra value attached to high-throughput flows
The equilibrium state is the solution to the optimization problem
U(x) = const - 2 / RTT^2 x^2
U(x)
severe penalty for allocating too little throughput x

18. Teleological description
DJW /24
Round trip time RTT
N TCP flows; flow r has rate xr
Service rate C
Buffer size medium
flows with large RTT are satisfied with just a little throughput
The equilibrium state is the solution to the optimization problem
U(x) = const - 2 / RTT^2 x^2
U(x)
flows with small RTT demand higher throughput x

19. Teleological description
DJW /24
Round trip time RTT
N TCP flows; flow r has rate xr
Service rate C
We should thus set up caches very close to the user, even if that means putting them in a congested part of the network
Buffer size medium
flows with large RTT are satisfied with just a little throughput
The equilibrium state is the solution to the optimization problem
U(x) = const - 2 / RTT^2 x^2
U(x)
flows with small RTT demand higher throughput x

20. Teleological description
DJW /24
Round trip time RTT
N TCP flows; flow r has rate xr
Service rate C
Buffer size medium
The equilibrium state is the solution to the optimization problem
U(x) = const - 2 / RTT^2 x^2 x
U(x)
there is no penalty until the link is overloaded, i.e. y>C
The network seeks to be overloaded, leading to poor quality for Internet telephone calls

21. Maths conclusion TCP chooses its own limit regime
DJW /24
TCP chooses its own limit regime
TCP, through its adaptive feedback mechanism, induces a fluid limit over the timescale of a round trip time
The queue dynamics change fluidly over the timescale of a round trip time, and stochastically over much shorter timescales
The choice of buffer size determines which of three queueing models is relevant

22. Design conclusion The TCP code admits description at two higher levels
DJW /24
The TCP code admits description at two higher levels
a dynamical system model at the macroscopic level
a welfare optimization problem at the teleological level
This gives us insight into how to extend TCP
make sure the dynamical system is stable
choose a more desirable optimization problem
These insights have led to new ideas about routing, load balancing, choice of buffer size in core routers, quality of service, high-speed TCP