Jiyong Park Seoul National University, Korea 2005. 2. 22. Preventing network performance interference of bidirectional TCP traffic with packet scheduling in a home network gateway using an asymmetric link Jiyong Park Seoul National University, Korea 2005. 2. 22.

Contents Overview of TCP Network model Performance problems of TCP on asymmetric link Causes Normalized asymmetry ratio Bottleneck buffer size Competing reverse traffic Proposed Solutions CBQ CBQ with simple implementation ACK filtering (AF) / ACK reconstruction (AR) CBQ+AF/AR ACQ+AF/AR Our problem

Related Papers T.V. Lakshmnan et al., Window-based error recovery and flow control with a slow acknowledgement channel: a study of TCP/IP performance, INFOCOM, 1997 --(1) L. Kalampoukas et al., Improving TCP Throughput over Two-Way Asymmetric Links: Analysis and Solutions, SIGMETRICS, 1998 --(2) H. Balakrishnan et al., The Effects of Asymmetry on TCP Performance, MobiCom, 1997 --(3)

TCP Overview Three key mechanisms of TCP Reliability Acknowledge Flow control (speed difference between sender and receiver application) Sliding window Window advertisement Congestion control (packet loss, delay, ...) Slow start Congestion avoidance

TCP Overview – 2. Flow Control Window Number of packets that can be sent without acknowledged Sliding window Upon receiving ACK, window slides Window advertisement Receiver advertises its window size on ACK (sender cannot send more than that)

TCP Overview – 2. Flow Control Sender Buffer Sent ACKed Sent Not ACKed Will send TCP output window (size 5) user output 1 2 3 4 5 6 7 8 9 10 11 11 12 12 13 14 13 14 sequence number ACK received: 6 9

TCP Overview – 3. Congestion Control Two kinds of windows Congestion window (cwnd) Advertised window (awnd) actual window size = min(cwnd, awnd) Upon receiving ACK, increases cwnd (try to send more) Slow start (aggressive) (cwnd++ upon receiving 1 ACK) Congestion avoidance (cautious) (cwnd ++ upon receiving cwnd ACKs) Congestion is caused by Data packet loss (detected by duplicated ACK) Long delay (detected by timer expiration) Upon detecting congestion, decreases cwnd = 1 (restart)

TCP Overview – 3. Congestion Control Enters congestion avoidance phase when W > Wt Wt := Wt / 2 when congestion is detected congestion window size congestion detected time

TCP Overview – Packet Processing TCP protocol handler performs two major jobs Upon receiving ACK, send data packet Upon receiving data, deliver it to user and send ACK These two jobs are handled by one handler (no interleaving) User process Data packet IP(input) queue Output queue data ACK ACK packet ACK data awaken by interrupt handler Protocol handler

Network Model (1/2) BS : 66 US : 800Kbps Local Host TS : 10ms Remote Host TF: 10ms UF : 2.1Mbps BF : 66 WL: 66 WR : 66 W (maximum window size): maximum number of packets that can be in the network U: transmission speed in bps B: queue size in packets T: propagation delay Data packet size: 1040 byte ACK packet size: 40 byte

Network Model (2/2) Application programs are assumed to be greedy. There are always packets to send. There are always rooms to receive packets.

Asymmetry Ratio (1/2) Asymmetry Ratio (k) Service speed for data packet on forward link / Service speed for ACK packet on reverse link Example Fast link can send 1000 data packets per second Slow link can send 25 ACK packets per second Then k = 1000/25 = 40

Asymmetry Ratio (2/2) At the receiving side, one ACK is generated for one data packet received. If k > 1, ACKs are queued. If k <= 1, no problem. 1000 data pkt/s 25 ACK pkts/s Local Host Remote Host ACK Data 1000 data pkt/s

Lets first consider the case when only download traffic is present.

sequence number of data Effect of Small k ACK arrives periodically and smoothly (sequence number increases by 1) Smooth transmission sequence number sequence number of data sequence number of ACK time

Effect of Large k Output buffer size of slow link is the key. Size is too small: (k-1) out of k ACKs are dropped Size is too large: entire ACKs are queued 4m pkt/s m pkt/s 9 5 1 6 2 7 3 dropped 8 4 9 8 7 6 5 4 3 2 1

Large k with Small Buffer (1/2) Since ACK is cumulative, sequence number increases abruptly by k  Bursty transmission (output buffer is needed) sequence number sequence number of data k burst sequence number of ACK k burst time

Large k with Small Buffer (2/2) Output buffer on fast link should larger than k Bursty packets are regulated by buffer queue size burst regulated time

Large k with Large Buffer Large buffer is problematic. Throughput is decreased. ACKs are very delayed. Sender has to wait to receive the ACKs. Idle time occurs. Example: Maximum window size: 100 packets. k = 10 Time to send full window of data packets: t seconds Time to send full window of ACK packets: 10t seconds RTT = 11t seconds. Achieved throughput = 100 / 11t = 9/t Ideal throughput = 100/t

Summary of Single Download Connection k <= 1 No problem (our case) k > 1 Size of output buffer on slow link is small Bursty transmission. Solved by using output buffer on fast link (size > k) Size of output buffer on slow link is very big Big problem. Very long delay of ACKs. Reduced throughput.

Now, proceed to the case of two-way traffic.

Two-way Connection On uplink, ACK packets for download traffic Data packets for upload traffic Size of output buffer on the link is small Upload speed is reduced. Size of output buffer on the link is large Download speed is reduced. Local Host ACK Data Case of our ADSL Modem

Two-way Connection – Case 1 Size of output buffer on slow link is small Upload speed is reduced. ACK and data packets are also dropped. Dropping ACK is not a big problem (solvable by buffer)  Download speed is not affected. Dropping data packet is a big problem Timeout  Retransmission  Upload speed is reduced.

Two-way Connection – Case 2 Size of output buffer on slow link is large Download speed is reduced. ACKs are queued behind the big data packets ACK compressing ACK packets for download data packets for upload 9 8 7 6 5 4 3 2 1 9 8 7 6 5 4 3 2 1 Very big delay

Two-way Connection - Our Case k = 0.1 (ACK is seldom dropped) Window size = 66 Upload buffer size = 100 Upload buffer is small  Upload speed is reduced. Buffer size : 100 34 data packets for upload traffic 66 ACKs for download traffic 23 data packets are dropped

Proposed Solutions Paper #1 Paper #2 Paper #3 Upload buffer size is small Upload throughput is reduced Solution: round-robin queue scheduling Paper #2 Upload buffer size is big Download throughput is reduced Solution: queue scheduling Paper #3 Solution: ACC, AF, ACK-first, Sender adaptation Similar to our approach (More flexible than us) Similar to our approach (Better performance than us) We can adopt some techniques from here

Solution in the Paper #1 Round-robin queue scheduling f is big ACK for download 9 8 7 6 5 4 3 2 1 f RR scheduler data for upload 9 8 7 6 5 4 3 2 1 1-f f is big Upload: slow (starvation) Download: good f is small Upload: good (1-f of the full speed) Download: slowdown (bursty transmission and small size of fast buffer)

Solution in the Paper #2 Emulate fair queuing scheduler by counter-based implementation Upload speed is guaranteed by (1-f) of the full speed. Download speed is reduced Slow buffer size is large Uses simple FIFO ACKs are delayed very long

Solution in the Paper #3 Ack Filtering (AF) queuing Ack-first scheduling AF Remove all redundant cumulative ACKs in the queue If there is an ACK packet in the queue, always transmit it first. 10 9 8 7 6 5 4 3 2 1

Solution in the Paper #3 Download: full speed (no delay of ACK) Upload: starvation ACKs are arrived faster than they are processed. So, there are always at least one ACK in the queue. ACK has strictly higher priority than data. Data has no chance to be transmitted.  starvation

My Previous Solution Solution: Give higher priority to ACK over data Result Other papers: download performance was bad. Our result: both traffic was good. Reason: k is very small in our environment. Bandwidth consumed by ACK packets is very low About 11% (94Kbps on 800Kbps link) Enough bandwidth to send data packet for upload traffic

Our New Solution AF + Queue scheduling Queue scheduling AF Guarantees upload throughput AF Minimizes ACK latency for download traffic

Better Solution Proposed (1/5) Fatma Louati, Chadi Barakat, Walid Dabbous, Handling two-way TCP traffic in asymmetric networks, Speed Networks and Multimedia Communications (HSNMC), 2004 AF + AR + ACQ

Better Solution Proposed (2/5) AF + AR + ACQ AF Minimizes ACK latency for download traffic AR (ACK Regeneration) Regulates burst transmission caused by AF ACQ (Adaptive Class-based Queuing) A kind of CBQ in which the bandwidth ratio is changed automatically and periodically RFC 3449 recommendations (by the authors of the paper #3)

Better Solution Proposed (3/5) Ratio update algorithm rn: bandwidth allocated to data packet of upload traffic current download speed current upload speed

Better Solution Proposed (4/5) Performance metric Link efficiency of upload traffic + link efficiency of download traffic from 0 to 2 ACQ+AF/AR : 1.72

Better Solution Proposed (5/5) strong asymmetry weak asymmetry