ECF: an MPTCP Scheduler to Manage Heterogeneous Paths Yeon-sup Lim1, Erich M. Nahum1, Don Towsley2, and Richard Gibbens3 1 IBM T.J. Watson Research Center 2 University of Massachusetts Amherst 3 University of Cambridge ACM CoNEXT, Incheon, Korea, Dec. 2017

Introduction Multi-Path TCP: simultaneously utilizes multiple interfaces Packets can be scheduled to each subflow according to several policies How do these scheduling policies affect MPTCP performance? App subflow-1 Wi-Fi Socket MPTCP subflow subflow … … WiFi LTE subflow-2 Cellular

MPTCP Default Scheduler Default scheduler sends packets over available subflow with lowest RTT Connection level send buffer Connection level subflow level send buffer MPTCP Scheduler subflow 1 with RTT1 MPTCP scheduler subflow subflow … . subflow 2 with RTT2

What is ideal average bit rate given bandwidths? MPTCP default scheduler provides ideal aggregate bandwidth? - Video Streaming Case Dynamic Adaptive Streaming over HTTP (DASH) Video : 22min, 6 resolutions, 5s chunks Required Bandwidth (bit rate) for each resolution Resolution 144p 240p 360p 480p 760p 1080p Bit Rate (Mbps) 0.26 0.64 1.00 1.60 4.14 8.47 What is ideal average bit rate given bandwidths? e.g. 0.3 Mbps WiFi + 8.6 Mbps LTE > 8.47 Mbps (1080p) All chunks served with 1080p 8.47Mbps* * Depends on ABR schemes, but ABR basic goal is to achieve this average bit rate

MPTCP default scheduler provides ideal aggregate bandwidth MPTCP default scheduler provides ideal aggregate bandwidth? - Video Streaming Case Measured average bit rate ratio to ideal 0.3Mbps WiFi & 8.6Mbps LTE Ideal = 0.3 + 8.6 > 8.47 (1080p) → 8.47 Measured = 2.30 Ratio = 2.30 / 8.47 = 0.28 0.3Mbps WiFi & 0.3 Mbps LTE Ideal = 0.3 + 0.3 ≈ 0.64 (240p) → 0.6 Measured = 0.55 Ratio = 0.55 / 0.6 = 0.92 Close to 1 (darker) -> Average bit rate is close to ideal

Motivation Default scheduler causes idle period of fast subflow < 𝑥 𝐶𝑊𝑁 𝐷 𝑓 times tx Connection level send buffer <RTTf RTTf 𝑥 idle … … CWND full Pkt for next GET Subflow with smaller RTT RTTs … CWND available Subflow with larger RTT send

Motivation Waiting for fast subflow can complete transfer earlier 𝑥 𝐶𝑊𝑁 𝐷 𝑓 times tx Connection level send buffer <RTTf RTTf 𝑥 … … CWND full Pkt for next GET Subflow with smaller RTT RTTs … CWND available Subflow with larger RTT send

Motivation Why do these fast subflow idle periods matter? Available bandwidth of fast subflow is not utilized during idle periods CWND frequently restarts by RFC 5681 Another bandwidth loss in fast subflow until CWND becomes sufficient value 0.3Mbps WiFi & 8.6Mbps LTE

Earliest Completion First (ECF) Scheduler If fastest subflow is available, just use it Otherwise, check subflow selected by default scheduler (second fastest subflow): If 𝑅𝑇 𝑇 𝑓 + 𝑥 𝐶𝑊𝑁 𝐷 𝑓 ×𝑅𝑇 𝑇 𝑓 <𝑅𝑇 𝑇 𝑠 i.e., waiting for fastest subflow can complete tx. earlier than using slower one at this moment Do not use second fastest subflow If not, Use second fastest subflow i.e., sender has enough large # of packets in send buffer to efficiently utilize both subflows

Experimental Setup Implemented in MPTCP Linux Kernel (version 0.89) Mobile device (Google Nexus 5) downloads several types of traffic using WiFi and LTE Streaming, HTTP file download, and Web page download Compares to Default, DAPS1, and BLEST2 schedulers Scenarios Controlled in-lab experiments with various conditions Evaluation in the wild [1] N. Kuhn et al., “DAPS:Intelligent delay-aware packet scheduling for multipath transport”, IEEE ICC’14 [2] S. Ferlin et al., “BLEST: Blocking estimation-based MPTCP scheduler for heterogeneous networks”, IFIP Networking’16

In-lab experiments – Streaming w/ fixed bandwidths Measured average bit rate relative to ideal (5 runs) Default Scheduler ECF Close to 1 (darker) -> Average bit rate is close to ideal

In-lab experiments – Streaming w/ fixed bandwidths ECF BLEST DAPS Improves for some cases, but worse for many others Obtains some gains, but ECF is still better

In-lab experiments – Streaming w/ fixed bandwidths LTE CWND Trace (0.3Mbps WiFi and 8.6Mbps LTE) Default’s LTE CWND frequently back to initial value BLEST works better than Default, but More often back to initial than ECF

In-lab experiments – Web browsing Replicated CNN home page (as of 9/11/2014) – 107 Web objects in page HTTP persistent connection is enabled (6 connections) 5Mbps WiFi & 5Mbps LTE (Similar paths) 1Mbps WiFi & 10Mbps LTE (Heterogeneous paths)

Evaluation in the Wild - Streaming DASH server at Washington DC Streaming Client at Cafe using public town WiFi in Amherst, MA 9 runs over two days * Sorted by avg. WiFi RTT small RTT difference - Default and ECF will yield similar performance

Evaluation in the Wild - Streaming similar Avg RTTs (70ms) WiFi has extremely large RTT (≈1sec).

Evaluation in the Wild – Web browsing Replicated CNN home page (as of 9/11/2014) at WDC server Object Download Completion Times from 30 runs 99.9% of object downloads complete in 17 seconds with ECF 30 seconds with Default

Summary Investigated reason of MPTCP performance degradation in presence of path heterogeneity Proposed and implemented new subflow scheduler (ECF) considering completion time Details for implementation in https://cs.umass.edu/~ylim/mptcp_ecf Evaluated ECF scheduler with several types of traffic ECF is never worse than default scheduler and substantially better with path heterogeneity More experimental results in paper HTTP Download workload, Out-of-order delay analysis, …

Thank you! Questions?

In-lab experiments – Streaming w/ fixed bandwidths Backup In-lab experiments – Streaming w/ fixed bandwidths ECF is working with more than two subflows? Experiments using two subflows over WiFi and two over LTE Regulated subflows over each interface to evenly provide designated bandwidth For 8.6 Mbps LTE, each subflow over LTE provides 4.3Mbps bandwidth 0.3 Mbps WiFi and [0.3, 8.6] Mbps LTE

In-lab experiments – Streaming w/ random bandwidths Backup In-lab experiments – Streaming w/ random bandwidths Average throughput during streaming Gains depend on how often heterogeneity happens Note: x axis is chunk index, not time

In-lab experiments – HTTP downloads Backup In-lab experiments – HTTP downloads File downloads using wget for single file of which size is 65KB-1MB In case of small file downloads (128KB) Download time is very short. Second flow is rarely used No difference between default and ECF Ratio of ECF download time to Default (128KB Downloads) More Red: ECF worse More Blue: ECF better White: no difference

In-lab experiments – HTTP downloads Backup In-lab experiments – HTTP downloads In case of large file downloads (256KB to 1MB) ECF works better with path heterogeneity Note: Only one idle period is expected at the end of transfer Significantly large file → Long transfer time → Small effect of idle period → Small gain from ECF 256KB 512KB 1MB