1. CUBS: Coordinated Upload Bandwidth Sharing in Residential Networks Enhua Tan 1, Lei Guo 2, Songqing Chen 3, Xiaodong Zhang 1 1 The Ohio State University 2 Yahoo! Inc 3 George Mason University ICNP’09, Princeton, NJ

2. Residential Networks: Broadband Everywhere 60% in 2005 93% in 2009 2

3. Residential Networks Bandwidth Residential networks download / upload bandwidth : asymmetric -- lower upload bandwidth: – Typical DSL: 384 Kbps upload, 1.5 Mbps download Affected by line quality and distance – Typical Cable: 512 Kbps upload, 7 Mbps download – Cable Network Bandwidth Allocation in U.S. : 3

4. Advantages of Limited Upload Bandwidth Most home users mainly utilize download bandwidth Residential network physical infrastructure fits limited upload bandwidth: – Cable network: tree structure Asymmetric bandwidth less attractive to business users: – Servers pay more because of upload traffic (symmetric) 4

5. Problems Caused by Limited Upload Bandwidth Increasingly demanding and dependable P2P- based applications demand equal upload to download bandwidth in principle – Large file download: slow upload peer suffers from slow download due to tit-for-tat incentive – VoIP: demands substantial and stable upload bandwidth to avoid jitters – Internet live streaming: slow upload leads to quality degradation 5

6. Case 1: VoIP Performance Deteriorates 274 ms jitter! Ping RTT increased by about 1,000 ms! 6 Our Internet experiments: computer connects to cable modem Upload traffic affects ping RTT (round-trip time) from 28ms to 1,000 ms or around: – VoIP demands < 150ms one-way delay: VoIP works poorly!

7. Case 2: BitTorrent Downloads Slowly BitTorrent tit-for- tat: download performance constrained by the cap of upload bandwidth – Our Internet experiments show this effect Download time increases when upload cap decreases 7

8. Would Upload Bandwidth Increase Effectively Address the Problems? Observed 25 residential networks for 21 days, we had two main findings: – Upload channel utilization is bursty and unbalanced: < 20% users highly active in cable network – Overall, about 50% - 80% upload bandwidth is idle Simply increasing the upload bandwidth is not a cost-effective solution: more idle upload bandwidth would be underutilized 8

9. Our Solution – CUBS: Coordinated Upload Bandwidth Sharing CUBS monitors availability of upload bandwidth from all members Utilize idle upload bandwidth from neighboring cable or DSL networks Bandwidth sharing is fair and flexible Significantly improve performance of upload intensive applications without affecting the performance of neighboring networks 9

10. Outline Problem Statement and Proposal Internet Measurements System Design Evaluation Conclusion 10

11. Residential Upload Bandwidth Measurement Collected IP addresses using Gnutella crawler Identified Top-7 ISPs: Charter, Comcast, Cox, Road Runner, Ameritech (AT&T), Pacbell, and Verizon Scanned 25 subnets by probing from Planet-Lab: up to 78.8% subnet IPs respond – 25 subnets with 2,040 IP addresses for 21 days (during February and March, 2008) – no control on the residential hosts: ICMP ping probing to estimate upload bandwidth – slightly underestimate the available bandwidth: on the conservative side 11

12. Measured Results Cable networks have higher upload capacity than DSL Average Idle upload bandwidth ratio: Cable higher than DSL Cable DSL 62% - 93% 52% - 83% 12

13. Available Upload Bandwidth Distribution < 10% hosts use more than half of upload bandwidth: Unbalanced DSL upload bandwidth varies among hosts (due to varied line length) Cable DSL Varies little along time: Chance to utilize 13

14. Feasibilities of CUBS Our measurements show that: In broadband networks, there exists plenty of unused upload bandwidth: 50% - 80%: – Suggests that CUBS is feasible to utilize the unused bandwidth Upload channel is utilized in an unbalanced way (less than 20% users fully utilized their upload bandwidth): – CUBS will be feasible to increase upload bandwidth without demanding additional bandwidth supply 14

15. Outline Problem Statement and Proposal Internet Measurements System Design Evaluation Conclusion 15

16. Challenges of CUBS Design How to share upload bandwidth? How to discover unused upload bandwidth? How to encourage users to participate? 16

17. Residential Networks: Wi-Fi Everywhere Jupiter Research: 65% of the U.S. households use Wireless connections in 2006 -- higher usage now A study in 2006: # of APs (Access Points) detected in residential areas ranges 2-20 Residential Network Cable/DSL Modem Wireless Access Point (AP) Wireless Station 17

18. CUBS System 18 Station A Station B Station C Local AP of A Local AP of B Local AP of C Periodically estimate local AP’s idle upload bandwidth CUBS System Associate with foreign AP Active using upload bandwidth now Associate with another foreign AP

19. CUBS Overlay: Discover Bandwidth Bootstrap: a tracker to record the local AP-ID and IP address of each node – The local AP-ID: {ESSID, AP MAC address} Overlay connections: new node queries tracker to connect with nearby nodes Idle bandwidth discovery: each node measures its local AP’s upload bandwidth usage to compute idle bandwidth, and shares this info on overlay 19

20. CUBS Node: Wireless for Sharing Dual connections: branching node’s upload traffic to – the local AP through one network interface – the foreign AP through another wireless interface Choose foreign AP for bandwidth sharing: – highest available upload bandwidth – acceptable signal strength Connection binding (network layer): – modifies the connect system call – binds new connection to one of the available network interfaces based on traffic load (or randomly) 20

21. Fair Usage of the Upload Bandwidth Serves as an incentive mechanism to encourage users to share bandwidth over CUBS Foreign AP owner is able to fully use upload bandwidth: – Tell other users to refrain from competitions when needed – Other user restrains further usage of the foreign AP, and tries to hand-off to another foreign AP 21

22. No Excessive Usage of Download For a new TCP connection: – If it is upload intensive or bidirectional, the connection can be bound to the foreign AP – Otherwise should be bound to local AP Requires prediction of a connection’s upload to download traffic ratio 22

23. No Excessive Usage of Download: Prediction Evaluation Trace-driven evaluations: – packet header trace of Dartmouth residential wireless network (2004) – uses the mean of the upload-to-download traffic ratios of the latest n connections to the same host for prediction 23 for more than 80% connections, the prediction difference is less than 20%

24. Discussions Overlay-based user authentication: – AP owner controls who can use the AP through AP web service to prevents abusive usage Multiplexing one wireless interface: – FatVAP (NSDI’08) allows one wireless interface to associate with multiple APs: utilize Power-Saving Mode buffering – Can be useful for CUBS user with only one interface 24

25. Discussions (Cont.) Mesh Networks for Sharing Long Distance APs: – CUBS node currently only connects to neighboring APs to harness available bandwidth – CUBS wireless mesh network can route an upload packet to the Internet via an AP which can be out of the signal range of the originator – Then upload bandwidth can be shared among any nodes in the mesh network 25

26. Outline Problem Statement and Proposal Internet Measurements System Design Evaluation Conclusion 26

27. Evaluation Setup To emulate the residential network, we use NISTNet to control the Internet connections of the APs Configure the upload and download capacity to be 500 Kbps and 7 Mbps – typical bandwidth caps for a cable subscription 27

28. BitTorrent Experiments on Planet-Lab Repeatable swarm: 1 seed and 18 peers on Planet-Lab Download lasts for one hour, w/o and w/ CUBS The average download throughput is improved by > 30%: – No matter it is load-based or hash-based (remote IP) 28

29. Fair Bandwidth Sharing in CUBS 29 AP owner gets full upload bandwidth quickly Idle bandwidth is detected and utilized again Utilize foreign AP’s upload bandwidth

30. Conclusion A low upload bandwidth cap in residential networks causes problems to P2P applications Measurements show the total upload is often underutilized – And the upload channel utilization is bursty and unbalanced CUBS enables a user to share idle upload bandwidth of its coordinated neighbors: – Improve the performance of delay-sensitive or P2P applications – Without demanding additional bandwidth supply from ISPs or infrastructure support – Flexibly shares idle upload bandwidth while providing fairness guarantees 30

31. Thank you! Enhua Tan: etan@cse.ohio-state.edu http://www.cse.ohio-state.edu/hpcs/ http://www.cse.ohio-state.edu/hpcs/ 31

32. Probing Ping RTT Distribution (When Upload Channel is Saturated) most RTTs are larger than 500ms: – implies that queue lengths of residential networks are very large – VoIP quality will degrade when upload is saturated 32