A Digital Fountain Approach to Reliable Distribution of Bulk Data

Slides:



Advertisements
Similar presentations
Computer Concepts – Illustrated 8th edition
Advertisements

GSA Pizza Talk - EPFL - Capillary routing with FEC by E. Gabrielyan 1 Capillary Multi-Path Routing for reliable Real-Time Streaming with FEC.
McGraw-Hill©The McGraw-Hill Companies, Inc., 2003 Chapter 11 Ethernet Evolution: Fast and Gigabit Ethernet.
Clustering of Source/Channel Rate Allocations for Receiver-driven Multicast under a Limited Number of Streams Philip A. Chou, Microsoft Research Kannan.
Fountain Coding-based Video Transmission System over Heterogeneous Wireless Networks Presented by Hyunchul Joo POSTECH
Multipath Routing for Video Delivery over Bandwidth-Limited Networks S.-H. Gary Chan Jiancong Chen Department of Computer Science Hong Kong University.
Justine Sherry*, Shaddi Hasan*, Colin Scott*, Arvind Krishnamurthy†,
Protocol layers and Wireshark Rahul Hiran TDTS11:Computer Networks and Internet Protocols 1 Note: T he slides are adapted and modified based on slides.
Streaming Video over the Internet
1 Haitao Zheng and Jill Boyce IEEE Transaction on Multimedia Leif 9/10/01 An Improved UDP Protocol for Video Transmission Over Internet-to-Wireless Networks.
The IP Revolution. Page 2 The IP Revolution IP Revolution Why now? The 3 Pillars of the IP Revolution How IP changes everything.
Chapter 1: Introduction to Scaling Networks
Home Gateway for Three-Screen TV Using H.264 SVC and Raptor FEC
Countering DoS Attacks with Stateless Multipath Overlays Presented by Yan Zhang.
LT Codes Paper by Michael Luby FOCS ‘02 Presented by Ashish Sabharwal Feb 26, 2003 CSE 590vg.
Digital Fountains: Applications and Related Issues Michael Mitzenmacher.
1 The ns-2 Network Simulator H Plan: –Discuss discrete-event network simulation –Discuss ns-2 simulator in particular –Demonstration and examples: u Download,
Company LOGO F OUNTAIN C ODES, LT C ODES AND R APTOR C ODES Susmita Adhikari Eduard Mustafin Gökhan Gül.
Improving Communication Quality with Reed Solomon Code in Internet Voice Broadcasting System Shingo Kashima Kyushu University, Japan Asia-Pacific Advanced.
Digital Fountain Codes V. S
Jump to first page A. Patwardhan, CSE Digital Fountains Main Ideas : n Distribution of bulk data n Reliable multicast, broadcast n Ideal digital.
D.J.C MacKay IEE Proceedings Communications, Vol. 152, No. 6, December 2005.
Multicast and Unicast Real-Time Video Streaming Over Wireless LANs Abhik Majumdar, Daniel Grobe Sachs, Igor V. Kozintsev, Kannan Ramchandran, and Minerva.
Scalable On-demand Media Streaming Anirban Mahanti Department of Computer Science University of Calgary Canada T2N 1N4.
Lava: A Reality Check of Network Coding in Peer-to-Peer Live Streaming Mea Wang, Baochun Li Department of Electrical and Computer Engineering University.
Performance Analysis of a Parallel Downloading Scheme from Mirror Sites Throughout the Internet Allen Miu, Eugene Shih Class Project December 3,
Scalable On-demand Media Streaming with Packet Loss Recovery Anirban Mahanti Department of Computer Science University of Calgary Calgary, AB T2N 1N4 Canada.
RAID- Redundant Array of Inexpensive Drives. Purpose Provide faster data access and larger storage Provide data redundancy.
A Server-less Architecture for Building Scalable, Reliable, and Cost-Effective Video-on-demand Systems Jack Lee Yiu-bun, Raymond Leung Wai Tak Department.
Typhoon: An Ultra-Available Archive and Backup System Utilizing Linear-Time Erasure Codes.
Computer Science 1 ShapeShifter: Scalable, Adaptive End-System Multicast John Byers, Jeffrey Considine, Nicholas Eskelinen, Stanislav Rost, Dmitriy Zavin.
Efficient and Flexible Parallel Retrieval using Priority Encoded Transmission(2004) CMPT 886 Represented By: Lilong Shi.
Streaming Video over the Internet: Approaches and Directions Dapeng Wu, Yiwei Thomas Hou et al. Presented by: Abhishek Gupta
Web Caching Schemes1 A Survey of Web Caching Schemes for the Internet Jia Wang.
Erasure Correcting Codes
Fountain Codes Amin Shokrollahi EPFL and Digital Fountain, Inc.
Computer Network Architecture and Programming
Fine-Grained Layered Multicast John Byers Dept. of Computer Science, Boston University Digital Fountain, Inc.
SomeCast A Paradigm for Real-Time Adaptive Reliable Multicast Presented by: Ibrahim Matta IEEE Real-Time Technology and Applications Symposium (RTAS ‘2000),
Reliable Transport Layers in Wireless Networks Mark Perillo Electrical and Computer Engineering.
1 Scalable Video Coding with Digital Fountain Kai-Chao Yang.
Digital Fountain with Tornado Codes and LT Codes K. C. Yang.
Informed Content Delivery Across Adaptive Overlay Networks John Byers Dept. of Computer Science, Boston University Joint work with.
Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads John Byers, Boston University Michael Luby, Digital Fountain, Inc.
1 CMSCD1011 Introduction to Computer Audio Lecture 10: Streaming audio for Internet transmission Dr David England School of Computing and Mathematical.
Repairable Fountain Codes Megasthenis Asteris, Alexandros G. Dimakis IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 32, NO. 5, MAY /5/221.
Exploring VoD in P2P Swarming Systems By Siddhartha Annapureddy, Saikat Guha, Christos Gkantsidis, Dinan Gunawardena, Pablo Rodriguez Presented by Svetlana.
1 Codes, Bloom Filters, and Overlay Networks Michael Mitzenmacher.
Min Xu1, Yunfeng Zhu2, Patrick P. C. Lee1, Yinlong Xu2
Computer Science Informed Content Delivery Across Adaptive Overlay Networks Overlay networks have emerged as a powerful and highly flexible method for.
On the use of Reliable Multicast for Content Distribution Vassilis Chatzigiannakis
Threshold Phenomena and Fountain Codes Amin Shokrollahi EPFL Joint work with M. Luby, R. Karp, O. Etesami.
Lan F.Akyildiz,Weilian Su, Erdal Cayirci,and Yogesh sankarasubramaniam IEEE Communications Magazine 2002 Speaker:earl A Survey on Sensor Networks.
CprE 545 project proposal Long.  Introduction  Random linear code  LT-code  Application  Future work.
Experiences with Multimedia Streaming over 2.5G and 3G Networks J. Chesterfield, R. Chakravorty, J. Crowcroft, P. Rodriguez, S. Banerjee Presented by Denny.
Peer-Assisted Content Distribution Pablo Rodriguez Christos Gkantsidis.
Page 1 The department of Information & Communications Engineering Dong-uk, kim A Survey of Packet Loss Recovery Techniques for Streaming.
March 2001 CBCB The Holy Grail: Media on Demand over Multicast Doron Rajwan CTO Bandwiz.
CoopNet: Cooperative Networking
A Mechanism for Communication- Efficient Broadcast Encryption over Wireless Ad Hoc Networks Johns Hopkins University Department of Computer Science Reza.
ITIS 1210 Introduction to Web-Based Information Systems Chapter 39 How Multicast IP and MBone Work.
INRIA Rhône-Alpes - Planète project 1 Design and evaluation of a Low Density Generator Matrix (LDGM) large block FEC codec Vincent Roca, Zainab Khallouf,
SURENDRA INSTITUTE OF ENGINEERING & MANAGEMENT PRESENTED BY : Md. Mubarak Hussain DEPT-CSE ROLL
Networked Multimedia Basics. Network Characteristics.
Coding for Multipath TCP: Opportunities and Challenges Øyvind Ytrehus University of Bergen and Simula Res. Lab. NNUW-2, August 29, 2014.
INTERNET PROTOCOL TELEVISION (IP-TV)
Solutions to Support Mobility in SDN-Based Dense Wireless Networks
MDC METHOD FOR HDTV TRANSMISSION OVER EXISTING IP NETWORK
INTERNET PROTOCOL TELEVISION (IP-TV)
Quick UDP Internet Connections
Presentation transcript:

A Digital Fountain Approach to Reliable Distribution of Bulk Data John Byers, ICSI Michael Luby, ICSI Michael Mitzenmacher, Compaq SRC Ashu Rege, ICSI

Application: Software Distribution New release of widely used software. Hundreds of thousands of clients or more. Bulk data: tens or hundreds of MB Heterogeneous clients: Modem users: hours Well-connected users: minutes

Primary Objectives Scale to vast numbers of clients No ARQs or NACKs Minimize use of network bandwidth Minimize overhead at receivers: Computation time Useless packets Compatibility Networks: Internet, satellite, wireless Scheduling policies, i.e. congestion control

Impediments Packet loss Receiver heterogeneity wired networks: congestion satellite networks, mobile receivers Receiver heterogeneity packet loss rates end-to-end throughput Receiver access patterns asynchronous arrivals and departures overlapping access intervals

Digital Fountain k Source Instantaneous Encoding Stream Transmission k Received Instantaneous Can recover file from any set of k encoding packets. k Message

Digital Fountain Solution Transmission 5 hours 4 hours 3 hours 2 hours 1 hour 0 hours File User 1 User 2

Is FEC Inherently Bad? Faulty Reasoning But… FEC adds redundancy Redundancy increases congestion and losses More losses necessitate more transmissions FEC consumes more overall bandwidth But… Each and every packet can be useful to all clients Each client consumes minimum bandwidth possible FEC consumes less overall bandwidth by compressing bandwidth across clients

DF Solution Features Users can initiate the download at their discretion. Users can continue download seamlessly after temporary interruption. Tolerates moderate packet loss. Low server load - simple protocol. Does scale well. Low network load.

Approximating a Digital Fountain k Source Encoding Time Encoding Stream (1 + c) k Received Decoding Time k Message

Approximating a DF: Performance Measures Time Overhead: Time to decode (or encode) as a function of k. Decoding Inefficiency: packets needed to decode k

Work on Erasure Codes Standard Reed-Solomon Codes Dense systems of linear equations. Poor time overhead (quadratic in k) Optimal decoding inefficiency of 1 Tornado Codes [LMSSS ‘97] Sparse systems of equations. Fast encoding and decoding (linear in k) Suboptimal decoding inefficiency

Tornado Z: Encoding Structure Irregular bipartite graph Irregular bipartite graph k stretch factor = 2 k = 16,000 nodes = source data = redundancy

Encoding/Decoding Process

Timing Comparison Tornado Z: Average inefficiency = 1.055 Encoding time, 1K packets Reed-Solomon Tornado Z Size 250 K 500 K 1 MB 2 MB 4 MB 8 MB 16 MB 4.6 sec. 19 sec. 93 sec. 442 sec. 30 min. 2 hrs. 8 hrs. 0.11 sec. 0.18 sec. 0.29 sec. 0.57 sec. 1.01 sec. 1.99 sec. 3.93 sec. Decoding time, 1K packets 2.06 sec. 8.4 sec. 40.5 sec. 199 sec. 13 min. 1 hr. 4 hrs. 0.24 sec. 0.31 sec. 0.44 sec. 0.74 sec. 1.28 sec. 2.27 sec. Tornado Z: Average inefficiency = 1.055 Both codes: Stretch factor = 2

Cyclic Interleaving Encoded Blocks Interleaved Encoding Blocks File Transmission Encoded Blocks Interleaved Encoding Blocks Encoding Copy 1 File Encoding Copy 2 Tornado Encoding

Cyclic Interleaving: Drawbacks The Coupon Collector’s Problem Waiting for packets from the last blocks: More blocks: faster decoding, larger inefficiency T B blocks

Scalability over File Size

Scalability over Receivers

Digital Fountain Prototype Built on top of IP Multicast. Tolerating heterogeneity: Layered multicast Congestion control [VRC ‘98] Experimental results over MBONE.

Research Directions Other applications for digital fountains Dispersity routing Accessing data from multiple mirror sites in parallel Improving the codes Implementation and deployment Scale to large number of clients Network interactions