End-to-end Publishing Using Bittorrent. Bittorrent Bittorrent is a widely used peer-to- peer network used to distribute files, especially large ones It.

Slides:



Advertisements
Similar presentations
Rarest First and Choke Algorithms are Enough Arnaud LEGOUT INRIA, Sophia Antipolis France G. Urvoy-Keller and P. Michiardi Institut Eurecom France.
Advertisements

The BitTorrent Protocol
The BitTorrent Protocol. What is BitTorrent?  Efficient content distribution system using file swarming. Does not perform all the functions of a typical.
The BitTorrent protocol A peer-to-peer file sharing protocol.
Incentives Build Robustness in BitTorrent Bram Cohen.
Bit Torrent (Nick Feamster) February 25, BitTorrent Steps for publishing – Peer creates.torrent file and uploads to a web server: contains metadata.
Ln -s /usr/timtimmah /usr/kevinstanonik. TORRENT BASICS  You want to get a file to other people.  BUT you don’t have the bandwidth to have everyone.
Kademlia: A Peer-to-peer Information System Based on the XOR Metric Petar Mayamounkov David Mazières A few slides are taken from the authors’ original.
The BitTorrent content distribution system CS217 Advanced Topics in Internet Research Guest Lecture Nikitas Liogkas, 5/11/2006.
CompSci 356: Computer Network Architectures Lecture 21: Content Distribution Chapter 9.4 Xiaowei Yang
Peer to Peer (P2P) Networks and File sharing. By: Ryan Farrell.
Spotlighting Decentralized P2P File Sharing Archie Kuo and Ethan Le Department of Computer Science San Jose State University.
Web Servers How do our requests for resources on the Internet get handled? Can they be located anywhere? Global?
Presented by Stephen Kozy. Presentation Outline Definition and explanation Comparison and Examples Advantages and Disadvantages Illegal and Legal uses.
Part 1: Overview of Web Systems Part 2: Peer-to-Peer Systems Internet Computing Workshop Tom Chothia.
A P2P file distribution system ——BitTorrent Fan Bin Sep,25,2004.
Lecture 10 Naming services for flat namespaces. EECE 411: Design of Distributed Software Applications Logistics / reminders Project Send Samer and me.
Browsing the World Wide Web. Spring 2002Computer Networks Applications Browsing Service Allows one to conveniently obtain and display information that.
 Proxy Servers are software that act as intermediaries between client and servers on the Internet.  They help users on private networks get information.
Fundamentals of Python: From First Programs Through Data Structures
1 Enabling Secure Internet Access with ISA Server.
The Bittorrent Protocol
NETWORK CENTRIC COMPUTING (With included EMBEDDED SYSTEMS)
FALL 2005CSI 4118 – UNIVERSITY OF OTTAWA1 Part 4 Web technologies: HTTP, CGI, PHP,Java applets)
BitTorrent Presentation by: NANO Surmi Chatterjee Nagakalyani Padakanti Sajitha Iqbal Reetu Sinha Fatemeh Marashi.
Peer to Peer Network Anas Hardan. What is a Network? What is a Network? A network is a group of computers and other devices (such as printers) that are.
BitTorrent.
BitTorrent Internet Technologies and Applications.
Application Layer – Peer-to-peer UIUC CS438: Communication Networks Summer 2014 Fred Douglas Slides: Fred, Kurose&Ross (sometimes edited)
BitTorrent How it applies to networking. What is BitTorrent P2P file sharing protocol Allows users to distribute large amounts of data without placing.
Forensics Investigation of Peer-to- Peer File Sharing Networks Authors: Marc Liberatore, Robert Erdely, Thomas Kerle, Brian Neil Levine & Clay Shields.
Distributed Systems Concepts and Design Chapter 10: Peer-to-Peer Systems Bruce Hammer, Steve Wallis, Raymond Ho.
1 BitTorrent System Efrat Oune Bar-Ilan What is BitTorrent? BitTorrent is a peer-to-peer file distribution system (built for intensive daily use.
Securing Data Authentication with Cryptographic Signatures in BitTorrent Du Wei cc.hut.fi / Supervisor : Professor Jukka Manner Instructor.
5 Chapter Five Web Servers. 5 Chapter Objectives Learn about the Microsoft Personal Web Server Software Learn how to improve Web site performance Learn.
ITIS 1210 Introduction to Web-Based Information Systems Chapter 38 How Music Sharing and File Sharing Work.
BitTorrent Dr. Yingwu Zhu. Bittorrent A popular P2P application for file exchange!
A P2P file distribution system ——BitTorrent Pegasus Team CMPE 208.
2: Application Layer1 Chapter 2: Application layer r 2.1 Principles of network applications r 2.2 Web and HTTP r 2.3 FTP r 2.4 Electronic Mail  SMTP,
Do incentives build robustness in BitTorrent? Michael Piatek, Tomas Isdal, Thomas Anderson, Arvind Krishnamurthy, Arun Venkataramani.
ITIS 1210 Introduction to Web-Based Information Systems Chapter 23 How Web Host Servers Work.
Bit Torrent A good or a bad?. Common methods of transferring files in the internet: Client-Server Model Peer-to-Peer Network.
11/10/20151 A Standalone Content Sharing Application for Spontaneous Communities of Mobile Handhelds Authors: Amir Krifa –
NoTorrent A Peer-to-Peer Web Caching System Modeled Partially After BitTorrent but Without the Need for.torrent Files Howie Vegter, hrv2101 Columbia University.
Scalable Web Server on Heterogeneous Cluster CHEN Ge.
1 Distributed Hash Tables (DHTs) Lars Jørgen Lillehovde Jo Grimstad Bang Distributed Hash Tables (DHTs)
Bittorrent Protocol Implementation. Bittorrent Bittorrent is a widely used peer-to- peer network used to distribute files, especially large ones It has.
B IT T ORRENT T ECHNOLOGY Anthony Pervetich. H ISTORY Bram Cohen Designed the BitTorrent protocol in April 2001 Released July 2, 2001 Concept Late 90’s.
Experience Sharing in Mobile Peer Communities EPI Planete, INRIA International Consortium Meeting (Oulou) 10 June, 2009.
Bit Torrent Nirav A. Vasa. Topics What is BitTorrent? Related Terms How BitTorrent works Steps involved in the working Advantages and Disadvantages.
2-Hop TorrentSmell A distributed tracking algorithm name:Raynor Vliegendhart date:July 10, 2009 event:Tribler Dev Meeting.
T3 data access via BitTorrent Charles G Waldman USATLAS/University of Chicago USATLAS T2/T3 Workshop Aug
INTERNET TECHNOLOGIES Week 10 Peer to Peer Paradigm 1.
PEAR TO PEAR PROTOCOL. Pure P2P architecture no always-on server arbitrary end systems directly communicate peers are intermittently connected and change.
End-to-end Publishing Using Bittorrent. Bittorrent Bittorrent is a widely used peer-to- peer network used to distribute files, especially large ones It.
Ch 2. Application Layer Myungchul Kim
General Architecture of Retrieval Systems 1Adrienn Skrop.
Design and Implementation of a High-Performance distributed web crawler Vladislav Shkapenyuk and Torsten Suel Proc. 18 th Data Engineering Conf., pp ,
November 19, 2016 Guide:- Mrs. Kale J. S. Presented By:- Hamand Amol Sambhaji. Hamand Amol Sambhaji. Pardeshi Dhananjay Rajendra. Pardeshi Dhananjay Rajendra.
An example of peer-to-peer application
Introduction to BitTorrent
BitTorrent Vs Gnutella.
SCHOOL OF ELECTRICAL ENGINEERING AND TELECOMMUNICATIONS
Angelo Sapello University of Delaware
The BitTorrent Protocol
Content Distribution Networks + P2P File Sharing
Chapter 2 Application Layer
Kademlia: A Peer-to-peer Information System Based on the XOR Metric
Content Distribution Networks + P2P File Sharing
Presentation transcript:

End-to-end Publishing Using Bittorrent

Bittorrent Bittorrent is a widely used peer-to- peer network used to distribute files, especially large ones It has a number of legal uses which separate it from other P2P

Practical Applications Distributing large files Podcasting Vlogging Disk images Legal distribution of movies (see bittorrent.com)

Traditional vs. Bittorrent One server provides many clients Many clients provide many clients

Terminology Swarm – clients downloading or uploading a given file through BIttorrent Tracker – centralized server that clients connect to to ask for lists of other clients connected to the swarm Seed – A client that has a complete copy of the file Peer (Leecher) – A client that does not have a complete copy of the file

Problem Torrents that are less popular may eventually “die” when there are no longer any complete copies of the file in the swarm

Everseed Permanent seed running on the same server as the tracker Guarantees that there will always be a complete copy of the file

Related Research The creator of Bittorrent wrote a paper on the process of downloading a file using Bittorrent at Maintainers of various Bittorrent clients wrote which is like the official specification except far more in depth Osprey ( seems to have thought of something similar, but haven't made any visible progresshttp://osprey.ibiblio.org/

Explanation The.torrent metadata file provides info about where to find the tracker and about the file being distributed Client connects to tracker Tracker gives client a list of other clients Client then downloads file from other clients (not a centralized server) Periodic update with tracker for new client list

Goals Complete internet publishing solution using Bittorrent Metadata file generator (.torrent) Tracker “Everseed” Web interface

.torrent File Official documention on bittorrent.org Metadata on the file to be downloaded (tracker URL, filename, size, checksum hashes) Stored as “bencoded” strings, integers, lists, dictionaries

Bencoding Integer: 6 => “i6e” String: “hello” => “5:hello” List: [“hello”,”world”] => “l5:hello5:worlde” Dictionary: {“hello”:”world”} => “d5:hello5:worlde”

Bencoding implementation Python, good string manipulation Structure of a.torrent file is a dictionary containing string keys and integer, string, list, and dictionary values Recursion to encode/decode

Tracker Makes use of the bencoding algorithm Handles two types of requests: “announce” and “scrape” Stores data on peers and torrents in a SQLite database No performance issues

Network performance Peer List Size

Database performance

Announce requests Used by a client to announce presence in a Bittorrent swarm Client sends an HTTP GET request to the announce URL in the.torrent file Tracker parses request, urldecodes data about the peer Tracker stores data in the database, sends appropriate response as bencoded string in a text/plain document Client bdecodes string, connects to other clients

Scrape requests Used by client to obtain info about the torrents the tracker is tracking Client sends an HTTP GET request to the scrape url found by transforming the announce url Tracker urldecodes and parses the request Tracker fetches data about torrent from the database Tracker returns a bencoded dictionary which the client decodes

Smart Peer List Response Seeds often disconnect from other seeds Tracker can also do this to some extent Announce requests contain a list of random peers If a client is seeding, it doesn't need IPs of other seeds Increased overall swarm performance

Peer List Compression Peer list in the tracker response to peer lists is normally ASCII encoded The peer list can be compressed to 4 bytes for the IP address, 2 bytes for the port Huge bandwidth savings, ~80% Greatly enhanced tracker performance Reduced tracker hardware requirements

Test Client Concurrent development of a test Bittorrent client written in Python Can send both announce and scrape requests Key-value pairs are easily configurable

Testing Generalized method of handling exceptions in the initialization methods Increased use of try/except statements to improve robustness Testing with incorrect or missing data

Summary Python Benefits of P2P technology “Everseed” concept.torrent files and bencoding Tracker