The Bittorrent Protocol A Technical Look Presented by Cody Sand

What is Bittorrent? Peer-to-Peer (P2P) File Transfer Protocol Centralized Discovery, Decentralized Distribution Currently uses TCP transfer. Designed to “facilitate file transfers among multiple peers across unreliable networks. “ Tit-for-Tat protocol Any downloaders MUST upload, discourages “leeching”. Uses Content Distribution (Downloading) Music, Movies, Software Valve/Blizzard Downloads Streaming video (beta)

History Designed by Bram Cohen in 2001. Assisted in Development of Valve’s Steam platform, before focusing on Bittorrent fulltime. First implementation released in July 2001. Accounted for roughly 33% of all internet traffic today. Nearly 100,000,000 users worldwide.

How it works Peer distributing the file breaks it into small pieces, typically 64k – 1MB in size. Pieces over 512k tend to decrease protocol efficiency Peer creates a checksum for each piece and records it in a file (a .torrent file) When another peer receives a piece, it is checked against this file for integrity purposes. This model allows uploading/downloading of one or more files by many peers.

Why Bittorrent? For large, popular files, more customers require more bandwidth. Bandwidth costs go up. Bittorrent’s solution: Unload some distribution to users Allows for savings in bandwidth and costs.

Structure Ordinary Web Server A static metainfo file (.torrent). A bittorrent tracker An original “downloader” or the “seed” Original content provider Web Browser Consumer (Downloader) with client software.

Web Server / Bittorrent Tracker Hosts the tracker, which is the most essential piece. Trackers implemented to run with all major servers. Tracker Assists communication between peers. Responds to HTTP Get Requests. Used to track peers, download statistics.

Tracker GET Requests Standard HTTP Get Request Parameters info_hash – 20byte SHA-1 hash code to ID torrent file peer_id – 20 character string, identifies user/client. ip – IP address of peer. port – the port the client is listening on. uploaded – total amount uploaded so far. downloaded – total amount downloaded so far left – number of bytes left to download event – state of torrent (ie. started, completed, stopped) numwant – Optional key stating the number of desired peers. default is 50. Info_hash – calculated using ‘info’ dictionary of .torrent Peer_id – unique id that identifies user and bittorrent client.

Example GET Request http://some.tracker.com:999/announce ?info_hash=12345678901234567890 &peer_id=UTABCXYZ &ip=174.250.2.22 &port=6888 &downloaded=1234 &left=98765 &event=stopped

Tracker Response Error Message (Failure) List of peers (Success) 3-digit error code human-readable error message List of peers (Success) Dictionary format interval – time client should wait before rerequesting peers – list of dictionaries coresponding to peers id – the peer_id of remote peer ip – IP address of remote peer port – port number 3-digit error – think 404.

Torrent file (.torrent) Static metainfo in BEncoded format. Provide client with information Tracker Location (URL) File(s) Information Storage information

BEncoding Format for loosely structured data 4 types Byte strings Integers Lists Dictionaries

BEncoded Byte String A sequence of bytes, not necessarily characters Format <length>:<contents> Example 5:hello 7:seminar

BEncoded Integer Format i<number>e Example i42e

BEncoded List Format Example l<contents>e l5:helloi42e7:seminare

BEncoded Dictionary Associative array Format Example d<contents>e <contents> alternating key/value Example d5:hello5:class4:falli2009ee hello => class fall =>2009

.torrent Files Structure BEncoded Dictionary announce – string value containing tracker URL. info – points to dictionary contain info about files Optional fields announce-list – a list of alternate trackers comment – any comment the author wants to include createdby – author of the .torrent file creationdate – the date the .torrent was created. Others Can put ANY field into .torrent. Ignored if not supported.

Single File Torrents info dictionary fields length – size of the file, in bytes name – a string containing the filename piece length – number of bytes in each piece pieces – string value containing concatenated 20-byte SHA-1 hashes for each piece of file.

Multiple File Torrents info dictionary fields files – dictionary containing length – size of the file, in bytes path – path to the file, including filename name – name of top-most directory piece length – number of bytes in each piece pieces – string value containing concatenated 20-byte SHA-1 hashes for each piece of file. For verification.

Peer Messaging Handshaking Peer states Peer-Wire Messages Peer connects to remote peer, citing specific file Peer states (Un)Choked – Is the remote peer allowed to send data? (Un)Interested – Is the peer interested in receiving data? Data Transfer takes place when one side is interested, and the other is not choking. Peer-Wire Messages Used to communicate between peers

Peer Wire Messaging Choke Unchoke Interested Uninterested Inform the other peer they are being choked Unchoke Inform the other peer they are no longer being choked Interested Inform remote peer that data is desired Uninterested Inform remote peer that data transfer is not desired

Peer Wire Messaging Have Bitfield Request Piece Cancel Announce that peer has a piece of the torrent Remote peer may then request that piece Bitfield Send a bitfield representing pieces peer needs/has. Request Official request for a specific piece(s). Piece Transfer of a piece(s) Cancel Sent if peer requested data, but is no longer interested.

Process Tracker goes online

Process Content Provider creates .torrent file using bittorrent client software

Client uploads the .torrent file to the server. Process Client uploads the .torrent file to the server.

Process Index Client re-downloads the .torrent file from the server 029317AHBS: first.torrent Tracker indexes the torrent Client re-downloads the .torrent file from the server and opens it using a bittorrent client.

Process Index 029317AHBS: first.torrent Client software sends HTTP Get request, sending information to server; PeerID, and lets tracker know we have full file

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 Adds seed to peer list Client software sends HTTP Get request, sending information to server; PeerID, and lets tracker know we have full file

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 More peers download the .torrent file

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 Peers open the .torrent file, and each send an HTTP Get request identifying that the want to start downloading the file.

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 111.222.33.44 222.124.232.1 125.55.66.42 122.45.77.99 Tracker adds peers to list

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 111.222.33.44 222.124.232.1 125.55.66.42 122.45.77.99 Tracker returns the list of peers to the new peers.

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 111.222.33.44 222.124.232.1 125.55.66.42 122.45.77.99 Peers begin forming direct connections with other peers, forming a “swarm”.

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 111.222.33.44 222.124.232.1 125.55.66.42 122.45.77.99 Peers begin exchanging messages ((choked)) ((unchoked)) ((Uninterested)) ((Interested))

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 111.222.33.44 222.124.232.1 125.55.66.42 122.45.77.99 Peers begin exchanging messages ((have)) ((bitfield)) ((Request)) ((Piece))

Peer to Peer Process Optional Data Transfer Begins State: Choked/Uninterested Handshake State: Choked/Interested ((interested)) State: Unchoked/Uninterested ((Unchoked)) Optional ((Bitfield)) / ((Have)) ((Request)) Data Transfer Begins ((Piece))

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 111.222.33.44 222.124.232.1 125.55.66.42 122.45.77.99 55.121.33.65 75.45.65.95 111.85.243.11 As peers continue to connect to tracker, more peers are added to the peer list.

New peers receive up-to-date peer lists. Process Index 029317AHBS: first.torrent Peers 123.234.12.34 111.222.33.44 222.124.232.1 125.55.66.42 122.45.77.99 55.121.33.65 75.45.65.95 111.85.243.11 New peers receive up-to-date peer lists. Existing peers periodically report to tracker, also receiving new peer list.

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 111.222.33.44 222.124.232.1 125.55.66.42 122.45.77.99 55.121.33.65 75.45.65.95 111.85.243.11 Peers continue exchanging messages and data until all peers receive the entire file. As pieces are received, the program calculates the SHA-1 hash for each piece and checks it against .torrent file. Any piece with incorrect checksum must be discarded and redownloaded.

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 111.222.33.44 222.124.232.1 125.55.66.42 122.45.77.99 55.121.33.65 75.45.65.95 111.85.243.11 As peers disconnect, they send another HTTP request, stating that they’re no longer sending.

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 111.222.33.44 222.124.232.1 125.55.66.42 122.45.77.99 55.121.33.65 75.45.65.95 Tracker removes peer As peers disconnect, they send another HTTP request, stating that they’re no longer sending.

Process Index 029317AHBS: first.torrent Peers 123.234.12.34 111.222.33.44 222.124.232.1 125.55.66.42 As peers disconnect, they send another HTTP request, stating that they’re no longer sending.

Future of Bittorrent Bittorrent continues to increase its presence online. Bram Cohen’s company, Bittorrent, Inc. is now in control of the protocol. UDP implementation Increased speeds without TCP’s congestion control Streaming Video Piece selection algorithms to increase speed and ensure sequential delivery.

The End. Thank You.