VoD BitTorrent Framework

Background BT is a very popular peer to peer protocol with many implementations: – Torrent_clientshttp://en.wikipedia.org/wiki/Comparison_of_Bit Torrent_clients The first one was created in Python by Bram Cohen.

Real world motivation BitTorrent is common and Free Good chance there is a swarm for your movie VoD servers are expensive and complicated Its better to use P2P for VoD

Some definitions Delay (D) = How much time I am allowed to hold before starting to play M (Movie). Prefetch Time (P) = How much time before I watch byte B from M, it should arrive. Playback Rate (R) = How many bytes/second I watch B(t) – the index of the bit that must arrive by t. B(t) = R *(P+t-D)

Rules and Examples B(t) = R * (P+t-D) If R=5 KB/sec and D=6 seconds and P=2 seconds, than: –At t=6 seconds, I must have 10KB continuous from the start of the movie. –At t=18 seconds I must have 70KB continuous from the start of the movie.

What does one VOD client do Your client tries to get from the Bittorent clients pieces of data needed to play the video. Each piece needs to arrive before its deadline. If the piece does not arrive by the deadline you can download it “instantly” from the “server”. Each VOD client counts the data from server (DFS).

What is my goal Each byte taken from the server costs money, therefore you need to minimize DFS. Goal - Minimize the accumulated DFS of all the VOD clients that run concurrently.

How do I create my client Download GROUP_VOD_CODE.tar.gz from “BitTornado - Start with this Code” link in the course web page. –It is also in /users/courses/hanochworkshop/group_vod GROUP_VOD_CODE.tar.gz is a regular BT with StreamWatcher.py to monitor DFS. Any changed section is marked ###### GROUP VOD ###### Modify it to minimize acumulated DFS. Do not change StreamWatched.py.

Standard BT parameters --saveas F --max_upload_rate F.torrent

VOD client additional params SET = defined inside run_all.sh, not a parameter. delay – Wait before prefetch, in seconds. SET TO 20 prefetchT – How much time in advance the byte must arrive. In percent of movie length. SET TO 5% rate – VOD rate in KB/sec. SET TO 90 KB. order – How many run when client spawned. SET BY run_all.sh. Parameter of run_multiple.sh. gap – How much to wait between spawns – unit is pewfetchT. alg – Name of the algorithm. Default is ORIG. verbose – Print and kill the whole swarm when complete. peers_num – how many regular peers. seeds_num – how many seeds. Vod_num – how many VoD peers.

How do I test my idea cd /users/courses/hanochworkshop/voddiversegroup/vod_ tests run_multiple.sh –your_client - example: ~/GROUP_VOD_CODE/btdownloadheadless.py –Gap – example: 0 –output_location example: ~/bt_out –iteration example: 4 –empty/full example: empty –peers_num example: 16 –seeds_num example: 2 –vod_num example: 14

Execution example run-all.sh Tracker BT VOD BT run_multiple.sh VOD BT VOD BT VOD BT

How does it work? Run tracker Run the “world” Run your clients

How does it work? Both tracker and the “world” run on the same computer. Running a test: –Reserve time on pc-hanoch-w3/w4 in: Or from course www - "Scheduling computers for tests.“

Output to screen ================================================================================== total time: 1 min 07 sec share rating: ( KB up / KB down) saving: picture-100.bmp (27.5 MB) percent done: 18.9 time left: 4 min 52 sec download to: /home/pc-hanoch-w4/home/hillelav/name/picture.pc-hanoch-w4-4.bmp download rate: 93.4 kB/s upload rate: 80.6 kB/s seed status: 2 seen now, plus distributed copies peer status: 28 seen now, 10.7% done at kB/s StreamWatcher Csv stats: ~/out/1/statistics-order-4-gap-0.csv DFS is: bytes DFS/Total is: 29 % FullPieces: 63 / 220 DirtyPieces: 1 / 220 FaildHashChecks: 0 Prefetching 5 % Playing point: 56 / 220 ( 25 %) Only last VOD client prints

Output to.csv In ~/out you will find vod_num directories: /1, /2, …/ iteration. In each directory, you have vod_num files. Statistics in - order- -gap-.csv Order is how many VOD clients ran before. is the parameter Gap.

.csv file alg,dfs,p2p ORIG,5,0 ORIG,10,2 ORIG,15,6 ORIG,19,11 ORIG,23,16 ORIG,28,20 ORIG,31,25 ORIG,34,31 ORIG,37,36 ORIG,39,41.

total DFS and total P2P GAP = 2 File percentage

GAP = 0

Local DFS GAP = 2

GAP = 0

run_all.sh defaults Start –Use in run_multiple.sh Stop –Kill all python threads Kill –Kill all python threads and scripts.

To run client separately: PATH_TO_YOUR_VOD_CLIENT/btdownloadheadless.py --saveas F --delay D --prefetchT P --rate R --out_dir O Can add the parameters --order X in the IDE --gap G --group_size S --alg ‘ALG' --verbose F.Torrent

Python Very common script language IDE: –PyDev –WINGWARE Python IDE – not free, but very good. References: –python.org : – – The book "Think like a Computer Scientist in Python" has a very good reputation:

Getting started Run the environment –ssh pc-hanoch-w3 or pc-hanoch-w4 –cd /users/courses/hanochworkshop/group_vod/vod_tests –run_multiple.sh MyClient GAP output-directory –Note tracker prints a message for each GET. You can connect to the tracker, from the CS network, with a browser at: – k :6970/ k :6970/ Do ls -l /home/pc-hanoch-w k /home/hillelav/$USER to see downloaded files

Directory Structure Root – –high level application Root/BitTornado – –services Root/BitTornado/BT1 – –Actual work

Client Operation btdownloadheadless.py – First to run. –Performs initializations. get_response – parse the.torrent Infohash – the ID of the.torrent startEngine – Start connection with peers startRerequester - Start connection with tracker –Run rawserver.listen_forever from RawServer.py. –listen_forever is: Polling port executing func() from a task queue.

Receiving messages From listen_forever loop it goes to: –BT1/Connecter.py/got_message CHOKE UNCHOKE INTERESTED NOT_INTERESTED BITFIELD REQUEST CANCEL PIECE HAVE

Influencing decisions Can be done in one of the callbacks which spring from Connecter.py / got_message: Downloader.py –HAVE->got_have->send_interested Tune for VoD?

BT tracker Send GET request in Rerequester.py / announce Receive GET data in thread in Rerequester.py / _rerequest

Policy Choker.py/_round_robin – decide who to choke / optimistically un choke / send_have to. Downloader.py / _request_more. PiecePicker.py / next - rarest piece, etc. Tune for VoD? got_unchoke->_request_more->next…

More about the code Start your project from “BitTornado - Start with this Code” link. Click “BitTornado - Legacy Code Guide” for basic explanations about the code.