X stream Project proposal

Project goals: Students Students: Academic Supervisor Academic Supervisor: Advisors: Developing and Implementing a large scale P-2-P scalable streaming protocol BarakSchlosserAmiadStern Barak Schlosser, Amiad Stern Prof. Danny Dolev Danny Bickson, Elan Pavlov Danny Bickson, Elan Pavlov Project ’ s web site:

Current protocols Many of the articles in this field, rely on trees or multiple parent topologies where the server or servers is multicasting the data to the recipients.

There are some disadvantages in those protocols: 1.They are built in a hierarchal manner causing intermediate tree nodes to work harder in forwarding the data.

There are some disadvantages in those protocols: 1.They are built in a hierarchal manner causing intermediate tree nodes to work harder in forwarding the data.

2.The depth of the tree can be large tree can be large causing some causing some users to wait a users to wait a long period of long period of time for the data. time for the data. There are some disadvantages in those protocols:

2.The depth of the tree can be large tree can be large causing some causing some users to wait a users to wait a long period of long period of time for the data. time for the data. There are some disadvantages in those protocols:

3. When a user from one of the from one of the upper levels of upper levels of the tree the tree disconnects, a disconnects, a lot of changes lot of changes are needed. are needed. There are some disadvantages in those protocols:

protocols: 3. When a user from one of the from one of the upper levels of upper levels of the tree the tree disconnects, a disconnects, a lot of changes lot of changes are needed. are needed.

We develop a distributed large scale streaming protocol that enables large number of users to log to a streaming content. This protocol reduces the work load on the server and designed to achieve maximal guarantee fairness for all users. Instead of using a structured multicast network we use unstructured peer to peer topology.

Protocol Design: connect connect... servers - 2 virtual servers. - Connect and get users list. - Start downloading data from users on list using ‘ Julia ’ (Animate)

Protocol Design: The exception to this rule is the first users who download the content from the server in addition for exchanging data between themselves.

Protocol Design: There are no assumptions taken regarding to users connecting time.... First.... Next time frame... And so on... (Animate)

A bottleneck can arise due to lack of users holding needed data … Protocol Design: First time frame Next one

Protocol Design: To solve this problem the server should take care that on connection time the ratio of – ‘ users demanding data ’ / ‘ users holding data ’ is constant. The server will manage this ratio by asking some large bandwidth users to virtually simulate 2 users

Protocol Design: servers First time frame Next time frame Connect … (Animate)

Protocol Design: Each user holds 2 different lists of users. Users ahead on stream offset Users downloading from him During the data transmission a gossiping method is used to spread out user ’ s details.

Protocol Design: The ContentServers are being used only for first ‘ group ’ users and in case a user or ‘ group ’ has a tight schedule problem. Or …

Expected challenges: - Developing a distributed communication protocol. - Debugging a distributed system. - Coping with real time constraints. - Coping with real time constraints. - Testing the protocol on a large distributed network (Planetlab). network (Planetlab). - Testing on a heterogeneous LAN with no delays.

Expected challenges: - Learning and using one of the newest video protocols – MPEG2,MPEG4 (divx). - Challenging existing streaming protocols - Building a working prototype in C++ and not just a simulation. - Engineering challenges like assembly and disassembly of video into frames. - Developing a streaming protocol that enables user to choose desired offset

Milestones: Learning phase (1 month) - Video protocols – MPEG, MPEG4. - Video protocols – MPEG, MPEG4. - Studying existing streaming and large scale content distribution network protocols. - Studying existing streaming and large scale content distribution network protocols. - Learning the details of Julia algorithm. - Learning the details of Julia algorithm. - Finding appropriate open source video players with programmable API. - Finding appropriate open source video players with programmable API.

Milestones: Developing the algorithm (3 months) Weeks 1-7: - Formulating and proving the correctness of the algorithm. - Formulating and proving the correctness of the algorithm. - Designing and implementing the UsersServer: - Designing and implementing the UsersServer: - Building the database holding the users. - Handling users connection. - Handling gossiping (updating the database). - Ensuring constant ratio between ‘ users groups ’.

Milestones: Developing the algorithm (3 months) Weeks 1-7: - User connection to the system: - User connection to the system: - Connection request to the UsersServer. - Designing the local database of the user and updating them as needed. - Gossiping – messages from user to the UsersServer.

Milestones: Developing the algorithm (3 months) Weeks 7-14: - Designing and implementing the ContentServer: - Designing and implementing the ContentServer: - Handling content distribution to the users of the first ‘ group ’. - Handling specific content pieces.

Milestones: Developing the algorithm (3 months) Weeks 7-14: - Users download of content from other users: - Users download of content from other users: - Managing local bitmap indicating existing pieces and missing ones. - Handling other users bitmaps and sending desired content pieces. - Identify ‘ emergency situation ’ and solve it by downloading from the server or connecting ‘ closest ’ user.

Milestones: Testing and Deployment (2-3 months) Local network workspace (~2 weeks) - Running local tests. Planetlab workspace (~2 weeks) - Compilation on Planetlab and verifying the code is compatible. - Learning to use Planetlab deployment mechanism.

Milestones: Testing and Deployment (2-3 months) - Comparing results with current protocols. - Improving results by changing different parameters values. - Gathering protocol statistics (server workload, user’s additional work etc.) Planetlab workspace (~4 weeks)

Division of labor: Developing the algorithm: Amiad: - Formulating and proving the correctness of the algorithm. - Formulating and proving the correctness of the algorithm. - UsersServer designing and implementation. - UsersServer designing and implementation. - ContentServer designing and implementation. - ContentServer designing and implementation. - Keeping a constant ratio between different ‘ users group ’. - Keeping a constant ratio between different ‘ users group ’.

- Learning video codecs. - Learning video codecs. - Finding open source API for showing the video and splitting into frames. - Finding open source API for showing the video and splitting into frames. Division of labor: Developing the algorithm: Amiad:

Division of labor: Developing the algorithm: Barak: - Formulating and proving the correctness of the algorithm. - Formulating and proving the correctness of the algorithm. - User’s connection to the system (user side). - User’s connection to the system (user side). - User’s content management and downloading. - User’s content management and downloading.

Division of labor: Testing and improving: Amiad: - Testing different user connection scenarios. - Testing different user connection scenarios. - Build the application that uses the protocol. - Build the application that uses the protocol. - Testing on Planetlab – connections, locating the data. - Testing on Planetlab – connections, locating the data.

Division of labor: Testing and improving: Barak: - Testing different user parameters – buffer size, bandwidth etc. - Testing different user parameters – buffer size, bandwidth etc. - Testing on Planetlab – data transfer. - Testing on Planetlab – data transfer. - Testing and improving performance comparing to current streaming protocols. - Testing and improving performance comparing to current streaming protocols.

Success criteria: - Creating one of the first truly distributed streaming solution. - Creating one of the first truly distributed streaming solution. - Comparing performance to current streaming protocols in server workload and scalability. - Comparing performance to current streaming protocols in server workload and scalability.

Success criteria: - Workload is divided equally between all users - Reducing server workload - Developing a streaming protocol that enables viewing of different streaming offsets

Thank you THE END