1. CS492: Special Topics on Distributed Algorithms and Systems Fall 2008 Lab 3: Final Term Project

2. Final Term Project  Goal In-depth study of distributed algorithms and systems  Each team chooses a topic of interest related to distributed systems or algorithms or one from example projects  Each team will submit a term paper which includes: Introduction and Motivation Problem Definition Related Work Your Solution and Evaluation Conclusions  Each team will have two presentations First presentation should include: motivation, problem definition, and related work Final Presentation at the end of semester

3. Evaluation  Analysis  Simulation  Emulation  Implementation

4. Project Ideas 1. Storage Management 2. PageRank in Large Dataset 3. PageRank in Dynamic Dataset 4. Paxos Project

5. 1. Storage Management  Many students faced with error caused by not enough disk storage while working on Lab 2.  Total amount of storage of 6 nodes are sufficiently large to store all data including input, output, and intermediate data.  How can we utilize storage effectively?

6. 2. PageRank in Large Dataset  In Lab 2, you worked with about 2,000,000 documents.  Suppose you have 10,000 times more documents.  What are the expected problems using PageRank algorithm and how can you solve those problems?

7. 3. PageRank in Dynamic Dataset  Web documents are created and removed as time goes.  Computing PageRank takes a large amount of time.  How can we compute PageRank efficiently when graphs change all the time?

8. 4. Paxos Project  What is “consensus”? The process of agreeing on one result among a group of process (or a group of state machine)  What is Paxos? Protocols for solving consensus in a network of unreliable processors  Paxos in wikipedia Paxos in wikipedia

9. Various Paxos Protocols  A family of Paxos protocols Basic, Multi, Cheap, Fast, Generalized Paxos  A number of optimization skills Reduce the number of messages between processors Reduce the number of calculations in a process

10. Applications Using Paxos (I)  Google Chubby Distributed lock service to keep replicas consistent Used in Google Analytics and other products http://labs.google.com/papers/chubby.html

11. Applications Using Paxos (II)  IBM SAN Volume Controller General-purpose fault-tolerant virtual machine Runs the configuration and control component of the storage virtualization services offered by the cluster IBM SAN volume controller

12. Paxos Project  Project Objectives Study the family of Paxos algorithms Choose a specific version of Paxos protocol and optimization skills Design appropriate services or systems and implement it using the Paxos algorithm You have to simulate or make some failure conditions and prove the system’s fault-tolerance  Experiment over PlanetLab?

13. Schedule 9. 10/30 Thu – Lab 2 review by TAs 10. Class on Chapters 2 & 3 11. Individual team meetings 12. Term project presentation #1 13. Class on Chapters 9 and 10 14. Individual team meetings (tentative) 15. Class on Chapters 9 and 10 (tentative) 16. Term project presentation #2 (tentative)