1. CIS 6930.5: Federated Distributed Systems Adriana Iamnitchi (Anda) anda@cse.usf.edu

2. 2 CIS6930.5: Federated Distributed Systems (Fall 2006) Contact Info Email: anda@cse.usf.edu Office: ENB 334 Office hours: by appointment (email me) Course page: http://www.csee.usf.edu/~anda/CIS6930.5

3. 3 CIS6930.5: Federated Distributed Systems (Fall 2006) CIS 6930.5: Course Goals l Primary –Gain deep understanding of fundamental issues that affect design of large-scale federated distributed systems –Map primary contemporary research themes –Gain experience in network research l Secondary –By studying a set of outstanding papers, build knowledge of how to present research –Learn how to read papers & evaluate ideas

4. 4 CIS6930.5: Federated Distributed Systems (Fall 2006) What I’ll Assume You Know l Basic Internet architecture –IP, TCP, DNS, HTTP l Basic principles of distributed computing –Asynchrony (cannot distinguish between communication failures and latency) –Partial global state knowledge (cannot know everything correctly) –Failures happen. In very large systems, even rare failures happen often l If there are things that don’t make sense, ask!

5. 5 CIS6930.5: Federated Distributed Systems (Fall 2006) Examples of Distributed Systems ATT webGnutella network The Internet A Sensor Network

6. 6 CIS6930.5: Federated Distributed Systems (Fall 2006) Definition (a version) l A distributed system is a collection of autonomous, programmable, failure-prone entities that are able to communicate through a communication medium that is unreliable. –Entity=a process on a device (PC, PDA, mote) –Communication Medium=Wired or wireless network l “Federated” – spanning multiple institutional or network (DNS) domains

7. 7 CIS6930.5: Federated Distributed Systems (Fall 2006) Outline l Case study (and project ideas): –Volunteer computing: SETI@home and BOINC –Grid computing –P2P systems l Administravia

8. 8 CIS6930.5: Federated Distributed Systems (Fall 2006)

9. 9 SETI@home Operations data recorder screensavers WU storage splitters DLT tapes data server science DBuser DB result queue acct. queue garbage collector tape archive, delete tape backup master DB redundancy checking RFI elimination repeat detection web site CGI program web page generator

10. 10 CIS6930.5: Federated Distributed Systems (Fall 2006) How does it work? l Fixed-rate data processing task l Low bandwidth/computation ratio l Independent parallelism l Error tolerance SETI@home Master-worker architecture

11. 11 CIS6930.5: Federated Distributed Systems (Fall 2006) History and Statistics l Conceived 1995, launched April 1999 l “scientific experiment that uses Internet-connected computers in the Search for Extraterrestrial Intelligence (SETI). You can participate by running a free program that downloads and analyzes radio telescope data. “ l No ET signals yet, but other results TotalLast 24 Hours (as of Wed Feb 23 07:04:51) Users5,361,3134,391 Results received1,779 millions5 million Total CPU time2.2 million years3610.717 years Average CPU time/work unit 10 hr 58 min 14.0 sec6 hr 19 min 30.1 sec

12. 12 CIS6930.5: Federated Distributed Systems (Fall 2006) Volunteer computing l Also called “public-resource computing” l Utilizes idle computing cycles over Internet l Other systems: –Original: GIMPS, distributed.net –Commercial: United Devices, Entropia, Porivo, Popular Power –Academic, open-source >Cosm, folding@home

13. 13 CIS6930.5: Federated Distributed Systems (Fall 2006) None of the popularity of SETI! l ET l How to get and retain users (from David Anderson, the leader of the SETI@home project) –Graphics are important (but monitors do burn in) –Teams: users recruit other users –Keep users informed l Science news l System management news l Periodic project emails l Reward users: –PDF certificates –Milestone pages and emails –Leader boards (overall, country, …)

14. 14 CIS6930.5: Federated Distributed Systems (Fall 2006) Millions and millions of computers! (Problems) l Server scalability l Dealing with excess CPU time l Cheating l Bad behavior: –Team recruitment by spam –Sale of accounts on eBay l Malfunctions l Network bandwidth costs money

15. 15 CIS6930.5: Federated Distributed Systems (Fall 2006) SETI@home: Summary l Master-worker design –Centralized solution >Master=central point of control >Single point of failure >Performance bottleneck l Incentives for participation –Mean sometimes incentives for cheating l Massive (“embarrassing”) parallelism l Low bandwidth/computation ratio Users do donate real resources: $1.5M / year consumed power l More information: http://setiathome.ssl.berkeley.edu

16. 16 CIS6930.5: Federated Distributed Systems (Fall 2006) BOINC l Berkeley Open Infrastructure for Network Computing “Open-source software for volunteer computing and desktop grid computing. “ http://boinc.berkeley.edu/ l Project idea: install and configure BOINC on a set of machines at USF to run large embarrassingly parallel applications. –Two candidate applications from mechanical engineering and physics (code already exists) –Report experience. Think along the following idea: would it be beneficial to use the administrative desktops for scientific computations at USF?

17. 17 CIS6930.5: Federated Distributed Systems (Fall 2006) Outline l Case study (and project ideas): –Volunteer computing: SETI@home and BOINC –Grid computing –P2P systems l Administravia

18. 18 CIS6930.5: Federated Distributed Systems (Fall 2006) Grid Computing: Current Status l The metaphor: power grid l Many deployed grids running in production mode l Scientists are the most traditional users l Users: –100s, 10s of institutions –Well-established communities l Resources: –Computers, data, instruments, storage, applications –Owned/administered by institutions l Applications: data- and compute- intensive processing l Approach: common infrastructure

19. 19 CIS6930.5: Federated Distributed Systems (Fall 2006) Why Don’t We Build a Huge Supercomputer? Top500 supercomputer list over time: l Zipf distribution: Perf(rank) ≈ rank -k

20. 20 CIS6930.5: Federated Distributed Systems (Fall 2006) Impact l Trend: it is increasingly interesting to aggregate the capabilities of the machines in the tail of this distribution. –A virtual machine that aggregates the last 10 in Top500 would rank 32 nd in ’95 but 14 th in ‘03 l Both Grid and P2P computing are results of this trend: –Grids: focus on assembling (a relatively small number of) resources to enable controlled, secure resource sharing –P2P focus: scale, deployability. Challenge: design services that offer the best of both worlds l complex, secure services, that deliver controlled QoS; are scalable and can be easily deployed.

21. 21 CIS6930.5: Federated Distributed Systems (Fall 2006) Outline l Case study (and project ideas): –Volunteer computing: SETI@home and BOINC –Grid computing –P2P systems l Administravia

22. 22 CIS6930.5: Federated Distributed Systems (Fall 2006) Peer-to-Peer Systems l Revived (?) by music sharing l A variety of applications deployed today l Def 1: “A class of applications that take advantage of resources (e.g., storage, cycles, content) available at the edge of the Internet.” –Edges often turned off, without permanent IP addresses, etc. l Def 2: “A class of decentralized, self-organizing distributed systems, in which all or most communication is symmetric.” l Lots of other definitions that fit in between

23. 23 CIS6930.5: Federated Distributed Systems (Fall 2006) P2P Impact (1) l Widespread adoption leading to –KaZaA – 170 millions downloads (3.5M/week) one of the most popular applications ever! l (almost) zero-cost data distribution … is forcing companies to change their business models … might impact copyright laws

24. 24 CIS6930.5: Federated Distributed Systems (Fall 2006) P2P Impact (2) l Killer application for broadband to consumers –P2P generated traffic may be the single largest contributor to Internet traffic today Internet2 Internet2 traffic statistics Source: www.internet2.eduwww.internet2.edu

25. 25 CIS6930.5: Federated Distributed Systems (Fall 2006) Applications (1) l File sharing –The ‘killer’ application to date –Too many to list them all: Napster, FastTrack (KaZaA, iMesh), Gnutella (LimeWire, Morpheus, BearShare), l Streaming: the user ‘plays’ the data as it arrives P2P approach Possible solution: The first few users get the stream from the server New users get the stream from the server or from users who are already receiving the stream source Oh, I am exhausted! Client/server approach

26. 26 CIS6930.5: Federated Distributed Systems (Fall 2006) Applications (2) l Performance benchmarking Problem: –Evaluate the performance of your Web site form end-user perspective >Multiple views on your site performance –Generate Internet statistics >Connectivity statistics >Routing errors, routing maps l Backup storage (HiveNet, OceanStore) l Collaborative environments (Groove Networks) l Instant messaging (Yahoo, AOL) l Web serving communities (uServ) l Spam filtering l Anonymous email l Censorship-resistant publishing systems (Ethernity, Freenet

27. 27 CIS6930.5: Federated Distributed Systems (Fall 2006) P2P Networks: Current Status l Users: –Millions –Anonymous individuals l Resources: –Computing cycles XOR files –Owned/administered (?) by user –Intermittent participation: >Gnutella: 60 min. (‘01) >MojoNation: 1/6 users always connected (‘01) >Overnet: 50% nodes available 70% of time over a week (‘02) l Applications: file retrieval or parallel computations l Approach: vertically integrated solutions (www.slyck.com, 06/14/’06) ???MP2P ???DirectConnect 645,120Overnet 2,219,539Gnutella 2,848606FastTrack 3,108,066 eDonkey2K NetworkUsers

28. 28 CIS6930.5: Federated Distributed Systems (Fall 2006) Trend: Large, Dynamic, Self-Configuring Grids Scale & volatility Functionality & infrastructure Grids P2P Large scale Weaker trust assumptions Ease of integration No centralized authority Intermittent resource/user participation Diversity in: Shared resources Sharing characteristics Variable technical support Infrastructure (sharable services) Support for diverse applications On Death, Taxes, and the Convergence of Grid and P2P Systems, Foster and Iamnitchi, IPTPS’03

29. 29 CIS6930.5: Federated Distributed Systems (Fall 2006) Challenges in Distributed Systems l Scale l Real problems: spam, denial of service attacks (and distributed), security, fault tolerance, etc. l We’ll look at latest solutions to such problems proposed in: –Top conferences in systems and networking: SIGCOMM, OSDI, NSDI –Top workshops (hot topics): IPTPS, HotOS –Other venues l (Digression: how do you tell when a conference is top?)

30. 30 CIS6930.5: Federated Distributed Systems (Fall 2006) Course Organization/Syllabus/etc.

31. 31 CIS6930.5: Federated Distributed Systems (Fall 2006) Administravia: Grading l Reviewing:30% l Discussion leading: 15% l Project: 55% –Aim high! –Have fun!

32. 32 CIS6930.5: Federated Distributed Systems (Fall 2006) Administravia: Paper Reviewing (1) l Goals: –Think of what you read –Get used to writing paper reviews l Reviews due by noon before class l Be professional in your writing l Have an eye on the writing style: –Clarity –Beware of traps: learn to use them in writing and detect them in reading –Detect (and stay away from) trivial claims. E.g., 1 st sentence in the Introduction: “The tremendous/unprecedented/phenomenal growth/scale/ubiquity of the Internet…”

33. 33 CIS6930.5: Federated Distributed Systems (Fall 2006) Administravia: Paper Reviewing (2) Follow the form provided when relevant. l State the main contribution of the paper l Critique the main contribution: Rate the significance of the paper on a scale of 5 (breakthrough), 4 (significant contribution), 3 (modest contribution), 2 (incremental contribution), 1 (no contribution or negative contribution). Explain your rating in a sentence or two. Rate how convincing the methodology is. l Do the claims and conclusions follow from the experiments? l Are the assumptions realistic? l Are the experiments well designed? l Are there different experiments that would be more convincing? l Are there other alternatives the authors should have considered? l (And, of course, is the paper free of methodological errors?)

34. 34 CIS6930.5: Federated Distributed Systems (Fall 2006) Administravia: Paper Reviewing (3) l What is the most important limitation of the approach? l What are the three strongest and/or most interesting ideas in the paper? l What are the three most striking weaknesses in the paper? l Name three questions that you would like to ask the authors. l Detail an interesting extension to the work not mentioned in the future work section. l Optional comments on the paper that you’d like to see discussed in class.

35. 35 CIS6930.5: Federated Distributed Systems (Fall 2006) Administravia: Discussion leading l Come prepared! –Prepare discussion outline –Prepare questions: >“What if”s >Unclear aspects of the solution proposed >… –Similar ideas in different contexts –Initiate short brainstorming sessions l Leaders do NOT need to submit paper reviews l Main goals: –Keep discussion flowing –Keep discussion relevant –Engage everybody (I’ll have an eye on this, too)

36. 36 CIS6930.5: Federated Distributed Systems (Fall 2006) Administravia: Projects l Combine with your research if relevant to the class l Get approval from all instructors if you overlap final projects: –Don’t sell the same piece of work twice –You can get more than twice as many results with less than twice as much work l Aim high! –Put one extra month and get a publication out of it –It is doable (we have proofs) l Try ideas that you postponed out of fear: it’s just a class, not your PhD.

37. 37 CIS6930.5: Federated Distributed Systems (Fall 2006) Administravia: Project deadlines (tentative) l Sept. 14: 1-page project proposal l Oct. 10: 3-page literature survey –Know relevant work in your problem area –If implementation project, list tools, similar projects l Nov. 13: 5-page Midterm project due –Have a clear image of what’s possible/doable –Report preliminary results l Last class(es):In-class project presentation –Demo, if appropriate l Dec. 15: –10-page write-up

38. 38 CIS6930.5: Federated Distributed Systems (Fall 2006) Next Class (Wed, August 30) l In-class discussion of papers: –“Automated Worm Fingerprinting”, OSDI ‘04. –“Planet Scale Software Updates”, SIGCOMM ’06. l Discussion of some project ideas l Need discussion leader to team up with me for the class next week: Real systems (1): BitTorrent –Exploiting BitTorrent For Fun (IPTPS’06) –A Case for Efficient Execution of Data-Intense Applications with BitTorrent on Computational Desktop Grids ()

39. 39 CIS6930.5: Federated Distributed Systems (Fall 2006) Questions?