1. Volunteer Computing with BOINC: a Tutorial David P. Anderson Space Sciences Laboratory University of California – Berkeley May 16, 2006

2. Tutorial overview ● Session 1: Introduction to BOINC ● Session 2: Creating a BOINC project ● Session 3: Developing a BOINC application ● Session 4: Processing work

3. Volunteer computing Projectstartwherearea peak #hosts GIMPS1994math 10,000 distributed.net1995cryptography 100,000 SETI@home I1999UCBSETI 600,000 Folding@home1999Stanfordbiology 200,000 United Devices2002commercialbiomedicine 200,000 CPDN2003Oxfordclimate change 150,000 LHC@home2004CERNphysics 60,000 Predictor@home2004Scrippsbiology 100,000 WCG2004commercialbiomedicine 200,000 Einstein@home2005LIGOastrophysics 200,000 SETI@home II2005UCBSETI 850,000 Rosetta@home2005U. Washbiology 100,000 SIMAP2005T.U. Munichbioinformatics 10,000

4. What is BOINC? ● Middleware for volunteer computing – Server-side: DB-driven software for distributing jobs, providing user web features, etc. – Client-side: ● Core client: manages work ● BOINC Manager: provides GUI ● Screensaver ● BOINC is open-source (LGPL) ● BOINC projects are autonomous

5. What are suitable applications? ● Coarse-grained parallelism – No direct communication between jobs ● Public appeal ● Low data/computation ratio – At least 1 CPU day per GB ● Fault tolerance

6. Projects ● Each project has a fixed Master URL – User-visible web site – Contains scheduler URLS: http://isaac.ssl.berkeley.edu/alpha_cgi/cgi ● Each project has its own server complex – Database (MySQL) – Scheduler – Daemons – Data server

7. Storage ● Files: input, output, executables, images, etc. – Names are unique within project – Immutable – May be replicated – XML description: foobar http://a.b.c/foobar http://x.y.z/foobar 123123123123 134423 200000 [ ]

8. File references ● From: – Workunits – Results – application versions ● XML description: foobar [ input ] [ ]

9. Platforms ● Represents a compilation target (OS + processor type) windows_intelx86 Microsoft Windows (95 or later) running on an Intel x86-compatible processor i686-pc-linux-gnu Linux running on an Intel x86-compatible processor powerpc-apple-darwin Mac OS 10.3 or later running on Motorola PowerPC i686-apple-darwin Mac OS 10.4 or later running on Intel sparc-sun-solaris2.7 Solaris 2.7 or later running on a SPARC-compatible processor

10. Applications – Each project can have multiple applications – Participants attach to projects, not applications – Each application has a set of ● Application versions ● Workunits (jobs) ● Results (job instances)

11. Application versions ● Linked to application, platform ● XML description:... [... ] foobar 4 program_1 library_12

12. Workunits ● Name (unique) ● Application ● List of input files ● Resource estimates (FLOPS, disk, memory) ● Delay bound ● Redundancy parameters: – Minimum quorum, maximum #instances

13. Results ● Name (unique) ● Workunit ● List of output files ● Server state – Unsent, in progress, done, error ● If done: – Host, exit status, CPU time, output file info

14. workunits applications results platforms Database summary application versions

15. Redundant computing ● Jobs may fail or return wrong results ● “Right” results may differ between hosts ● Do each job at least twice; if results don't agree, issue more copies of job ● Validation: compare set of results; choose “canonical result” ● Assimilation: handle canonical result ● Homogeneous redundancy

16. Locality scheduling ● For applications with – A few large input files – Each file has many workunits ● Special scheduling policy: – if a host has file X, try to send it more work for file X – Minimize network traffic ● Einstein@home

17. Trickle messages ● For long-running applications (like CPDN) ● Application can generate trickle-up messages – Partial credit, output summary ● Server handles trickle-up messages, can generate trickle-down messages – Abort job – Messages to volunteer ● Messages are sent/received asynchronously

18. BOINC client software Core client screensaver servers Manager Application

19. client Scheduler RPC scheduler List of applications list of files list of results ack of completed results new preferences Platform account key host ID List of completed results host hardware description host usage description work request (seconds)