Volunteer Computing with BOINC: a Tutorial David P. Anderson Space Sciences Laboratory University of California – Berkeley May 16, 2006

Tutorial overview ● Session 1: Introduction to BOINC ● Session 2: Creating a BOINC project ● Session 3: Developing a BOINC application ● Session 4: Processing work

Volunteer computing Projectstartwherearea peak #hosts GIMPS1994math 10,000 distributed.net1995cryptography 100,000 I1999UCBSETI 600, ,000 United Devices2002commercialbiomedicine 200,000 CPDN2003Oxfordclimate change 150,000 60, ,000 WCG2004commercialbiomedicine 200, ,000 II2005UCBSETI 850,000 Washbiology 100,000 SIMAP2005T.U. Munichbioinformatics 10,000

What is BOINC? ● Middleware for volunteer computing – Server-side: DB-driven software for distributing jobs, providing user web features, etc. – Client-side: ● Core client: manages work ● BOINC Manager: provides GUI ● Screensaver ● BOINC is open-source (LGPL) ● BOINC projects are autonomous

What are suitable applications? ● Coarse-grained parallelism – No direct communication between jobs ● Public appeal ● Low data/computation ratio – At least 1 CPU day per GB ● Fault tolerance

Projects ● Each project has a fixed Master URL – User-visible web site – Contains scheduler URLS: ● Each project has its own server complex – Database (MySQL) – Scheduler – Daemons – Data server

Storage ● Files: input, output, executables, images, etc. – Names are unique within project – Immutable – May be replicated – XML description: foobar [ ]

File references ● From: – Workunits – Results – application versions ● XML description: foobar [ input ] [ ]

Platforms ● Represents a compilation target (OS + processor type) windows_intelx86 Microsoft Windows (95 or later) running on an Intel x86-compatible processor i686-pc-linux-gnu Linux running on an Intel x86-compatible processor powerpc-apple-darwin Mac OS 10.3 or later running on Motorola PowerPC i686-apple-darwin Mac OS 10.4 or later running on Intel sparc-sun-solaris2.7 Solaris 2.7 or later running on a SPARC-compatible processor

Applications – Each project can have multiple applications – Participants attach to projects, not applications – Each application has a set of ● Application versions ● Workunits (jobs) ● Results (job instances)

Application versions ● Linked to application, platform ● XML description:... [... ] foobar 4 program_1 library_12

Workunits ● Name (unique) ● Application ● List of input files ● Resource estimates (FLOPS, disk, memory) ● Delay bound ● Redundancy parameters: – Minimum quorum, maximum #instances

Results ● Name (unique) ● Workunit ● List of output files ● Server state – Unsent, in progress, done, error ● If done: – Host, exit status, CPU time, output file info

workunits applications results platforms Database summary application versions

Redundant computing ● Jobs may fail or return wrong results ● “Right” results may differ between hosts ● Do each job at least twice; if results don't agree, issue more copies of job ● Validation: compare set of results; choose “canonical result” ● Assimilation: handle canonical result ● Homogeneous redundancy

Locality scheduling ● For applications with – A few large input files – Each file has many workunits ● Special scheduling policy: – if a host has file X, try to send it more work for file X – Minimize network traffic ●

Trickle messages ● For long-running applications (like CPDN) ● Application can generate trickle-up messages – Partial credit, output summary ● Server handles trickle-up messages, can generate trickle-down messages – Abort job – Messages to volunteer ● Messages are sent/received asynchronously

BOINC client software Core client screensaver servers Manager Application

client Scheduler RPC scheduler List of applications list of files list of results ack of completed results new preferences Platform account key host ID List of completed results host hardware description host usage description work request (seconds)