1. Introduction to the BOINC software David P. Anderson Space Sciences Laboratory University of California, Berkeley

2. Outline Abstractions The BOINC server software The BOINC client software and runtime system

3. Design goals A few applications, lots of jobs High performance – millions of jobs per day Scalability Fault tolerance

4. Abstractions Platform App version – a collection of files, one of which is an executable main program – associated with a platform App – a set of app versions that all perform roughly the same computation – may have versions for different platforms – may have different versions for one platform (GPU, non-GPU)

5. Abstractions Workunit (job) – a collection of input files – associated with an app (not an app version!) – attributes resource estimates and bounds latency bound Result (job instance) – a collection of output files – associated with a workunit

6. Anatomy of a BOINC project MySQL database project root/ bin/ cgi-bin/ download/ 00/.. 3ff/ html/ log_*/ templates/ upload/ 00/.. 3ff/ servers daemons and periodic tasks clients

7. Work generator MySQL database project root/ bin/ cgi-bin/ download/ 00/.. 3ff/ html/ log_*/ templates/ upload/ 00/.. 3ff/ work generator Creates input files Creates workunits One per app Flow control – disk space – DB size

8. Specifying a job Workunit template – XML document describing input files (logical, physical names) job attributes Result template – XML document describing output files create_work() – specifies templates, app, input files

9. Validator Check result validity Compare replicas May be app-specific MySQL database project root/ bin/ cgi-bin/ download/ 00/.. 3ff/ html/ log_*/ templates/ upload/ 00/.. 3ff/ validator

10. Validation Clients may – return bad results – exaggerated claimed credit Strategies – app-specific consistency checking – replication fuzzy comparison homogeneous redundancy – adaptive replication

11. Assimilator Processes completed results App-specific MySQL database project root/ bin/ cgi-bin/ download/ 00/.. 3ff/ html/ log_*/ templates/ upload/ 00/.. 3ff/ assimilator

12. Summary Create app, app versions for different platforms Develop work generator Develop validator Develop assimilator Isn’t there a simpler way?

13. Single-job submission Assemble your input files and executable, then boinc_submit --input foo --output blah program How this works: – uses “wrapper” app – executable is part of workunit – templates are created automatically What it doesn’t do: – multi-platform – validation

14. Job dispatch MySQL database project root/ bin/ cgi-bin/ download/ 00/.. 3ff/ html/ log_*/ templates/ upload/ 00/.. 3ff/ transitioner feeder scheduler (CGI or FastCGI) share-memory job cache clients

15. File transfer project root/ bin/ cgi-bin/ download/ 00/.. 3ff/ html/ log_*/ templates/ upload/ 00/.. 3ff/ Apache file upload handler clients

16. Janitorial daemons MySQL database project root/ bin/ cgi-bin/ download/ 00/.. 3ff/ html/ log_*/ templates/ upload/ 00/.. 3ff/ file deleter DB purger

17. Ways to deploy a BOINC server Linux server Server VM for VMWare Server VM for Amazon EC2

18. The BOINC runtime system Directory structure: BOINC/ projects/ lhcathome/ physical_name0 physical_name1 setiathome/ slots/ 0/ logical_name0 (link file) logical_name1 1/ BOINC client application share-memory message-passing BOINC runtime fraction done CPU time suspend resume quit

19. Basic API boinc_init() – creates a thread that handles messages boinc_finish() – creates a “finish file” boinc_resolve_filename() – maps logical to physical file names

20. Checkpointing boinc_time_to_checkpoint() – call at points where you can checkpoint boinc_checkpoint_done() – call when you’re finished checkpointing

21. Compound applications Examples: – coordinator program runs several worker programs in sequence – “switcher” program probes CPU architecture, selects which executable to run Variants of boinc_init() let you specify which app is main program, and how messages are handled Each message type must be handled by 1 process

22. Long-running applications Trickle-up messages Trickle-down messages Intermediate file transfers

23. Legacy applications The BOINC wrapper – takes XML “job file” – handles all messages

24. GPU and multithread apps Server – you supply a function that takes an app version and a host, and returns resource usage and estimated FLOPS – the BOINC scheduler chooses the best version Client – senses and reports coprocessors (e.g. NVIDIA GPUs) – coprocessor-aware scheduling and work fetch