1. The 3 rd Pan-Galactic Workshop

2. Schedule 9:00 – 10:00BOINC status report 10:00 – 10:15Break 10:15 – 11:30BOINC plans 11:30 – 12:30Discussion 12:30 – 1:30Lunch 1:30 – 2:15Breakout session 1 2:30 – 3:15 Breakout session 2 3:30 – 4:00reports 4:00 – 5:00Wish list, discussion, wrap-up

3. BOINC status report David P. Anderson davea@ssl.berkeley.edu Sept. 6, 2007

4. Outline The state of volunteer computing The state of BOINC What’s happened since the last workshop  and what hasn’t happened What’s in the works What’s in the future

5. The state of volunteer computing Folding@home leads the way  200 TFLOPS from PCs, 50 TFLOPS from GPUs, 600 TFLOPS from PS3s  Storage@home BOINC-based activity  ~40 projects  500 TFLOPS from PCs No big discoveries so far Grid.org shuts down April 07 IBM WCG transitions to BOINC Volunteer population: stagnant No major (> 10K hosts) projects in last year

6. The state of BOINC Über-goals  use all computers in the world for important work  enable new science  involve the public in science BOINC’s role  provide software technology  catalyze/enable communities Partners: projects, BAM!, GridRepublic, translators, alpha testers, volunteers,... Staff: me, Rom Walton, Charlie Fenton 50% Contributors  Kevin Reed, John McLeod, Rytis Slatkevicius, Nicolas Alvarez, David Barnard, Frank Weiler, many others Funding: new NSF grant, 9/07-8/10

7. Requests for BOINC projects If you need a new feature:  talk to me about it first  develop it yourself but stay in synch Upgrade server software frequently Participate in email lists Include some BOINC funding in your grant proposals Tell me about publications

8. What’s happened since the last workshop?

9. BOINC now uses trac Trac bug-tracking system replaces BOINCzilla Subversion replaces RCS  sandboxes Milestones Documentation is Wiki-fied  please participate!

10. Facilitating project creation VMware BOINC server  run on Windows/Mac/Linux  all software pre-installed, compiled Wrapper application  handles sequence of “legacy” apps, which use stdin/stdout or named files  checkpointing at “sub-job” level make_project improvements  reasonable config.xml defaults

11. Client software releases 5.6 (Sept 06)  new CPU scheduler  Mac sandbox 5.8 (Dec 06)  Simple view  Snooze button  CPU throttling  memory usage prefs 5.10 (May 07)  Project selection menu  Advanced prefs dialog  Grid-based views

12. Multi-platform support clientscheduler supported platforms: win64 win32 application version number platform Some hosts can run >1 type of binary

13. Homogeneous redundancy Multiple HR types  each classifies hosts based on numerical properties  per-application HR types Feeder reserves shared-memory slots per HR class in proportion to RAC “Census” program computes ratios Feeder DB queries scans all unsent results feeder win32 schedule r linux/x86 macnew shared mem schedule r...

14. Load-aware scheduler clientscheduler jobdeadlineCPU time remaining 1174 23512... For each candidate job J: if run J with existing jobs, are any new deadlines missed (or missed by more)? If not, accept J

15. Client simulator Simulates 1 client and N projects Uses actual client code Web interface Uses:  study new scheduling policies  debug client  optimize preferences for a particular client project description host description cc_config.xml global_prefs.xml simulator options graphical timeline message log figures of merit: idle CPU wasted CPU share violation monotony

16. Recruitment and retention Newsletter support Reminder email support  dropouts  initial failures Grass-roots PC magazine publicity campaign  http://boinc.berkeley.edu/trac/wiki/BoincPr

17. BOINC-wide teams http://boinc.berkeley.edu/teams  create BOINC-wide teams here  offers list of teams/founders by Web RPC team_update.php (periodic task)  fetches team list  creates users and teams as needed BOINC-wide teams are automatically created on new projects

18. Skype-based “customer support” Volunteer registration  languages  projects Server polls Skype status Users can call or email Users can rate volunteers

19. Scheduler and daemons Nov 2007 2006 OctAugJulJunMayAprMarFebJanDec use reliable hosts for retries homogeneous redundancy changes multi-platform support abort, abort-if-unstarted EDF simulation based on workload

20. Client Nov 2007 2006 OctAugJulJunMayAprMarFebJanDec memory-aware scheduling “suspend if idle” preference security-related changes auto-update log availability info improved network BW estimates improved slot dir management

21. Manager Nov 2007 2006 OctAugJulJunMayAprMarFebJanDec per-application slide show project selection menu grid-based views complete skin mechanism preferences dialog, day-of-week prefs

22. API and runtime system Nov 2007 2006 OctAugJulJunMayAprMarFebJanDec support memory-aware apps kill unresponsive apps message-passing more robust multi-job wrapper

23. Web features Nov 2007 2006 OctAugJulJunMayAprMarFebJanDec certificate of computation private messages forum moderation features team foundership transfer show all projects on user page Paypal donation system mass host merge

24. Web RPCs and AMS support Nov 2007 2006 OctAugJulJunMayAprMarFebJanDec AMS can specify prefs, venue, resource share team history delta BOINC-wide teams Google gadget

25. Simulators Nov 2007 2006 OctAugJulJunMayAprMarFebJanDec client simulator SIMBA (global simulator)

26. What did we not get around to? Document app development using Cygwin, MinGW, FORTRAN Disk space management Revised credit system  better FLOPS accounting  generalized credit system Community features  buddy lists  cross-project profiles, message boards BOINC protocols as Internet standards

27. What’s in the works?

28. Version 6 application graphics Old: graphics in separate thread New: graphics in separate program  takes –fullscreen option  has logical name “v6graphics” Library support: supply render function, input- handling functions Support for shared memory between app and graphics BOINC client graphics app main app BOINC manager, screensaver shmem

29. Other V6 changes Windows  Sandboxing: create unprivileged account for BOINC  Move data directory out of C:Program Files  Wix-based installer Mac/Linux  Use memory-mapped files instead of shmat()

30. V5/V6 transition V6 client is compatible only with V6 apps Scheduler change:  API version is attribute of app version  Can have both V5 and V6 app versions  Scheduler will send only V5 apps to pre-V6 clients Projects can maintain backwards compatibility as long as they want

31. What’s in the future?

32. Berkeley@home Campus-level “meta-project” Applications  6 pilot apps: climate, geophysics, nanotechnology, genetics, phylogenomics, scientific decision systems Volunteers  1,000 instructional PCs  5,000 faculty/staff  30,000 students  400,000 alumni  general public NSF proposal submitted

33. Multi-threading support What’s in a $1000 PC?  2007: dual-core CPU, 4 GFLOPS, 1 GB RAM  2010: 80-core CPU, 100 GFLOPS, 8 GB RAM  Volunteer computing provides a use for all those cores, but you may run out of RAM BOINC support for multi-thread apps Languages/libraries for parallel programming  Open MP  Titanium, Cilk,... core client app Try to use N cores OK, I’m using M cores

34. Non-PC platforms Sony Playstation 3  Support from Sony: on hold  PS3Grid.org Other game consoles TV set-top boxes Hand-held devices  not clear if feasible

35. Energy and volunteer computing Computing == energy usage [== carbon emission ] Energy-aware computing  dynamic clock rate  shut-off when idle  disk spin-down BOINC must be energy-aware friendly  “compute while in use” by default  allow auto-shutoff  don’t assume constant clock rate Make sure volunteers are aware of energy cost Computer as space heater

36. How to validate results and credit? Redundancy  homogeneous redundancy: messy  heterogeneous: must write fuzzy validator  wastes computing power Non-redundancy (CPDN, Rosetta)  result consistency checking (application-specific)  granted credit: constant, or based on project-wide distribution Variable redundancy?  hosts start with 2X redundancy, asymptote to 1X redundancy as trust increases  (doesn’t solve comparison problem)

37. Skill aggregation (human computing) Web-based vision tasks  Stardust@home, Clickworkers, galaxy classification Amazon “Mechanical Turk” Validation Formulation as multi-person game  Louis von Ahn: image tagging Motivational axes: competition community

38. Rosetta@home plan Protein structure prediction  low-res: combinatorial, spatial, intuitive; humans do better than computers  high-res: computers do better Interactive “protein manipulation” program Teams as management structures  tasks are given to (possibly multiple) teams  managers organize and schedule sub-groups with particular skills or resources  communication paths between sub-groups

39. Berkeley Open System for Skill Aggregation (BOSSA) DB-driven web-based system Tools for community, competition, collaboration, and organization  Teams, groups, sub-groups  Communication mechanisms  Tasks, validation, credit Integration with BOINC for tasks that involve large computation or storage

40. How to verifiably train volunteers? Interspersed lessons and exercises Optimal refresh for long-term retention Analytics for content assessment/refinement Adaptive teaching Volunteer computing and skill aggregation provide an ideal venue for education research lesson exercis e lesson exercis e lesson exercis e (3 weeks) (1 week) exercis e lesson exercis e

41. Berkeley Open Learning Technology (BOLT) DB-driven CMS and analytics engine for web- based teaching content (lessons, exercises) course structure (XML) teaching engine (PHP) Sequencing, navigation student info, interaction DB Students analytical tools Educators

42. DB and web integration Accounts, teams and groups Communication Credit and competition BOINC hosts applicatio ns jobs BOLT lessons courses BOSSA tasks

43. Background utility compatibility Background utilities  disk defrag  disk indexing  virus scanning  web pre-fetch  disk backup Most run only when computer is idle  volunteer computing ==> they never run A) ignore zero-priority CPU activity B) Background manager  intelligent decision about when to run activities