1. Einstein@home Bruce Allen, UWM
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee

2. LIGO Scientific Collaboration, UW - Milwaukee
Overview
Like but for LIGO/GEO data. When computer idle, screensaver does ‘real work’.
American Physical Society (APS) will publicize as part of World Year of Physics (WYP) 2005 activities, and will help with web site & graphics.
Use infrastructure/help from developers for the distributed computing parts (BOINC).
Goal: pulsar searches using ~1 million clients. Support for Windows, Mac OSX, Linux clients.
Why?
From LSC & Grid we can get ~ thousands of CPUs. From hope to get order(s) of magnitude more at low cost.
Great outreach and science education tool. Let people contribute actively not just learn passively.
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee

3. LIGO Scientific Collaboration, UW - Milwaukee
Screensaver
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee

4. How does it work?
User goes to web site, “signs up” in form, giving address.
User is sent Account ID via , and prompted to download “core client” for Windows, Mac OSX or Linux.
User installs BOINC core client, enters Project URL and Account ID.
Users computer starts downloading data, doing computation, and uploading results.
TOTAL USER TIME: A FEW MINUTES
Optionally: user can join a team, create a Profile, add additional machines from home or work.
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee

5. LIGO Scientific Collaboration, UW - Milwaukee
Who is helping?
AEI
Bernd Machenschalk (Windows builds)
Marialessandra Papa (ComputeFStatistic)
Badri Krishnan (Data preparation)
Reinhard Prix (Linux builds and ComputeFStatistic modifications)
Steffen Grunewald (Validator, setting up European Data Server)
BERKELEY
David Anderson (BOINC infrastructure)
LIGO LAB
Stuart Anderson (NSF proposal)
Peter Shawhan (NSF proposal and GUI tool)
Teviet Creighton (Publicity)
UWM
David Hammer (Screensaver, Einstein server)
Xavier Siemens (Data preparation/comparison with standalone)
Scott Koranda (Data preparation)
VASSAR
Eric Myers (Screensaver, Einstein server)
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee

6. LIGO Scientific Collaboration, UW - Milwaukee
Publicity
This is why we didn’t start years ago!
APS has made one of their half-dozen WYP2005 activities. Coordination: James Riordan, in charge of media relations at APS.
Hardest part now: delaying publicity until we are ready for large numbers of users
Half-dozen magazines planning articles (Astronomy, Wired, others). Lead time 4 months.
Nova (PBS) doing a TV episode on distributed computing, and will feature as one of a handful of projects
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee

7. LIGO Scientific Collaboration, UW - Milwaukee
Scaling to more users
Currently about 50 active test users from LSC
Ultimate limits to number of users set by:
Load on Scheduling Server (must be a single machine)
Load on Upload Server (single machine?).
Validator CPU time (single machine?)
Note: Download Servers (SFT data) can be easily distributed.
Currently:
Getting about 2500 CPU hours/day
On average, we upload 1 result file (150 kB average) per minute, so 2.5kB/s. File systems, hardware and networking should sustain 10 MB/s, which is a factor of 4000 more IO. So current setup should scale to about 200,000 users.
Other BOINC projects are comfortably supporting ~50,000 users.
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee

8. LIGO Scientific Collaboration, UW - Milwaukee
Search Code (1 of 2)
Based on ‘tried and tested’ ComputeFStatistic frequency domain search code.
Modified (in CVS HEAD branch of LALAPPS) to:
Interface with BOINC
Do periodic checkpointing and restart cleanly from checkpoints
Industrial strength error detection:
Catch signals (SIGSEGV, SIGBUS, etc) and print stack traces
Systematic error code returns
Customized LAL error handler
Add screensaver code (separate thread)
Modified build procedure for thread-safe code, reduced dependency on shared system libraries, use of C++ include files from BOINC.
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee

9. LIGO Scientific Collaboration, UW - Milwaukee
Search Code (2 of 2)
Until Oct 26th, Windows application code crashed unpredictably (Linux & Darwin seemed stable)
Bug #1: invalid memory access
Bug #2: non-LAL running median code
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee

10. Are users/credits/teams just window dressing?
When I first learned about BOINC, I didn’t think this was important
I was VERY wrong: it’s REALLY important
It’s how people can tell if their setup works and if they are actually making a contribution
Homo Sapiens is a competitive species
“Teams” feature gets people to sign up their friends
BOINC provides ‘redundant computing’. Can require that more than one client get ‘the same result’ before credits are granted.
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee

11. LIGO Scientific Collaboration, UW - Milwaukee
Changes to search code
Short term
Make Linux graphics more efficient (shared OpenGL libraries in executable)
Restructure code so that instead of doing ComputeFStatistic once for a single IFO, we run it twice for two different IFOs, and generate the list of only coincident events. This reduces size of uploaded data set.
Running CFS twice is trivial. Modify ‘polka’ to return list of coincident events and include into client code.
Long term
CW search group must finalize what search to do with
Current testing is an all-sky coherent search using 10 hours of S2 data. Can be generalized to longer/more sensitive data
Would be good to make this part of a hierarchical search. Either use as follow-up for Hough stage, or incorporate incoherent stage into application code
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee

12. LIGO Scientific Collaboration, UW - Milwaukee
What needs to be done?
Add VALIDATOR. From (two or more) results it chooses the ‘cannonical’ (“correct”) result. Has been written by Steffen Grunewald, using ndiff and heuristics. Now being “commissioned”
BOINC under active development. Need to track changes in server code AND application library
BOINC doesn’t yet support application graphics under Mac OSX, but will soon. Need to rebuild/test app.
Upgrade server to current CVS head from 3 months old tagged release (Done, yesterday!)
After this, begin adding test users from the general public, a few thousand at a time. Change in plans: we’ll do testing continuously from now until the end of the year, adding users continuously.
Deploy additional servers to spread download load, and serious server hardware for main server.
Get NSF funding to support we need a couple of professional staff to maintain it.
LSC Meeting Nov 4-7, 2004
LIGO Scientific Collaboration, UW - Milwaukee