Einstein@home Bruce Allen, UWM LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee

LIGO Scientific Collaboration, UW - Milwaukee Overview Like SETI@home, but for LIGO/GEO data. When computer idle, screensaver does ‘real work’. American Physical Society (APS) will publicize as part of World Year of Physics (WYP) 2005 activities, and will help with web site & graphics. Use infrastructure/help from SETI@home developers for the distributed computing parts (BOINC). Goal: pulsar searches using ~1 million clients. Support for Windows, Mac OSX, Linux clients. Why? From LSC & Grid we can get ~ thousands of CPUs. From Einstein@home hope to get order(s) of magnitude more at low cost. Great outreach and science education tool. Let people contribute actively not just learn passively. LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee

LIGO Scientific Collaboration, UW - Milwaukee Screensaver LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee

How does it work? User goes to web site, “signs up” in form, giving email address. User is sent Account ID via email, and prompted to download “core client” for Windows, Mac OSX or Linux. User installs BOINC core client, enters Project URL and Account ID. Users computer starts downloading data, doing computation, and uploading results. TOTAL USER TIME: A FEW MINUTES Optionally: user can join a team, create a Profile, add additional machines from home or work. LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee

LIGO Scientific Collaboration, UW - Milwaukee Who is helping? AEI Bernd Machenschalk (Windows builds) Marialessandra Papa (ComputeFStatistic) Badri Krishnan (Data preparation) Reinhard Prix (Linux builds and ComputeFStatistic modifications) Steffen Grunewald (Validator, setting up European Data Server) BERKELEY David Anderson (BOINC infrastructure) LIGO LAB Stuart Anderson (NSF proposal) Peter Shawhan (NSF proposal and GUI tool) Teviet Creighton (Publicity) UWM David Hammer (Screensaver, Einstein server) Xavier Siemens (Data preparation/comparison with standalone) Scott Koranda (Data preparation) VASSAR Eric Myers (Screensaver, Einstein server) LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee

LIGO Scientific Collaboration, UW - Milwaukee Publicity This is why we didn’t start Einstein@home years ago! APS has made Einstein@home one of their half-dozen WYP2005 activities. Coordination: James Riordan, in charge of media relations at APS. Hardest part now: delaying publicity until we are ready for large numbers of users Half-dozen magazines planning articles (Astronomy, Wired, others). Lead time 4 months. Nova (PBS) doing a TV episode on distributed computing, and will feature Einstein@home as one of a handful of projects LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee

LIGO Scientific Collaboration, UW - Milwaukee Scaling to more users Currently about 50 active test users from LSC Ultimate limits to number of users set by: Load on Scheduling Server (must be a single machine) Load on Upload Server (single machine?). Validator CPU time (single machine?) Note: Download Servers (SFT data) can be easily distributed. Currently: Getting about 2500 CPU hours/day On average, we upload 1 result file (150 kB average) per minute, so 2.5kB/s. File systems, hardware and networking should sustain 10 MB/s, which is a factor of 4000 more IO. So current setup should scale to about 200,000 users. Other BOINC projects (Predictor@home) are comfortably supporting ~50,000 users. LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee

LIGO Scientific Collaboration, UW - Milwaukee Search Code (1 of 2) Based on ‘tried and tested’ ComputeFStatistic frequency domain search code. Modified (in CVS HEAD branch of LALAPPS) to: Interface with BOINC Do periodic checkpointing and restart cleanly from checkpoints Industrial strength error detection: Catch signals (SIGSEGV, SIGBUS, etc) and print stack traces Systematic error code returns Customized LAL error handler Add screensaver code (separate thread) Modified build procedure for thread-safe code, reduced dependency on shared system libraries, use of C++ include files from BOINC. LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee

LIGO Scientific Collaboration, UW - Milwaukee Search Code (2 of 2) Until Oct 26th, Windows application code crashed unpredictably (Linux & Darwin seemed stable) Bug #1: invalid memory access Bug #2: non-LAL running median code LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee

Are users/credits/teams just window dressing? When I first learned about BOINC, I didn’t think this was important I was VERY wrong: it’s REALLY important It’s how people can tell if their setup works and if they are actually making a contribution Homo Sapiens is a competitive species “Teams” feature gets people to sign up their friends BOINC provides ‘redundant computing’. Can require that more than one client get ‘the same result’ before credits are granted. LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee

LIGO Scientific Collaboration, UW - Milwaukee Changes to search code Short term Make Linux graphics more efficient (shared OpenGL libraries in executable) Restructure code so that instead of doing ComputeFStatistic once for a single IFO, we run it twice for two different IFOs, and generate the list of only coincident events. This reduces size of uploaded data set. Running CFS twice is trivial. Modify ‘polka’ to return list of coincident events and include into client code. Long term CW search group must finalize what search to do with Einstein@home Current testing is an all-sky coherent search using 10 hours of S2 data. Can be generalized to longer/more sensitive data Would be good to make this part of a hierarchical search. Either use as follow-up for Hough stage, or incorporate incoherent stage into application code LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee

LIGO Scientific Collaboration, UW - Milwaukee What needs to be done? Add VALIDATOR. From (two or more) results it chooses the ‘cannonical’ (“correct”) result. Has been written by Steffen Grunewald, using ndiff and heuristics. Now being “commissioned” BOINC under active development. Need to track changes in server code AND application library BOINC doesn’t yet support application graphics under Mac OSX, but will soon. Need to rebuild/test app. Upgrade Einstein@home server to current CVS head from 3 months old tagged release (Done, yesterday!) After this, begin adding test users from the general public, a few thousand at a time. Change in plans: we’ll do testing continuously from now until the end of the year, adding users continuously. Deploy additional servers to spread download load, and serious server hardware for main Einstein@home server. Get NSF funding to support Einstein@home: we need a couple of professional staff to maintain it. LSC Meeting Nov 4-7, 2004 LIGO Scientific Collaboration, UW - Milwaukee