1. Collaborative Expedition Workshop, NSF, Aug. 19, 2008 Deploying a Lightweight, Context-Linked Event Notification System for a Distributed Scientific Team Cecilia R. Aragon, Ph.D. Computational Research Division Lawrence Berkeley National Laboratory

2. 2 Collaborators Sarah Poon, Rollin Thomas, Brian Lee; Nearby Supernova Factory collaboration: –US: G. Aldering, C. Aragon, S. Bailey, S. Bongard, M. Childress, S. Loken, P. Nugent, S. Perlmutter, K. Runge, R. Thomas, B. Weaver (LBNL) R. Kessler (Chicago) C. Baltay, N. Ellman, D. Rabinowitz, R. Scalzo (Yale) –France: C. Buton, Y. Copin, E. Gangler, G. Smadja (IPNL, Lyon) P. Antilogus, J. Li, R. Pain, R. Pereira, C. Wu (LPNHE, Paris) E. Pecontal, G. Rigaudier (CRAL, Lyon)

3. 3 Talk Outline Motivation and challenges Proposed solution: context-linked tools Astrophysics case study Science background Telescope operational challenges Design and implementation of “Bert” User evaluation Discussion and conclusions

4. 4 Motivation Distributed teams need to analyze increasingly vast and complex data streams –Often working under time pressure –Examples: first responders, pilots, air traffic controllers, hospital staff, astrophysicists operating large telescopes Software tools can facilitate collaborative decision-making Lightweight tools needed –low cost –low barriers to entry

5. 5 Challenges in Developing Collaborative Tools Difficulty of gaining user acceptance of custom tools Cost Event notification must be appropriately but not overly intrusive –Especially in time-critical, distributed “control room” situations Need to integrate with existing environment

6. 6 Proposed Solution: Context-Linked Tools Aviation analogy: “scene-linked” symbology on head-up displays improves pilot performance and situation awareness [McCann 95, Aragon 05] Context-linked tools: both task and context information included in shared communication space –Task: immediately pertinent to the job –Context: background processes, events in environment

7. 7 Lightweight, Context-Linked Event Notification System “Bert” - virtual assistant/event notification system Developed for Nearby Supernova Factory –international astrophysics collaboration –largest data volume supernova search currently in operation –processes 50GB of image data per night –time-critical telescope operation Integrated with instant messaging (“chat”) client already used by collaboration for communication

8. 8 Science Background: Supernovae Stellar explosions Can be used to measure the expansion rate of the universe Answer fundamental questions about the universe

9. 9 Stellar explosions appearing as bright spots near galaxies Rare: 1-2 per millennium Random and fleeting: wax and wane within several weeks Type Ia Supernovae [slide from R. Romano]

10. 10 Fundamental Questions about the Universe Einstein 1915: Why doesn’t gravity cause the universe to collapse? –The cosmological constant Λ Hubble 1929: The universe is expanding, no need for Λ –Next question: will the universe ever contract? –Need to measure the deceleration rate Riess, Perlmutter 1998: The rate of expansion is accelerating Question: Why?

11. 11 Universe Composition

12. 12 The Nearby Supernova Factory An experiment to collect data on more Type Ia supernovae than have ever been studied in a single project before and answer some fundamental cosmological questions about the nature of the universe and “dark energy.”

13. 13 SNfactory supernova search data flow [slide by S. Bailey]

14. 14 Telescope Operational Challenges Astronomers must monitor complex, dynamic operational data streams Multi-million dollar telescope with custom hardware –Equipment failures occur (e.g. telescope gets stuck, filters or cooling systems fail) Telescope must be operated within constraints or damage may occur –sunrise, maneuvering limitations Located on remote mountaintops –adverse weather may require immediate closing –hypoxia –night operations Critical decisions may need to be made rapidly –in as little as 45 seconds

15. 15 Telescope Remote Operation (VNC)

16. 16 Further Challenges Telescope time is expensive ($4K/night) Supernovae are rare and fleeting –occur about once per millennium in a typical galaxy –visible for a few weeks Optimal schedule necessary for maximum scientific benefit Unpredictable variations in schedule caused by: –upper air turbulence, fog, changing cloud cover

17. 17 Cross-Cultural Collaboration Nearby Supernova Factory –about 30 collaborators, half in U.S., half in France Telescope in Hawaii typically remotely operated by two to six people on a given night, across geographically separated time zones (US west/east coast, France) Correct decisions must be made collaboratively –some team members have never met –cultural differences, language differences –unfamiliarity with telescope interface –diurnal rhythms, environmental cues are opposite to telescope environment (up to 11 hour time difference)

18. 18 Bert Lightweight virtual assistant in the chat developed to assist with time-critical science tasks and general awareness Two main functions –announces relevant events, e.g. completed exposures, number of minutes till sunrise –responds to user queries for information In operational use since 2005

19. 19 Previous Work Scientific collaboratories –Wulf’s “center without walls” [Wulf 93] –Upper Atmospheric Research Collaboratory (UARC) [Finholt 95, Olson 01] The use of chat –Grounding and orienting tool –Keep distributed members aware of status at physical site [Bimholtz 05] –Trouble shooting, answering technical questions –Tickertape CVS notifications for distributed software development team [Fitzpatrick 06] –Persistent chat benefits [Erickson 99, Geyer 04, Halverson 04, Fono 06]

20. 20 Bert Implementation Written in Perl, uses standard CPAN modules (Comprehensive Perl Archive Network) Communicates with chat network via OSCAR protocol employed by AOL Instant Messenger system At start-up, Bert connects to chat network, finds preset chat room Forks a listener process that accepts connections from processes on the summit machine and forwards them to Bert

21. 21 Bert Implementation (cont’d) Event notification –Programs send messages to Bert via simple Unix file socket –Unix “at” command used to trigger a Bert message at pre-specified time

22. 22 Bert Implementation (cont’d) User commands to Bert –Users type Bert’s name in a chat message –Primitive syntax “bert, where is the moon?” “bert, what is humidity?” “bert, where is the telescope?” “bert, when is sunrise” –Get help from telescope observing experts (sends SMS (text) message) “bert, SOS [name]”

23. 23 Bert Implementation (cont’d) User interface –Cross-cultural user-centered design –Greetings –Eliza [Weizenbaum 66]

24. 24 Bert Implementation (cont’d) Speech synthesis (iChatter) Integration with telescope automated instrument control system (AIC) Logging of chats for training purposes

25. 25 User Evaluation: Methodology Analysis of chat logs –Three years of logs –449,684 comments (users, Bert, system) –Bert comments: 145,667 or about 32%of total –Focused on April - December 2007 logs (current state of operations) 21 unique users 150,644 comments Interviews with four observers (“shifters”) –Three have been observing since before Bert’s integration

26. 26 User Evaluation Synchronous and asynchronous uses of chat Comparison with pre-Bert chat logs –typical problem: confusion about sunrise Art: So sunrise is at 6:00 am. And now it’s 5. Art: We are on spectrum 2 of 4, about 1/2 way through, which is 2500 s left. John: No, it’s 4 Art: Eh? Hawaii is 2 h behind... Art: It’s 7 here John: read director... Art: Wow I am really just crazy. John: yes... Art: I have no sense of time or distance.

27. 27 Categories of Synchronous Bert Use 1. Collective Decision Making Stef: Rene, did the sky spectra of the other day been useful? Rene: I haven’t looked at it yet Stef Stef: ok Rene: are you wondering if it would be useful to take an other one? Stef: yes Rene: I would say yes if we have time for it and if it is before 18 deg twilight Stef: bert when is sunrise BERT: 18deg twilight is at 2007-04-13 14:51:00 UTC (in 30 min). BERT: 12deg twilight is at 2007-04-13 15:17:00 UTC (in 56 min). BERT: Sunrise is at 2007-04-13 15:57:00 UTC (in 96 min). Stef: yep Rene: good Rene: lets’ go for it then

28. 28 Categories of Synchronous Bert Use (cont’d) 2. Collective Trouble Shooting BERT: aic: ERROR: command [ point_object ] returned an error. BERT: aic: ERROR: event 18 terminating on fatal error at 2007-05-03 10:40:22 UTC Chi: yes Chi: Stef, can you log in? Stef: yes coming in Chi: seen Fatal error Stef: ok I am in Chi: don't match? Stef: weird: there are stars Stef: just try it again Chi: OK BERT: aic: New aic session started 2007-05-03 10:43:13 UTC with schedule file aicSched/sch.070503.tcl. BERT: aic: Starting event 18 (SNF20070326-012, type Supernova) Stef: should be far from moon Stef: Dec is off by 4 degrees Stef: 4 minutes Chi: yes Stef: that's a lot Stef: I think telescope is stuck

29. 29 Categories of Synchronous Bert Use (cont’d) 3. Keeping Track of Events –short responses to Bert’s announcements 4. Analysis Discussions –Bert announcements lead to focused discussion –45-second time limit to make determination whether or not to take supernova spectrum BERT: Prepare for check_match target confirmation, be prepared with stop_script! Chi: ok Dom: don't worry, you get another window just like it in Analysis. Chi: nothing in... Dom: bummer Chi: i stop ! 5. Humor

30. 30 Asynchronous Uses of Bert 1. Orientation –Bert announcements provide context 2. Training/Increasing Group Knowledge –Senior scientists review chat logs and email feedback –Experts at first did not realize the steep learning curve required of non-experts and beginner shifters –Scientists in U.S. could not monitor chat all night - used logs to unmask training problems –Informed design and development of groupware

31. 31 Context-Linked Information Group awareness of: –task information, e.g. AIC notifications –context information, e.g. time until sunrise

32. 32 Discussion and Benefits Context-linked information aids in communication and collaborative work –Provides continuous environmental data –Informs awareness of time, schedule –Crucial in coordinating time-critical tasks –Reduction of coordination efforts to establish common ground (less time spent figuring out state of tasks, beneficial when working under time pressure) Lightweight, easy-to-use –Did not require elaborate setup

33. 33 Context-Linked Information Aids time-critical work –facilitates task prioritization (environmental state notices inform on task urgency) –easy to ask for help or information w/o context switch –voice synthesis avoids overloading user’s visual bandwidth Presented within persistent, shared communication space –enables rich, asynchronous analysis of shared work –chat communication reveals shifters’ frame of mind –Bert’s notifications allow link with actual events and work products

34. 34 Further Work Detect degree of shifter’s cognitive overload and interruptibility –adaptive verbosity for Bert More in-depth analysis of half a million lines of chat logs Controlled studies on the effect of context-linked information in distributed teams

35. 35 Acknowledgments We would like to thank the scientists of the SNfactory collaboration for their time and detailed feedback. The authors wish to recognize and acknowledge the very significant cultural role and reverence that the summit of Mauna Kea has always had within the indigenous Hawaiian community. We are most fortunate to have the opportunity to conduct observations from this mountain. This work was supported in part by the Director, Office of Science, Office of Advanced Scientific Computing Research, of the U.S. Department of Energy under Contract No. DE- AC02-05CH11231, and by the Director, Office of Science, Office of High Energy Physics, of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231, and by a grant from the Gordon & Betty Moore Foundation. This research used resources of the National Energy Research Scientific Computing Center, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231.

36. Collaborative Expedition Workshop, NSF, Aug. 19, 2008 Questions? Cecilia R. Aragon CRAragon@lbl.gov