1. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Cyberinfrastructure Requirements and Best Practices Lessons from a study of TeraGrid Ann Zimmerman Research Assistant Professor UM School of Information OGF Workshop, May 27, 2009

2. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Outline n Background n Challenges n People n Methods used n Analysis n Conclusions

3. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Background: Learning from TeraGrid n NSF-funded study to examine: –The TeraGrid collaboration –user needs and requirements –impact on research practice & outcomes –education, outreach & training activities n Research Team –Tom Finholt, PI; Ann Zimmerman, co-PI –Magia Krause, PhD student

4. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Key Questions: User Needs n What factors affect users’ computing needs and requirements? n What factors affect users' behavior as it relates to their use (or non-use) of TeraGrid/HPC? n How are the needs of users expected to change over the next five years?

5. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Key Questions Continued n Where do users currently spend time that does not count as doing science? n What research questions do they want to answer but currently cannot? What are the barriers?

6. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Data Collection June 2006-May 2007 n 7 site visits, including 4 TeraGrid sites n Interviews (n=~90) n Participant observations n User workshop n Document analysis and review n Surveys –Survey of current TeraGrid users –Surveys of tutorials at TG ’06 & TG ‘07

7. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu TeraGrid Planning Process n The goal of the planning process was to –develop options for delivering TeraGrid resources and services –based on the diverse needs of science and engineering communities n http://www.teragridfuture.org/

8. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Challenges n Heterogeneous users n Potentially thousands of users n Distributed environment

9. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu People included in the study n Current TeraGrid users n “Target” TeraGrid users –Non-users –Science gateway developers n Cyberinfrastructure “experts” n TeraGrid personnel

10. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu METHODS

11. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Workshops n Strengths –Learn by seeing users interact –Gain both broad & detailed information –Relatively efficient n Challenges –Require careful & creative planning (pre- and post-workshop) –Invite 3-4 times as many people as you want to participate in the workshop

12. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Interviews n Strengths –Gain detailed information –Information informs survey development n Challenges –Time-consuming (to conduct & to analyze data) –Resources limit the number of people who can be interviewed

13. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu CategoryDefinition Number of interviewees TeraGrid Users  Individual Researchers Individuals associated with a project that had a TeraGrid allocation at the time of the interview 26 TeraGrid Users  Science Gateway Developers Individuals who on a day-to- day basis spend some portion of their time working on a project designated as a TeraGrid Science Gateway 27 TeraGrid PersonnelIndividuals employed by one of the TeraGrid RP sites who have a formal or informal role in the TeraGrid project 26 Non-TeraGrid Users of HPC Resources Individuals who use HPC computing resources other than TeraGrid 3 Cyberinfrastructure ExpertsIndividuals with extensive knowledge of high- performance computing 4

14. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Survey n Advantages –Findings are generalizable across a larger population –Inexpensive (in $ not in person time) n Challenges –Developing a good survey is hard –Getting a good response rate takes a lot of effort

15. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Participant Observation n Advantages –Learn about all the factors that affect the ability to serve users –Learn about user needs from a variety of sources n Challenges –Time-consuming –Capturing and analyzing data

16. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu FINDINGS

17. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Components of User Behavior n The nature of the research problem n Alignment between infrastructure and scientific practice n Computational readiness n Ease of use

18. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Achieving Transformative Science n “Easy” things can be show stoppers n Many complexities to manage –virtual organization –diverse user needs –changes in science

19. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Conclusions n Different methods provide different kinds of information n Involve more than users in your study n Current methods are effective, but time- consuming and resource-intensive –New methods required. For example, “mine” and analyze sources of information (wikis, user support logs and databases, user sites)

20. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu More information n TeraGrid evaluation study reports & Planning Process workshop reports –http://deepblue.lib.umich.edu (browse for documents by Ann Zimmerman) n Other TeraGrid Planning Process materials –http://teragridfuture.org

21. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Acknowledgments n TeraGrid n Research participants n NSF grants OCI-0602525 and OCI- 0724300

22. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Contact me! n Sorry I couldn’t be here! n Contact me at: –asz@umich.edu –http://www-personal.si.umich.edu/~asz/index.html