1. Enabling Science Through Campus Bridging A case study with mlRho Scott Michael July 24, 2013

2. Outline What is Campus Bridging? Why is Campus Bridging important? The gap between Campus Champion and Extended Collaborative Support What does Campus Bridging look like in practice? –Use case example with mlRho

3. What is Campus Bridging? NSF Task Force on Campus Bridging Report – Campus bridging is the seamlessly integrated use of cyberinfrastructure operated by a scientist or engineer with other cyberinfrastructure on the scientist’s campus, at other campuses, and at the regional, national, and international levels as if they were proximate to the scientist, and when working within the context of a Virtual Organization (VO) make the ‘virtual’ aspect of the organization irrelevant (or helpful) to the work of the VO. Campus Bridging makes a supercomputer as easy to use as your desktop!

4. What is Campus Bridging? Stewart et. al. 2012 defined a series of goals based on use cases –Authentication –Training and information dissemination –Interactive computing –Data transfer –Distributed workflows –Resource sharing –User support

5. What is Campus Bridging? Campus Bridging efforts are yielding results from a number of tools –Globus Online –Global Federated File System (GFFS) –Cluster installation distributions What is missing from many descriptions of Campus Bridging is explicit support for users to transition their applications to increasingly complex systems

6. Why is Campus Bridging Important? Researchers in fields relatively new to HPC do not always have the necessary expertise scale their workflow to massive systems

7. Why is Campus Bridging Important? What is driving new areas of research and fields of science to use HPC resources? –Data, not tightly coupled computation Researchers with massive amounts of data are prime candidates for campus bridging Some example fields –Biology –Humanities –Social media

8. Campus Champions and ECSS Campus Champions are primarily providers of information and points of communication – Source of local, regional and national high-performance computing and cyberinfrastructure information on your campus – Source of information regarding XSEDE resources and services that will benefit research and education on your campus – Source of start-up accounts on your campus to quickly get researchers and educators using their allocations of time on XSEDE resources – Conduit for the campus high-performance computing needs, requirements and challenges, with direct access to XSEDE staff Champions may not have the necessary domain or technical knowledge to effectively bridge between campus and XSEDE resources

9. Campus Champions and ECSS XSEDE Extended Collaborative Support allocations provide –“expert staff members for an extended period to work together to solve challenging science and engineering problems through the application of cyberinfrastructure” Require an allocation request Are targeted towards a well defined and novel problem

10. The Gap Between Campus Champions and Extended Collaborative Support What’s missing? –Assistance in preparing allocation request –Scaling up to large numbers of processors –Managing massive data –Improving performance at scale

11. The mlRho Use Case Work at IU with the Michael Lynch lab nicely demonstrates potential of investing in Campus Bridging mlRho is a serial application developed jointly between IU researchers and researchers at the Max Planck Institute for Evolutionary Biology It uses a maximum likelihood approach to estimate linkage disequilibrium rates for a variety of species

12. mlRho from Desktop to XSEDE Lynch lab researchers were running 100s of serial mlRho jobs Quarry They needed larger resource to speed up their research We helped them prepare an XSEDE allocation request OrganismSize of Profile (GB)Distance/sec F. cylindrus (diatom)0.720.323 P. ornithorhynchus (platypus) 110.020 C. familiaris (dog)310.005

13. Preparing a Successful Allocation Proposal We bundled serial jobs using the BigJob framework We assisted the researchers in estimating the SUs the would require The most appropriate XSEDE machine to target Assisted in performing scalability studies Helped with file system issues

14. Application Optimization Once the application was running at scale we assisted with performance optimization We traced the application using Vampir and gave the results to the developer

15. Application Optimization The developer improved the runtime by more than 50x We probably did less work than what would be considered an ECSS collaboration

16. Future Explorations We have begun exploring running multiple serial instances of mlRho on the Xeon Phi acclerators BigJob is not available for the Phi so some low level scripting was required Not overly difficult but too much for most biologists Lynch lab is working to develop a mlRho gateway

17. Conclusions and Questions Campus Bridging is taking researchers from desktop computing, through campus computing, to computing with XSEDE resources Researchers new to HPC need help at all stages of an XSEDE allocation Campus Bridging tools will help with many pain points in this transition, with a modest investment of human resources there is huge potential ROI Campus Bridging activities can be a stepping stone for gateways

18. Conclusions and Questions What is the best way to provide these resources? How should Campus Bridging Specialists fit into the ecosystem of Campus Champions and ECSS? Comments? Questions? scamicha@iu.edu

19. Acknowledgements and Citations Sen Xu Abhinav Thota Robert Henschel Thomas Doak BigJob development team TACC support staff Rich Knepper Craig Stewart Craig A. Stewart, Richard Knepper, James Ferguson, Felix Bachmann, Ian Foster, Andrew Grimshaw, Victor Hazlewood, and David Lifka. 2012. What is campus bridging and what is XSEDE doing about it?. In Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment: Bridging from the eXtreme to the campus and beyond(XSEDE '12). ACM, New York, NY, USA,, Article 47, 8 pages. NSF Advisory Committee for Cyberinfrastructure Task Force on Campus Bridging. Final Report. March 2011. Available from: http://www.nsf.gov/od/oci/taskforces/TaskForceReport_CampusBridging.pdf