Scaling Science Communities Lessons learned by and future plans of the Open Science Grid Frank Würthwein OSG Executive Director Professor of Physics UCSD/SDSC

Professor of Physics, experimental particle physics with CMS @ LHC About Me Professor of Physics, experimental particle physics with CMS @ LHC HWW, WW, ttbar, SUSY SUSY convener CMS 2013/14 Executive Director of Open Science Grid Co-PI of Pacific Research Platform Executive Team member of SDSC Most of the talk is written in my role as ED of OSG but I stray occasionally without prior warning.

The OSG Vision in a nutshell Across the nation, institutions invest into research computing to remain competitive Science is a team sport, and institutions with an island mentality will underperform. Integration is key to success OSG provides services to advance this vision

Integrating Clusters Worldwide

Integrating Clusters Worldwide Integrating hardware across more than 100 institutions. Supporting science across more than 30 domains. Supporting Scientists across more than 100 institutions. Monitoring networking with 279 perfSonar instances. Organized into 16 meshes each for bandwidth & latency. A total of 9982 end-to-end paths are monitored. Within last 12 months: 134 million jobs executed. 1.5 Billion hours of compute time. 2 Billion data transfers. display.opensciencegrid.org We’re doing this since 2004

Lessons Learned

Be open to resource providers at all scales Be Open to All Be open to resource providers at all scales from small colleges to large national labs Be open to user communities at all scales from individual students to large research communities domain science specific and across many campuses campus specific and across many domain sciences Be open to any business model sharing, allocations, purchasing preemption is an essential part of operations

One tool does not fit all We host services on our hardware for you. We host services on your hardware for you. We provide an integrated software stack for you to deploy to host services on your hardware for you. “for you” = “for you and your friends” in all of the above statements. You control dynamically who you consider your friends. In all cases, seamless integration is key !

Open Source is mandatory We must integrate software from many sources. Research computing infrastructure software can not be provisioned from just one source !!! Some of our sources disappear on us. external software providers have their own objectives and timescales. Science needs to continue even when software providers disappear. we adopt orphaned software, and continue to maintain it until an orderly transition to a replacement can be executed.

Stay engaged with software providers and IT shops As the scale increases, we need to work with providers on scaling out their software capabilities. Need to offer at scale testbeds to benchmark capabilites. Need to be willing to do the benchmarking both in our shop and in yours, to verify that what works when we deploy it also works when you deploy it.

Funding Agencies are Fickle Science outlasts agency timelines Try to build consortium structures that can survive the changing winds in D.C.

Actively maintain relationships with international partners. Future Goals Integrate all compute, storage, and networking resources at US Research Institutions and Commercial Cloud providers. Do so for all of science. Actively maintain relationships with international partners. let science collaborations define the partners that we need to integrate with.