1. Scaling Science Communities Lessons learned by and future plans of the Open Science Grid Frank Würthwein OSG Executive Director Professor of Physics UCSD/SDSC

2. Professor of Physics, experimental particle physics with CMS @ LHC
About Me
Professor of Physics, experimental particle physics with LHC
HWW, WW, ttbar, SUSY
SUSY convener CMS 2013/14
Executive Director of Open Science Grid
Co-PI of Pacific Research Platform
Executive Team member of SDSC
Most of the talk is written in my role as ED of OSG
but I stray occasionally without prior warning.

3. The OSG Vision in a nutshell
Across the nation, institutions invest into research computing to remain competitive
Science is a team sport, and institutions with an island mentality will underperform.
Integration is key to success
OSG provides services
to advance this vision

4. Integrating Clusters Worldwide

5. Integrating Clusters Worldwide
Integrating hardware across more than 100 institutions.
Supporting science across more than 30 domains.
Supporting Scientists across more than 100 institutions.
Monitoring networking with 279 perfSonar instances.
Organized into 16 meshes each for bandwidth & latency.
A total of 9982 end-to-end paths are monitored.
Within last 12 months:
134 million jobs executed.
1.5 Billion hours of compute time.
2 Billion data transfers.
display.opensciencegrid.org
We’re doing this since 2004

6. Lessons Learned

7. Be open to resource providers at all scales
Be Open to All
Be open to resource providers at all scales
from small colleges to large national labs
Be open to user communities at all scales
from individual students to large research communities
domain science specific and across many campuses
campus specific and across many domain sciences
Be open to any business model
sharing, allocations, purchasing
preemption is an essential part of operations

8. One tool does not fit all
We host services on our hardware for you.
We host services on your hardware for you.
We provide an integrated software stack for you to deploy to host services on your hardware for you.
“for you” = “for you and your friends” in all of the above statements.
You control dynamically who you consider your friends.
In all cases, seamless integration is key !

9. Open Source is mandatory
We must integrate software from many sources.
Research computing infrastructure software can not be provisioned from just one source !!!
Some of our sources disappear on us.
external software providers have their own objectives and timescales.
Science needs to continue even when software providers disappear.
we adopt orphaned software, and continue to maintain it until an orderly transition to a replacement can be executed.

10. Stay engaged with software providers and IT shops
As the scale increases, we need to work with providers on scaling out their software capabilities.
Need to offer at scale testbeds to benchmark capabilites.
Need to be willing to do the benchmarking both in our shop and in yours, to verify that what works when we deploy it also works when you deploy it.

11. Funding Agencies are Fickle
Science outlasts agency timelines
Try to build consortium structures that can survive the changing winds in D.C.

12. Actively maintain relationships with international partners.
Future Goals
Integrate all compute, storage, and networking resources at US Research Institutions and Commercial Cloud providers.
Do so for all of science.
Actively maintain relationships with international partners.
let science collaborations define the partners that we need to integrate with.