1. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Grids in Context Dr. Francine Berman Director, San Diego Supercomputer Center Professor and HPC Endowed Chair, UCSD OSG AHM 07

2. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Comments Dan Katz: I would call staged/linked apps workflow apps, since workflow tools are often used to run these tools. However, workflow tools can (in theory) be adaptive also, though they often are not yet. There is also a lot of work going on now with optical networks and even without them, with multi-cluster MPI apps - apps which may not run as well as they would on a single cluster, but since there is not a single cluster large enough, they allow new problem sizes to be run. Miron: I-WAY not the start

3. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Seeds of an Idea: Computational Grids and Electric Power Grids, circa 1995 The Computational Grid is like the Electric Power Grid Electric power is ubiquitous Don’t need to know the source of the power or the power company that serves it Everything works together seamlessly The Computational Grid is different from the Electric Power Grid Wider spectrum of components and services More integrative activities in the research domain Not clear how to measure performance or success Policy, business model, standards, etc. not obvious

4. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman A First Shot: The I-Way @ SC95 Conference First large-scale “modern” Grid experiment provided the basis for modern Grid infrastructure efforts I-Way included A Grid of 17 sites connected by vBNS 60+ application groups OC-3 backbone Large-scale use of immersive displays CAVE and I-Desk I-Soft programming environment Pioneered security, scheduling ideas Scheduling done with a “human-in-the-loop” (Warren Smith!)

5. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Emergence of the Grid Community g IPG

6. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Fast Forward to 2007: National-scale Computational Grid Efforts 9 U.S. sites comprising > 100 TFs of computing capability and > 15 PB storage TG objectives: “deep science, wide impact, open environment”. Based on CTSS; born from PACI partnership experiences > 50 compute and storage sites in US, Asia, South America OSG Goal: to “build a sustained production national infrastructure of shared resources, benefiting a broad set of scientific applications” Based on VDT; born from GriPhyN, iVDGL, PPDG project experiences

7. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Wikipedia on Grid Computing Grid computing is an emerging computing model that distributes processing across a parallel infrastructure. Throughput is increased by networking many heterogeneous resources across administrative boundaries to model a virtual computer architecture. For a computing problem to benefit from a grid, it must require either large amounts of computation time or large amounts of data, and it must be reducible to parallel processes that do not require intensive inter- communication. Today resource allocation in a grid is done in accordance with SLAs (service level agreements).parallel networkingresourcesresource allocationservice level agreements As of 3/4/07, there are 45,400,000 entries for “grid computing” in

8. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman An Informal Assessment Grid Computing: the Good, the Not-Quite-Ready-for-Prime-time, and the Really Challenging

9. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Grid computing – The Good Grid computing has come of age – accepted as a viable infrastructure paradigm and a compelling source of research problems Real national-scale Grids and Grid software exist and are in daily use Applications which have no other alternatives are successfully using Grids (e.g. distributed data applications, staged applications, etc.) Real breakthroughs have been accomplished in Grid environments Community has extensive experience coordinating distributed groups of people to provide Grid resources, support, and services.

10. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Grid Programs What programs run better in a grid environment (than at a single site under centralized administration)? Programs which traditionally have used the Grid effectively include Minimal Communication applications Includes embarrassingly parallel apps, parameter sweeps Staged/linked applications (do part A then do part B) Includes remote instrument applications (get input from instrument at site A, compute/analyze data at site B) Access to resources (get stuff from/do something at site A) Portals, uniform access mechanisms

11. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Not Quite Ready for Prime Time Adaptive Programs Programs which can dynamically adapt to resources which will provide “good”/”best”/”sufficient” performance Sophisticated allocation of resources Grid economies Allocations which satisfy multiple objective functions (the individual, the group, priorities, etc.) Metaphoric goal: Don’t need to know the source of the power or the power company that serves it

12. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Not-adequately-tapped opportunities for greater leverage of/collaboration with the Private Sector Private sector innovation in distributed programs, multi-user, real-time dynamic environments, portals, etc. attacking some of the same issues as the Grid world

13. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Really Challenging Accurate representation/prediction of the “state of the Grid” Analysis of load/performance of grid components Aggregate performance measures, etc. New statistics and modes of analysis What is “good enough”? Program Development and Optimization Environments How do we compile for the Grid? How do we debug a distributed Grid program? When we find a bug in Grid software, who fixes it? How do we capture enough and use information to tune the performance of Grid programs Metaphoric goals: Don’t need to know the source of the power or the power company that serves it Everything works together seamlessly

14. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman More Really Challenging Issues What happens when the Grid becomes even more heterogeneous? The “e-hospital” – sensors and sensornets, wireless and wired, heterogeneous instruments with different performance profiles, new statistical models of state and behavior, increased failover requirements, impact of power/cost, etc. Policy boundaries National policy, institutional policy, open source policy, IP policy, licensing policy, etc. How are conflicts resolved (in policy and otherwise) What is success?

15. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Application Drivers are the Key Applications provide specific and quantifiable goals for grid software. Applications can help define functionality, usability, capability, capacity and other requirements for useful grids Application goals New infrastructure capabilities motivate New application goals enable New infrastructure capabilities motivate

16. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Success Our Grids should be as easy to use as our cars All components should work together to provide end-to-end performance Things should be basically where you expect them to be – you can get in almost any car and drive it with a minimum of effort There should be an instruction manual that’s user friendly It should be relatively easy to find someone who can fix it when it breaks It’s more about where you’re going than whether your car works By ?????

17. UCSD SAN DIEGO SUPERCOMPUTER CENTER Fran Berman Welcome to SDSC – Have a Great Meeting!