1. The Grid and the Future of Business Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago http://www.mcs.anl.gov/~foster

2. Grid Computing

3. Abstract By enabling the integration of services and resources within and across enterprises, the Grid promises to enable new ways of doing business powered by on- demand access to computing, seamless access to corporate data, and dynamic outsourcing to service providers. In this visionary keynote, the founder of Grid computing reviews the origins of Grid computing in e- science, discusses its likely impact in e-business and the commercial marketplace, and describes progress towards the integration of Grid technologies and Web services within the Open Grid Services Architecture.

4. Technical Universe, Circa 1890 Ubiquitous communication infrastructure –The telegraph Local power generation –Electric power generators serve at most city blocks

5. Technical Universe, Circa 2000 Ubiquitous comms infrastructure –Internet, email, Web Ubiquitous power distribution –(Reliable?) standard access –Tremendous variety of devices Local computing –Most computing and storage on internal enterprise computers

6. Exponentials (and Coefficients) Network vs. computer performance –Computer speed doubles every 18 months –Network speed doubles every 9 months –Difference = order of magnitude per 5 years 1986 to 2000 –Computers: x 500 –Networks: x 340,000 2001 to 2010 –Computers: x 60 –Networks: x 4000 Scientific American (Jan-2001)

7. Therefore: A Computing Grid On-demand, ubiquitous access to computing, data, and services New capabilities constructed dynamically and transparently from distributed services “When the network is as fast as the computer's internal links, the machine disintegrates across the net into a set of special purpose appliances” (George Gilder)

8. My Presentation The emergence of the Grid concept –Origins in eScience, and the Globus Toolkit Grids and e-business –Opportunities & requirements Technology convergence –Open Grid Services Architecture Summary

9. My Presentation The emergence of the Grid concept –Origins in eScience, and the Globus Toolkit Grids and e-business –Opportunities & requirements Technology convergence –Open Grid Services Architecture Summary

10. E-Science: The Original Grid Driver Pre-electronic science –Theorize &/or experiment, in small teams Post-electronic science –Construct and mine very large databases –Develop computer simulations & analyses –Access specialized devices remotely –Exchange information within distributed multidisciplinary teams  Need to manage dynamic, distributed infrastructures, services, and applications

11. And Thus: The Grid “ Resource sharing & coordinated problem solving in dynamic, multi- institutional virtual organizations”

12. Early 90s –Gigabit testbeds, metacomputing Mid to late 90s –Early experiments (e.g., I-WAY), academic software projects (e.g., Globus), applications 2002 –Dozens of application communities & projects –Significant technology base (Globus Toolkit TM ) –Global Grid Forum: ~500 people, 20+ countries The Grid: A Brief History

13. Sloan Digital Sky Survey Analysis

14. Size distribution of galaxy clusters? Sloan Digital Sky Survey Analysis Galaxy cluster size distribution Chimera Virtual Data System + iVDGL Data Grid (many CPUs)

15. A Large Virtual Organization: CERN’s Large Hadron Collider 1800 Physicists, 150 Institutes, 32 Countries 100 PB of data by 2010; 50,000 CPUs?

16. Data Grids for High Energy Physics Tier2 Centre ~1 TIPS Online System Offline Processor Farm ~20 TIPS CERN Computer Centre FermiLab ~4 TIPS France Regional Centre Italy Regional Centre Germany Regional Centre Institute Institute ~0.25TIPS Physicist workstations ~100 MBytes/sec ~622 Mbits/sec ~1 MBytes/sec There is a “bunch crossing” every 25 nsecs. There are 100 “triggers” per second Each triggered event is ~1 MByte in size Physicists work on analysis “channels”. Each institute will have ~10 physicists working on one or more channels; data for these channels should be cached by the institute server Physics data cache ~PBytes/sec ~622 Mbits/sec or Air Freight (deprecated) Tier2 Centre ~1 TIPS Caltech ~1 TIPS ~622 Mbits/sec Tier 0 Tier 1 Tier 2 Tier 4 1 TIPS is approximately 25,000 SpecInt95 equivalents

17. Lift Capabilities Drag Capabilities Responsiveness Deflection capabilities Responsiveness Thrust performance Reverse Thrust performance Responsiveness Fuel Consumption Braking performance Steering capabilities Traction Dampening capabilities Crew Capabilities - accuracy - perception - stamina - re-action times - SOPs Engine Models Airframe Models Wing Models Landing Gear Models Stabilizer Models Human Models Grids at NASA: Aviation Safety

18. NETWORK IMAGING INSTRUMENTS COMPUTATIONAL RESOURCES LARGE DATABASES DATA ACQUISITION PROCESSING, ANALYSIS ADVANCED VISUALIZATION Life Sciences: Telemicroscopy

19. Underlying Technical Requirements Dynamic formation and management of virtual organizations Online negotiation of access to services: who, what, why, when, how Configuration of applications and systems able to deliver multiple qualities of service Autonomic management of distributed infrastructures, services, and applications

20. State of the Art: Globus Toolkit TM (since 1996) Small, standards-based set of protocols –Authentication, delegation; resource discovery; reliable invocation; etc. Information-centric design –Data models; publication, discovery protocols Open source implementation & community –With commercial support Enabler of services and applications

21. Grid Projects in eScience

22. My Presentation The emergence of the Grid concept –Origins in eScience, and the Globus Toolkit Grids and e-business –Opportunities & requirements Technology convergence –Open Grid Services Architecture Summary

23. And What About Business? Fragmentation of enterprise infrastructure –Specialized platforms -> commodity servers –“Intelligence” embedded in networks The rise of the “eUtility” (IBM, HP, …) –Outsourcing, economies of scale Business-to-business computing –Especially complex virtual organizations Ever more challenging QoS requirements –“Green-screen” -> “ubiquitious web presence”

24. Today’s Enterprise Computing Environment

25. The Business Opportunity On-demand computing, storage, services –Significant savings due to reduced build-out, economies of scale, reduced admin costs –Greater flexibility => greater productivity Entirely new applications and services –Based on high-speed resource integration Solution to enterprise computing crisis –Render distributed infrastructures manageable

26. Grids and Industry: Early Examples Butterfly.net: Grid for multi-player games Entropia: Distributed computing (BMS, Novartis, …)

27. Realizing the Promise Requires Significant Innovation Automation of infrastructure operation to achieve economies of scale Management and component models for distributed service provisioning New applications and tools powered by distributed services and resources Business and service models to support specialization of function

28. My Presentation The emergence of the Grid concept –Origins in eScience, and the Globus Toolkit Grids and e-business –Opportunities & requirements Technology convergence –Open Grid Services Architecture Summary

29. Grid Evolution: Open Grid Services Architecture Refactor Globus protocol suite to enable common base and expose key capabilities Service orientation to virtualize resources and unify resources/services/information Embrace key Web services technologies: standard IDL, leverage commercial efforts Result: standard interfaces & behaviors for distributed system management

30. Transient Service Instances “Web services” address discovery & invocation of persistent services –Interface to persistent state of entire enterprise In Grids, must also support transient service instances, created/destroyed dynamically –Interfaces to the states of distributed activities –E.g. workflow, video conf., dist. data analysis Significant implications for how services are managed, named, discovered, and used

31. Open Grid Services Architecture Defines fundamental (WSDL) interfaces and behaviors that define a Grid Service –Required + optional interfaces = WS “profile” Defines WSDL extensibility elements –E.g., serviceType (a group of portTypes) Open source Globus Toolkit 3.0 –Leverage GT experience, code, community And also commercial implementations

32. The Grid Service = Interfaces/Behaviors + Service Data Service data element Service data element Service data element Implementation GridService (required) Service data access Explicit destruction Soft-state lifetime … other interfaces … (optional) Standard: - Notification - Authorization - Service creation - Service registry - Manageability - Concurrency + application- specific interfaces Binding properties: - Reliable invocation - Authentication Hosting environment/runtime (“C”, J2EE,.NET, …)

33. Example: Database Service DBaccess Grid service supports at least two portTypes –GridService –DBaccess Each has service data –GridService: basic introspection, lifetime, … –DBaccess: database type, current load, …, … Maybe other portTypes as well –E.g., NotificationSource (SDE = subscribers) Grid Service DBaccess DB info Name, lifetime, etc.

34. Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider “I want to create a personal database containing data on e.coli metabolism”...... Database Factory

35. Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... “Find me a data mining service, and somewhere to store data” Database Factory

36. Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Handles for Mining and Database factories Database Factory

37. Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... “Create a data mining service with initial lifetime 10” “Create a database with initial lifetime 1000” Database Factory

38. Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Miner “Create a data mining service with initial lifetime 10” “Create a database with initial lifetime 1000”

39. Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Miner Query

40. Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Miner Query Keepalive

41. Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Miner Keepalive Results

42. Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Miner Keepalive

43. Data Mining for Bioinformatics User Application BioDB n Storage Service Provider Database Factory Mining Factory Community Registry Database Service BioDB 1 Database Service...... Compute Service Provider...... Database Keepalive

44. GT3: OGSA-Based Globus Toolkit GT3 Core –Grid service interfaces –Reference impln of evolving standard –Multiple hosting envs: Java/J2EE, C, C#/.NET? GT3 Base Services –Globus capabilities Many other services GT3 Core GT3 Base Services Other Grid Services GT3 Data Services

45. OGSA: Current Status Grid service specification & other documents moving forward in GGF –www.gridforum.org/ogsi-wg Globus Project on track for open source OGSA-based GT3 release end of 2002 –www.globus.org/ogsa IBM committed to various OGSA-compliant software releases (e.g., WebSphere) Other industrial efforts underway

46. My Presentation The emergence of the Grid concept –Origins in eScience, and the Globus Toolkit Grids and e-business –Opportunities & requirements Technology convergence –Open Grid Services Architecture Summary

47. Summary: The Grid Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations –On-demand, ubiquitous access to computing, data, and services –New capabilities constructed dynamically and transparently from distributed services Evolved to be dominant eScience, now transitioning to industry (think Web in 1994)

48. Open Grid Services Architecture Open Grid Services Architecture represents next step in Grid evolution Service orientation enables unified treatment of resources, data, and services Standard interfaces and behaviors (the Grid service) for managing distributed state Open source Globus Toolkit implementation (and numerous commercial value adds)

49. For More Information Grid Book –www.mkp.com/grids Survey articles –www.mcs.anl.gov/~foster The Globus Project™ –www.globus.org Global Grid Forum –www.gridforum.org –Edinburgh, July 22-24 –Chicago, Oct 15-17