1. Argonne National Laboratory
The Grid
by Ian Foster
Argonne National Laboratory
University of Chicago
Globus Alliance
SJT: Updated title

2. Grid Computing is in the News …

3. Why Should You Care? 1) Grid is a disruptive technology [Vision]
It ushers in a virtualized, collaborative, distributed world
2) Grid addresses pain points now [Reality]
Grids are built not bought, but are delivering real benefits in commercial settings
3) An open Grid is to your advantage [Future]
Standards are being defined now that will determine the future of this technology

4. Why Should You Care? 1) Grid is a disruptive technology [Vision]
It ushers in a virtualized, collaborative, distributed world
2) Grid addresses pain points now [Reality]
Grids are built not bought, but are delivering real benefits in commercial settings
3) An open Grid is to your advantage [Future]
Standards are being defined now that will determine the future of this technology

5. The Power Grid: On-Demand Access to Electricity
Decouple production &
consumption, enabling
On-demand access
Economies of scale
Consumer flexibility
New devices
Quality, economies of scale
Time

6. But Computing Isn’t Really Like Electricity!
How about “access computing resources like we access Web content”?
We have no idea where a website is, or on what computer or operating system it runs
Two interrelated opportunities
1) Enhance economy, flexibility, access by virtualizing computing resources
2) Deliver entirely new capabilities by integrating distributed resources
I import electricity but must export data
“Computing” is not always interchangeable

7. Virtualization Application Virtualization
Automatically connect applications to services
Dynamic & intelligent
provisioning
Applications:
Delivery
Application
Services:
Distribution
Infrastructure Virtualization
Dynamic & intelligent
provisioning
Automatic failover
Servers:
Execution
Source: The Grid: Blueprint for a New Computing Infrastructure (2nd Edition), 2004

8. Distributed System IntegrationUK
Financial Institution NY
Financial Institution – Insurance Group NY
Financial Institution –
Capital Markets Group

9. And Ultimately …

10. “The Anatomy of the Grid”, Foster, Kesselman, Tuecke, 2001
“Resource sharing & coordinated problem solving in dynamic … virtual organizations”
Enable integration of distributed service & resources
Using general-purpose protocols & infrastructure
To achieve useful qualities of experience
“The Anatomy of the Grid”, Foster, Kesselman, Tuecke, 2001

11. Terminology (1)
Cluster Grid Enterprise Grid Global Grid

12. Terminology (2) Grid Utility Computing Utility Computing Autonomic
Service-
Oriented
Architecture
Service-
Oriented
Architecture

13. Why Should You Care? 1) Grid is a disruptive technology [Vision]
It ushers in a virtualized, collaborative, distributed world
2) Grid addresses pain points now [Reality]
Grids are built not bought, but are delivering real benefits in commercial settings
3) An open Grid is to your advantage [Future]
Standards are being defined now that will determine the future of this technology

14. Grid Addresses Pain Points Now
Low utilization of enterprise resources
High cost of provisioning for peak demand
Inadequate resources prevent use of advanced applications
Lack of information integration

15. Grid Deployment Trends
Corporate
Corporate
Mission Criticality
Scientific
Department Enterprise Collaboration Internet

16. Early Commercial Applications
Leading adopters (Oct 2003) *
Financial services: 31%
Life sciences: 26%
Manufacturing: 18%
*Grids 2004: From Rocket Science To Business Service, The 451 Group
Manufacturing
Mechanical/ Electronic Design
Process Simulation
Finite
Element Analysis
Failure Analysis
Financial
Services
Derivatives
Analysis
Statistical Analysis
Portfolio Risk Analysis
LS /
Bioinformatics
Cancer Research
Drug Discovery
Protein Folding
Protein Sequencing
Other
Web Applications
Weather Analysis
Code Breaking/ Simulation
Academic
Energy
Seismic
Analysis
Reservoir Analysis
Entertainment
Digital Rendering
Massive Multi-Player
Games
Streaming Media
Grid Services Market Opportunity 2005
“Gridified” Infrastructure
Sources: IDC, 2000 and Bear Stearns- Internet /01 Analysis by SAI

17. Grid Deployment Strategies
A range of excellent commercial & open source products for resource federation
Federate enterprise computing resources
Federate enterprise information resources
Globus Toolkit®: inter-enterprise sharing
But, “Grids are built, not bought”
Integration with other enterprise systems is needed to deliver complete solution
Start small & with well-defined ROI case
Grow based on experience

18. Example: Novartis PC Grid links 3,700 desktop systems
Research & development applications
Potentially mainstream business computing
> 5 teraflop/s computing power
Estimate savings of $200M over 3 years
“We have projects we calculate would take 6 years on a single supercomputer. Today, the run time is 12 hours.”
Peter Sany, Novartis CIO

19. Example: University of Texas

20. Example: Royal Dutch/Shell
Improve accuracy and speed of summarization & scientific modeling applications
More robust, scalable IT infrastructure that adjusts as volumes fluctuate
Open standards ease integration of software
Cut processing time of seismic data, while improving the quality of the output
Focus employees on key scientific, not IT problems
“Grid computing is important to Shell because it offers the potential to create a truly unlimited resource, with a uniform interface to a variety of services. This is a significant opportunity for Shell to engage its independent companies in closer cooperation.”
J.N. Buur, Principal Research Physicist, Shell International Exploration and Production B.V.

21. Example: Kansai Electric Power Co.
Japan’s second largest electric utility company has various information in a heterogeneous, distributed database environment
Integrate information across departments and affiliated companies to enable information sharing
Application
DB Client
CRM
Accounting
Workflow
Federated Database
Oracle
Server
DB2
Server
ADABAS
Server
NOTES
Server

22. Scheduled Workload Reallocation
Start of Quarter:
General Ledger
Order Entry
End of Quarter:
General Ledger
Order Entry
Source: Oracle

23. NEES (Network for Earthquake Engineering Simulation) Collaboratory
U.Nevada Reno

24. NEESgrid at Work f2 f1 m1 UIUC U. Colorado Experimental Model
NCSA Computational Model

25. CMS High Energy Physics
Generate data for a physics experiment
Simulate 1.5 million events for physics ~500 sec per event
2 months continuous running across 5 sites
Managed by a single person

26. Why Should You Care? 1) Grid is a disruptive technology [Vision]
It ushers in a virtualized, collaborative, distributed world
2) Grid addresses pain points now [Reality]
Grids are built not bought, but are delivering real benefits in commercial settings
3) An open Grid is to your advantage [Future]
Standards are being defined now that will determine the future of this technology

27. Grid Vision, Marketing, and Reality
Computing & data resources can be shared like content on the Wb
Marketing
Have we got a [Data, compute, knowledge, information, desktop, PC, enterprise, cluster, …] Grid for you!
Reality
Commercial products mostly noninteroperable
Open source tools offer de facto standards, but are also far from a complete solution

28. For many different types of resource
Standards Matter!
Open, standard protocols
Enable interoperability
Avoid product/vendor lock-in
Enable innovation/competition on end points
Enable ubiquity
In Grid space, must address how we
Describe, discover, & access resources
Monitor, manage, & coordinate, resources
Account & charge for resources
For many different types of resource

29. Open Grid Services Architecture
Define a service-oriented architecture …
the key to effective virtualization
… that addresses vital “Grid” requirements
AKA utility, on-demand, system management, collaborative computing
in particular, distributed service management
… building on Web services standards
extending those standards where needed
“The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration”, Foster, Kesselman, Nick, Tuecke, 2002

30. Latest Step Forward: WS-Resource Framework
A family of six Web services specifications
A design pattern to specify how to use Web services to access “stateful” components
Message-based publish-subscribe to Web services
Groups
References
Notification
Faults
Properties
Lifetime
WS-Resource
Framework

31. WS-Renewable References
WSRF & the Web Services Stack
Service Composition
WS-Service Group
WS-Notification
BPEL4WS
Quality of
Experience
(QoX)
WS-Reliable Messaging
WS-Transaction
WS-Security
WS-Resource Lifetime
Description
WS-Resource Properties
WS-Base Faults
XSD
WSDL
WS-Policy
WS-Metadata Exchange
Messaging
XML
SOAP
WS-Addressing
WS-Renewable References
Transports
HTTP/HTTPS
SMTP
RMI / IIOP
JMS

32. WS-Resource Framework Completes Grid-WS Convergence
OGSI
GT2
GT1
HTTP
WSDL,
WS-*
WSDL 2,
WSDM
Have been
converging
Started far apart in apps & tech
WSRF
Web
The definition of WSRF means that Grid and Web
communities can move forward on a common base

33. OGSA Development Continues
Domain-Specific Services
Program
Execution
Data Services
Core Services
Open Grid Services Infrastructure
WS-Resource Framework
Web Services Messaging, Security, Etc.

34. OGSA Standards: Status
WSDL 1.2 in progress in W3C
OGSI 1.0 completed July 2003 in GGF
WSRF Jan 2004, OASIS submission soon
WS-Agreement draft in GGF
WS-Management drafts in OASIS
OGSA Data Access & Integration in GGF
WS-Security specifications in OASIS
SAML & XACML in OASIS
Other work in progress

35. Bringing it All Together
Scenario: Resource management & scheduling
Grid “Jobs” and “tasks” are also modeled using WS-Resources and
Resource Properties
Grid Scheduler is a
Web Service
Local processor manager
is “front-ended” with
A Web service interface
Other kinds of resources are also
“modeled” as WS-Resources J
Grid
Scheduler A
Service
Level
Notification
WS-Resource used to “model” physical processor resources
Service Level
Agreement
is modeled as a WS-Resource
Blades R
Cluster R
Storage R
WS-Notification can be used to “inform” the scheduler when processor utilization changes
Lifetime of SLA Resource tied to the duration of the agreement
WS-Resource Properties “project” processor status (like utilization)

36. Standards: Summary Grid and Web Services are merging
Grid is an aggressive use case of Web Services
WSRF completes common infrastructure
Web Services standards landscape is in flux
Uncertain status of security & policy standards continues to be a big source of concern
Grid services standards landscape heating up
Agreement, management, data access, …
W3C, OASIS, GGF are key standards orgs
Open source software important for adoption

37. OGSA Status: Implementations
Globus Toolkit v3: Linux for the Grid
Open source middleware, commercial support
A range of computation & data management, registry, and security functions
Some nice announced OGSI-based products
IBM, Avaki, Platform, Sun, NEC, HP, UD, Entropia, DataSynapse, Insors, Oracle, etc.
Read the fine print: “Intent is to use OGSI-based products,” “OGSA-compliant software,” “Embraces fundamental OGSA concepts”

38. Pop Quiz: The Grid Is …
A collaboration & resource sharing infrastructure for scientific applications
A distributed service integration and management technology
A disruptive technology that enables a virtualized, collaborative, distributed world
An open source technology & community
A marketing slogan
All of the above

39. Summary: Why You Should Care
1) Grid is a disruptive technology [Vision]
It ushers in a virtualized, collaborative, distributed world
2) Grid addresses pain points now [Reality]
Grids are built not bought, but are delivering real benefits in commercial settings
3) An open Grid is to your advantage [Future]
Standards are being defined now that will determine the future of this technology

40. To Learn More Background information
Deep technology & strategic information
Working, research groups
Three meetings a year
Weekly & web newsletter
Commercial conf & expo
May 24-26, Philadelphia
2nd Edition
Background
information
~foster