1. GCC 2006 Panel: Grid Research and Engineering Vs Standards Dr. Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Laboratory Dept. of Computer Science and Software Engineering The University of Melbourne, Australia www.gridbus.org www.gridbus.org Gridbus Sponsors

2. 2 Panel Questions Q1: How can research creativity be reconciled with the need for, and use of, standards and standard software? Q2: What are the most important challenges (and possible solutions) in managing in an integrated manner the middleware, data, users, and resources that make up a Grid? Q3: How can Grid-user/developers be provided with an integrated “on the Grid” development environment? Q4: What are the essential aspects of the “engineering science” needed for Grid systems that are interoperable, scalable, robust, sustainable, and maintainable?

3. 3 Q1: Research Creativity Vs Standards Standard are great as long as they are restricted to: Protocols Interfaces Formats Successful ICT standards: ASCII (languages), IEEE (devices), IETF (networking), W3C (web and web services), OGF (in progress, yet to settle) Yes, Research(ers) should embrace standard as long as purpose is served, if not, “invent” new one. If we push for a specific software as a standard, It creates “one world” rules all mentality! They can “kill” many creative works  leads to loss of diversity. So called de-facto standards make extremely hard for creative ideas to get in.

4. 4 What does Grid promises Resource sharing across multiple administrative boundaries Effective utilisation of existing resources Dynamic provisioning Application Acceleration Scalability Reliability Virtualisation: applications, services, resources,…

5. 5 E.g., The Use of Grid as a Cyberinfrastructure for e-Science Distributed instruments Distributed computation Distributed data Peers sharing ideas and collaborative interpretation of data/results 2100 2100 2100 Remote Visualization Data & Compute Service Cyberinfrastructure E-Scientist

6. 6 Q2: Multi-institution Collaboration Challenges Security Resource Allocation & Scheduling Data locality Network Management System Management Resource Discovery Uniform Access Computational Economy Application Construction

7. 7 Why are they challenging ? : Grid characteristics, Sources of Resource Management Challenges Numerous resources Different security requirements & policies Resources are heterogeneous Geographically distributed Different resource management policies Connected by heterogeneous, multi-level networks Owned by multiple organizations & individuals Unreliable resources and environments Slide figure by Hiro

8. 8 Some Open-Source Grid Middleware Solutions

9. 9 What does Grid players want? Grid Consumers Execute jobs for solving varying problem size and complexity Benefit by utilizing distributed resources wisely Tradeoff timeframe and cost Strategy: minimise expenses Grid Providers Contribute resources for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements & market opportunity Strategy: maximise return on investment

10. 10 Solution 1: Service Oriented Architecture (SOA) A SOA is a contractual architecture for offering and consuming software as services. There are four entities that make up an SOA service provider, service registry, and service consumer (also known as service requestor). The functions or tasks that the service provider offers, along with other functional and technical information required for consumption, are defined in the service definition or contract. provider registry consumer contract

11. 11 Solution 2: Market-Oriented Grid Computing - (a) Sustained Resourced Sharing and (b) Effective Management of Shared Resources Grid Economy

12. 12 The Gridbus Project @ Melbourne: Enable Leasing of ICT Services on Demand WWG Pushes Grid computing into mainstream computing Gridbus

13. 13

14. 14 Q3 & Q4: “on the grid” app. dev. Environment and “Engineering Science” Interoperability: Do use standards Scalable Decentralised management, make sure “application” scale from “desktop” to “global grids” Robust Fault-management, persistence, state-saving (like Windows!)… Sustainable and Maintainable Make a clear separation between different concerns E.g: application programming interface must be independent of software used for deploying and executing “user” abstract services, resources, “device independent” “applications” and “execution management/scheduling engine”, low-level middleware. Support “old” and “new” standard at the same time!

15. 15 Gridbus Broker: Separating “applications” from “different” remote service access enablers and schedulers Alchemi Gateway UnicoreData Store Access Technology Grid FTP SRB -PBS -Condor -SGE Globus Job manager fork()batch() Gridbus agent Data Catalog -PBS -Condor -SGE -XGrid SSH fork() batch() Gridbus agent Single-sign on security Home Node/Portal Gridbus Broker fork() batch() -PBS -Condor -SGE -Alchemi -XGrid Application Development Interface Scheduling Interfaces Alogorithm1 AlogorithmN Plugin Actuators

16. InterGrid: Internetworking of Islands of Grids Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Laboratory Dept. of Computer Science and Software Engineering The University of Melbourne, Australia www.gridbus.org/gridsim 5th International Conference on Grid and Cooperative Computing Changsha, China, October 2006

17. 17 The Outcomes of Grid Computing thus Far: Islands of Grids

18. 18 InterGrid: An architecture for Internetworking of “islands” of Grids

19. 19 The InterGrid: How to get there?

20. 20 New Research Challenges Decentralised “service” discovery from gateways-based to “leaf” nodes Protocols and Business models for “Peering” between Grids Application models that support growing and shrinking of application Resource management and scheduling SLA based resource allocation.

21. 21 Thanks for your attention! We Welcome Cooperation in Research and Development! http:/www.gridbus.org

22. 22 Layers of Grid Architecture Grid resources Desktops, servers, clusters, networks, applications, storage, devices + resource manager + monitor Security Services Authentication, Single sign-on, secure communication Job submission, info services, Storage access, Trading, Accounting, License Resource management and scheduling Grid programming environment and tools Languages, API, libraries, compilers, parallelization tools Grid applications Web Portals, Applications, Adaptive Management Application Development and Deployment Environment Distributed Resources Coupling Services Core Middleware User-Level Middleware System level User level Autonomic/ Grid Economy