Overview of Grid Computing Environments Proposed GGF Information Document G.Fox, D. Gannon, M. Pierce, M. Thomas PTLIU Laboratory for Community Grids Geoffrey Fox Computer Science, Informatics, Physics Indiana University, Bloomington IN

Source of Report Last Half of 2001: Call from GCE RG for Papers for Journal special issues – 28 papers submitted and reviewed Published in Concurrency and Computation: Practice and Experience Vol. 14, Grid Computing Environments Special Issue 13-15, Fall Augmented by chapters (about 14) in Fran Berman, Geoffrey Fox and Tony Hey, ‘Grid Computing: Making the Global Infrastructure a Reality’, ISBN , John Wiley & Sons Ltd, Chichester, See Other key papers which we knew about

Raw (HPC) Resources Database Aggregation Portal System Services Application Service Portal Services Portal Services Grid Computing Environments User Services “Core” Grid Application Service OGSA (OGSI) Interfaces The fuzzy definition of Grid Computing Environments

Two Major Areas “Programming the User Side of the Grid” which involves discussing computing model for Grid Controlling user interaction – rendering any output and allowing user input in some (web) page. –This includes aggregation of multiple data sources in a single portal page (Jetspeed). –Has natural overlaps with information (commercial) portals

Classification of GCE Papers in Programming the User Side of the Grid Technology for building GCE systems -Interface with backend Infrastructure e.g. Community Grid Kits, GPDK Problem Solving Environments –Domain specific collection of tools and user interface. E.g. XCAT, Polder, SCIRun, Astrophysics Collaboratory GCE Tools –Support parameter sweep, visualization, job status, files, security, workflow.. GCE Shell Portals providing a general interface to many Grid capabilities –Analogous (not usually command line) to role of UNIX shell providing access to UNIX tools and user programs, files … –Note UNIX has core system and higher level tools accessed by Shell –E.g. Unicore, Hotpage, Mississippi Portal –PSE’s often built on top of GCEShell portals

Aspects of Programming Model I See review of programming model by GGF APM RG –Chapter 21 of Wiley Book and web page Handling of the basic components of a distributed computing system – files, computing and data resources, programs, and accounts. –The GCE will typically interface with an environment like Globus or a batch scheduler like PBS to actually handle the back-end resources. –However the GCE will present the user interfaces to handle these resources. We can follow the lead of UNIX (and Legion) and define a basic GCEShell providing access to the core distributed computing functions. –JXTA also builds some Grid-like capabilities with a modification of UNIX shell model. –GCEShell can have a command line or more visually appealing graphical user interface.

Implications of 3-Tier Model The 3-tier model implies that any given capability (say run a matrix inversion program) can appear at multiple levels. –Maybe there is a backend parallel computer running an MPI job; this is front-ended perhaps as a service by some middle-tier component running on a totally different computer, which could even be in a different security domain. –One can “interact” with this service at either level; a high performance I/O transfer at the parallel computing level and/or by a slower middle-tier protocol like SOAP at the service level. –These two (or more) calls (component interactions) can represent different functions or the middle tier call can be coupled with a high performance mirror. Typically the middle tier provides control and the back end “raw data transfer”.

Raw (HPC) Resources Database Aggregation Portal System Services Application Service Portal Services Portal Services Grid Computing Environments User Services “Core” Grid Application Service Application Metadata Actual Application OGSA (OGSI) Interfaces The fuzzy definition of Grid Computing Environments

Technology for building GCE Systems CoG Kits for Java Perl Python CORBA …. GPDK extending CoG kits with JavaBean and JSP middleware Grid Portal Toolkit Other such interfaces with also C and XML tools Event Service Simulations

GCEShell Portals and PSE’s One can divide GCE capabilities into generic (copy file) or domain specific (generate the multigrid solver suitable to simulate Middle Earth) Correspondingly one finds portal frameworks or GCEShell Portals presenting general capabilities Grid or web-based Problem Solving Environments optimized for a particular domain –PSE’s are often (and perhaps should be) built on top of GCFShell Portals

Typical Layered Architecture GCEShell Portals Globus GT2 Services Manage backend resources Interface Middleware with backend resources Core Middleware Services Application Services Problem Solving Environments User Interface-Client Aggregate component Interfaces Jetspeed

GCE Tools Data Management –File manipulation in all tiers including client, middleware and back end Security –In all tiers and providing interfaces to core Grid Security Workflow or “Programming the Grid” Grid versions of MPI Higher-level tools include visualization or support of parameter sweep applications –UNIX Shell has mix of lower level and higher level tools

More on Programming Models Basic 2-level programming model is assumed by most projects –First you use classic (parallel) programming to produce “simulation” nuggets (wrapped as application web services) –JDBC / OGSA-DAI to package data resources as objects or services Then you need to compose (orchestrate) the control and dataflow between nuggets –Many different models for how this is to be done and can call this workflow – Next thrust of GCE RG? Some work on optimization between levels as in GrADS project

Research in “2-level” Programming Models Basic user interface to middle tier proxies controlling backend (software) resources Component Models like ICENI (Imperial College) or DoE Common Component Architecture Commercial workflow engines as in BPEL4WS Scripting front-ends as in Matlab or Python Network server model as in NetSolve or Ninf Computational Economies Semantic Grid technologies (ontology based integration of resources) as in MyGrid or DiscoveryNet Agents?

Programming Infrastructure (Hosting Environment) Different approachs assume different run-time (implementation) and user programming “ether” BPEL4WS thinks about specifying interactions between components; Matlab interface invokes components from a script Java Interface to OGSI WG emphasizes that we don’t have an established implementation even if you pick a language –Jini –JXTA –Servlet –Enterprise JavaBeans EJB –Or perhaps some future SJB (Scientific JavaBean) supporting high volume dataflow better than EJB (short transactions)

Web Services as a Portlet Each Web Service naturally has a user interface specified as “just another port” –Customizable for universal access This gives each Web Service a Portlet view specified (in XML as always) by WSRP (Web services for Remote Portals) So component model for resources “automatically” gives a component model for user interfaces –When you build your application, you define portlet at same time Application or Content source WSDL Web Service S R W P Application as a WS General Application Ports Interface with other Web Services User Face of Web Service WSRP Ports define WS as a Portlet Web Services have other ports (Grid Service) to be OGSI compliant

Online Knowledge Center built from Portlets Web Services provide a component model for the middleware (see large “common component architecture” effort in Dept. of Energy) Should match each WSDL component with a corresponding user interface component Thus one “must use” a component model for the portal with again an XML specification (portalML) of portal component A set of UI Components

Content Provider WSDL Web Service F I U O F I R O Portal Aggregate WS-User Facing Fragments Render Other WS User Facing Ports Other WS Resource or Service-facing Ports User-facing Ports User Facing Ports for Web Service

User Profile Application or Content source WSDL Web Service F I U O F I R O Render Portal (Aggregator) Selector Filter Control Channel Customized View Selection View Control Channel Customized View Customized User-Facing Ports As used in Universal Access

Portlet XML RSS, OCS, or other Local or remote HTML Local files JSP or VM Local templates WebPage Remote HTML Portlet Portlets User implemented using Portal API Portlets Data PortletController Screen Manager HTML PSML PortletControl ECS JSP template ECS ECS Root to HTML ECS Turbine Servlet Jetspeed Architecture

Portlets and Portal Stacks User interfaces to Portal services (Code Submission, Job Monitoring, File Management for Host X) are all managed as portlets. Users, administrators can customize their portal interfaces to just precisely the services they want. Core Grid Services User facing Web Service Ports Application Grid Web Services Aggregation Portals (Jetspeed) Message Security, Information Services

Jetspeed Computing Portal: Choose Portlets 4 available portlets linking to Web Services I choose two

Choose Portlet Layout Choose 1-column Layout Original 2-column Layout

Lists user files on selected host, noahsark. File operations include Upload, download, Copy, rename, crossload Tabs indicate available portlet interfaces. File management

Sample page with several portlets: proxy credential manager, submission, monitoring

Provide information about application and host parameters Select application to edit Administer Grid Portal