Holding slide prior to starting show

Grid Projects at WeSC: Synergies and Opportunities David W. Walker School of Computer Science Cardiff University

26 November 2003WeSC Seminar3 Overview of Activities Software development of middleware, tools, and problem-solving environments. –Triana – UDDIe– JACAW –SWFL/JISGA – G-QoSM– MEDLI Funded research projects (RC&EU): –GSiB – WOSE– BD-World –GridLab – PASOA– e-HPTX –GridOneD – GENSS Five collaborative industrial projects (DTI). Also work in patterns and operators, performance and evaluation, semantic web technologies, and Grid economies.

26 November 2003WeSC Seminar4 Collaborative Industrial Projects Constraint-Oriented Negotiation in an Open Information Services Environment (CONOISE-G). Alex Gray Collaborative Virtual Teams (COVITE). Alex Gray and John Miles Environment for Industrial Design Optimisation (DIPSO). Omer Rana Grid-Enables Computational Electromagnetics (GECEM). David Walker Resource-Aware Visualisation Environment (RAVE). David Walker

26 November 2003WeSC Seminar5 Staff Researchers in COMSC Nick Avis: Medical imaging, collaborative visualisation. Alex Gray, Andrew Jones, Jianhua Shao: Bio- Informatics, information/knowledge management Yan Huang: Jini-based Grid middleware, workflow description, composition, deployment and enactment Omer Rana: QoS frameworks, provenance and metadata issues, agent technologies Ian Taylor: APIs for Grid computing, workflow, composition. David Walker: PSEs/portals, workflow, visualisation and Grid applications.

26 November 2003WeSC Seminar6 Other Cardiff Researchers Sathyaprakash (PHYSX) Peter Kille (BIOSC) John Miles (ENGIN) Also interest in PSYCH,ENCAP, ARCHI, EARTH, and UWCM.

26 November 2003WeSC Seminar7 Grid Middleware Lightweight service-oriented architecture for grids based on Jini and P2P technologies. Workflow tools and description languages Grid execution environments Quality of service frameworks Provenance and other metadata issues Collaborative visualisation and collaborative working

26 November 2003WeSC Seminar8 Web Services Everything is a (Web/Grid) service. This includes: –Computation routines –Access to files and databases –Components of the Grid infrastructure, such as workflow enactment engines, resource monitors, etc.

26 November 2003WeSC Seminar9 A Common Approach to Workflow Visual service composition. Service interfaces and other metadata expressed in an XML-based service description document. Services registered with, and discovered through, a registry.

26 November 2003WeSC Seminar10 Workflow Within an SOA Workflow description –SWFL Visual composition of services –Triana Aggregation –Higher level services and applications PSE (or portal) for deploying. Managing,and monitoring services, applications and grid resources. –GSiB

26 November 2003WeSC Seminar11 Workflow in GSiB and JISGA Visual Service Composition Environment Validator Create SWFL Drawing Screen swfl Job swfl Job Workflow Engine Workflow Engine Execution Environment

26 November 2003WeSC Seminar12 SWFL2Java It converts the description of a job in SWFL into executable Java code. SWFL Description Document Java Executable Code Dataflow Graph ControlFlow Graph FlowModel

26 November 2003WeSC Seminar13 JISGA JISGA consists of two main parts: –A WorkflowEngine service –A JobProcessor service Grid application is submitted to a WorkflowEngine service as SWFL. Sequential jobs are handled directly by the WorkflowEngine service. Parallel jobs involve multiple JobProcessor services.

26 November 2003WeSC Seminar14 What the Workflow Engine Is Processes service-based applications described in an XML-based workflow language. Determines the order of execution and generates the harness code. Executes the code which includes discovering services, invoking services, receiving results and sending results to clients.

26 November 2003WeSC Seminar15 What the Workflow Engine Does Converts a SWFL description of a composite service-based job into an executable Java code, and executes it. Java Executable Code <JFlowModel …………… …………. ………. …. SWFL description Intermediate FlowModel object SWFL2Graph Graph2Java SWFL2Java To be continued…

26 November 2003WeSC Seminar16 WOSE Workflow Optimisation Services for e-Science Applications. Middleware Open Call. Collaboration with Imperial College and Daresbury Lab. £400k, 2 years, one postdoc at each site Status: –Advertising for postdocs –1 Dec 2003 start date

26 November 2003WeSC Seminar17 WOSE Overview Draws together JISGA and Triana work at CU, with ICENI at IC, and portal expertise at DL. Topics addressed –Service aggregation and deployment –Runtime discovery and late binding of services –Service discovery and selection from multiple semantically equivalent services

26 November 2003WeSC Seminar18 Quality of Service Framework Service discovery using QoS properties Guarantees QoS at the application, middlware and resource levels (similar to DiffServ), and establishes Service Level Agreements Support for QoS adaptation Implementation using GARA/DSRT, NRM/Diffserv BB, and UDDIe UDDIe supports the description of a service through service properties, and service discovery based on these properties.

26 November 2003WeSC Seminar19 Provenance and Metadata Important in many middleware and application projects. Two main middleware projects –PASOA: Provenance-Aware Service-Oriented Architecture –GENSS: Grid-Enabled Numerical and Symbolic Services Also key in BD-World application project.

26 November 2003WeSC Seminar20 PASOA Provenance-Aware Service-Oriented Architecture. Fundamental Computer Science for e- Science call. Collaboration with Southampton Univ. £443k, 3 years, one postdoc and student each. Status: –Recently funded –Aiming for 1 Feb 2004 start date.

26 November 2003WeSC Seminar21 PASOA outline Execution and service provenance in relation to workflow enactment. Algorithms to reason over provenance data,to help scientists to achieve better utilisation of Grid resources for their specific tasks. Generating provenance data in workflow enactment. Properties that can be deduced from provenance-based data. Prototype that supports provenance generation and reasoning in Grid environments.

26 November 2003WeSC Seminar22 Collaborative Visualisation Central to two joint industrial projects –RAVE: Resource-Aware Visualisation Environment. –GECEM: Grid-Enabled Computational Electromagnetics.

26 November 2003WeSC Seminar23 RAVE Project Resource-Aware Visualisation Environment Status: –Started 1 April –Collaboration agreement in place. Partners: SGI and ORNL Duration: 3 years Partner contribution: £150,000 (SGI) EPSRC/DTI contribution: £186,534 Staff: Dr Ian Grimstead hired as postdoc.

26 November 2003WeSC Seminar24 RAVE Overview Aims to develop a collaborative visualization environment that scales across a wide range of network-enabled devices. Will respond to changes in network bandwidth and capabilities of the target display device. Will start by examining VizServer and COVISE systems

26 November 2003WeSC Seminar25 GECEM Project Grid-Enabled Computational Electromagnetics Status: –Start date 1 May –Collaboration agreement not in place yet Partners: Swansea University, BAE Systems, Hewlett- Packard, and Singapore Institute of High Performance Computing Duration: 2 years Partner net contributions: £113,750 (BAE Systems), £113,750 Hewlett-Packard EPSRC/DTI contribution: £227,500 Staff: Postdoc in place at Swansea; waiting for work permit for CU postdoc

26 November 2003WeSC Seminar26 GECEM Overview Aims to use and develop Grid technology as an enabler of large-scale and globally-distributed scientific and engineering research. The focus of the project will be collaborative numerical simulation and visualisation between the UK and Singapore.

26 November 2003WeSC Seminar27 Two Hard Problems Semantic specification of applications Scheduling of workflow nodes on distributed resources. –Early binding model: bind to specific service/platform at composition time (“validation”). –Intermediate binding model: bind at “compile” time (when converting from XML to executable form). –Late binding model: bind dynamically at runtime. Later binding allows the use of more up-to- date information to make scheduling decisions.

26 November 2003WeSC Seminar28 Key Research Problems Semantic specification of applications Scheduling of workflow nodes on distributed resources. –Early binding model: bind to specific service/platform at composition time (“validation”). –Intermediate binding model: bind at “compile” time (when converting from XML to executable form). –Late binding model: bind dynamically at runtime. Later binding allows the use of more up-to-date information to make scheduling decisions. How to deal with “volatile” services Also need to discuss limitations of workflow approach.

26 November 2003WeSC Seminar29 A WeSC Service Repository We need to create a service repository for the publishing and discovering services created in WeSC projects. Initially based on UDDI 3.0. Will allow projects to make use of and experiment with services developed by other projects.

26 November 2003WeSC Seminar30 Summary of Activities Lightweight Grids. Visual Service Composition Environment for creating services and applications based on workflow Workflow enactment and execution environments. Collaborative visualisation. Quality-of-Service. Provenance and metadata.

26 November 2003WeSC Seminar31