Support for Automatic Workflow Composition in Semantic Grid Environemnt Tomasz Gubała, Marian Bubak, Maciej Malawski Institute of Computer Science and Cyfronet, AGH University of Science and Technology, Kraków, Poland

5 th Cracow Grid Workshop, CGW 2005, Kraków Grid Workflows  Grid workflows Provide a method to (de)compose applications Provide a method to (de)compose applications Naturally support parallel computations Naturally support parallel computations Allow to program grid through specifying inter- component dependencies Allow to program grid through specifying inter- component dependencies  Multiple levels of abstraction in workflows Separate execution details from application logic Separate execution details from application logic Allow for workflow reusability (the application is able to overcome Grid dynamic nature) Allow for workflow reusability (the application is able to overcome Grid dynamic nature) Ease the application execution process as the user does not has to learn all the midleware-specific details Ease the application execution process as the user does not has to learn all the midleware-specific details

5 th Cracow Grid Workshop, CGW 2005, Kraków Motivation  Motivation (Grid Invisibility) It is sometimes not trivial to compose a new application manually (even using nice graph-based widgets) It is sometimes not trivial to compose a new application manually (even using nice graph-based widgets) Users not always realize the actual needs (requirements) of a certain computation Users not always realize the actual needs (requirements) of a certain computation It is difficult task to choose (using a browser) among hundreds of grid resources dispersed worldwide It is difficult task to choose (using a browser) among hundreds of grid resources dispersed worldwide  Use scenario Specify what you need as a result choosing a concept Specify what you need as a result choosing a concept Machine builds several possible solutions as workflow Machine builds several possible solutions as workflow Choose one with an aid of a user assistant tool Choose one with an aid of a user assistant tool

5 th Cracow Grid Workshop, CGW 2005, Kraków Solution – Step 1  The user wants a meteo application execution  The application is supposed to provide weather prediction data  The user chooses WeatherData concept using User Assistant Agent  The basic workflow skeleton (see right) is generated WeatherData GridJob Begin

5 th Cracow Grid Workshop, CGW 2005, Kraków Solution – Step 2 The tool identifies a single data dependency The tool identifies a single data dependency It contacts a registry to find a class of services being capable of delivering WeatherData It contacts a registry to find a class of services being capable of delivering WeatherData Newly found building block is inserted into the workflow, but... Newly found building block is inserted into the workflow, but......it generates new dependencies...it generates new dependencies MM5Boundary Conditions WeatherData MM5.calc Topological DataUploaded

5 th Cracow Grid Workshop, CGW 2005, Kraków Solution – Step 3 Using follow-up technique the tool constructs the entire workflow step-by-step Using follow-up technique the tool constructs the entire workflow step-by-step Support for both control and data flow constructs Support for both control and data flow constructs Several patterns supported: Several patterns supported:  XOR-merge  AND-merge  AND-split  loop

5 th Cracow Grid Workshop, CGW 2005, Kraków Result  As a result the newly created workflow supports parallel execution, supports parallel execution, is a valid PetriNet workflow in a specified language, is a valid PetriNet workflow in a specified language, is abstract and non-executable, is abstract and non-executable,  the tool operates on abstract notion of service class operations  it needs a specific mapper, another tool that concretizes it may be reused many times, may be reused many times,  as there may be many different instances of certain type of service still includes some red parts where a user has to supply important data. still includes some red parts where a user has to supply important data.

5 th Cracow Grid Workshop, CGW 2005, Kraków Semantic Service Discovery  IOPE technique IOPE = (input, output, precondition, effect) IOPE = (input, output, precondition, effect)  Every data request sets several constraints Usually they constrain content, format and storing mechanism Usually they constrain content, format and storing mechanism  Every service operation defines its input and output parameters as data templates With similar constraints on content, format and storage place With similar constraints on content, format and storage place

5 th Cracow Grid Workshop, CGW 2005, Kraków Templates and Instances  Data template  Describes an abstract class of data (all the data that fits)  Data instance  Metadata description of real piece of data stored somewhere Data Template content constraint What is the meaning of data content format constraint storage constraint Data Instance contains has format is stored in What is the format the data has Where is the data stored

5 th Cracow Grid Workshop, CGW 2005, Kraków Comparison  Comparison guidelines  A certain operation output fits perfectly if it produces a data template with equal or more restrictive constraints then needed  Inexact match  When fits, but  is different (we need an additional transformation activity in between), and/or  is different (we need an additional relocation activity in between)  Transformation/relocation  generic operation changing format or storage properties of data and not concerned about the content of the data  E.g..tex to.html converter or file-from-url retriever content constraint format constraint storage constraint

5 th Cracow Grid Workshop, CGW 2005, Kraków Current State, Future Work  First prototype of the workflow compositor to be finished this winter  Stable release due to end of 2006 Better matchmaking capabilities based on more sophisticated data/service descriptions Better matchmaking capabilities based on more sophisticated data/service descriptions Some more workflow patterns properly supported Some more workflow patterns properly supported To be deployed as a standalone service (is just a.jar now) To be deployed as a standalone service (is just a.jar now) Possibility of employing semantic workflow comparison algorithms in order to reuse similar solution constructed so far Possibility of employing semantic workflow comparison algorithms in order to reuse similar solution constructed so far

5 th Cracow Grid Workshop, CGW 2005, Kraków Thank you very much for your attention