State of Service Oriented Science Tools Open Source Grid Cluster Conference Oakland

Agenda l Presentation l Demo l Questions

Service-Oriented Science People create services (data or functions) … which I discover (& decide whether to use) … & compose to create a new function... & then publish as a new service.  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!  I hope that this “someone else” can manage security, reliability, scalability, … !! “Service-Oriented Science”, Science, 2005

Creating Services Introduce + gRAVI Shannon Hastings Scott Oster David Ervin Stephen Langella Kyle Chard Ravi Madduri

Introduce Overview l A framework which enables fast and easy creation of Globus based grid services l Provide easy to use graphical service authoring tool. l Hide all “grid-ness” from the developer l Utilize best practice layered grid service architecture l Integration with other core grid services and architecture components u GAARDS Security Infrastructure (Dorian, GridGrouper, CSM) u Globus Index Service u Global Model Exchange (GME) u Cancer Data Standards Repository l Extension Framework for integrating with other architecture components

Inside the Introduce created service l Services have many moving and configurable parts which support features such as: u Advertisement u Discovery u Invocation u Security (Authentication/Authorization) u Stateful Resources l The Introduce Toolkit can keep all these features in sync as the developer creates and modifies the grid service

Introduce Features l Supports modification of operations u Adding operations u Removing Operations u Updating Operations u Importing Operations l Graphical Configuration u Advertisement u Security u Service Metadata Specification u Service Metadata Editing u Service Configuration Properties l Auto Generates Code for Service l Auto generates a client API for service. l Graphical Deployment of Service u Globus u Tomcat u JBoss

Created Skeleton Layout = introduce managed = globus/axis managed = developer implements

Appln Service Create Index service Store Repository Service Advertize Discover Invoke; get results Introduce Container Transfer GAR Deploy gRAVI l Grid Remote Application Virtualization Interface l Builds on Introduce u Define service u Create skeleton u Discover types u Add operations u Configure security l Wrap arbitrary executables

Discovering Services People create services (data or functions) … which I discover (& decide whether to use) … & compose to create a new function... & then publish as a new service.  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!  I hope that this “someone else” can manage security, reliability, scalability, … !! “Service-Oriented Science”, Science, 2005

Discovering Services WS-MDS+ Taverna Laura Pearlman Mike Darcy myGrid Team

 The ultimate arbiter?  Types, ontologies  Can I use it?  Billions of services Discovering Services Assume success Semantics Permissions Reputation A B

Discovery (1): Registries Globus

Discovery (2): Standardized Vocabularies Core Services Grid Service Uses Terminology Described In Cancer Data Standards Repository Enterprise Vocabulary Services References Objects Defined in Service Metadata Publishes Subscribes to and Aggregates Queries Service Metadata Aggregated In Registers To Discovery Client API Index Service Globus

Composing Services People create services (data or functions) … which I discover (& decide whether to use) … & compose to create a new function... & then publish as a new service.  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!  I hope that this “someone else” can manage security, reliability, scalability, … !! “Service-Oriented Science”, Science, 2005

Composing Services Taverna + GT4 Taverna team Wei Tan Ravi Madduri

Composing Services Globus

Hosting Services People create services (data or functions) … which I discover (& decide whether to use) … & compose to create a new function... & then publish as a new service.  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!  I hope that this “someone else” can manage security, reliability, scalability, … !! “Service-Oriented Science”, Science, 2005

Provisioning Services WS-GRAM + VWS Martin Feller Stuart Martin Kate Keahey Tim Freeman Joshua Boverhof

Provisioning using WS-GRAM l gRAVI uses JSDL for Application Description l Generates a method on the generated service called GRAM l Generates implementation that creates a GramJob from Application Description l Uses the bootstrapped community credential to run the application as a grid job l Used widely in realizing the usecases from caBIG community

Invoke; get results Container With Application Service gRAVI & WS-GRAM

Repository Service Using VWS/Cloud Computing l gRAVI service u Wrap the application as Service u Create the GAR and put in repository l Provision resources u Use clouds (VMM) l start service u Transfer gar, deploy l Index Service u Grid service registers itself nimbusstratus Index service Transfer, deploy Register service Create VM provision discovery portal GAR

gravi portlets stratusnimbus Index service VM Repository Service GAR WAR Portal register service Invoke executable

Cloud Computing

User Communities and Early Adopters

Advanced Photon Source l APS Requirements align well with goals of SOS l They have data, analytics and workflows l We wrapped one of the 2 applications as a service l Goal is to create and run a workflow with services generated using gRAVI and provision the workflow using Nimbus

Cancer Bio-informatics Grid l Primary usecase for Service Oriented Science l Demo at the Architecture F2F meeting u geWorkbench from Columbia University u Hierarchical Clustering Service u gRAVI  WS-GRAM on TeraGrid l Integrating into caGrid Tools l Taverna Integration for composition u Use gRAVI to wrap more popular services into Grid Services

caGrid Service

TeraGrid Aware caGrid Service

Creating the Gateway Service l Manually stage the binary (jar file) on TeraGrid u Takes in.ser files as input u Produces results also in a.ser file l Used the gRAVI plugin for Introduce to create the gateway service u l Gateway gridFTPs input data and parameters from geWorkbench to TeraGrid u geWorkbench passes input to the gateway in geWorkbench’s native format (caDSR compliant) u Gateway serializes the input before gridFTPing to TeraGrid l Gateway invokes the staged binary l Gateway gridFTPs results back to geWorkbench u Gateway deserializes the result file u Gateway returns results to geWorkbench in its native format l Gateway service is a secured caGrid service which in turn invokes TeraGrid with a caBIG community account

Steps to establishing geWorkbench/caGrid/TeraGrid Interface

caGrid Security (GTS, Grid Grouper, Dorian, CDS) php?title=GAARDS:Main

Future Work (Near Term) l Plan for integrating Virtual Workspace services as a provisioning mechanism u Details of the Plan in the following slides l caBIG demo in June 2008 using both WS- GRAM and VWS as backends l Google Summer of Code Projects – AJAX front-end to gRAVI and Service Composition l Two summer students

Future Work l gRAVI portlets u Plugin also creates a grid portlet u interface to the generated client u Or simple portal exposing the WSDL operations l Publish Services & Portlets u Add window for publishing via Introduce u Publish gRAVI services to a Repository u Entry contain service (GAR) and portlet (WAR) u Metadata describing resource requirements, etc l Use a Service from Portal u User selects the service(s) from Repository u Identify or Create resources (VM) based on metadata u Deploy service(s) u Discover service(s) u User access the service through portlet or client.

Success stories from Early Adopters l From Brian Tieman at APS: I was able to muddle through and get it working. Total time spent ~2 hours. I'm happy with that, and I'm sure future applications will go even faster! I've only run the sample client and I'm not trying to pass arguments yet so there's still a bit more to do before I can try integrating this into my application, but so far so good! l From Paul Burkhardt at NCI Inc : We used the gRAVI extension to the Introduce Grid Service Authoring Toolkit to demonstrate the ease of deploying arbitrary executables as Grid Services and GRAM jobs. Migrating legacy applications to a Web Services model presents an imposing hurdle to adoption. Many important legacy applications would require a significant amount of man-hours to rewrite as Grid Services. An alternative is to create JNI interfaces to wrap the executables but this approach still requires effort and expertise to create the necessary WSDL and corresponding Java implementation. This development cost can be prohibitive. The gRAVI tool has provided us the solution to low-cost and rapid deployment of legacy applications for Grid infrastructures.