1. Building Science Gateways Marlon Pierce Community Grids Laboratory Indiana University

2. Slide URL

3. Concept #1: Web Portal Web container that aggregates content from multiple sources into a single display. o “Start Pages” Typically consume RSS/Atom news feeds. More powerful versions these days support Flickr, calendars, games, etc. o Gadgets, widgets Examples: iGoogle, Netvibes, My Yahoo!

4. Gadget RSS Feeds

5. Concept #2: Grid Computing Grid computing software is designed to integrate large supercomputing facilities. TeraGrid, Open Science Grid, EGEE, etc. This is done via network services Key Service Components Authentication and authorization framework (MyProxy) Remote process access and control (GRAM, Condor) Remote file, I/O access (GridFTP, SRB, RFT) Additional Services Information services, replica management, database federation, storage management, schedulers, etc. Example Grid Software Stacks: CTSS and VDT

7. Science Portals and Gateways Science Gateways adapt Web portal technology to build user interfaces to the Grid. Science portals resemble standard portals, but must also o Support access to computing and storage resources. o Allow users remote, Unix-like access to these resources. o Provide access to science applications and data sets. And we must provide value added services as well as user interfaces.

8. Example Science Gateways Many listed here: o http://www.teragrid.org/programs/sci_gateways/ http://www.teragrid.org/programs/sci_gateways/ Cover many different scientific fields: o Atmospheric science, geophysics, computational chemistry, bioinformatics, etc See also GCE07 and earlier proceedings o http://www.collab- ogce.org/gce07/index.php/Main_Page http://www.collab- ogce.org/gce07/index.php/Main_Page o GCE08 accepted workshop for Supercomputing 2008.

9. Portlets + Client Stubs DB Service JDBC DB Job Sub/Mon And File Services Operating and Queuing Systems WSDL Browser Interface WSDL Visualization Service DB WSDL Host 1Host 2Host 3 My 2002 “octopus” SOA diagram, from the archives. SOAP/HTTP HTTP(S) WSDL

10. Terminology Portlet: this is a standard Java component that generates HTML and can also act as a client to a remote service. o Lives in a portal container. o I will also use this term generically. Web Service: a remotely invokeable function on the Internet. o SOAP: the XML message envelop for carrying commands over HTTP. o WSDL: describes the service’s API in XML. o REST: A variation of this approach. Lots more info: http://grids.ucs.indiana.edu/ptliupages/prese ntations/I590WebService.ppt http://grids.ucs.indiana.edu/ptliupages/prese ntations/I590WebService.ppt

11. But Why? Three-tiered Service Oriented Architecture is the network equivalent of the the famous Model-View- Controller design pattern. o View: the user interface components. o Controller: Web service middleware o Model: the backend resources. Independence of tiers gives flexibility o Services can be reused with alternative user interfaces  Workflow composers like Taverna, Xbaya, Kepler o User interfaces can work with different service implementations. Drawback: reliability and robustness are issues.

12. Two Approaches to the Middle Tier Grid Service Backend Resource Web Service Portal Comp. Grid Client Backend Resource Fat ClientThin Client Grid Protocol (SOAP) Grid Client HTTP + SOAP Grid Protocol (SOAP)

13. Disloc output converted to KML and plotted.

14. GeoFEST Finite Element Modeling portlet and plotting tools

15. What’s In the Screenshots? GeoFEST and Disloc Portlets o Live on gf7.ucs.indiana.edu o Manage the user’s display: Web forms, links to output, graphics. o Save user session state persistently. QuakeTables Fault DB Web Service o Lives on gf2.ucs.indiana.edu o Contains geometric fault models. GeoFEST and Disloc Execution Web Services o Lives on gf19.ucs.indiana.edu o Generates input files from fault models. o Runs and manages codes.

16. Best Practice for Scientific Web Services There are many tools to choose from. o.NET, Apache Axis, Sun WS, Ruby on Rails, etc. Make them self-contained. o If possible, generate input files within the service. o Or have an input file generating service. o Remember that they may be used by other people with other client tools. Communicate data files with URLs. Be very careful about exposing the state of the service. o Don’t assume persistent connections.

17. Components for Portals Open Grid Computing Environments Examples. See http://www.collab- ogce.org/http://www.collab- ogce.org/

18. Components for Science Portals OGCE is founded on the principal that portals should be built out of reusable parts. Key standard in our first phase: the JSR 168 portlet specification. Portlets can run in multiple containers o uPortal, Sakai, GridSphere, LifeRay, etc. Allows us to build Grid specific components and deploy along side other goodies: Sakai collaboration tools, contributed portlets, etc. Future: Open Social compliant Google Gadgets

19. OGCE GPIR portlet can interoperate with TeraGrid and your own GPIR services.

20. Manage TeraGrid MyProxy credentials with the OGCE ProxyManager portlets.

21. OGCE file management client portlets interact with TeraGrid GridFTP servers.

22. General purpose batch and interactive job submission to GRAM, WS-GRAM is supported.

23. Dashboard Portlet 23 The dashboard portlet allows users to track jobs on the selected resource. The user can view either his own set of jobs or get information on all submitted jobs.

25. Queue forecasting portlets work with the NWS QBETS to predict wait times and deadlines.

26. PURSe portlets manage user requests for portal accounts and Grid credentials.

27. Condor and Condor-G

28. OGCE IFrame Portlet can be used to integrate external sites.

29. Client Libraries for Grid Computing

30. Two Major Grid Client Efforts The Java COG Kit o Supports several versions of Globus and SSH. Condor-G o Has a Web Service interface (BirdBath) and Java client libraries. o Supports Globus (v2 and v4) and several other Grid middleware systems. You can build either portlets or Web services with either of these. OGCE portlets use primarily COG We prefer Condor-G based Web services for long running jobs.

31. CoG Abstraction Layer CoG CoG Data and Task Management Layer CoG Gridfaces Layer CoG CoG GridIDE GT2 GT3 (X) GT4 WS-RF CondorUnicore Applications SSHOthers Nano materials Bio- Informatics Disaster Management Portals CoG Abstraction Layer CoG CoG Data and Task Management Layer CoG Gridfaces Layer CoG CoG GridIDE Development Support CoG Abstraction Layers

32. Task Handler Service Task Specification Security Context Service Contact The class diagram is the same for all grid tasks (running jobs, modifying files, moving data). Classes also abstract toolkit provider differences. You set these as parameters: GT2, GT4, etc.

33. Coupling CoG Tasks The COG abstractions also simplify creating coupled tasks. Tasks can be assembled into task graphs with dependencies. o “Do Task B after successful Task A” Graphs can be nested.

34. Grid Client Development Problems Grid portlets typically wrap each single Grid capability in a separate portlet Problem is that Grid portlets need to combine these operations o Portlets are entire web applications, so we need a component model for portlets: reusable portlet parts Even with the COG Abstraction Layer, we must still do a lot of coding to build new applications. To address these problems we have adopted Java Server Faces o Provides several nice Model-View-Controller features o JSF provides an extensible framework (tag libraries) for making reusable components. o Apache JSF portlet bridge allows you to convert standalone JSF applications (development phase) into portlets (deployment phase).

35. GTLAB JSF Example …. <o:myproxy id=”pr” hostname=”gf1.ucs.indiana.edu” port=”7512” lifetime=”2” username=“mnacar” password=”***” /> <o:jobsubmit id=”task” hostname=”cobalt.ncsa.teragrid.org” provider=”GT4” executable=”/bin/ls” stdout=”tmp/result stderr=”tmp/error” /> 35

36. Grid TagsAssociated Grid BeansFeatures ComponentBuilderBean Creating components, job handlers, submitting jobs MonitorBean Handling monitoring page actions MultitaskBean Constructing simple workflow MultitaskBean Defining dependencies among sub jobs MyproxyBean Retrieving myproxy credential FileOprationBean Providing Gridftp operations JobSubmitBean Providing GRAM job submissions FileTransferBean Providing Gridftp file transfer ResourceBean Describes common properties among all tags and beans. Passing values given by standard visual JSF components.

37. Managing Scientific Workflows

38. Scientific Workflows Portal interfaces encode scientific use cases. If you have a rich set of services, it is a lot of work to make portlets for all possible use cases.  And power users will have always want something more. Example: our CICC project has dozens of chemical informatics Web services. http://www.chembiogrid.org.wiki Workflow composers can simplify this. o Allow users to encode and execute their own use cases.

39. Web Services and Workflows Perform a similarity search on the NIH DTP Human Tumor data. Filter the results based on Pharmacokinetic properties (FILTER) Convert to 3D (OMEGA) Docking into a pre- defined protein (FRED) Visualize (JMOL). Taverna workflow connects remote services.

40. OGCE’s XBaya Workflow Composer

41. Future of Science Gateways Web 2.0, Clouds, and other Stuff

42. Social Gadgets+AJAX DB Service JDBC DB Job Sub/Mon And File Services Operating and Queuing Systems REST Browser Interface REST WSDL REST Visualization Service DB REST Host 1Host 2Host 3 Updating the Octopus RSS,JSON/HTTP HTTP(S) REST

43. Enterprise ApproachWeb 2.0 Approach JSR 168 PortletsGadgets, Widgets Server-side integration and processing AJAX, client-side integration and processing, JavaScript SOAPRSS, Atom, JSON WSDLREST (GET, PUT, DELETE, POST) Portlet ContainersOpen Social Containers (Orkut, LinkedIn, Shindig); Facebook; StartPages User Centric GatewaysSocial Networking Portals Workflow managers (Taverna, Kepler, etc) Mash-ups Grid computing: Globus, condor, etcCloud computing: Amazon WS Suite, Xen Virtualization Semantic Web: RDF, OWL, ontologies Microformats, folksonomies

44. Sample Grid Gadgets in iGoogle

45. Microformats, KML, and GeoRSS feeds used to deliver SAR data to multiple clients.

46. More Information Contact me: mpierce@cs.indiana.edumpierce@cs.indiana.edu See what I’m up to: http://communitygrids.blogspot.com/ http://communitygrids.blogspot.com/ OGCE software: http://collab-ogce.org/http://collab-ogce.org/ QuakeSim: http://www.quakesim.org/http://www.quakesim.org/ CICC: http://www.chembiogrid.org/wiki/http://www.chembiogrid.org/wiki/ Lots of people worked on all of these.