Open Grid Computing Environments Tutorial Marlon Pierce, Suresh Marru, Gopi Kandaswamy, Gregor von Laszewski, and Tom Scavo

Tutorial Overview Part 1: OGCE Portal (Marlon) o Introduction and overview o Portal Demo o How to build Grid portlets and gadgets Part 2: OGCE Workflow Tools (Suresh and Gopi) o Part A: Demo and overview o Part B: Hands on demo, deploying new applications Part 3: Web 2.0 for Grid Portals (Gregor) Part 4: GridShib and Gateway security (Tom)

Goals of Tutorial Goals o Review portal and gateway technologies old and new. o Demonstrate how to execute workflows and wrap TeraGrid applications. o Discuss security o Hear from TG Gateway community: what do you want? We have a lot of material to cover so we will sample many things. o You will have enough information to get going as a homework assignment. o Please also check with us at TG08 Please see Nancy’s “Gateway Parade” this afternoon.

Slides and Demo Site Tutorial slides are available from We run a permanent demo portal at o Also aliased as Demo services are also available from Community o [NEED URLS] Accounts train30-train49 have been created for the workshop. Password is the same as the account name. We also have training accounts with same names that can be used to retrieve TG proxy credentials.

Portal and Gadget Demos Demonstration of basic capabilities

Getting the Code

What You Can Download OGCE Portal o Various portlet samplers Web and Grid Services o XRegistry: Web Service Registry o GFAC: Application Factory Service o Grid Portal Information Repository Programming Libraries o Java and JavaScript COG o Grid Tag Libraries and Beans (GTLAB) o Information Service Client APIs

Project Release Schedule We have two major releases per year with intermediate updates. o TeraGrid (June) o Supercomputing (November) Focus of this release cycle: the workflow suite tools shown in Part 2. We do not currently package services and portlets together. o These are separate downloads.

Project Build and Test Source Code Version Control and Building: We use Maven 2 and SourceForge’s SVN o Anonymous checkouts o We are Open Source Nightly Build Testing: We use the NMI Build and Test facility at UW o Mac and Linux flavors System Testing: We use Apache JMeter to verify portal builds function correctly. o You can download this from us o You can also view the nightly dashboard

Project Information Web site: Announcement Blog: ogce.blogspot.comhttp://collab- ogce.blogspot.com RSS/Atom Feeds: ogce.blogspot.com/atom.xmlhttp://collab- ogce.blogspot.com/atom.xml o This is also embedded in the wiki as “News” Contact us:

Building the Portal

Choosing a Host Computer Select your host computer. o You will need a real IP address o VPN also OK (?) o Know general Globus et al firewall issues.  I recommend turning firewalls and SELinux off for development. Operating Systems o We test with Mac OS X and many Linux flavors o Other Unix should be fine (assuming Java works) o We have not had any requests for Windows. Software Environment o The portal download comes with everything you need EXCEPT Java. You need the JDK, not the JRE. o We test with Java 1.5. Don’t use Java 1.4. The more memory, the better.

Downloading and Building the Portal Download the portal from o Unpack in $HOME. You should also download the Maven repository. o This is optional but recommended Edit ogce-portal- home/pom.xml Run the command “mvn clean install”. o Use “mvn –o clean install” to build offline. Build will typically take 5 minutes (new Mac) to 30+ minutes (old Linux PC). You can also check out the latest code from SVN. Check the “Nightly Build” dashboard to verify.

Practical Considerations Increase your JVM Memory o export JAVA_OPTS="-server -Xms512m -Xmx1024m - XX:MaxPermSize=256m” o Xmx2048m, 4096m, etc if you can. Monitor catalina.out for problems o tail –f catalina.out Make sure that Tomcat shuts down o Kill if necessary NFS can really slow down building o Use /usr/local/ if you have problems

Editing pom.xml You only need to edit the top level ogce-portal- only/pom.xml. You only need to change for local settings. You MUST update the IP address. You may want to add or remove GRAM and GridFTP hosts. Update project.home if you are building someplace besides $HOME …. …. ${env.HOME}/ogce-portal-only

Rebuilding the Portal Running “mvn clean install” will completely wipe out the existing installation. o You may not want to do this. If you want to rebuild individual components, just run the following command from the ogce- portal-only directory. o mvn clean install –f portlets/myportlet/pom.xml You can do this for all in ogce-portal- deploy/pom.xml

OGCE Portal Project Layout We follow standard Maven 2 conventions for project layouts for each component. o moduleName/src/main/webapp o moduleName/src/main/java Modules are listed on next slide o See ogce-portal-only/pom.xml. o Each module is a separate directory and has its own pom.xml.

ModuleDescription Portal_deployThis is where your portlets are deployed into Tomcat. ContainersSpecific build instructions for containers. Currently we only have GridSphere here. Lnf_portalControls look and feel for the container. Global-configDeploys COG jars into /shared/lib and applies Tomcat custom configs. CertificatesCA certs. Includes download of all TG certs. PortletsEach portlet has its own module.

Developing Grid User Interface Components Portlets, gadgets, etc.

Under the Hood: Building Components OGCE portlets use the Java COG Abstraction Layer to access Grid services. o Provides a buffer over different Grid toolkits and versions. o Provides ability to compose graph-based workflows. o Tutorial in itself Portlets can be built with our Apache Velocity bridge. o “Template” or “steal this code” approach. We have also developed Grid Tag Libraries and Beans (GTLAB) to simplify this process.

GTLAB Features Extends Java Server Faces. o Tag components wrap major COG Abstraction Layer features Allows you to do standalone development. Use JSF portlet bridge to convert into portlets  No new coding, just change, add config files and jars. Or forget about portlets and o Convert into Google Gadgets o Develop as a Facebook application o Etc.

Installing GTLAB You need to install the OGCE portal o GTLAB depends on many shared jars. Download the release code from or check out from SVN: o svn checkout Edit GTLAB/jsf_standalone/pom.xml o should point to your OGCE portal server. Edit GTLAB/jsf_standalone/src/main/resources/resources.properti es. Run “mvn clean install” in GTLAB/jsf_standalone/

Levels of Abstraction in GTLAB code You can use prepackaged gadgets/portlets. You can mix and match repeatable sections of tags via “include” statements. You can develop new applications using existing tag libraries. You can hack the code Note JSF is compatible with JSP, so you can mix and match. And you can embed JavaScript o YUI, Scriptaculous user interface goodies. o Google and Facebook APIs.

MyProxy Example <o:submit id="submit" value="Submit" binding="#{builder.body}" action="#{builder.junkAction}" > <o:myproxy id="mypr" hostname="#{resource.myproxyHostname}" lifetime="2" password="#{resource.password}" port="7512" username="#{resource.username}"/>

Show the Proxy Credential <h:dataTable value="#{myproxy.storedProxyList}" border="1" binding="#{myproxy.storedProxyTable}" var="proxyEntry" id="proxyTable"> <h:commandButton id="deleteProxy" value="Remove” actionListener="#{myproxy.deleteCredential}"/>

Executing Multiple Task <o:submit id="submit" value="Submit" binding="#{builder.body}" action="#{builder.junkAction}" > <o:multitask id="multi" persistent="true" taskname="#{resource.taskname}"> <o:jobsubmit id="js" myfaces="true" arguments="#{resource.arguments}" executable="#{resource.executable}" hostname="#{resource.hostname}" provider="#{resource.provider}" stdout="#{resource.stdout}"/> <o:filetransfer id="ft" myfaces="true" from="#{resource.from}" to="#{resource.to}" />

GTLAB ExampleDescription MyProxyExampleGet a credential and store in OGCE global proxy store. JobSubmitExamplesVarious examples showing how to submit a grid job and get output. Dashboard ExampleRuns showq on Big Red and formats output. DirectoryBrowserDisplays and downloads files on a remote host via GridFTP MultipleTaskExampleShows how to couple file transfers, operations, and TaskMonitorExampleView task state and metadata. These will work as gadgets, portlets, or standalone applications

Making GTLAB Gadgets into Portlets Do all development in jsf_standalone. When you are ready to convert: o cd GTLAB/transition o mvn clean process-resources o cd GTLAB/portlets o mvn clean install You will need to manually edit portlet.xml, group.xml, and layout.xml in GTLAB/portlet/src/main/webapp/WEB-INF/ That’s it.

Making GTLAB Google Gadgets A Gadget is a standalone application that is integrated into your personal iGoogle display. o They can run on your Web server. o You can use SSL/HTTPS Google Gadgets have two versions o http: these use Google’s JavaScript APIs o url: these wrap external applications with IFrames. Limitations: o Not much real estate (but location, location, location) o Cookies manage state on gadgets from the same server but we need a way to integrate multiple independent servers.

Some GTLAB Gadgets

Example Gadget Config File <ModulePrefs title="MyProxy Gadget Example" scrolling="true" height="500"/> <Content type="url" href=" Save this as MyProxy.xml and place in a Web accessible place.

Some Web 2.0 Resources See recent tutorials by Geoffrey Fox and me: o ntations/Web2.0Tutorial.ppt ntations/Web2.0Tutorial.ppt o ntations/CTSTechFutures_May19-08.pptx ntations/CTSTechFutures_May19-08.pptx o Includes material on Google Gdata, Facebook API, Open Social, Microformats, etc. See also full examples at

TeraGrid Information Services: Portlets and Gadget Libraries Slides courtesy of Maytal Dahan

TG Information Services What are TG Information Services? o Services that publish metadata about TeraGridd  TG Software, TG resources, TG Services o Software and applications can query TG Info services for latest information Existing TG Info Services: o Currently we are relying on the REST interface of TG Info Services  TG Also has WebMDS interfaces o Service includes CTSS4 information

Libraries for TG Info Services Using the REST Interfaces we are o Developing JavaScript Libraries that automatically query and return service information o Handling all the querying and xml parsing necessary and return objects through libraries o API will include the ability to (and more)  getTGSites() – Get a list of all TG sites  getResoures() – Get list of resources at a site  getServices() – Get list of TG resources Late summer availability Demo portlets available now

Sample Portlet Interfaces

Talks, Demos, and BOFs 37 Monday1:00 pmScience Gateways on Parade (Nancy Wilkins-Diehr and a cast of 1000’s) Tuesday11:00 amTALK: TeraGrid Open Life Science Gateway (Wenjun Wu) Tuesday1:30 pmTALK: Engaging with the LEAD Science Gateway Project: Lessons Learned in Successfully Deploying Complex System Solutions on TeraGrid Tuesday3:30 pmTALK: Reliable and Scalable Job Submission: LEAD Science Gateway's Testing and Experiences with WS GRAM on TeraGrid Resources Tuesday3:30 pmTALK: The LEAD-WxChallenge Pilot Project: The Potential of Grid- Enabled Learning Tuesday5:30 pmBOF: TeraGrid User Portal BOF (Maytal Dahan) Wednesday10:30 amTALK: Workflow Infrastructure for Multi-scale Science Gateways Wednesday2:00 pmTALK: Increasing TeraGrid User Productivity through Integration of Information and Interactive Services (Maytal Dahan) Wednesday4:00 pmTALK: On-Demand Severe Weather Forecasts Using TeraGrid via the LEAD Portal Wednesday6:30 pmPOSTER/DEMO: TeraGrid Open Life Science Gateway Wednesday6:30 pmPOSTER/DEMO: Fault Tolerance and Recovery of Scientific Workflows in LEAD (Gopi)