Building and Testing OGCE Software on the NMI Build and Test Facility Marlon Pierce Indiana University

OGCE’s portal release includes several Grid portlets built with Java COG, our Velocity Bridge, and GTLAB JSF libraries

The OGCE Gadget Container allows you to build portals out of public and private Google Open Social gadgets. Supports HTTPS. Downloadable, packaged software.

OGCE’s Cyberaide JavaScript provides a Grid abstraction layer for developing mashups and gadgets. Downloadable, packaged software.

The XBaya workflow composer allows you to build scientific workflows from services running across the TeraGrid. This is part of our workflow suite. OGCE Tools for Science Workflows

OGCE Packaging, Building and Testing All component builds are designed to be self contained – Uses Apache Maven 2.x – Download includes everything you need. – Built with a single command: mvn clean install – Builds are modular Code is managed by SourceForge’s SVN. Apache JMeter test suite for the portal. – Need more than just unit tests – Run against your installation – Automated tests nightly

Basic NMI Build and Test Script Structure submitOGCE.sh is the master script. This is called as a cron job. – This calls the.submit script for each component For each OGCE component, we have 3 metronome submit scripts – Nonmac (linux), mac-x86, mac-ppc – Example submit script on next slide – This stages in inputs to the destination machine, runs SVN Finally, we run a build-specific shell script that does the actual Maven build. You can get all of this from our SourceForge SVN: – scripts

Example NMI submit file script project = OGCE component = Trunk Build component_version = description = Automated test of OGCE build/runtime execution run_type = build inputs = ogcePortal.svn, buildOGCEPortal-nonmac.scp, OGCEJMeterTestFile.scp, OGCEJMeterResults.scp, jakarta-jmeter-2.scp, maven.scp remote_task = buildOGCEPortal-nonmac.sh remote_task_args = "" platforms = x86_64_fc_4, x86_64_fc_5, x86_64_rhap_5, x86_64_rhas_3, x86_64_rhas_4, x86_64_sles_8, x86_cent_4.2, x86_deb_3.1, x86_deb_4.0, x86_fc_2, x86_fc_3, x86_fc_4, x86_fc_5, x86_rh_7.2, x86_rh_8.0, x86_rh_9, x86_rhap_5, x86_rhas_3, x86_rhas_4, x86_slc_3, x86_sles_8, x86_sles_9, x86_slf_3, x86_suse_10.0, x86_suse_10.2, x86_ubuntu_5.10 prereqs = java-1.5.0_08 notify = remote_task_timeout = 120m

Practical Lessons o Need to modify your build script for some platforms o JDK versions were different between Linux platforms, Macintosh x86, and Macintosh ppc platforms respectively o Made it necessary to use three different NMI scripts o Maven program had to be copied separately for Linux platforms because of NMI incompatibilities with OGCE’s bundled maven o Once again, be self-contained

Practical Lessons o Environment variables need to be specifically assigned o $JAVA_HOME needed to be set and added to path o $HOME directory needed to be set to current NMI build directory because OGCE portal builds in the $HOME by default o Keep trying to build even if project fails the first time o OGCE svn download would occasionally fail, so we had each script try 'rebuilding' the portal up to three times every night

Practical Lessons o Have your script "sleep" if runtime programs aren't loading o Apache Tomcat server would often fail to start o Issuing a "sleep 20" command gave Tomcat enough time to load

Some Sample Timings Linux platforms o Average build time per platform = minutes o Average total time w/ queue wait (all platforms) = minutes Macintosh OSX 10.4 (PPC) o Average build time = 10.3 minutes o Average total time (including queue) = minutes Macintosh OSX 10.4 (x86) o Average build time = 6.2 minutes o Average total time (including queue) = 96.5 minutes

Critique of the Build and Test System Help support has always been great Documentation should be improved. – Difficult to remember how to find out which Linux platforms are available. – Difficult to remember how to find out which Java versions are supported. – Need some “getting started” scripts I really don’t care too much about specific OS and Java versions. It would be nice to have “use all linux”, “use newest Java” options. – Would also allow me to use the same scripts for all my platforms. I would like to revisit our systems testing strategy

More Information OGCE Web Site: Blog/News Feed: ogce.blogspot.com/ ogce.blogspot.com/ See our poster at the SDCI All Hands Meeting Thursday night Rishi Verma did most of the original build and test work