Building and Testing OGCE Software on the NMI Build and

Test FacilityMarlon Pierce

Indiana University

OGCE’s portal release includes several Grid portlets built with Java COG, our Velocity Bridge, and GTLAB

JSF libraries

The OGCE Gadget Container allows you to build portals out of public and private Google Open Social gadgets. Supports HTTPS.

Downloadable, packaged software.

OGCE’s Cyberaide JavaScript provides a Grid abstraction layer for developing mashups and gadgets. Downloadable, packaged

software.

The XBaya workflow composer allows you to build scientific workflows from services running across the TeraGrid. This is part of our workflow suite.

OGCE Tools for Science Workflows

OGCE Packaging, Building and Testing• All component builds are designed to be self

contained – Uses Apache Maven 2.x– Download includes everything you need.– Built with a single command: mvn clean install– Builds are modular

• Code is managed by SourceForge’s SVN.• Apache JMeter test suite for the portal.

– Need more than just unit tests– Run against your installation– Automated tests nightly

Basic NMI Build and Test Script Structure

• submitOGCE.sh is the master script. This is called as a cron job.– This calls the .submit script for each component

• For each OGCE component, we have 3 metronome submit scripts– Nonmac (linux), mac-x86, mac-ppc– Example submit script on next slide– This stages in inputs to the destination machine, runs SVN

• Finally, we run a build-specific shell script that does the actual Maven build.

• You can get all of this from our SourceForge SVN:– http://ogce.svn.sourceforge.net/viewvc/ogce/metronome-

scripts

Example NMI submit file scriptproject = OGCEcomponent = Trunk Buildcomponent_version = 1.0.0description = Automated test of OGCE build/runtime execution

run_type = build

inputs = ogcePortal.svn, buildOGCEPortal-nonmac.scp, OGCEJMeterTestFile.scp, OGCEJMeterResults.scp, jakarta-jmeter-2.scp, maven.scp

remote_task = buildOGCEPortal-nonmac.sh

remote_task_args = ""

platforms = x86_64_fc_4, x86_64_fc_5, x86_64_rhap_5, x86_64_rhas_3, x86_64_rhas_4, x86_64_sles_8, x86_cent_4.2, x86_deb_3.1, x86_deb_4.0, x86_fc_2, x86_fc_3, x86_fc_4, x86_fc_5, x86_rh_7.2, x86_rh_8.0, x86_rh_9, x86_rhap_5, x86_rhas_3, x86_rhas_4, x86_slc_3, x86_sles_8, x86_sles_9, x86_slf_3, x86_suse_10.0, x86_suse_10.2, x86_ubuntu_5.10

prereqs = java-1.5.0_08

notify = riverma@indiana.edu

remote_task_timeout = 120m

Practical

Lessons

o Need to modify your build script for some platformso JDK versions were different between Linux

platforms, Macintosh x86, and Macintosh ppc platforms respectively

o Made it necessary to use three different NMI scriptso Maven program had to be copied separately for

Linux platforms because of NMI incompatibilities with OGCE’s bundled maveno Once again, be self-contained

Practical Lessons

o Environment variables need to be specifically assignedo $JAVA_HOME needed to be set and added to patho $HOME directory needed to be set to current NMI

build directory because OGCE portal builds in the $HOME by default

o Keep trying to build even if project fails the first timeo OGCE svn download would occasionally fail, so we

had each script try 'rebuilding' the portal up to three times every night

Practical Lessons

o Have your script "sleep" if runtime programs aren't loadingo Apache Tomcat server would often fail to starto Issuing a "sleep 20" command gave Tomcat

enough time to load

Some Sample Timings

• Linux platformso Average build time per platform = 52.19 minuteso Average total time w/ queue wait (all platforms) =

223.2 minutes

• Macintosh OSX 10.4 (PPC)o Average build time = 10.3 minuteso Average total time (including queue) = 25.29 minutes

• Macintosh OSX 10.4 (x86)o Average build time = 6.2 minuteso Average total time (including queue) = 96.5 minutes

Critique of the Build and Test System

• Help support has always been great• Documentation should be improved.

– Difficult to remember how to find out which Linux platforms are available.

– Difficult to remember how to find out which Java versions are supported.

– Need some “getting started” scripts• I really don’t care too much about specific OS and Java

versions. It would be nice to have “use all linux”, “use newest Java” options.– Would also allow me to use the same scripts for all my

platforms.• I would like to revisit our systems testing strategy

More Information

• Email: mpierce@cs.indiana.edu• OGCE Web Site: www.collab-ogce.org• Blog/News Feed:

http://collab-ogce.blogspot.com/• See our poster at the SDCI All Hands Meeting

Thursday night• Rishi Verma did most of the original build and

test work