OGCE Workflow Suite

GopiKandaswamySuresh MarruSrinathPerera

ChathuraHerathMarlon Pierce

TeraGrid 2008

Workflow Suite

• Generic Service Toolkit– Tool to wrap command-line applications as web services– Handles file staging&job submissions– Extensible runtime for security, resource brokering& urgent computing– Generic Factory service for on-demand creation of application services

• XRegistry– Information repository for the OGCE workflow suite– Register, search, retrieve&share XML documents– User & hierarchical group based authorization

• XBaya– GUI based tool to compose&monitor workflows– Extensible support for compiler plug-ins like BPEL &Jython

2OGCE TeraGrid08 Tutorial

Features

• Security– Authentication and authosrization– Secure invocations between services– Support for gateway community accounts– Support for multiple user accounts

• Reliability– Retry job submissions and file staging– Fault Tolerance and Recovery service

• Over-provisioning and migration

• Compatibility– Taverna, Kepler and Trianna

3OGCE TeraGrid08 Tutorial

Applications

• LEAD– Lower entry barrier to using weather analysis tools– Improve detection, analysis & prediction of mesoscale weather

• Motif-Network– Transformation of sequenced genomes to “domain-space”

• Cyber-Infrastructure Evaluation– Performance evaluation of future supercomputer architectures

• ADAM– Algorithms for feature extraction, data normalization,

classification and normalization

4OGCE TeraGrid08 Tutorial

88D Radar Re-

mapper

Satellite Data Re-mapper

NIDS Radar Re-mapper

Radar data (Level II)

Surface data, upper air mesonet data and

wind profiler data

Radar data (Level III)

Satellite data

ADAS

Terrain Preprocessor

3D Model Data

Interpolator (Initial

Boundary Conditions)

3D Model Data

Interpolator(lateral Boundary

Conditions)

Terrain data files NAM, RUC, GFS data

WRF Static Preprocessor

ARPS to WRF Data

Interpolator

ARPS Plotting Program

IDV

Surface, terrestrial data files

1

WRF to ARPS DataInterpolator

2

3

4

5

6

7

9

11

12

13

Run once per forecast region

Repeat periodically

for new data

Triggered if a storm is detected

Visualization on users request

ADAM

Data mining:look for storm

signature

WRFWRF

WRFWRF

14

ARPS Ensembl

e Generato

r

15

Static data Initialization Forecast VisualizationReal time data Analysis Data Mining

10

8

LEAD: A Weather Forecasting Workflow (1/2)

5

WRF-Static running on Tungsten

LEAD: A Weather Forecasting Workflow (2/2)

6OGCE TeraGrid08 Tutorial

Motif-Network: Whole Genome workflow

• Domain webs of large genomes– Input list of amino acid sequences– Identify all known domains– Construct webs

Parallel processingCapability-type computing

Ensemble-type processing(minimal network reqs)

Capacity-type computing

7Jeff Tilson, RENCIOGCE TeraGrid08 Tutorial

CI: Execute Sub-Workflow

• Input a campaign step filename• Execute GAMESS per step

specification

8Jeff Tilson, RENCIOGCE TeraGrid08 Tutorial

More Information

OGCE TeraGrid08 Tutorial 9

Tuesday 1:30 pm Engaging with the LEAD Science Gateway Project: Lessons Learned in Successfully Deploying Complex System Solutions on TeraGrid

Tuesday 3:30 pm Reliable and Scalable Job Submission: LEAD Science Gateway's Testing and Experiences with WS GRAM on TeraGrid Resources

Tuesday 3:30 pm The LEAD-WxChallenge Pilot Project: The Potential of Grid-Enabled Learning

Wednesday 10:30 am Workflow Infrastructure for Multi-scale Science Gateways

Wednesday 4:00 pm On-Demand Severe Weather Forecasts Using TeraGrid via the LEAD Portal

Questions?

10OGCE TeraGrid08 Tutorial