State of Service Oriented Science Tools Open Source Grid Cluster Conference Oakland

State of Service Oriented Science Tools

Open Source Grid Cluster Conference

Oakland

Agenda

Presentation Demo Questions

Service-Oriented Science

People create services (data or functions) …

which I discover (& decide whether to use) …

& compose to create a new function ...

& then publish as a new service.

I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!

I hope that this “someone else” can manage security, reliability, scalability, …!!

“Service-Oriented Science”, Science, 2005

Creating Services

Introduce + gRAVIShannon Hastings

Scott OsterDavid Ervin

Stephen Langella

Kyle ChardRavi Madduri

Introduce Overview A framework which enables fast and easy creation of Globus

based grid services Provide easy to use graphical service authoring tool. Hide all “grid-ness” from the developer Utilize best practice layered grid service architecture Integration with other core grid services and architecture

components GAARDS Security Infrastructure (Dorian, GridGrouper, CSM) Globus Index Service Global Model Exchange (GME) Cancer Data Standards Repository

Extension Framework for integrating with other architecture components

Inside the Introduce created service

Services have many moving and configurable parts which support features such as: Advertisement Discovery Invocation Security (Authentication/Authorization) Stateful Resources

The Introduce Toolkit can keep all these features in sync as the developer creates and modifies the grid service

Introduce Features Supports modification of operations

Adding operations Removing Operations Updating Operations Importing Operations

Graphical Configuration Advertisement Security Service Metadata Specification Service Metadata Editing Service Configuration Properties

Auto Generates Code for Service Auto generates a client API for service. Graphical Deployment of Service

Globus Tomcat JBoss

Created Skeleton Layout= introduce managed= globus/axis managed

= developer implements

ApplnService

Create

Index service

Store

Repository ServiceAdvertize

Discover

Invoke;get results

Introduce

Container

Transfer GAR

Deploy

gRAVI

Grid Remote Application Virtualization Interface

Builds on Introduce Define service Create skeleton Discover types Add operations Configure security

Wrap arbitrary executables

Discovering Services






I hope that this “someone else” can manage security, reliability, scalability, …

!!“Service-Oriented Science”, Science, 2005

Discovering ServicesWS-MDS+ Taverna

Laura Pearlman

Mike Darcy

myGrid Team

The ultimate arbiter?

Types, ontologies

Can I use it?

Billions of servicesDiscovering Services

Assume success

Semantics

Permissions

Reputation

A B

Discovery (1):Registries

Globus

Discovery (2):Standardized Vocabularies

Core Services

Grid Service

Uses TerminologyDescribed In

Cancer DataStandardsRepository

EnterpriseVocabularyServices

ReferencesObjectsDefined in

Service Metadata

Publishes

Subscribes toand Aggregates

Queries Service

Metadata Aggregated In

Registers To

Discovery Client API

IndexService

Globus

Composing Services








Composing Services

Taverna + GT4Taverna team

Wei TanRavi Madduri

Composing Services

Globus

Hosting Services








Provisioning ServicesWS-GRAM + VWS

Martin FellerStuart Martin

Kate KeaheyTim Freeman

Joshua Boverhof

Provisioning using WS-GRAM

gRAVI uses JSDL for Application Description Generates a method on the generated service

called <appName>GRAM Generates implementation that creates a

GramJob from Application Description Uses the bootstrapped community credential

to run the application as a grid job Used widely in realizing the usecases from

caBIG community

Repository Service

Using VWS/Cloud Computing

gRAVI service Wrap the application as

Service Create the GAR and put in

repository Provision resources

Use clouds (VMM) start service

Transfer gar, deploy Index Service

Grid service registers itself

nimbus stratus

Index service

Transfer,deploy

Registerservice

Create VM

provision

discovery

portal GAR

gravi portlets

stratus

nimbus

Index service

VM

Repository Service

GAR

WAR

Portal

register

service

Invoke executable

Cloud Computing

User Communities and Early Adopters

Advanced Photon Source

APS Requirements align well with goals of SOS

They have data, analytics and workflows We wrapped one of the 2 applications as a

service Goal is to create and run a workflow with

services generated using gRAVI and provision the workflow using Nimbus

Cancer Bio-informatics Grid

Primary usecase for Service Oriented Science Demo at the Architecture F2F meeting

geWorkbench from Columbia University Hierarchical Clustering Service gRAVI WS-GRAM on TeraGrid

Integrating into caGrid Tools Taverna Integration for composition

Use gRAVI to wrap more popular services into Grid Services

caGrid Service

TeraGrid Aware caGrid Service

Creating the Gateway Service Manually stage the binary (jar file) on TeraGrid

Takes in .ser files as input Produces results also in a .ser file

Used the gRAVI plugin for Introduce to create the gateway service

http://www-unix.mcs.anl.gov/~neillm/ravi/ Gateway gridFTPs input data and parameters from

geWorkbench to TeraGrid geWorkbench passes input to the gateway in

geWorkbench’s native format (caDSR compliant) Gateway serializes the input before gridFTPing to

TeraGrid Gateway invokes the staged binary Gateway gridFTPs results back to geWorkbench

Gateway deserializes the result file Gateway returns results to geWorkbench in its

native format Gateway service is a secured caGrid service which in

turn invokes TeraGrid with a caBIG community account

http://www-unix.mcs.anl.gov/~neillm/ravi/

Steps to establishing geWorkbench/caGrid/TeraGrid Interface

caGrid Security (GTS, Grid Grouper, Dorian, CDS)

http://www.cagrid.org/mwiki/index.php?title=GAARDS:Main

Future Work (Near Term)

Plan for integrating Virtual Workspace services as a provisioning mechanism Details of the Plan in the following slides

caBIG demo in June 2008 using both WS-GRAM and VWS as backends

Google Summer of Code Projects – AJAX front-end to gRAVI and Service Composition

Two summer students

Future Work

gRAVI portlets Plugin also creates a grid portlet interface to the generated client Or simple portal exposing the WSDL operations

Publish Services & Portlets Add window for publishing via Introduce Publish gRAVI services to a Repository Entry contain service (GAR) and portlet (WAR) Metadata describing resource requirements, etc

Use a Service from Portal User selects the service(s) from Repository Identify or Create resources (VM) based on metadata Deploy service(s) Discover service(s) User access the service through portlet or client.

Success stories from Early Adopters From Brian Tieman at APS: I was able to muddle through and get it

working. Total time spent ~2 hours. I'm happy with that, and I'm sure future applications will go even faster! I've only run the sample client and I'm not trying to pass arguments yet so there's still a bit more to do before I can try integrating this into my application, but so far so good!

From Paul Burkhardt at NCI Inc : We used the gRAVI extension to the Introduce Grid Service Authoring Toolkit to demonstrate the ease of deploying arbitrary executables as Grid Services and GRAM jobs. Migrating legacy applications to a Web Services model presents an imposing hurdle to adoption. Many important legacy applications would require a significant amount of man-hours to rewrite as Grid Services. An alternative is to create JNI interfaces to wrap the executables but this approach still requires effort and expertise to create the necessary WSDL and corresponding Java implementation. This development cost can be prohibitive. The gRAVI tool has provided us the solution to low-cost and rapid deployment of legacy applications for Grid infrastructures.

Documents

State of Service Oriented Science Tools Open Source Grid Cluster Conference Oakland