Lightweight Grid Computing Worksop

2nd May 2006, Losehill Hall, Derbyshire

Requirements and Expectations from Workflows

Asif Akram

e-Science Grid Technology Group

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

WOSE Project

Workflow Optimisation for Services in e-Science EPSRC funded project In collaboration with Imperial College and Cardiff

University CCLRC is investigating the user requirements Developing Use Cases based on existing e-

Science Application e-HTPX: An e-Science Resources for High

Throughput Protein Crystallography

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Best Practices for Workflows

Modular Design

Exception Handling

Compensation Mechanism

Adaptive Workflow

Flexible Workflows

Management of Workflow

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Modular Design

Thorough investigation of different activities Group similar or related activities Group activities with minimum side effects Module components should be scalable Module components can be replaceable Reliable to serve as repeatable units Modules operate as “black boxes” in the overall

workflow, with their own variables, computational logic, dependency constraints, and event handlers.

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Exception Handling

Exceptions can be due to inconsistent data, divergence of tasks from the workflow model, unexpected contingencies and un-modeled changes in the environment.

Type of Exceptions: “expected exceptions” (“variations”) and “unexpected exceptions”

Exception Handlers: Global Exception Handlers, Scoped Exception Handlers and Inline Exception Handlers.

Exception handling should be localized

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Compensation Mechanism

“undoing steps in the business process that have already completed successfully”

Reverse the effects of successful activities in the abandoned workflow

Un-handled Exceptions requires compensation Compensation behavior must be defined by the

services to ensure reversal of a operation. Synchronous operations (i.e. a single connection

request/reply) have short lifetimes and do not require compensation to release resources.

Asynchronous communications requires some sort of compensation (unluckily we have normally RPC style Web Services)

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Adaptive Workflow

Adaptive nature of a workflow can be static or dynamic depending on whether changes are accommodated at design time or runtime respectively.

Redesign and optimization of a process to gain greater efficiency and effectiveness requires adjustment in the workflows; in fact the final goal is to constantly improve the workflow/ process by evolutionary redefinition.

Modifications break a pre-defined process in an attempt to solve the problem in better way.

Dynamically adaptive workflows adjust at runtime; i.e. due to unexpected results

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Flexible Workflows

Flexibility is required during the early stages of a workflow design when services are un-stable.

Flexibility in the data types and service endpoint. Flexibility in data, is achieved by applying generic

“base” or “parent” data types Flexibility in service endpoint, requires integration of

additional and re-located services through WS-Addressing.

Potential callouts and logic for selecting partner service has to be built into the business process as external configuration file .

Assuming all the services have “similar” interfaces which may not always be the case.

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Management of Workflow

Workflow management involves the addition of Breakpoints arbitrary location crucial to the process Steering capabilities generalizes breakpoints and

involves reset, re-schedule, restart, undo, abort, complete, recover, ignore or jump operations.

Persistence of state for fault tolerance, to allow re-execution with different experimental data sets, and to facilitate inspection of intermediate data values

Monitoring for optimization and analysis of internal state, data flow, inspection of values, re-scheduling of tasks and re-ordering of tasks

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Portals & Web Services

Inte

rnet

Inte

rnet

Repository for Authorisation and Policies

Workflow

Service End-points

Clients

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Business Process Execution Language

Builds on top of XML and Web Services Abstract, which allow specification of the public

message exchange between participating services and doesn’t include the internal details of process flows.

Executable, which allows specification of the exact details of a workflow. Normally BPEL is used for executable processes.

Can utilize other Web Services specification for reliable messages, transactions etc

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

BPEL

Provides different types of activities Primitive activities can be structured in Modules BPEL support different types of structured

activities i.e. “sequence”, “flows” and “scope” Extensive support for Exception handling Can trigger Events from “Exception Handler” Compensation Handlers can be called from

Exception Handler Limited monitoring is possible through different

primitive activities like “onMessage”, “onAlarm” and “pick”

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

BPEL Extension Different implementation has varying level of

extensions Oracle BPEL engine supports “notification” Oracle supports “NFlow” for parallel execution of

any module “n times” Oracle has proprietary management capabilities

and user interaction functionality

Engines provides API to query and interact with process at runtime

Proprietary API to populate SOAP Headers Limited support for WS-Security Support for WSIF and EJB

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Limitations of BPEL

Limited reusability of primitive and structured activities

Can’t re-execute activities defined earlier <onMessage> or <onAlarm> events are not to

modify the workflow at runtime BPEL specification does not explicitly support

user interactions and notifications Difficult to integrate security without support from

Vendors BPEL is fairly complex specification

Presenter Name

Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire

Future Work Moving to Document Style Web Services Evaluating WSRF for e-HTPX e-HTPX mainly passes URL of images in different

services (candidate for Resource Properties) Moving EPR across services is more efficient Built in support for Notifications Support for persistence (flat files but can be

extended for DB); e-HTPX is using Hibernate WS-Core supports XML Database Xindice Support for different level and type of security