Lightweight Grid Computing Worksop
2nd May 2006, Losehill Hall, Derbyshire
Requirements and Expectations from Workflows
Asif Akram
e-Science Grid Technology Group
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
WOSE Project
Workflow Optimisation for Services in e-Science EPSRC funded project In collaboration with Imperial College and Cardiff
University CCLRC is investigating the user requirements Developing Use Cases based on existing e-
Science Application e-HTPX: An e-Science Resources for High
Throughput Protein Crystallography
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Best Practices for Workflows
Modular Design
Exception Handling
Compensation Mechanism
Adaptive Workflow
Flexible Workflows
Management of Workflow
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Modular Design
Thorough investigation of different activities Group similar or related activities Group activities with minimum side effects Module components should be scalable Module components can be replaceable Reliable to serve as repeatable units Modules operate as “black boxes” in the overall
workflow, with their own variables, computational logic, dependency constraints, and event handlers.
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Exception Handling
Exceptions can be due to inconsistent data, divergence of tasks from the workflow model, unexpected contingencies and un-modeled changes in the environment.
Type of Exceptions: “expected exceptions” (“variations”) and “unexpected exceptions”
Exception Handlers: Global Exception Handlers, Scoped Exception Handlers and Inline Exception Handlers.
Exception handling should be localized
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Compensation Mechanism
“undoing steps in the business process that have already completed successfully”
Reverse the effects of successful activities in the abandoned workflow
Un-handled Exceptions requires compensation Compensation behavior must be defined by the
services to ensure reversal of a operation. Synchronous operations (i.e. a single connection
request/reply) have short lifetimes and do not require compensation to release resources.
Asynchronous communications requires some sort of compensation (unluckily we have normally RPC style Web Services)
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Adaptive Workflow
Adaptive nature of a workflow can be static or dynamic depending on whether changes are accommodated at design time or runtime respectively.
Redesign and optimization of a process to gain greater efficiency and effectiveness requires adjustment in the workflows; in fact the final goal is to constantly improve the workflow/ process by evolutionary redefinition.
Modifications break a pre-defined process in an attempt to solve the problem in better way.
Dynamically adaptive workflows adjust at runtime; i.e. due to unexpected results
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Flexible Workflows
Flexibility is required during the early stages of a workflow design when services are un-stable.
Flexibility in the data types and service endpoint. Flexibility in data, is achieved by applying generic
“base” or “parent” data types Flexibility in service endpoint, requires integration of
additional and re-located services through WS-Addressing.
Potential callouts and logic for selecting partner service has to be built into the business process as external configuration file .
Assuming all the services have “similar” interfaces which may not always be the case.
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Management of Workflow
Workflow management involves the addition of Breakpoints arbitrary location crucial to the process Steering capabilities generalizes breakpoints and
involves reset, re-schedule, restart, undo, abort, complete, recover, ignore or jump operations.
Persistence of state for fault tolerance, to allow re-execution with different experimental data sets, and to facilitate inspection of intermediate data values
Monitoring for optimization and analysis of internal state, data flow, inspection of values, re-scheduling of tasks and re-ordering of tasks
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Portals & Web Services
Inte
rnet
Inte
rnet
Repository for Authorisation and Policies
Workflow
Service End-points
Clients
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Business Process Execution Language
Builds on top of XML and Web Services Abstract, which allow specification of the public
message exchange between participating services and doesn’t include the internal details of process flows.
Executable, which allows specification of the exact details of a workflow. Normally BPEL is used for executable processes.
Can utilize other Web Services specification for reliable messages, transactions etc
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
BPEL
Provides different types of activities Primitive activities can be structured in Modules BPEL support different types of structured
activities i.e. “sequence”, “flows” and “scope” Extensive support for Exception handling Can trigger Events from “Exception Handler” Compensation Handlers can be called from
Exception Handler Limited monitoring is possible through different
primitive activities like “onMessage”, “onAlarm” and “pick”
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
BPEL Extension Different implementation has varying level of
extensions Oracle BPEL engine supports “notification” Oracle supports “NFlow” for parallel execution of
any module “n times” Oracle has proprietary management capabilities
and user interaction functionality
Engines provides API to query and interact with process at runtime
Proprietary API to populate SOAP Headers Limited support for WS-Security Support for WSIF and EJB
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Limitations of BPEL
Limited reusability of primitive and structured activities
Can’t re-execute activities defined earlier <onMessage> or <onAlarm> events are not to
modify the workflow at runtime BPEL specification does not explicitly support
user interactions and notifications Difficult to integrate security without support from
Vendors BPEL is fairly complex specification
Presenter Name
Facility NameLightweight Grid Computing Workshop, 2nd May 2006, Losehill Hall, Derbyshire
Future Work Moving to Document Style Web Services Evaluating WSRF for e-HTPX e-HTPX mainly passes URL of images in different
services (candidate for Resource Properties) Moving EPR across services is more efficient Built in support for Notifications Support for persistence (flat files but can be
extended for DB); e-HTPX is using Hibernate WS-Core supports XML Database Xindice Support for different level and type of security