Upload
oswin-mclaughlin
View
217
Download
2
Tags:
Embed Size (px)
Citation preview
WS-VLAM Introduction presentation
WS-VLAM Workflow Engine
System and Network Engineering group
Institute of informatics
University of Amsterdam
WS-VLAM Introduction presentation
Introduction
• This presentation contains information related to WS-VLAM engine
• It helps the reader understand how the workflow engine works, describes the current state and present the ongoing activities and planned ones.
• NOTE: We are currently still working on the User’s and developer’s
Guide, they are not available at this time. The presentations do not replace the documentation it just give a quick introduction.
Do not hesitate to contact us for further information
Thanks for taking some of your time to read about the WS-VLAM work, any comments are more than welcome
WS-VLAM developers [email protected]
WS-VLAM Introduction presentation
Needed background …
• To be able to follow the content of this presentation you will need to know about:
WSRF= Web Service Reference Framework
GT4= Globus Toolkit version 4
• More detailed description of the presented work can be found in the section of publication of the WS-VLAM home page
WS-VLAM Introduction presentation
Objective of this presentation…
• Why you should look at this presentations You want to know how WS-VLAM works and
interacts with the Grid middleware.
• Why you don’t have to worry if you do not understand the content on this presentation: You are an end-user: it will be enough for you to
read the User ’s guide or the WS-VLAM composer presentation
You are an application developer: it will be enough for you to read the Developer ’s guide & the WS-VLAM developers presentation
WS-VLAM Introduction presentation
Outline
• Introduction • Architecture of the WS-VLAM• Hierarchical workflow compositions &
execution • Conclusions
UvA, Amsterdam June 2007 WS-VLAM Introduction presentation
Process Management Stack
Objectives: A WSRF enabled workflow engine
Data Management Stack
Network & Computing ResourcesNetwork & storage Resources
Grid Middleware: Process & resource management
Grid Middleware: Data management
Workflow execution
Workflow composition
Workflow Management system
GridServices
GridServices
GridServicesApplication Templates
Web service
Web Service Interface
OGSA DAI
WorkflowEngine
Web service
WorkflowEngine
Web service
Web Service Interface
Application web service
Distributed Workflow
WS-VLAM Introduction presentation
WS-VLAM Engine characteristics
• Implemented as GT4 WSRF service
• Uses GT4 delegation service for security
• Uses GSI enabled, private VNC for Graphical modules
• Uses notifications mechanisms provides by GT4 for monitoring the execution
UvA, Amsterdam June 2007 WS-VLAM Introduction presentation
GRAMservices
GT4 Java Container
RTSMFactory
Delegationservice
Worker nodes
pre-ws-GRAM
Cli
ent
Job
functions
Delegate
Service host(s) and compute element(s)
WS-VLAM Engine: architecture
RTSM Instance
Workflowcomponents
WS-VLAM Introduction presentation
A three step execution process
Client Delegation Service
RTSM Factory
GRAM RTSM Instance
Step 1
Step 2
• Step 1: Create Delegated Grid Credentials
• Step 2: Instantiate the workflow components
• Step 3: Monitor the workflow execution
Workflowcomponents
Step 3
UvA, Amsterdam June 2007 WS-VLAM Introduction presentation
Sequence-diagram
WS-vlamClient
GT4 DelegationService
RTSM Instance
RTSM Factory
1. Create: delegation credential
2. Submit workflow execution plan
GT4GRAM
Get the delegation credential EPR
Get the RTSM instance EPR 4. Create: RTSM instance
3. Create: RTSM instance
5. subscribe: to notification events
Get the notification events
Step 1
Step 2
Step 3
WS-VLAM Introduction presentation
Current Setting…
Not yet available
DAS-3
Sara
SRB
VU
TUDELFTUvA
Leiden University
Server host Where the WS-VLAM engine
is running
WS-VLAM Introduction presentation
Step1: Create Delegated Credential
VLAMClient RTSM
Factory.
proxy
•Initialize user proxy cert.
•Create delegated credential resource
•Credential EPR returned
EPRGRAM
Worker nodes
Delegation
Credential
RP
GT4 Service Container
EPR = End Point Reference
WS-VLAM Introduction presentation
Step 2: instantiates the workflow components and Create a RTSM instance
VLAMClient
Delegation
RTSMFactory.
Credential
RP
• Request the creation RTSM instance• Pass delegated credential EPR
•RTSM instance EPR returned
EPR
•Access delegated credential resource
•Contact GRAM to instantiates the workflow components
GRAM
RTSMInstance
Worker nodes
•GRAM instantiates the workflow components
Workflowcomponents
Workflowcomponents
EPR = End Point Reference
GT4 Service Container
WS-VLAM Introduction presentation
Step3: monitors the application workflow
VLAMClient
RTSMFactory.
•Contact RTSM instance and subscribe for notification events
GRAM
RTSMInstance
Worker nodes
•RTSM instance subscribes for notification events from GRAM and Workflow components
Workflowcomponents
Workflowcomponents
•GRAM and Workflow components generate events
Notificationevents
Delegation
Credential
RP
GT4 Service Container
WS-VLAM Introduction presentation
Workflow components with graphical output
VLAMClient
RTSMFactory.
GRAM
RTSMInstance
Delegation
Credential
RP
Worker nodes
Workflowcomponents
Workflowcomponents
RTSM Instance
GUIproxy
GT4 Service Container
vnc server
GSI enabled connection
WS-VLAM Introduction presentation
Support hierarchical workflows …
• workflow components can be composed of a collection of other components
• The advantage is now both the composition & the execution of complex workflows become easier
WS-VLAM Introduction presentation
Server host Where the WS-VLAM engine
is running
Server host Where the WS-VLAM engine
is running
Setting for distributed hierarchical workflows …
DAS-3
Sara
SRB
VU
TUDELFTUvA
Leiden University
UvA, Amsterdam June 2007 WS-VLAM Introduction presentation
Step2: instantiates the distributed workflow components & Create a RTSM instances
GT4 Service Container
VLAMClient
RTSMFactory
EPRGRAM
RTSM Instance
Worker nodes
Workflowcomponents
Workflowcomponents
GRAM
Worker nodes
Workflowcomponents
Workflowcomponents
GT4 Service Container
RTSM Instance
GUIproxy
Dataproxy
GUIproxy
Dataproxy
Distributed RTSM
RTSMFactory
Distributed RTSM
UvA, Amsterdam June 2007 WS-VLAM Introduction presentation
Step2: instantiates the distributed workflow components & Create a RTSM instances
GT4 Service Container
VLAMClient
RTSMFactory
EPRGRAM
RTSM Instance
Worker nodes
Workflowcomponents
Workflowcomponents
GRAM
Worker nodes
Workflowcomponents
Workflowcomponents
GT4 Service Container
RTSM Instance
GUIproxy
Dataproxy
Dataproxy
GUIproxy
Distributed RTSM
RTSMFactory
Distributed RTSM
WS-VLAM Introduction presentation
Conclusions
• WSRF compliant middleware such as GT4 offers useful basic services to develop a truly grid enabled workflow engine:
Delegation service & ws-notification mechanisms proved to be very useful
• Separating the workflow composition from execution suits better the Grid environments
Some References
1. Science Z Zhao, A Belloum, M Bubak Editorial: Special section on workflow systems and applications in e-Future Generation Computer Systems 25 (5), 525-527
2. A. Wibisono, D. Vasyunin, V. Korkhov, AS.Z. Belloum WS-VLAM: a GT4 based workflow management system , The 2nd International Workshop on Scientific Workflows, In conjunction with ICCS 2007, Beijing, China, 2007
3. A.S.Z. Belloum, V. Korkhov, S. Koulouzis, M. A Inda, and M. Bubak Collaborative e-Science experiments: from scientific workflow to knowledge sharing JULY/AUGUST, IEEE Internet Computing, 2011
4. Ilkay Altintas, Manish Kumar Anand, Daniel Crawl, Shawn Bowers, Adam Belloum, Paolo Missier, Bertram Ludascher, Carole A. Goble, Peter M.A. Sloot, Understanding Collaborative Studies Through Interoperable Workflow Provenance, IPAW2010, Troy, NY, USA
UvA, Amsterdam June 2010 WS-VLAM Introduction presentation
Some References
5. A. Belloum, Z. Zhao, and M. Bubak Workflow systems and applications , Future Generation Comp. Syst. 25 (5): 525-527 (2009)
6. Z. Zhao, A.S.Z. Belloum, et al., Distributed execution of aggregated multi domain workflows using an agent framework The 1st IEEE International Workshop on Scientific Workflows, Salt Lake City, U.SA, 2007
7. Zhiming Zhao, Adam Belloum, Cees De Laat, Pieter Adriaans, Bob Hertzberger Using Jade agent framework to prototype an e-Science workflow bus Authors Cluster Computing and the Grid, 2007. CCGRID 2007
UvA, Amsterdam June 2010 WS-VLAM Introduction presentation
UvA, Amsterdam June 2007 WS-VLAM Introduction presentation
http://www.vl-e.nl/