1 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

Part V

Overview of Scientific Workflow Systems

AAAI-08 Tutorial on Computational Workflows for

Large-Scale Artificial Intelligence Research

2 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

TAVERNAhttp://taverna.sourceforge.net

3 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

TAVERNA [Goble et al 07; Hull et al 06; Oinn et al 00]   Workflows of services

•  Web services, REST services, etc.

  Bioinformatics applications in myGrid •  3,000+ 3rd party services: EMBL-EBI, NCBI, BioMOBY, KEGG, …

  Workflows described in Scufl language •  Workflows composed of “processors” •  DAG + control links •  Map item to list, item to list element, flatten lists of lists, cross

product iteration, dot product

  Components: •  Service discovery: UDDI registry, GRIMOIRES, etc. •  Feta service discovery and selection uses semantics •  FreeFluo workflow execution

4 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

Architecture of Taverna and Associated myGrid Components [Oinn et al 07]

5 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

Taverna Workflow Editor

6 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

ASKLALONhttp://www.askalon.org

7 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

ASKALON [Fahringer et al 07]   Research focus is on resource management, scheduling,

performance analysis and prediction   Abstract Grid Workflow Language (AGWL)   Complex control constructs evaluated at run-time:

•  if-then, switch, while loop, parallel-for

  Assumptions made about control constructs at scheduling time

•  If assumptions not met, workflow is rescheduled

  HEFT scheduling algorithm [Zhao and Sakellariou 03]

8 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

ASKALON Architecture [Fahringer et al 07]

9 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

ASKALON: UML-Based Workflow Editor

10 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

TRIANAhttp://www.trianacode.org

11 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

TRIANA [Taylor et al 07]   Seamless integration of local Java tools and remote web

services on same user workspace   Workflow components represent algorithm, process, or

service (the smallest granularity of work)   Special components provided for loops and branching

12 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

TRIANA [Taylor et al 07]

13 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

KEPLERhttp://www.kepler-project.org

14 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

Kepler [Ludaescher et al 06]   Builds on the Ptolemy-II system for component oriented

concurrent programming (http://ptolemy.berkeley.edu)   Workflow steps implemented as “actors” with I/O ports

and parameters •  Actors may be data sources, sinks, data transformers, analytical

steps, or arbitrary computational steps.

  Includes WebService actors, grid actors, database actors, data transformation actors, ingestion actors

•  Ptolemy-II provides Matlab and Python actors

15 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

Kepler User Interface

16 Yolanda Gil (gil@isi.edu) AAAI-08 Tutorial July 13, 2008 USC Information Sciences Institute

Other Workflow Systems   A Taxonomy of Workflow

Management Systems for Grid Computing, Jia Yu and Rajkumar Buyya, Journal of Grid Computing, Volume 3, Numbers 3-4, 2005.

  Workflows for e-Science: Scientific Workflows for Grids, Taylor, I. J., Deelman, E., Gannon, D. B., and M. Shields (Eds). Springer Verlag, 2007.