10
Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen Microsoft Research Keith Grochow, Mark Stoermer, Ed Lazowska University of Washington Joby Thomas and Murali Narayanan Aditi Technologies

Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen

Embed Size (px)

Citation preview

Page 1: Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen

TridentScientific Workflow Workbench

Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van IngenMicrosoft Research

Keith Grochow, Mark Stoermer, Ed LazowskaUniversity of Washington

Joby Thomas and Murali NarayananAditi Technologies

Page 2: Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen

Visually program workflows, through a web browser

Libraries of activities, workflows and servicesSocial annotations and search, connect to myexperiment.

Abstract parallelism, for HPC & many core (CCR)

Adaptive workflows, to detect and respond to events

Automatic provenance capture, open provenance model

Costing model, resources include time, power, data xfer

Integrated data storage and access

Integrated visualization tools

Fault tolerance, facilitate smart reruns, what-if analysis

Distribution, moving the work closer to the data source

Factory scheduling of workflows

TridentScientific Workflow Workbench

Page 3: Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen
Page 4: Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen
Page 5: Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen

Registry Architecture

Trident UIUI or WF Engine

Unix

SQL Server Cloud DB Others

Mac

Linux

XML

Data ModelAPI (.Net)

Connection API

DataStorage

C# GenCode

ExtensionsConnection

Manager

Data Store Interface

SQLConnection

Cloud DBConnection

XMLFile

Web Service

Notation:

Not in the scope yet

XML to generate data access code and database stored procedures

WorkflowWorkbench

WorkflowMonitor

AdminConsole

Custom UI

SilverlightExcelWeb PartsHTML/XMLetc.

HPC WFManager

Registry UIWorkflowLauncher

DirectQuery

Web Service

Others

Page 6: Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen

One of the largest visible light telescopes4 unit telescopes acting as one1 Gigapixel per telescope

Surveys entire visible universe in 1 weekCatalog solar system, moving objects/asteroidsps1sc.org: UHawaii, Johns Hopkins, …

Pan-STARRS Sky Survey

Page 7: Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen
Page 8: Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen
Page 9: Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen
Page 10: Trident Scientific Workflow Workbench Nelson Araujo, Roger Barga, Tim Chou, Dean Guo, Jared Jackson, Nitin Gautam, Yogesh Simmhan, Catharine Van Ingen