Trident Scientific Workflow For Neptune Roger S. Barga Architect, Technical Computing Group...

TridentTridentScientific Workflow Scientific Workflow For NeptuneFor NeptuneRoger S. BargaArchitect, Technical Computing GroupMicrosoft Corporation

Trident Project Trident Project ContributorsContributors

Monterrey Bay Aquarium Research InstituteJim Bellingham

Yanwu Zhang

Mike Goding

University of WashingtonKeith Grochow, Dept of Computer Science

Mark Stoermer, Dept. of Oceanography

Donald Averill, Dept. of Oceanography

Microsoft Technical Computing GroupLuciano Digiampietri, intern, UNICAMP Brazil

Nolan Li, intern, Johns Hopkins University

Roger Barga

Project NeptuneProject Neptune

North East Pacific Time-Series Undersea Networked Experiment

The world’s first plate-scale undersea observatory

Project Neptune: ThemesProject Neptune: Themes

Engineering ChallengesDelivering power and internet into the ocean, uninterrupted operation, device lifetime,

data processing and storage

Scientific ResearchPlate tectonic processes Regional ocean/climate dynamicsGas hydrates, etc

Disparate, high volume data sets and streams

Several types of CTD devices, ROVs, AUVsADCPZAP (vertical and horizontal)8MPix moveable digital still camera3-way hydrophone arrayHDTV camera

From raw data to useable data productsData cleaning, analysis, regridding, interpolation

Support real time, on-demand visualization

Technical ChallengesTechnical Challenges

Scalar

Complex

Streaming

Technical ChallengesTechnical Challenges

Support a variety of users interacting with systemCommunity Easy access to regularly generated data products (model output, images, visualizations, etc.)

Researchers access to quasi-live or historical data through thin client (web browser), ability to both access data and create visualizations on demand, author both data analysis pipelines and visualizations remotely

PIs direct access to their own instrument for live access, add new instruments, introduce new analysis codes and algorithms into the system

Never been done before in oceanographyRequirements not easily obtained

Principal Goals Of TridentPrincipal Goals Of Trident

Allow users toAutomate tedious data cleaning and analysis pipelines.

Explore and visualize data, regardless of source.

Compose, run and catalog experiments, save results.

A workflow starter kit, one that will allow users to easily extend Trident functionality.

Learn by exploring and visualizing ocean & model data.

By…Allowing experts to author custom workflow activities, but basic users aren’t forced to see the details.

Allow user access mostly through a web portal, one that is intuitive and requires nominal local resources.

A Quick Look At TridentA Quick Look At TridentScientific workflow workbench for Scientific workflow workbench for oceanographyoceanographyPopulate Windows WF with custom activities

Introduce gridded data structures;

Define basic operators (data transformations);

Implemented as custom activities;

Introduce parameterized activitiesEasier for users to design workflows

Tool to convert custom to parameterized activities

Invoke and author workflows via web browser

Persistent workflows, checkpoints (stop-revise-rerun)

Internal Data Structures for Internal Data Structures for Spatiotemporal DataSpatiotemporal Data

Oceanographic DataSpatiotemporal information stored in CDF and NetCDF files, in various formats

Internal data structuresISTem3DCollection: A collection of spatiotemporal points. For each point there is an collection of objects to represent measured values;

HyperCube4DOfDoubles: A four dimensional hypercube (grid). For each point there is an array of doubles to represent measured values;

Custom Activities In Windows Custom Activities In Windows WFWF

Parameterized ActivityParameterized Activity

Converting Custom To Converting Custom To Parameterize ActivitiesParameterize ActivitiesThe conversion is made automatically:

Custom Activity

Parameterized Activity (web accessible)

Converting Custom To Converting Custom To Parameterize ActivitiesParameterize ActivitiesParameterized , and now web accessibleParameterized , and now web accessible

This tool is designed to assist users with workflow creation and execution in the described system. It rovides a simple point and click interface and does not require users to write any code or XML.

Remote Authoring Via Web Remote Authoring Via Web BrowserBrowser

Luciano Digiampietri (UNICAMP), Nolan Li (JHU)Interns, Summer 2007Technical Computing Group at Microsoft

To Sum UpTo Sum Up

Tridentis a very young project (3 months old)growing by application pull from its contributorsmost features are now being designed jointlyIs not quite an alpha release but deployed at both UW and MBARI

The screen shots were the baby pictures!

There is a lot more to cover and work on…

© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after

the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft Research Faculty Summit 2007