Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

Cabled Ocean Observatories

The latest on VENUS and NEPTUNE

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

What is VENUS?

• Shallow-water test bed for NEPTUNE• But more than just engineering (lots of science topics)

• Easy access (Saanich Inlet and Strait of Georgia)

• Financed at ~10 MCAD

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

VENUS Project Status

• Saanich Inlet wet plant installation end of October• Shore station near IOS (in a Navy building!)• About 6-7 different types of instruments initially• Test bed for HDTV camera• Data backhaul 100Mb/s to UVic and CANARIE (as

budgeted); seeking DF• Strait of Georgia line in 2006

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

What is NEPTUNE?

• International Project • ~300 MCAD, 30% Canada-70% US, but ∆T

≥ 3 years• Interoperability between the two stages

• NEPTUNE Canada: First of its kind RCO• ~5 nodes, 100’s instruments, 24/7 ops,

100kW power, ~8Gb/s, 2500m depth, >25 years lifetime

• Themes • Science: geology (seismic, tsunamis), gas

hydrates, climate, biology• Engineering: power, lifetime, uninterrupted

operation, Internet in the Ocean, data storage & distribution

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

NEPTUNE Project Status

• Wet Plant:• Selection of main contractor for

wet plant completed. Contract being negotiated.

• Installation in Q2/Q3 ‘07 with a few test instruments

• Shore station in Port Alberni, BC• ~40 MCAD available

• Instrumentation:• Selection of science

instrumentation programs on-going

• Most installed in ‘08• ~7 MCAD available

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

What is DMAS?

DMAS (budget ~7MCAD) has four elements:

• Instrument access & sensor data archival• User data access and retrieval (implementation of observatory data

access policy)

• Instrument control (exploitation of various instrument modes)

• Observatory monitoring (NOC), resource management (power & bandwidth)

➜CI for NEPTUNE Canada

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

DMAS Status

• Budget Issue:

• Wet plant funds short due to NRE costs not entirely shared with US partner

• Savings possibly required from both science instruments and DMAS

• Trying to compensate for shortfall with responses to call for funding proposals

—CANARIE: project on Service Oriented Architecture and workflow management (sensor and archive access through WS, workflow for event management) for DMAS. (With IBM Canada).

—CANARIE: project on sub-sea HDTV camera control and data transfer using WS and UCLP. (McGill)

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

DMAS status

• Spiral Development:• Prototype system with data acquisition, archival, data retrieval

completed in February• Prototype currently being adapted for VENUS use (DB

redesigned)• NEPTUNE system will evolve gradually from the prototype and

the VENUS experience• Preparing accelerated design of the overall NEPTUNE cyber-

infrastructure in response to CANARIE call for proposal

• Instrument Data Acquisition and Control (SOA/Web services)

• Orchestration for Grid dispatching and decomposition of ocean science work-flows (Kepler, GTK, GridX1)

• Evolve DMAS into a data grid bus/hub for persistence and real time (RSS) feeds to subscribers

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

A few OperationalConstraints

• Ops & data center at UVic (not at shore station)• Shore station mostly unmanned, remotely monitored• Three types of users:

• Public access to “pre-cooked” products (images, plots, …)• General scientist ccess to quasi-live or historical data through

thin (web browser), special-purpose middleware (e.g., MATLAB with web and DB i/f) or fat clients (data analysis packages with WS access)

• PIs direct access to own instrument for live access, through shore station (bypassing data center)

• Access security to be guaranteed: • military nervous about anyone in a position to access the shore

station and wet plant remotely, while they require access to intercept hydrophone stream!

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

DMAS Conceptual Design

Users

Privil. Public

Observatory Ops

NOC

Data Center

Post-Proc.

Shore Station

Event detect/

reaction agent

Instrumentagents

Node / extension

agentsWet Plant

(nodes, ext.,

instruments Instrument

agents

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

DMAS Conceptual Design

Driver/agent

Driver/agent

Pre-processing

Pre-processing

Publisher

ArchiveSubscrib

er

MetadataDB

FilesArchiv

e

Real-time

SubscriberReal-

timeSubscrib

erReal-time

Subscriber

Publisher

Driver/agent

Driver/agent

Driver/agent

WarehouseDB

Streaming Instruments

Non-streaming Instruments

ArchiveAccess

ArchiveAccess

Event detection

Ins. Monitoring

Event detection

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

VENUS Instruments

• First Array: (Saanich inlet)

• 3 types of CTDs (one with Oxygen sensor)• ADCP• ZAP (vertical and horizontal)• 8MPix moveable digital still camera• 3-way hydrophone array• HDTV camera

Scalar

Complex

Streaming

Data in warehouseMeta-data in DB

Data in file storageMeta-data in DB

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

Expected Usage ofNEPTUNE/VENUS

• Some limited direct interaction to take place:• Occasional control of move-able instruments: cameras, ROVs• Restricted to instrument PI or their team members• But humans typically available only 8 hrs/day!

• Expected long-term use mostly through automated event detection and scripted routine maneuvers:• Value checks against hard-coded thresholds or the instrument’s own

long-term trends or correlation of multiple instruments outputs• Reactions as commands to e.g., turn on cameras and lights when

“something” is detected• Reactions contingent to other phenomena: e.g., activate water column

profiler winch when water current strength small enough• Automated extraction from video and audio stream (either live or off-

line), populating “content databases” that can subsequently be searched (e.g., instantaneous fish count in still images or video)

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

Networking Issues

• Access security

• Protect assets: authentication, authorizations

• US Navy, DND requests: remote access to shore station: re-direction of data flows at specific times

Joint Techs Workshop, Vancouver, BC, July 19, 2005

Benoît Pirenne, Ass. Director, IT, NEPTUNE Canada

Conclusions

• VENUS & NEPTUNE development progressing

• Projects are exciting from a scientific, technological and education/outreach point of view

• Need to convince science community to change its approach towards doing ocean science: exploit the power!

• Looking at opportunities to create a very modern cyber-infrastructure