MAYER SLCTALK FINAL - Ocean Leadershipoceanleadership.org/wp-content/uploads/2009/07/ooi_di_mayer... · but scientific community input will be required to ... approach to marine science

ORION/OOIORION/OOICYBERINFRASTRUCTURECYBERINFRASTRUCTURE

CyberinfrastructureCyberinfrastructure: the coordinated aggregate : the coordinated aggregate of software, hardware and other technologies, as well of software, hardware and other technologies, as well as human expertise, required to support current and as human expertise, required to support current and future discoveries in science and engineering. future discoveries in science and engineering.

Fran Berman Fran Berman --SDSCSDSC

ORION D&I WorkshopORION D&I Workshop 27 March 200627 March 2006

Focused on sharing and making greater capabilities Focused on sharing and making greater capabilities available across the science and engineering research available across the science and engineering research communitiescommunities

Allows applications to interoperate across institutions Allows applications to interoperate across institutions and disciplinesand disciplines

Ensures that data and software acquired at great Ensures that data and software acquired at great expense are preserved and easily available to all expense are preserved and easily available to all

Empowers enhanced collaboration over distance and across disciplines.

MORE THAN HIGHMORE THAN HIGH--ENDEND COMPUTING AND CONNECTIVITYCOMPUTING AND CONNECTIVITY

Report of the National Science Foundation BlueReport of the National Science Foundation Blue--Ribbon Advisory Panel on Ribbon Advisory Panel on CyberinfrastructureCyberinfrastructure

Support interactive measurement and integration of Support interactive measurement and integration of physical, chemical, biological, and geological phenomena physical, chemical, biological, and geological phenomena across a range of interacross a range of inter--connected instrument networks.connected instrument networks.

3 types of observatories3 types of observatories

In context of ORION/OOI:In context of ORION/OOI:

Many 10’s of sitesMany 10’s of sites

Many 10’s of instruments typesMany 10’s of instruments types 1000’s of sensors1000’s of sensors

Many 100’s of investigators Many 100’s of investigators For 10’s of yearsFor 10’s of years


The MREFC Account supports stateThe MREFC Account supports state--ofof--thethe--art tools that areart tools that arecentralized in nature, integrate systems of leadingcentralized in nature, integrate systems of leading--edge instruments, and/or sharededge instruments, and/or shared--use networked use networked infrastructureinfrastructure in advancing one or more fields of scientific study. in advancing one or more fields of scientific study. Examples include accelerators, telescopes, research vessels, airExamples include accelerators, telescopes, research vessels, aircraft, craft, earthquake simulators, networked highearthquake simulators, networked high--tech research platforms, and tech research platforms, and advanced computing resources.advanced computing resources. To qualify for MREFC To qualify for MREFC investment, networked infrastructure must exhibit investment, networked infrastructure must exhibit systems characteristics greater than inferred simply systems characteristics greater than inferred simply by the connectivity of its parts.by the connectivity of its parts.

THE MREFC ACCOUNTTHE MREFC ACCOUNT


ORION CYBERINFRASTRUCTURE

COMMITTEE DESIGN GROUPIlkay Altintas UCSD Matthew Arrott UCSDMatthew Arrott UCSD Alan Chave WHOISuzanne Carbotte LDEO John Graybeal MBARIYi Chao JPL Eric Guillemot NEPTUNE CAWu-chi Feng Portland State Ingolf Krueger UCSDJohn Graybeal MBARI Benoit Pirenne NEPTUNE CAMatt Howard NOAA PMELJason Leigh Univ. IllinoisAndy Maffei WHOILarry Mayer UNHJohn Orcutt OSC LiaisonTom Palmatier JHU/APLBenoit Pirenne NEPTUNE CA

CI FORUMCI FORUM

TUESDAY EVENINGTUESDAY EVENING

8PM 8PM

Grand Ballroom FGrand Ballroom F


ORION CYBERINFRASTRUCTURE ORION CYBERINFRASTRUCTURE

Overarching Principles: “the BILITIES”Overarching Principles: “the BILITIES”

•• InteroperabilityInteroperability•• ScalabilityScalability•• ExtensibilityExtensibility•• ReliabilityReliability•• Public AccessPublic Access


ibilityibility


There must be a “centralized facility” that takes There must be a “centralized facility” that takes responsibility for observatory operations responsibility for observatory operations –– “The “The Common Operating Infrastructure”Common Operating Infrastructure”

OVERARCHING STATEMENTSOVERARCHING STATEMENTS::


There must be a “focused authority” that takes There must be a “focused authority” that takes responsibility for observatory operations responsibility for observatory operations –– “The “The Common Operating Infrastructure”Common Operating Infrastructure”

account management account management (what experiments are being done?)

performance and fault monitoring performance and fault monitoring –– (is it working the way it should?)

governance and security policy governance and security policy –– ((is it being used the way it should be?)

resource management resource management –– ((what is available -including scheduling conflicts and real-time configuration changes?)

workflow management workflow management –– (what is the most efficient way to operate the system?)

OVERARCHING STATEMENTSOVERARCHING STATEMENTS::

GOVERNANCE AND CYBERGOVERNANCE AND CYBER--SECURITYSECURITY

“The Common Operating Infrastructure” will establish “The Common Operating Infrastructure” will establish a governance infrastructure that:a governance infrastructure that:

•• controls access and registry for all services controls access and registry for all services and data productsand data products

•• includes, authorization authentication, includes, authorization authentication, auditing, and identity management including auditing, and identity management including different levels of access to users with different levels of access to users with different levels of securitydifferent levels of security

•• addresses national security concernsaddresses national security concerns


“The Common Operating Infrastructure” will provide authority and ability to resolve scheduling conflicts and limited resource conflicts including:

Tracking and allocation of resource usage

A policy-based decision support system must be established including security -- they know how….

but scientific community input will be required to establish appropriate policies

RESOURCE MANAGEMENTRESOURCE MANAGEMENT

DATA RETURN DATA RETURN –– Near realNear real--time data return from all observatories at time data return from all observatories at designed network latencies (best available delivery cycle designed network latencies (best available delivery cycle ---- for global for global observatories add satellite latencies, for moorings add reportinobservatories add satellite latencies, for moorings add reporting g latency)latency)

TWOTWO--WAY AND ONWAY AND ON--DEMAND COMMUNICATIONS DEMAND COMMUNICATIONS –– the ability to the ability to send commands for instrument control at designed network latencisend commands for instrument control at designed network latencies es (best available delivery cycle (best available delivery cycle ---- for global observatories add satellite for global observatories add satellite latencies, for moorings add command latency). latencies, for moorings add command latency). INTERACTIVITY!!!INTERACTIVITY!!!

““The Common Operating Infrastructure” will support:The Common Operating Infrastructure” will support:


INSTRUMENT TO INSTRUMENT COMMS:INSTRUMENT TO INSTRUMENT COMMS:Peer-to-peer communications and networks of

sensors – need to account for fact that instruments may invoke activity of other instruments without human intervention (fleet of AUV’s).

“What is going on in the terrestrial wireless world today will be ported to the seafloor over the next decade.” (A.C.)


DATA INGEST:

All data collected under ORION/OOI will be acquired by, deposited in, and distributed from the “ORION Data Repository” -- most likely a distributed grid of storage devices. “The Common Operating Infrastructure” will be responsible for:

1- spatial and temporal indexing and registration with appropriate catalogue construction and provenance information (metadata) of all data sets as provided by PI or system).


““The Common Operating Infrastructure” will specify interface The Common Operating Infrastructure” will specify interface and metadata requirements for output of domain specific and metadata requirements for output of domain specific productsproductsAt a minimum this metadata must address data provenance, descripAt a minimum this metadata must address data provenance, description, tion, quality, maturity level (raw data streams, automated QC, processquality, maturity level (raw data streams, automated QC, processed, ed, derivative products), collection context, and support interoperaderivative products), collection context, and support interoperability with bility with other observatories, archives, and databasesother observatories, archives, and databases

METADATAMETADATA

Metadata is an essential key to QA, data integration and Metadata is an essential key to QA, data integration and analysis analysis ---- metadata that meets or exceeds national metadata that meets or exceeds national requirements (e.g., FGDC ) must be provided with all data.requirements (e.g., FGDC ) must be provided with all data.


2- Provide initial QA/QC – tools for automated validation

3- Support of delayed mode QA/QC by instrument providors or others through notification and archive updates

4- Support of “associated datasets” which can (at the Observatory’s discretion) be incorporated into “The ORION Data Repository”

5- Track multiple versions of data sets (version control)

DATA INGEST: “The Common Operating Infrastructure” will:


DATA DISTRIBUTION:A data distribution process that will provide:

1- a robust data archive mechanism

2- online access in accordance with ORION DATA POLICY and in a form that supports interoperability

3- both Query and Subscribe – data distribution services


••The Ocean Observatory Initiative, by establishing an The Ocean Observatory Initiative, by establishing an interactive, distributed and integrated network of ocean interactive, distributed and integrated network of ocean sensors and software, represents a fundamentally new sensors and software, represents a fundamentally new approach to marine science that requires new policies approach to marine science that requires new policies regarding data access and distribution. regarding data access and distribution.

••Both the nature of the science to be addressed under Both the nature of the science to be addressed under OOI, and the auspices under which the data are collected, OOI, and the auspices under which the data are collected, demand that demand that OOI data be rapidly disseminated, open, OOI data be rapidly disseminated, open, andand freely available freely available (within constraints of national (within constraints of national security).security).

ORION/OOI DATA POLICYORION/OOI DATA POLICY


Rapidly disseminatedRapidly disseminated implies that data will be made implies that data will be made available as soon as technically feasible (i.e., there will be available as soon as technically feasible (i.e., there will be no delays in distribution for nonno delays in distribution for non--technical, nontechnical, non--security security reasons). reasons).

OpenOpen implies that data distribution will be unrestricted implies that data distribution will be unrestricted (unless for reasons of national security).(unless for reasons of national security).

Freely availableFreely available means that data will be distributed means that data will be distributed without charge. There may be charges for the cost of without charge. There may be charges for the cost of reproduction and delivery unless access is obtained through reproduction and delivery unless access is obtained through the Internetthe Internet

ORION/OOI DATA POLICYORION/OOI DATA POLICY

DATA INTEGRATION AND ANALYSIS:A basic set of applications that will:

1- provide efficient 4-D data discovery, integration, analysis, and visualization capability including semantic cross-referencing capabilities

22-- facilitate data access in forms accessible facilitate data access in forms accessible via other data fusion systems, programmatic via other data fusion systems, programmatic interfaces, and services (e.g., GIS, visualization, interfaces, and services (e.g., GIS, visualization, etc)etc)

33-- provide efficient interaction with other data, provide efficient interaction with other data, databases and MODELS databases and MODELS –– e.g. IODP, OAPS, e.g. IODP, OAPS, NGDC, CLASS, DAAC, ROMSNGDC, CLASS, DAAC, ROMS

• Lewis Rothstein (U. Rhode Island) - Chair• John Allen (Oregon State U.)

• Fei Chai (U. Maine) • Shuyi Chen (U. Miami)

• Bruce Cornuelle (Scripps)• Katja Fennel (Rutgers)

• Pierre Lermusiaux (Harvard) • Raghu Murtugudde (U. Maryland)

• Yvette Spitz (Oregon State U.) • Cisco Werner (U. North Carolina)

ORION Modeling Committee

• To help establish the role of each of a hierarchy of coupled numerical physical-biogeochemical ocean/atmosphere/geophysical models (including global, regional and coastal models) in the evolving plan for ocean observatories.– “Forward” Models– Ocean System Simulation Experiments (OSSEs)

• To provide advice on the current status of the different types of models required to achieve the objectives of the ORION program, and the primary issues or needs for development of those models.

• To explore ways in which the ocean modeling community should interface with the atmospheric and geophysical modeling community for achieving the objectives of the ORION program.

Terms of Reference

General Recommendations

Foster a scientific approach where ocean observatories are fully integrated with ocean modeling for better understanding and predictions through:

Dynamical interpolation and synthesis of multiple data sets

Utilization of models to iteratively guide the sampling design and adaptive sampling plans (e.g. OSSEs)

Evaluation and improvement of models, including error estimates

General RecommendationsUtilize models within ORION to study, synthesize, discover and resolve (multi)-scale interactions, physical-biogeochemical-ecological coupling and ocean/earth processes, e.g.:

Quantify interactions among coastal and global scale, mesoscale and sub-mesoscale that are not directly observable

Assess uncertain, or discover new, interdisciplinary ocean coupling

Monitor, explain and better forecast climate dynamics, ecosystem evolution, and earthquakes

How to do it - General•• Encourage modeling proposals as integral part Encourage modeling proposals as integral part of ORION observatory and/or analysis of ORION observatory and/or analysis RFPsRFPs

• Facilitate modeling evaluation, assessment and sharing of knowledge and skill levels between observatories

• Serve resource management needs by providing prediction capability for specific regions

• Foster collaborations among different universities, institutions, government agencies and modeling centers

How to do it - OperationalPlan modeling continuity through the long life of ORION

•Support efficient & cooperative transition of methodologies & technologies among research & operational centers, involving universities, institutions and other agencies

•Preliminary proposal: Establish ‘Modeling Centers’ for:Maintaining hierarchy of evolving interdisciplinary models (e.g.

from ‘reduced’ process-oriented models to operational forecast systems)

•Model repositories with support for writing modeling manuals, etc.

Archiving and disseminating model data sets Flexible formats for wide range of research endeavors

Linking to other IOOS activities & operational centers (e.g. NCEP)Coordinating between the ORION research community and the

‘Modeling Centers’ through:•Visiting scientist and community postdoc programs•Student workshops

Modeling ForumModeling Forum

Wednesday 8:00PMWednesday 8:00PM

Grand Ballroom FGrand Ballroom F

Documents

MAYER SLCTALK FINAL - Ocean Leadershipoceanleadership.org/wp-content/uploads/2009/07/ooi_di_mayer... · but scientific community input will be required to ... approach to marine science