Content-Infused OGC Web Services Enabling Dynamic Quality Assessment in Observing Systems

Content-Infused OGC Web Services

Enabling Dynamic Quality Assessment in Observing Systems

Janet J. Fredericks Applied Ocean Physics & Engineering Woods Hole Oceanographic Ins=tu=on

Carlos Rueda

Monterey Bay Aquarium Research Ins=tute

Workshop on Sensor Web Enablement 2011 (SWE 2011) As part of The 2011 Cybera Summit on Data For All -‐ Opening up the Cloud October 6-‐7, 2011, Banff, AB, Canada

1

Research and survey data served with

associated metadata in a few speci5ic formats with

associated software installations

Sensor Manufacturers <html/> and manuals

NOAA/NDBC provides 24/7 QC;

Feeds National IOOS backbone;

NOAA/NODC provides national archival for valued

data sets (they can determine the value)

NSF/OOI; NSF/R2R; NSF/BCODMO provides community-‐based integration with tools and QC, along with discovery

and mapping opportunities

Real-‐time Rapid Response integration can be accomplished quickly and reliably by communicating metadata

in standards-‐based systems

Modeling using translation tools from the cloud, modelers have access to a broader

source of information

ANYONE By fully describing data, sensors and

processing with associated provenance, data can be discovered and explored for

any program

Data Provider Nightmare!

User-‐based Output 2







Data Provider (and Consumer)

Nightmare!

User-‐based Output







IOOS

GEOSS

3

















any program

Data Provider (and Consumer)

Nightmare!

User-‐based Output 4

Standards-‐based (machine-‐to-‐machine harves=ng)


associated metadata in a community-‐

adopted, standards-‐based framework

Sensor Manufacturers and domain experts develop sensor and

processing descriptions in standards-‐based

encodings













any program

Converters; QC algorithms; vocabularies & ontologies; analysis and

visualization tools

GOAL: two paths Described well enough for assessment of

data for specified use and for a repurposed applica<on

User-‐based Frameworks 5

Data Provider needs to communicate how the sensible proper=es were turned into observa=ons!

Sensor Logging/Processing

Web Service

6

Quality Assurance (QARTOD)

QC Tests

MetaData Requirements

Seman=c Tools (MMI)

Vocabulary Registry & Term Mapping

Ontology Development & Registry

Standards (OGC)

Syntactac=c Interoperabilty (SensorML/O&M)

Standards-‐based web services (SOS)

Guides/Implementa=on (OOStethys/OGC-‐OIE/OpenIOOS)

So_ware Packages/ cookbooks

Observa=ons Based SOS

Quality – to – OGC (Q2O) -‐ IntegraKon of sensor & processing descripKons aimed towards the ability to assess quality of observaKons

Project to Address Data Quality in Sensor Web Enablement Frameworks

BACKGROUND

7

Domain Experts

Sensor Mfgrs

Operators

IT Specialists

Content Specifica=ons/

SWE Implementa=on

Model

What informaKon is needed to assess quality of data and how do we implement it into an Sensor ObservaKon Services (SOS)?

Community-‐based Development

8

OWL/RDF Vocabularies/ontologies

SensorML

Data

HARVESTS

REFERENCES RESOLVES

DescribeSensor (SensorML)

GetObserva@on (O&M)

CL I ENT

SOS

9


SensorML

Data

SOS

HARVESTS

REFERENCES RESOLVES

DescribeSystem (SensorML)

GetObserva@on (O&M)

CL I ENT

<sml:output name="swell"> <swe:Quan=ty defini=on="hkp://mmisw.org/ont/mvco/proper=es/swell"> <swe:uom code="cm"/> </swe:Quan=ty> </sml:output>

10


SensorML

Data

SOS

HARVESTS

REFERENCES RESOLVES

DescribeSystem (SensorML)

GetObserva@on (O&M)

CL I ENT

11

Building ontologies

12

Five Role-‐based Categories of SensorML

Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on)

Process Files (SensorML -‐> DescribeSensor)

QC Tests -‐ are classified as QC tests (QcCategory) and may have associated QC flags as output

Processing Descrip=ons -‐ to describe how an observa=on is derived from sensor output

Sensor/Deployment Files (SensorML -‐> DescribeSensor)

Original Equipment Manufacturer (OEM) File Descrip=on of Sensor Model

Configura=on/Ownership/Deployment (CONDEP)File Descrip=on of Sensor Configura=on, Deployment and

Event History Details

Observable Proper=es

SML system

13











SML system OEM Model DescripKon Created by

manufacturer and available for anyone using the par<cular model – accuracy; error analysis etc

specific to the model

14











SML system

ConfiguraKon and Deployment File Working with OEM

file/Sensor Manufacturers and Marine Operator

describe this instance: contacts (operator), parameters (set-‐up specifica=on that can affect accuracy or

relevance to repurposed

applica=on), posi=ons, events rela=ng to sensor health

15











SML system

QC Tests Data manager

describes QC tests and associated flags; inputs

to tests and parameters are specified – the

parameters can be =me-‐stamped

16











SML system

Processing DescripKons

Data managers and domain experts provide authorita<ve reference and descrip<ons of processing used for derived proper=es

17











SML system

18

How does this model enable dynamic quality assessment? 1)  ROLES -‐ Provides a template for instrument manufacturers/data managers/

marine operators to describe details that describe quality related informa=on in a standards-‐based encoding

2)  CONNECTIONS -‐ Through the connec=ons list in SensorML, the QC flags can be associated with the QC tests with associated parameters

3)  ENABLING SEMANTIC MAPPINGS -‐ Through inclusion of associated URLs encoded with each term, ontologies and mappings can be built to define rela=onships across poli=cal and research domains promo=ng interoperability and interdisciplinary research for all geospa=al, sensor-‐based observa=ons.

4)  Encoding thorough descrip=ons of processing and process lineage promotes beker understanding of the observa=ons, which enhances the value and reliability of the data.

The original provider has no knowledge of how the data may be used! We need to communicate enough informa=on to enable assessment for a par=cular use beyond the project design! Did they sample fast enough for the new applica=on? Or long enough? Is the repor=ng frequency adequate?

19

Next Steps •  Build beker SensorML editors and registries -‐-‐ making things easier and promo=ng fully-‐described sensor and processing lineage. This will promote adop<on of the use of standards and more fully-‐described systems!

•  Encourage manufactures, data managers and domain experts to create meaningful vocabularies including authorita=ve references to processing algorithms, with figures, equa=ons, etc. and to register the vocabularies, providing resolvable links in a standards-‐based encoding (OWL)

•  Provide tools and opportuni=es for domain experts to create and register ontologies, associa=ng terms in RDF (Is ThisQCtest the same as ThatQCtest? Does this QC flag have th same meaning as thatQCflag)

20

Conclusions •  Structured Q2O (hkp://q2o.whoi.edu) SensorML serves as a

model for any sensor-‐based, in situ observa=ons; each component can be implemented by the responsible party and adop=on of the model can happen in stages.

•  By associa=ng QC flags with qc tests, processing methods with observa=ons, and fully-‐describing how observable proper=es become observa=ons knowledge about quality will be shared.

•  By referencing (encoding) resolvable terms, ontologies can be built and registered to foster interoperability across-‐domains and poli=cal boundaries.

The ability to dynamically assess data quality will provide a

trusted founda<on for observing systems

21

Technology

Content-Infused OGC Web Services Enabling Dynamic Quality Assessment in Observing Systems