iMarDIS Integrated Marine Data and Information System http ... · Image / Video –uav and camera...

iMarDISIntegrated Marine Data and

Information System

iMarDISInfrastructure to deliver information solutions

Dave MillsThomas Prebble, Graham Worley

&Cathy Blakey, Gwyn Roberts

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarISIntegrated Marine Data and

Information System

SEACAMS DataMeasure once, use many times

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDISIntegrated Marine Data and Informtion

System

SEACAMS Data• data collected over the lifetime of SEACAMS1 & SEACAMS2

• existing data from School of Ocean Science

Water quality (nutrients, chlorophyll, microbiology and contaminants

Trawl samples (plankton, micro nekton)

Tide gauges (tidal height)

ADCP acoustic doppler current profiler (current, suspended sediments and waves)

CTD (salinity and temperature depth profiles)

Wave buoy (near real-time data including wave height, period, direction, water temperature)

Geophysical – (high resolution bathymetry and backscatter from multibeam)sub-bottom seismic

High resolution laser scanning of intertidal zonegrab sampling – bottom classification, particle size analysis, contaminant analysis, ground truthing, benthic sampling

Image / Video – uav and camera footage of seabed flora and fauna, sediment type, topographic information Side Scan Sonar - Seafloor imaging

Single beam - ecological dataLISST (Laser In-Situ Scattering and Transmissometer) –Suspended Sediments

Cetacean data – visual and acoustics

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Making Use of Data

Data Download Service

Derived Data Products

Analytic Tools

Modelling Tools – Combining data and modelling

iMarDISIntegrated Marine Data and

Information System

Data Download Service

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

The Bad and Ugly

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

The New and Shiny

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

The New and Shiny

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Derived Data Products

iMarDISIntegrated Marine Data and

Information System

Digital Terrain Map One informative terrain map to replace a patchwork of files

Data available from EMODNET and GEBCOFine resolution bathymetry available from SEACAMS

Data Selection Algorithm

Automated production of hybrid DTM (variable resolution)

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Analytical Products

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

DataApply Analytical

Function

User FocusedInformationDashboardsDownloadsMobile Apps

Obtain Process Disseminate

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Data• Area Classification• Wave/Tidal Energy

Distribution• Depth• Socio-economic • Cetacean

Abundance• Distance from shore• Seabed

classification

Function mapping data to a 0-100% index of suitability

Interactive Heat-map

ofSuitable areas

Obtain Process Disseminate

Site Classification Tool

Integrated Marine Data and Information Systemhttp://www.imardis.org

Tidal Current

Depth

Distance to inter-connect

Cetacean Abudance

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Modelling Products

iMarDISIntegrated Marine Data and Infoation

SystemModelling

High Resolution

Data

StreamedObservatory

Data

SEACAMSModelling

Wave Height

Tidal Height

Tidal FlowMet Data

HPC OrchestrationHPC OutputTransformations

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDISIntegrated Marine Data and Infoation

System

Integrated Marine Data and Information Systemhttp://www.imardis.org

SEACAMS Model – Higher Resolutionhttp://www.renewables-atlas.info

iMarDISIntegrated Marine Data and Infoation

System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Example - Model Dashboard (http://www.chonos.org)

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Managing SEACAMS Data

iMarDISIntegrated Marne Data and

Information System

Review schemas for metadata – MEDINShadow the workflow from collection through processing to analysis

Single Record – Many Items

SEACAMS Data

Data Collection

Post-Processing

Final Data Output

Generic Metadata (e.g. Cruise)

Processed Data

Package

Raw Data Metadata

Raw Data Package

Processed Data

Metadata

Integrated Marine Data and Information Systemhttp://www.imardis.org

• Create MEDIN compliant metadata with minimal effort

• Ingest large data files

• QA and workflow of data uploads

• Publish to a searchable Metadata Catalogue

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Data Management - Aims

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDIS Metadata Manager

Templated data

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Over 4 TB Ingested so far

Largest file over 12GB

iMarDIS Metadata Manager

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Metadata Workflow management

iMarDIS Metadata Manager

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDIS Metadata Manager

iMarDISIntegrated Marine Data and

Information System

Where to put our data?

Data is high-resolution but not spatially very largeNeed to access data both for download by a person and by machine

for analytics

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDISIntegrated Marine Data and

Information System

Micro-Services Approach

Integrated Marine Data and Information Systemhttp://www.imardis.org

http://www.imardis.org

Integrated Marine Data and Information System

http://www.imardis.org

Integrated Marine Data and Information System

http://www.imardis.org

Integrated Marine Data and Information System

http://www.imardis.org

Integrated Marine Data and Information System

http://www.imardis.org

Integrated Marine Data and Information System

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDIS API

http://www.imardis.org

Integrated Marine Data and Information System

• Data management• Ingestion• Cataloguing• Publishing

• Data management• Ingestion• Cataloguing• Publishing

• Data Download Service • Discover• Download

http://www.imardis.org

Integrated Marine Data and Information System

• Data management• Ingestion• Cataloguing• Publishing

• Data Download Service • Discover• Download

• Bespoke Products & Services

http://www.imardis.org

Integrated Marine Data and Information System

• Data management• Ingestion• Cataloguing• Publishing

• Data Download Service • Discover• Download

• Bespoke Products & Services

• Open Access to API

http://www.imardis.org

Integrated Marine Data and Information System

[{

"sampleTime": "2018-01-23T12:59:55.000Z","shortName": "sea_level","vocabRef": "SDN:P01::ASLVODNP","irn": "f119c7c7-0107-4278-835a-7c2757c43087","uri": "irn:dataparameter:f119c7c7-0107-4278-835a-7c2757c43087","value": 4.892463684192322,"unit": "m"

},{

"sampleTime": "2018-01-23T12:53:56.000Z","shortName": "wind_speed","vocabRef": "SDN:P01::EWSBSS01",

"irn": "64e71b9a-c80d-4d72-ac89-2711929583b4","uri": "irn:dataparameter:64e71b9a-c80d-4d72-ac89-2711929583b4","value": 5.68475292748536,"unit": "mS-1"

},{

"sampleTime": "2018-01-23T12:53:56.000Z","shortName": "wind_direction_from","vocabRef": "SDN:P01::EWDASS01","irn": "d54921c7-c167-491c-a9b3-a74a3690fd00","uri": "irn:dataparameter:d54921c7-c167-491c-a9b3-a74a3690fd00","value": 238.7238721196367,"unit": "degree"

}]

Integrated Marine Data and Information Systemhttp://www.imardis.org

API Access to Data - JavaScripthttps://api-param.imardis.org/rest/latest/getLatestParametersForDeployment/52cca744-4c97-44f4-8c18-7ca85778bf35

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

http://www.imardis.org/raynes.html

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

API Access to Data - MATLABurl = 'https://api-param.imardis.org/rest/latest/getLatestParametersForDeployment/52cca744-4c97-44f4-8c18-7ca85778bf35';payload = '["64e71b9a-c80d-4d72-ac89-2711929583b4"]';options = weboptions('MediaType', 'application/json', 'RequestMethod', 'POST',

'KeyName', 'Accept', 'KeyValue', 'application/json', 'ArrayFormat', 'json', 'KeyName', 'Content-Type', 'KeyValue', 'application/json');

data = webwrite(url, payload, options);sample = strcat({'Latest Wind Speed '}, num2str(data(1).value));sample = strcat(sample, data(1).unit);sample = strcat(sample, {' taken at '}, num2str(data(1).sampleTime));sample

http://www.imardis.org

Integrated Marine Data and Information System

Build Your own Products

iMarDISIntegrated Marine Data and

Information System

3rd Party Derived Products

Cloud-accessible APIs available to ALLDocumentation and Access Keys will be made available

Collaborative R&D with Software Developers / Consulting Companies?

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDISIntegrated Marine Data and Informtion

System

Data LicensingAllows use of our data for business use

Metadata describing our data will always be completely open

Restrictions on re-distribution

Embargo on academic use in publications for 2 years

Publications based on SEACAMS data to always carry attribution

Significant academic publications to carry co-authorship

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDIS

Integrated Marine Data and Information System

Progress To DateTeam Assembled

Review of Data

Metadata Schema Developed (based on MEDIN)

Metadata Manager Developed

Collation of Metadata – Cataloguing the data

Design of some REST Services

Implementation of some REST Services

Upload of Data to Servers CommencedDesign of back-end services

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Publish APIsDocument and Publish APIs?

Next Steps….

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Publish APIsDocument and Publish APIs?

Next Steps….

Data Download ServiceWhat does the MRE community need? Commission front-end UI

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Publish APIsDocument and Publish APIs?

Next Steps….

Data Download ServiceWhat does the MRE community need? Commission front-end UI

Data DiscoveryContinue upload and cataloguing - Publish MEDIN metadata to aggregator sites

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Publish APIsDocument and Publish APIs?

Next Steps….

Data Download ServiceWhat does the MRE community need? Commission front-end UI

Data DiscoveryContinue upload and cataloguing - Publish MEDIN metadata to aggregator sites

Products and ServicesDecide on exemplar applications to highlight API capability that benefits MRE companies

iMarDISIntegrated Marine Data and

Information SystemDirectorDave Mills

d.mills@bangor.ac.uk

Project Manager and Solutions ArchitectGraham Worley

G.Worley@bangor.ac.uk

Gwyn Roberts Software Developers Thomas Prebblegwyn.Roberts@bangor.ac.uk t.prebble@bangor.ac.uk

Cathy BlakeyData Librarian

C.Blakey@bangor.ac.uk

Integrated Marine Data and Information Systemhttp://www.imardis.org

iMarDISIntegrated Marine Data and

Information System

Integrated Marine Data and Information Systemhttp://www.imardis.org

Questions