Data Integration on the Semantic Sensor Web
Alasdair J G GrayInformation Management Group
University of Manchester
Seminar at Imperial College London7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 2
Overview of the Talk• Motivation: Estuarine Flooding
• Semantic Sensor Web – SemSorWeb
– Requirements
– Architecture
• Ontology-based query access over sensor, streaming, and stored sources
• Demo7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 3
The Solent
7 April 2011
Image: http://www.chandlery-online.com/
Image: http://www.combinedops.com/
• Strait separating the Isle of Wight from English mainland
• Complex tidal and wave patterns
Data integration on the SemSorWeb – Alasdair J G Gray 4
The Solent
7 April 2011
Images: http://www.wikipedia.com/
Data integration on the SemSorWeb – Alasdair J G Gray 5
Estuarine Flood Warning• Financial implications
– Damage– Loss of business
• Personal factors– Emotional
• Flood prediction– Early warning/forecasting– Locations– Severity
• Requires correlating– Sea-state data– Weather forecasts– Details of sea defences
• Response Planning– Evacuation routes– Personnel deployment– …
• Requires more data– Traffic reports– Shipping– …
7 April 2011
Image: http://www.metro.co.uk/
Data integration on the SemSorWeb – Alasdair J G Gray 6
Flood Risk Alert: English South Coast
• Detect conditions likely to cause a flood
“Provide details of overtopping events in the Solent region with high wind speed observations” • Need to correlate data from
multiple sources• Sensor data• Web feeds• Databases and other
sources• Present data model in terms of
the user domain: e.g. Flood risk assessment
...
...
...
Flood defences data (database)
Other sources
Meteorological forecasts
7 April 2011
Real-time sensor data
Wave,Wind,Tide
Data integration on the SemSorWeb – Alasdair J G Gray 7
1. Accurate characterisation of conditions that define an event
2. Correlation of data of differing modalities3. Integrating data from heterogeneous data
models4. Discovery of relevant data sources5. Presentation and control of information
7 April 2011
Sensor Web Requirements“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”
“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”
“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”
“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”
“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”
“Provide flood risk details of overtopping events in the Solent region with high wind speed observations”
Data integration on the SemSorWeb – Alasdair J G Gray 8
Ontology Network: Reconcile Terminology
SSN
SWEET
Service
Coastal Defences
Ordnance Survey
Additional Regions
Role
DOLCE UltraLite
Schema
FOAF
Upper
External
SSG4Env infrastructure
Flood domain
7 April 2011
9
Application Tier
Middleware Tier
Data Tier
ApplicationServices
Semantic Registry
Semantic Integrator
Data Source
Connectivity Bridge
Applications
Concrete Resource
SemSorWeb Architecture
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 10
DATA SOURCES
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 11
Channel Coastal Observatory: Observing Sea-State
43 sensors deployed around UK coast• Measuring
– Tides (7)– Waves (24)– Weather conditions (12)
• On and off shore• Bespoke hardware• Fixed functionality• Fixed (low) data rate
– No bursts• Central data centre
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 12
Publishing CCO Data• Relational data streams
– envdata_sandownpier_tide (ts:int, Observed:float, Tz:float, Hs:float, HMax:float, Tp:float)
• Streaming data service• Semantically annotated metadata
description– Service description– Spatiotemporal data coverage– Thematic data coverage
• Tide height7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 13
Environment Agency: Flood Defences Database (NFCDD)
• Flood defence details– Type of defence
• Wall• Sand banks• …
– Location– Height– …
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 14
Publishing NFCDD Data• Relational database
– locations (id:int, latitude:decimal, longitude:decimal,location:string,storm_threshold:decimal)
• Stored data service• Semantically annotated metadata
description– Service description– Spatiotemporal data coverage– Thematic data coverage
• Sea defences7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 15
INTEGRATING DATA
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 16
Abstract Problem
Stored dataSensor
Network
Integrator
7 April 2011
Sensor Network
Stored data service
Streaming data service
Streaming data service
Data integration on the SemSorWeb – Alasdair J G Gray 17
Data sourceData
streamQuery
capabilitiesData
access
Types of Heterogeneity
Stored dataSensor
Network
Integrator
7 April 2011
Sensor Network
Stored data service
Streaming data service
Streaming data service
Data semantics
Data integration on the SemSorWeb – Alasdair J G Gray
Querying Approach• Use ontologies as common model
Requires:– Representation of RDF stream– Expressing continuous queries over an
RDF stream– Establishing mappings between
ontology models and data source schemas
– Accessing data sources through queries over ontology model
7 April 2011 18
Data integration on the SemSorWeb – Alasdair J G Gray 19
RDF Stream• Named graph• Continuously updating• Triples annotated with timestamp
7 April 2011
STREAM http://www.semsorgrid4env.eu/ccometeo.srdf......( <ssg4e:Obs1, rdf:type, cd:Observation>, ti ),( <ssg4e:Obs1, cd:observationResult, “34.5”>, ti ),( <ssg4e:Obs2, rdf:type, cd:Observation>, ti+1 ),( <ssg4e:Obs2, cd:observationResult,”20.3”>, ti+1 ),......
cd:Observation
xsd:double
cd:observationResult
Data integration on the SemSorWeb – Alasdair J G Gray 20
SPARQLStream
PREFIX cd: <http://www.semsorgrid4env.eu/ontologies/CoastalDefences.owl#>PREFIX sb: <http://www.w3.org/2009/SSN-XG/Ontologies/SensorBasis.owl#> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> RSTREAMSELECT ?windspeed ?windts FROM STREAM <http://www.semsorgrid4env.eu/ccometeo.srdf> [ NOW – 1 MINUTE TO NOW STEP 5 MINUTES ] WHERE { ?WindObs a cd:Observation; cd:observationResult ?windspeed; cd:observationResultTime ?windts; cd:observedProperty ?windProperty; cd:featureOfInterest ?windFeature. ?windFeature a cd:Feature; cd:locatedInRegion cd:SolentCCO. ?windProperty a cd:WindSpeed. }
7 April 2011
cd:Observation
xsd:double
cd:observationResult
cd:Feature
cd:featureOfInterest
cd:Property
cd:observedProperty
cd:Region
cd:locatedInRegion
“Every 5 minutes give me with the wind speed observations over the last minute in the Solent Region ”
Data integration on the SemSorWeb – Alasdair J G Gray 21
Semantic Integrator
Semantic Integrator
7 April 2011
Streaming Source
Stored data
Stored data
Streaming Source
Tuples
Tuples
Tuples
Tuples
Q
SPARQLStream query over ontology
Data Translator
[[Q]]
SPARQL bound variables
Query Translator
q
S2O Mappings
SNEEql query over source
schema
SNEE Query Evaluation
Engine
[[q]]
Tuples
Data integration on the SemSorWeb – Alasdair J G Gray 22
Query TranslationRSTREAMSELECT ?xFROM STREAM <http://…> [NOW–1 MIN TO NOW STEP 5 MIN] WHERE { ?y a cd:Observation; cd:observationResult ?x.}
• Conjunctive queries
• Extended with windows
• Extended with window-to-stream
7 April 2011
Query Execution
7 April 2011 Data integration on the SemSorWeb – Alasdair J G Gray 23
PREFIX cd: <http://www.semsorgrid4env.eu/ontologies/CoastalDefences.owl#>PREFIX sb: <http://www.w3.org/2009/SSN-XG/Ontologies/SensorBasis.owl#> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> SELECT ?waveheight ?wavets ?lat ?lon FROM STREAM <http://www.semsorgrid4env/ccometeo.srdf> WHERE { ?WaveObs a cd:Observation; cd:observationResult ?waveheight; cd:observationResultTime ?wavets; cd:observationResultLatitude ?lat; cd:observationResultLongitude ?lon; cd:observedProperty ?waveProperty; cd:featureOfInterest ?waveFeature. ?waveFeature a cd:Feature; cd:locatedInRegion cd:SouthEastEnglandCCO. ?waveProperty a cd:WaveHeight. }
Ontologies Streamsvvvvenvdata_rhylflats
Timestamp: longHs : floatLon: floatLat: float
envdata_hornsea
envdata_milford
envdata_chesil
envdata_westbay
S2O Mapping
Observation
WaveHeightProperty
observedPropert
y
hasObservationResult
xsd:float
locatedInRegion
Region
Feature
SPARQLStream
(SELECT Lon,timestamp,Hs,Lat FROM envdata_rhylflats) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_hornsea) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_milford) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_chesil) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_perranporth) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_westbay) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_pevenseybay)
SNEEql
Data Translation
7 April 2011 Data integration on the SemSorWeb – Alasdair J G Gray 24
<ns9:sparql> <ns9:head> <ns9:variable name="waveheight"/> <ns9:variable name="wavets"/> </ns9:head> <ns9:results> <ns9:result> <ns9:binding name="waveheight"> <ns9:literal datatype=xs:double>4.850</ns9:literal> </ns9:binding> <ns9:binding name="wavets"> <ns9:literal datatype=xs:long>1272588663</ns9:literal> </ns9:binding> </ns9:result> <ns9:result> <ns9:binding name="waveheight"> <ns9:literal datatype=xs:double>2.1230</ns9:literal> </ns9:binding> <ns9:binding name="wavets"> <ns9:literal datatype=xs:long>1272587400</ns9:literal> </ns9:binding> </ns9:result> </ns9:results></ns9:sparql>
waveheight wavets
4.850 1272588663
2.1230 1272587400
Data integration on the SemSorWeb – Alasdair J G Gray 25
FLOOD APPLICATION
“Smashing it all together”
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 26
Flood Web Application• Discover relevant sources
– Role– Task– Location
• Mashup data– Layers on a map
• Limited execution environment– Application services provide REST
access to Web services7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 27
Relevant Data Sources
Sensor/Streaming Data
• Sea-state sensor data– CCO– Port authority– WaveNet
• Shipping– AIS
• Traffic Alerts– RSS feed
Contextual Data• Base Maps• Flood defense details• Land usage• Weather forecasts• Flooding forecasts
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 28
DEMO
http://www.semsorgrid4env.eu/services/dynamic-demo
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 33
Conclusions: SemSorWeb1. Accurate characterisation of conditions that
define an event– Declarative queries
2. Correlation of data of differing modalities– Query evaluation over streaming and stored
3. Integrating data from heterogeneous data models
– Ontology-based access to streaming data
4. Discovery of relevant data sources– Semantic registry (not covered in this talk)
5. Presentation and control of information– Application service to support, e.g. smash-ups
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 34
Conclusions• Query-based access to RDF streams
– SPARQLStream • Continuous query language
– S2O mappings• Relational streams to ontology• Global-as-View approach
– Query translation to SNEEql • Query substition
– Query execution by SNEE-DQP
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray 35
Acknowledgements
• Christian Brenninkmeijer• Alvaro Fernandes• Ixent Galpin• Norman Paton
• Jean-Paul Calbimonte• Oscar Corcho
7 April 2011
For more information: http://www.semsorgrid4env.eu/
Data integration on the SemSorWeb – Alasdair J G Gray 36
References1. I. Galpin, C. Y. A. Brenninkmeijer, A. J. G. Gray, F. Jabeen,
A. A. A. Fernandes, and N. W. Paton. SNEE: A query processor for wireless sensor networks. Distributed and Parallel Databases, 29(1-2):31–85, 2011. Special issue on Query Processing in Sensor Networks.
2. A. J. G. Gray, R. García-Castro, K. Kyzirakos, M. Karpathiotakis, J.-P. Calbimonte, K. Page, J. Sadler, A. Frazer, I. Galpin, A. A. A. Fernandes, N. W. Paton, O. Corcho, M. Koubarakis, D. De Roure, K. Martinez, and A. Gómez-Pérez. A semantically enabled service architecture for mashups over streaming and stored data. In European Semantic Web Conference (ESWC 2011), pages 300–314, 2011. To appear.
3. J.-P. Calbimonte, Ó. Corcho, and A. J. G. Gray. Enabling ontology-based access to streaming data sources. In Proceedings of 9th International Semantic Web Conference (ISWC 2010), volume 6496 of LNCS, pages 96–111, Shanghai, China, November 2010. Springer. (Alphabetical authorship, equal responsibility).
7 April 2011