Upload
ledang
View
213
Download
0
Embed Size (px)
Citation preview
1Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Building Virtual Earth Observatories Using Scientific Database and Semantic Web Technologies
Kostis [email protected]
Database Architectures group
2Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Motivation
3Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Motivation (cont’d)
~20 PB
4Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
About gmes and data geese and golden eggshttp://esamultimedia.esa.int/docs/EarthObservation/Open_Data_Study_Final_Report.pdf
About Copernicus and Data: Geese and golden eggs
5Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
About gmes and data geese and golden eggshttp://esamultimedia.esa.int/docs/EarthObservation/Open_Data_Study_Final_Report.pdf
About Copernicus and Data: Geese and golden eggs
€
6Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
About gmes and data geese and golden eggshttp://esamultimedia.esa.int/docs/EarthObservation/Open_Data_Study_Final_Report.pdf
About Copernicus and Data: Geese and golden eggs
€
7Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
About gmes and data geese and golden eggshttp://esamultimedia.esa.int/docs/EarthObservation/Open_Data_Study_Final_Report.pdf
About Copernicus and Data: Geese and golden eggs
€
8Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
About gmes and data geese and golden eggshttp://esamultimedia.esa.int/docs/EarthObservation/Open_Data_Study_Final_Report.pdf
About Copernicus and Data: Geese and golden eggs
€
9Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
About gmes and data geese and golden eggshttp://esamultimedia.esa.int/docs/EarthObservation/Open_Data_Study_Final_Report.pdf
About Copernicus and Data: Geese and golden eggs
€
10Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
About gmes and data geese and golden eggshttp://esamultimedia.esa.int/docs/EarthObservation/Open_Data_Study_Final_Report.pdf
About Copernicus and Data: Geese and golden eggs
€
11Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
About gmes and data geese and golden eggshttp://esamultimedia.esa.int/docs/EarthObservation/Open_Data_Study_Final_Report.pdf
About Copernicus and Data: Geese and golden eggs
€
Free and Open Data Policy
for the Sentinel Satellite Data
12Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
GMES Value Chain
Contributing Missions
Sentinel Satellites
Value-Adding or
downstreamProcessing
Sensors
Users
Satellite Imagery
In-Situ data
GMES Services
Information
Products
About gmes and data geese and golden eggshttp://esamultimedia.esa.int/docs/EarthObservation/Open_Data_Study_Final_Report.pdf
13Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
14Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Cataloguing Service& Metadata Creation
Processing Chain(SciQL based)
HotSpots
Back End: MonetDB / Strabon
Fire monitoring applicationAdvancements - Integration of the TELEIOS technologies
Data Vault
Eumetsat @ 9.5°East
Raw Data
• Corine Landcover• Admin Boundaries• POIs
External Sources
Front End: GUIMap Element
• Search & Display
• Search for raw & Processing
• Real-time Fire Monitoring
• Refinement (Post-Processing)
• Linked Data
GeospatialOntology
Web accessbased on Semantics
Linked Geospatial Data
Semantictechnologies
15Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Data Acquisition
16Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Data Ingestion: Data Vaults
17Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Data Processing
Cropping Georeferencing Classification
18Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
SciQL: A Query Language for Science ApplicationsClassification
Classification
19Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Improvements
Improving the fire monitoring service using Semantic Web technologies
Representing fire related products using ontologies
Enriching products with linked geospatial data
Improving accuracy with respect to:
Underlying land cover/land use
Persistence in time
http://bit.ly/FiresInGreece
20Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
StrabonA Scalable Geospatial RDF Store
stRDFgraphs
stSPARQL/GeoSPARQL
queries
Query Engine
Parser
Optimizer
Evaluator
Transaction
Manager
Storage Manager
Repository
SAIL
RDBMS
Strabon
PostGIS
GeneralDB
Sesame
21Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Product ingestion, processing and refinement is completed in less than 12 seconds
More refinement operations to be added later given the five minutes time frame
Fire Monitoring ServicePreliminary evaluation – Processing times
101
102
103
104
Re
sp
on
se
Tim
e (
mse
c)
June 2007 - August 2011
StoreMunicipalitiesDelete in sea
Invalid for firesRefine in coast
Time persistence
22Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Discussion
Use higher-level languages, stop worrying about how to store and manage metadata, just focus on the actual processing
Express common Earth Observation operations easily using the SciQL and stSPARQL/GeoSPARQL queries instead of using a lengthy C program
Rapid prototyping and new refinement modules without the need to recompile everything
23Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Following the advice of Jim Gray
“How can you, a computer scientist, engage with a domain scientist (or group)?…
Well, first you have to learn a bit of their language. This generally involves mastering the introductory text for that domain – it is painful and you can skip this step if you are in a hurry, but you will end up doing this work in any case. Doing it early is the most efficient way. In parallel you have to form a working relationship with the domain experts (scientists.) You need to put in enough face time so that they are not surprised to see you. This goes hand-in hand with developing a common language. The converse of this, the domain scientists you are working with need to explore some of the things done in computer science and in other disciplines so that they have a sense of what is possible and what is almost possible.”
Quote from article:Where the Rubber Meets the Sky: Bridging the Gap between Databases and Science.Jim Gray, Alexander S. Szalay. MSR-TR-2004-110, October 2004.IEEE Data Engineering Bulletin, December 2004, Vol. 27.4, pp. 3-11.
23
24Building Virtual Earth Observatories Using Scientific Database and Semantic Web TechnologiesCWI Scientific Meeting, March 27, 2015
Thank you for your attention!
Kostis [email protected]