Download ppt - Implementation through European Community Research Projects

State of the art State of the art Technologies for GIS Technologies for GIS

Management SystemsManagement Systems

Implementation through EuropeanCommunity Research Projects

Hellenic Center for Marine Research

Lykiardopoulos Angelos, Iona Athanasia, Lakes Vasilis, Balopoulos Efstathios, Kalkavouras Constantine

IMDIS 2008 HCMR / Hellenic National Oceanographic Data Center

State of the art Technologies for GIS Management Systems:

Implementation through European Community Research Projects

Data Management Historical Brief

At the end of 80’s and early 90’s several Oceanographic Data centers had created Databases with mass storage of spatial Information which concerned Oceanographic research outcomes. This was en enormous step of digital storage because big data volumes organized in Database Systems.

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions




• In other words the data centers owned databases – Based on various RDBMS mechanisms– With their own database Schemas– With their own data vocabulary (if existed)– With raw data in various formats– With Applied various processes of data quality

control (or with no data quality control)

• This effort of course leaded into the creation of several data bases without any Interoperability among them.

Data Management Historical Brief

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




The centralized Data Management Idea…

• But the oceanographic community had the need of homogenized and comprehensive data sets for various regions and even world wide.

• Due to the limitations of information technologies this goal was achieved in the framework of several E.U. projects– Central accumulation of data from various Data Centers took

place– Data sets were processed in common data formats and more or

less same Q.C. procedures. – Finally the dissemination of these Data sets was carried out

mainly via digital media (CD-ROMs, etc).

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




The centralized Data Management Idea…

During this process:• Diverse Data form a wide range of Data Centers were

gathered.• Common homogenization and QC procedures were passed.• And finally some comprehensive Data sets were produced

and disseminated.

But…• The produced Data Sets were stand alone products stored in

digital media.• Without any dynamic mechanism of maintenance and

enhancement.• And without any online access.

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




From Centralized to Distributed model

• Thus, the idea of a distributed Oceanographic Data network among the Data Centers started growing up.

• Since Web technologies were being developed rapidly, became apparent that the above network has to be web based.

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




Main obstacles…

The main obstacles that came through for this idea to be fulfilled were:

• Data bases were in different Schemas and Technologies specific per Data center so there was not any common data interchange mechanism.

• Data were not homogenized among Data centers.• Data quality and Data vocabulary differed from

Database to Database

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




From Non Spatial to Spatial Enabled Databases…

• At the same time another obstacle was that due to premature Data Base technology many RDMBS systems lacked spatial capabilities.

• So it was very difficult for the end user:– To query the Database with spatial criteria– To represent geographically the spatial information and

produce geospatial products– To work interactive with Geo Data through interfaces with

geospatial capabilities– To combine geospatial data with synthetic data products

• Hence, for a distributed Oceanographic Data network to be established, the necessity of GIS enabled Databases to become apparent as well.

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




From Non Spatial to Spatial Enabled Databases…

• At the end of 90’s most of the common used RDBMS were enriched with spatial capabilities and became then powerful mechanisms of integration, exploitation and presentation of spatial information

• Oceanographic data centers, exploited this advantage and started upgrading their databases

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




Interoperability obstacle is remaining…

• But still the luck of interoperability was unsolved. The spatial databases made researchers’ life easier with regard to querying data or representing graphically geospatial information

• …but they were acting without any interconnection among the various Data Centers.

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




Web Services…

At the same time with the efforts of the Oceanographic community to overcome the obstacle of interoperability of diverse data management systems and during the last five years new technologies instead of traditional web access tools aroused and became common known as:

Web ServicesWeb Services

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




SEA DATA SEA DATA NETNET

SEASEARCSEASEARCHH

MEDAR/MEDAR/MEDATLAS IIMEDATLAS II

MEDATLAMEDATLASS

Summarizing the historical brief throughout important E.U. projects

1994

•Efforts in QC•Data homogenization•Centralized management•Distribution with CD ROMs

1998

•Enrichment of existing data sets•Establishment of common protocols in QC and formatting•Enhancement of communication facilities (www, ftp, e-mail etc)

2002

2006

•Development of a pan-european network for ocean and marine data & information management.•Improve the exchange, availability and accessibility of ocean & marine data and information.•Expand the online metadatabases with input of research institutes.•Improve the online accessibility of Sea-Search services & products.

•To network existing oceanographic data centres already nationally funded. •To develop an efficient distributed pan-European marine data management Infrastructure (virtual data center).•To provide on-line access to integrated databases of standardised quality by using adapted communication & information technology (Web Services).

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




Web Services fundamentals

First of all what is a Web Service?

• The hidden idea behind WS is the existence of a universal messaging mechanism, capable to deliver and translate messages for diverse and different systems.

• This feature came to open the road for the solution of interoperability.

• If different and diverse systems can interpret and process requests from each other let them stay different.

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• There are various theoretical definitions of What a Web Service is– “A software system designed to support

interoperable machine-to-machine interaction over a network. It has an interface described in a machine - processable format (specifically WSDL). Other systems interact with the Web service in a manner prescribed by its description using SOAP messages, typically conveyed using HTTP with XML serialization in conjunction with other Web-related standards.” (W3C, 2004)

What is a Web Service?

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




What is a web Service?

• “Web services are a distributed computing architecture. Only this particular architecture makes use of loosely coupled applications, as opposed to tightly coupled applications, to enable applications to communicate. This tightly coupled concept radically affects how information systems will work in the future”

(Clabby 2003)

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




What is a web Service?

• Generalizing we may define that Web services provide a convenient and standardised way of exposing business logic over a network (the Internet) by use of specific components of communication such as

SOAP,UDDI,WSDL

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• UDDI (Universal Description, Discovery and Integration) which provides the ability to dynamically select a service at runtime.

• SOAP (Simple Object Access Protocol) which provides a Standardized messaging structure based on XML.

• WSDL (Web Services Description Language) which describes how the client communicates with a Web service or what a service does or where is located.

Web Service Architecture…PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions



Implementation through European Community Research Projects• In the most simple form a web

service architecture may be depicted as the diagram

Web Service Architecture…PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




Geospatial Web Services

• The rapid development of technologies towards WS concept has produced several sets of Web Services with regard to the area of the WS applicability.

• Especially for Geospatial data, Open Geographical Consortium (OGC) has defined two basic sets of Geospatial Web Services.

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• The Web Map Services (WMS) which are concerned with transforming spatial data into maps (images).

• The Web Feature Services (WFS) which are concerned with direct access to data - reading, writing, and updating data.

WMS and WFS…

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• Furthermore, more Web services are under finalization process, one of the most important of them is Web Processing Services

• WPS is designed to standardize the way that GIS calculations are made available to the Internet. WPS can describe any calculation (i.e. process) including all of its inputs and outputs, and trigger its execution as a Web Service. WPS supports simultaneous exposure of processes via GET, POST, and SOAP, thus allowing the client to choose the most appropriate interface mechanism.

Web Processing Services

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




HNODC…

The HNODC data management case.

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• Hellenic National Oceanographic Data Center (HNODC) is a National Public Oceanographic Data provider and at the same time a member of the International Net of Oceanographic Data Centers (IOC/IODE)

• HNODC owns a very big volume of Data and Relevant information about the Marine Ecosystem

HNODC…PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• For the efficient management and exploitation of these Data, the first step was a relational Data Base to be constructed.

• After the necessary stage of data quality control took place, following the International standards, as they agreed during several EU Projects or International Organizations.

• Finally a mass volume of over 300.000 station data concerning , physical, chemical and biological Oceanographic information, was stored in a Relational Data Base.

HNODC…PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• The next target aroused then, was an Application to be constructed capable to represent

- the Geospatial aspects of this information - together with the non spatial information

• The necessary analysis took place and a traditional Web Application was developed by the use of the convenient web tools and technologies.

• Geographical representation was achieved by the use of a Middle Layer tool (ArcIMS) over the Data Base Mechanism.

HNODC…PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• The convenient Multilayer Architecture was followed – The Relational data base as the back end.– ArcIMS as the Geospatial representation

Mechanism.– Apache and Tomcat in heart of the Middle-Tier

System.– On the top a web Interface with selection criteria

for querying specific data sets in specific geographical areas.

HNODC - Architecture…PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




Data Base

ArcIMS

TomcatApache


Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




HNODC – Web page…PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




HNODC…

• Concluding we may say that the whole Architecture , served it’s purpose to:– Handle data management issues– Expose data dynamically to the web– Offer user easy and online data access– Represent geospatial information

graphically

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• Of course some technical issues aroused, regarding the Application’s performance, due to the inefficiency of the Geographical mechanism to handle big volumes of data sets as fast as a Web Application needs.

• For HNODC this approach is today considered as the first Version (V1) of an Integrated Web Based Environment for it’s Data Management and Data exploitation.

HNODC…

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• Thus and at the time that Web Services became a fact, a second Version was planned

• The target of this effort was an Integrated environment to be developed following at least the basic concepts of a Service Oriented Application (SOA) which are:– The interface contract to any Web Service

is platform-independent. – Any Web Service can be dynamically

located and invoked. – Any Web Service invoked is self-contained.

That is, the service maintains its own state.

HNODC…

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• For the development, new software components and tools replaced the “old” ones:– Geospatial and non Spatial Web

Services mechanisms took the place in the Middle- Layer.

– Geospatial open source tools and custom development was employed in presentation layer.

HNODC…

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




At the same time the application may now interact with any other SOA application either in sending or receiving Geospatial Data, since it inherits the big advantage of interoperability between Web Services systems.

HNODC…

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




The Architecture can be denoted as follows:– At the back End Open source PostgreSQL DBMS

(and not only..) stands as the data storage mechanism.

– UMN Map Server and Geoserver are the mechanisms for

• Representing Geospatial Data via Web Map Service (WMS)

• Querying and Navigating in Geospatial and Meta Data Information via Web Feature Service (WFS)

• And in the near future Transacting and processing new or existing Geospatial Data via Web Processing Service (WPS)


Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




– MapBender, a geospatial portal site management software for OGC and OWS architectures, acts as the integration module between the Geospatial Mechanisms.

– MapBender comes with an embedded data model capable to manage interfaces for displaying, navigating and querying OGC compliant web map and feature services (WMS and transactional WFS).


Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• Apache and Tomcat stand again as the Web Service middle Layers

• Apache Axis2 with it’s embedded implementation of the SOAP protocol acts as the No spatial data Mechanism of Web Services. These modules of the platform are still under development but their implementation will be fulfilled in the near future.


Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• And a new Web user Interface for the end user based on enhanced and customized version of a MapBender GUI.


Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




PostGreSQLSpatial

PostGreSQLNon Spatial

UMN Map Server

Internet

GeoServer

Apache

Web Application

Back End

Geospatial Mechanism

Middle Layer

End User

External Web Services

Internal Web Services(Apache/Axis)

Non Spatial Mechanism


Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




HNODC Web Interface PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




HNODC Web Interface PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




HNODC Web Services

• For HNODC the interoperability of Web Services is the big advantage since it is capable to act in the future as provider and consumer of Web Services in both ways:– Either as data products provider for

external SOA platforms. (e.g. SeaDataNet platform)

– Or as consumer of data products from external SOA platforms for new applications to be developed or for existing applications to be enhanced.

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• Further more and when the technology of Web Processing Service (WPS), will be mature enough and applicable for development, the derived data products will be able to have any kind of GIS functionality for consumers across the network.

HNODC Web ServicesPASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




Internet / Network

HNODC DB

HNODC Web Services External Web Services

Other Data Sources

SEA DATA NET Web Services

Application

Data Harmonization(Humboldt Framework)

Unified Messaging System

An Overall Architecture

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




A good example of the previous architecture, is the Oil spill Drift forecasting Use Case Scenario for Aegean sea.

An implementation:The “Oil Spill Scenario”

PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• Several and diverse Data Sets will be used as inputs in the scenario Application:

– ASAR satellite images – Oceanographic data– Wave model (WAM) data– Textual Information– GIS data about the nearby coasts layout

(roads, urban or protected areas etc)


PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• The scenario aims to deliver to the end user synthetic information in near real-time regarding:– Details about oil spill detection – Forecasts about the drift of the oil spill.– Additional information about the characteristics and the infrastructure of the

nearby coast area

• The scope of the scenario is to provide the decision makers with all the necessary information to handle the crisis

The output of the Application will be Synthetic GIS and non GIS information • Either via Map images presentation• Or textual information

Web Services will be present as discrete Layers

of Information (input or output) from the different

Data Sources


PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




A big obstacle aroused was the Harmonization of the

various datasets.Here comes E.U. HUMBOLDT Project which will contribute to harmonization since it aims to:

– make an inventory of the data harmonisation issues in the Scenario,

– assess their importance in the Scenario – create Scenario-specific data model, – develop harmonization software tools


PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




Internet / Network

HNODC DB

HNODC Web Services External Web Services Other Data SourcesSEA DATA NET Web Services

Application

Data Harmonization(Humboldt Framework)

Unified Messaging SystemScientific

CommunityDecision Makers ? ?


PASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• In conclusion we may denote that by embedding these new technologies to data management model we can exploit the advantages of:– Inheriting interoperability between among

Data Centers and Data Providers– Interchanging and combining Data Sets with

minimum programming efforts– Producing comprehensive and self explained

new Data Products combing various and diverse data sources

– Helping end user to navigate through a wide range of information

ConclusionsPASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




• Thus we are happy to say that HNODC joins the global scientific community by providing and consuming application Independent data products.

ConclusionsPASTPAST

Historical Brief

Centralized D.M.



Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface



An example

Conclusions




Thank you for your attention!

The end…