Transcript
Page 1: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

State of the art State of the art Technologies for GIS Technologies for GIS

Management SystemsManagement Systems

Implementation through EuropeanCommunity Research Projects

Hellenic Center for Marine Research

Lykiardopoulos Angelos, Iona Athanasia, Lakes Vasilis, Balopoulos Efstathios, Kalkavouras Constantine

Page 2: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Sections of this presentation

• Data management and Web Services

• Quality control in HNODC data base   

• Poseidon system, a short description

Page 3: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Data Management and Web Services

a Historical Brief

At the end of 80’s and early 90’s several Oceanographic Data centers had created Databases with mass storage of spatial Information which concerned Oceanographic research outcomes.

This was en enormous step of digital storage because big data volumes organized in Database Systems.

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 4: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

In other words the data centers owned databases – Based on various RDBMS mechanisms– With their own database Schemas– With their own data vocabulary (if existed)– With raw data in various formats– With Applied various processes of data quality control (or

with no data quality control)

This effort leaded into the creation of several data bases without any Interoperability among them.

Data Management Historical Brief

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 5: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

The centralized Data Management Idea…

But the oceanographic community had the need of homogenized and comprehensive data sets for various regions and even world wide.

Due to the limitations of information technologies this goal was achieved in the framework of several E.U. projects– Central accumulation of data from various Data Centers

took place– Data sets were processed in common data formats and

more or less same Q.C. procedures. – Finally the dissemination of these Data sets was carried

out mainly via digital media (CD-ROMs, etc).

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 6: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

The centralized Data Management Idea…

During this process: Diverse Data form a wide range of Data Centers

were gathered. Passed common homogenization and QC

procedures. And finally some comprehensive Data sets were

produced and disseminated.But… The produced Data Sets were stand alone

products stored in digital media. Without any dynamic mechanism of maintenance

and enhancement. And without any online access.

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 7: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

From Centralized to Distributed model

Thus, the idea of a distributed Oceanographic Data network among the Data Centers start growing up.

And since Web technologies were being developed rapidly, became apparent that the above network had to be web based.

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 8: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Main obstacles…

The main obstacles that came through for this idea to be fulfilled were:

Data bases was in different Schemas and Technologies specific per Data center so there was not any common data interchange mechanism.

Data were not homogenized among Data centers. Data quality and Data vocabulary differed from

Database to Database

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 9: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

From Non Spatial to Spatial Enabled Databases…

At the same time another obstacle was that due to premature Data Base technology many RDMBS systems lacked spatial capabilities.

So it was very difficult for the end user: To query the Database with spatial criteria To represent geographically the spatial information and

produce geospatial products To work interactive with Geo Data through interfaces with

geospatial capabilities To combine geospatial data in synthetic data products

Hence, for a distributed Oceanographic Data network to be established, the necessity of GIS enabled Databases became apparent as well.

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 10: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

From Non Spatial to Spatial Enabled Databases…

At the end of 90’s most of the common used RDBMS enriched with spatial capabilities and became then powerful mechanisms of integration, exploitation and presentation of spatial information

Worldwide oceanographic data centers, exploit this advantage and started upgrading their databases

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 11: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Interoperability obstacle is remaining…

But still the luck of interoperability was unsolved. The spatial databases made researchers’ life easier with regard to querying data or representing graphically geospatial information.But…

they were acting without any interconnection among the various Data Centers.

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 12: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

SEA DATA SEA DATA NETNET

SEASEARCSEASEARCHH

MEDAR/MEDAR/MEDATLAS IIMEDATLAS II

MEDATLAMEDATLASS

Summarizing the historical brief throughout important E.U. projects

1994

•Efforts in QC•Data homogenization•Centralized management•Distribution with CD ROMs

1998

•Enrich existing data sets•Establishment of common protocols in QC and formatting•Enhance communication facilities (www, ftp, e-mail etc)

2002

2006

•Development of a pan-european network for ocean and marine data & information management.•Improve the exchange, availability and accessibility of ocean & marine data and information.•Expand the online metadatabases with input of research institutes.•Improve the online accessibility of Sea-Search services & products.

•Develop a standardized distributed system for managing and disseminating the large and diverse data sets•Enhance the currently existing infrastructures with web services

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 13: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Web Services…

At the same time with the efforts of the Oceanographic community, during the last five years new technologies aroused and became common known as:

Web ServicesWeb Services

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 14: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Web Services fundamentals

First of all what is a Web Service?

• The hidden idea behind WS is the existence of a universal messaging mechanism, capable to deliver and translate messages for diverse and different systems.

• This feature came to open the road for the solution of interoperability.

• If different and diverse systems can interpret and process requests from each other let them stay different.

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 15: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

There are various theoretical definitions of What Web Service is such as:

“A software system designed to support interoperable machine-to-machine interaction over a network. It has an interface described in a machine - processable format (specifically WSDL). Other systems interact with the Web service in a manner prescribed by its description using SOAP messages, typically conveyed using HTTP with XML serialization in conjunction with other Web-related standards.” (W3C, 2004)

What is a Web Service?

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 16: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

What is a web Service?

or

“Web services are a distributed computing architecture. Only this particular architecture makes use of loosely coupled applications, as opposed to tightly coupled applications, to enable applications to communicate. This tightly coupled concept radically affects how information systems will work in the future”

(Clabby 2003)

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 17: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

What is a web Service?

Generalizing we may define that: “Web services provide a convenient and standardised way of

exposing business logic over a network (the Internet) by use of specific components of communication”

The basic components of WS are SOAP (Simple Object Access Protocol)

• which provides a Standardized messaging structure based on XML

UDDI (Universal Description, Discovery and Integration) • which provides the ability to dynamically select a

service at runtime WSDL (Web Services Description Language)

• which describes how the client communicates with a Web service or what a service does or where is located.

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 18: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

In the most simple form a web service architecture may be depicted as the diagram

Web Service Architecture…PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 19: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Geospatial Web Services

The rapid development of technologies towards WS concept has produced several sets of Web Services with regard to the area of the WS applicability.

Especially for Geospatial data, Open Geographical Consortium (OGC) has defined two basic sets of Geospatial Web Services.

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 20: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

The Web Map Services (WMS) which are concerned with transforming spatial data into maps (images).

The Web Feature Services (WFS) which are concerned with direct access to data - reading, writing, and updating data.

WMS and WFS…

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 21: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

• Furthermore, more Web services are under finalization process, one of the most important of them is WPS (Web Processing Service)

• WPS is designed to standardize the way that GIS calculations are made available to the Internet.

• Can describe any calculation (i.e. process) including all of its inputs and outputs, and trigger its execution as a Web Service.

• Supports simultaneous exposure of processes via GET, POST, and SOAP, thus allowing the client to choose the most appropriate interface mechanism.

Web Processing Services

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 22: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

HNODC…

The HNODC data management case.

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 23: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Hellenic National Oceanographic Data Center (HNODC) is a National Public Oceanographic Data provider and at the same time a member of the International Net of Oceanographic Data Centers (IOC/IODE)

HNODC owns a very big volume of Data and Relevant information about the Marine Ecosystem

HNODC…PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 24: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

For the efficient management and exploitation of these Data, the first step was a relational Data Base to be constructed.

After the necessary stage of data quality control took place, following the International standards, as they agreed during several EU Projects or International Organizations.

Finally a mass volume of over 300.000 station data concerning , physical, chemical and biological Oceanographic information, stored in a Relational Data Base.

HNODC…PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 25: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

The next target aroused then, was an Application to be constructed capable to represent either

- the Geospatial aspects of this information - together with the non spatial information

The necessary analysis took place and a traditional Web Application was developed by the use of the convenient web tools and technologies.

Geographical representation was achieved by the use of a Middle Layer tool (ArcIMS) over the Data Base Mechanism.

HNODC…PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 26: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

• The convenient Multilayer Architecture was followed – The Relational data base as the back end.– ArcIMS as the Geospatial representation Mechanism.– Apache and Tomcat in heart of the Middle-Tier System.– On the top a web Interface with selection criteria for

querying specific data sets in specific geographical areas.

HNODC - Architecture…PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 27: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Data Base

ArcIMS

TomcatApache

HNODC - Architecture…PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 28: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

HNODC – Web page…PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 29: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

HNODC…

Although it’s geospatial capabilities this Application stranded only as a data geographical presentation and queering mechanism – without any Data management and Data

processing abilities.– and also without any Data interchanging with other

applications– and with it’s own data vocabulary

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 30: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

• At the same time technical issues aroused, regarding the Application’s performance, due to the inefficiency of the Geographical mechanism to handle big volumes of data sets as fast as a Web Application needs.

• For HNODC this approach is today considered as the fist Version (V1) of an Integrated Web Based Environment for it’s Data Management and Data exploitation.

HNODC…

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 31: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

• Thus and at the time that Web Services became a fact, a second Version was planned

• The target of this effort was an Integrated Web Based environment to be developed, following at least the basic concepts of a Service Oriented Application (SOA) which are:– The interface contract to any Web Service is

platform-independent. – Any Web Service can be dynamically

located and invoked. – Any Web Service invoked is self-contained.

That is, the service maintains its own state.

HNODC…

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 32: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

For the development, state of the art software components and tools replaced the “old” ones:– Geospatial and no Spatial Web Services

mechanisms took the place in the Middle- Layer.– Geospatial open source tools and custom

development was employed in presentation layer.

HNODC…

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 33: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

At the same time the application may now interact with any other SOA application either in sending or receiving Geospatial Data, since it inherits the big advantage of interoperability between Web Services systems.

HNODC…

PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 34: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

The Architecture can denoted as follows:– At the back End Open source PostgreSQL DBMS (and

not only..) stands as the data storage mechanism with more than one Data Base Schemas cause of the separation of the Geospatial Data and the non Geospatial Data.

– UMN Map Server and Geoserver are the mechanisms for

• Represent Geospatial Data via Web Map Service (WMS)

• Querying and Navigating in Geospatial and Meta Data Information via Web Feature Service (WFS)

• And in the near future Transacting and processing new or existing Geospatial Data via Web Processing Service (WPS)

HNODC - Architecture…PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 35: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

– WhereGroup MapBender, a geospatial portal site management software for OGC and OWS architectures, acts as the integration module between the Geospatial Mechanisms.

– MapBender comes with an embedded data model capable to manage interfaces for displaying, navigating and querying OGC compliant web map and feature services (WMS and transactional WFS).

HNODC - Architecture…PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 36: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Apache and Tomcat stand again as the Web Service middle Layers

Apache Axis2 with it’s embedded implementation of the SOAP protocol acts as the No spatial data Mechanism of Web Services.

(These modules of the platform are still under development but their implementation will be fulfilled in the near future.)

Finnaly Web user Interface for the end user developed based on enhanced and customized version of a MapBender GUI.

HNODC - Architecture…PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 37: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

PostGreSQLSpatial

PostGreSQLNon Spatial

UMN Map Server

Internet

GeoServer

Apache

Web Application

Back End

Geospatial Mechanism

Middle Layer

End User

External Web Services

Internal Web Services(Apache/Axis)

Non Spatial Mechanism

HNODC - Architecture…PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 38: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

HNODC Web Interface PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 39: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

HNODC Web Interface PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 40: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

HNODC Web Interface PASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 41: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

• As conclusion we may denote that embedding these new technologies to data management model we can exploit the advantages to:– Inherit interoperability between among

Data Centers and Data Providers– Interchange and combine Data Sets with

minimum programming efforts– Produce comprehensive and self explained

new Data Products combing various and diverse data sources

– Help end user to navigate through a wide range of information

ConclusionsPASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 42: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

We are happy to say that HNODC joins the global scientific community by providing and consuming application Independent data products.

ConclusionsPASTPAST

Historical Brief

Centralized D.M.

From Centralized to Distributed Model

From Non Spatial to SpatialPRESENTPRESENT

Web Services

W.S. Architecture

Geospatial W.S.

WMS/WFS

WPS

HNODC General

HNODC Architecture

HNODC Interface

HNODC W.S.FUTUREFUTURE

The new architecture

An example

Conclusions

Page 43: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Quality Control in HNODC

The quality control procedures that HNODC uses to manage oceanographic data and information follow the international standards recommended by IOC, ICES and EU:

Documentation of the data sets

Format Conversions (Medatlas)

Quality Control on data and metadata

Page 44: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Quality Control in HNODC

Quality Control at Metadata and Data (automatic and visual):

• Format, codes and completeness of information

• Cruise and stations headers

• Observed data of vertical profiles and time series of fixed moorings

The results of the QC are added as quality flags to each numerical value for the location, date, bottom depth and the data points of vertical profiles and time series.

The quality controlled data set is archived in the

HNODC’s multidisciplinary database.

Page 45: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

The main software tool deployed for QC is “Scoop” Scoop is a open source application aimed in

moderating and visual handling of Quality Control aspects like :– Checks in various data format– Duplication entries in datasets– Visual observation of vertical or time series profiles – …etc

Quality Control in HNODC

Page 46: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Format Checks

Page 47: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Vertical Profiles

duplicates cruises within a preset space-time radius

date of the stations

ship velocity between two

consecutive stations

bottom sounding (ETOPO5)

land position (GEBCO)

Time Series

duplicates entriesSensor depth check

Series duration check

land position (GEBCO)

Automatic Checks of Cruise and Stations headers

Page 48: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

at least two parameters required for the qc (vertical

reference + measurement)

constant profiles or times series (sensor stuck)

data out of regional values

comparison with pre-existing climatological statistics

(LEVITUS, MODB, MEDATLAS). Time series are compared

with internal statistics.

decreasing reference parameter (pressure or time)

pressure must not exceed the bottom depth

spikes

vertical instability (for vertical profiles)

Observed data of vertical profiles and time series of

fixed moorings

Automatic Checks of Observed Data

Page 49: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Visual Checks of Cruise and Stations headers

Page 50: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Visual Checks of Observed Data

Page 51: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Poseidon System

Poseidon is a system of : Monitoring

Forecasting

Various Information production

For the Greek seas

Page 52: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Poseidon is based on an established network of observation buoys

And on a specialized operational center for the processing of the data collected and the production of forecasts,

POSEIDON system is an infrastructure at the leading edge of modern oceanography in Europe .

The network of observation buoys records continuously the physical, biological and chemical parameters of the Greek seas.

Those data are then transmitted to the operational center where they are sorted and fed into forecasting models.

Poseidon System

Page 53: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

POSEIDON system is a unique planning tool in the endeavour for the protection of the marine environment.

It also provides a competitive advantage for the development of business activity, the prevention of disaster, and the safeguarding of human life.

In the frame of the world-wide trend for the development of operational oceanography, POSEIDON network places Greece among the leading countries in this field

Poseidon System

Page 54: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

The system under its full functionality can provide :

immediate (on line) and dependable information

reliable forecast

protection of marine ecosystem

support of marine research

Poseidon System

Page 55: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

http://www.poseidon.hcmr.gr

7 Seawatch buoys

3 Wavescan buoys

The “POSEIDON” buoy network system

The observation buoys are equipped with sensors that monitor:

Air-pressure

Air-temperature

Wind speed and direction

Wave height, period and direction

Sea surface salinity and temperature

Surface current speed and direction

Sea surface dissolved oxygen

Light attenuation with fluorescence

Salinity and temperature in depths 20-1000 m

Chlorophyll-A in depths 0-100 m

Dissolved oxygen in depths 20-100 m

Current speed and direction in depths 20-50 m

Nutrients

Radioactivity

Page 56: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Data are transmitted to the Operational Center of POSEIDON using 3

telecommunications systems:

- INMARSAT-C satellite

- GPRS

- Irridium

Page 57: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Exports – Decodes DATA

Creates a 24h-file with the previous day – Data which are

to pass the Quality Check process

Inputs file to

Quality Check

The data pass through some tests

Page 58: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

QC checkMissing Data are replaced with 99.999

Sensor-range test

The data are tested against physical limitation of sensors

fail

Physical-range test

The data have to range within thresholds according to regional

climatology

Rate of change test

The difference between the test value with the previous and the next one may not exceed the

timeseries std or a pre-arranged threshold

fail

Stuck value test

The value may not remain constant compared with a number of previous values

fail

fail

Flag 9 assigned

Flag 2 assigned

Flag 4 assigned

Flag 3 assigned

Flag 3 assigned

Flag 1 assigned

Produce the file with flag indicators

pass

pass

pass

pass

Generate output file in Medatlas format

Update Database

•MERSEA

•Coriolis

•Ocean SITES

•SEPRISE …etc

Examples

Page 59: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Contains

Contains

The parameter data pass through a perl script into a mysql

database

Insitu Data DatabaseNormalized Metadata

Database

Metadata database normalization process applied to the insitu

metadata database

Metadata Database

Metadata cleaning (containing name uniformity, filling missing

values)

Parameters Measured Stored

into files

Server 1

Server 2

Normalization process applied to the insitu

data

Inputs data

Updates flagged data

Inputs Metadata

Page 60: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center POSEIDON, 2007

A. Telecommunications (Satellite, GSM modems)B. Computer infrastructure:

o Altix 3700 High-Performance Server 128 CPUs [email protected] GHz 128 GΒ RAM 20 TB Hard Disk Drives

10 Dual socket Linux Workstations

C. Information management (S/W, D/B)

DATA VIAINMARSAT-C

DATA VIA GSM

DATARECEPTIONODA LINK

POSEIDONDATA BASE

OCEAN CHECK(DATA QUALITY

CONTROL)

OCEAN INFO

NODC

Page 61: State of the art Technologies for GIS Management Systems Implementation through European Community Research Projects Hellenic Center for Marine Research

HCMR / Hellenic National Oceanographic Data Center

Thank you for your attention!

The end…


Recommended