Data Integration and Analysis System (DIAS)

Toshio KoikeThe University of Tokyo

GEO Alliances and Harmonization WorkshopWashington DC, USA, 11-12 November, 2009

National Key Technologies The 3rd Basic Program for Science and Technology of Jap

The Mission of DIAS

•to coordinate the cutting-edge information scienceand technology and the various research fieldsaddressing the earth environment;•to construct data infrastructure that can integrate earth observation data, numerical model outputs, and socio-economic data effectively;•to create knowledge enabling us to solve the earth environment problems; and•to generate socio-economic benefits.

4

DIAS, tackling a large increase in volumeof the earth observation data

DIAS is developing a core system for data integration and analysis that includes the supporting functions of life cycle data management, data search, information exploration, scientific analysis, and partial data down-loading.

国家基幹技術「海洋地球観測探査システム」データ統合・解析システム

Global Earth Observation System of Systems（GEOSS）

WCRP CMIP3 MetBrokersatellite simulation In-situ ocean

Disk Array

Storage Layer

File System LayerData Management Layer

•Storage Management System•Power management System

•DBMS

Application Layer

•Data Manager•Data Navigator•Meta Data Manger

•Visualizer(w display wall)•Discovery Work Flow Assist•Data Quality Manager

Common SoftwareUser Apps.

A Prototype of Data Integration and Analysis

User Apps. User Apps. User Apps. User Apps.

•Data Transformer•Data Crawler•ETL

7

DB

DB

0

50

100

150

200

250

300

350

1992

1994

1996

1998

2000

2002

2004

2006

2008

2010

2012

2014

2016

2018

2020

A1:高度成長社会

A2：多元化型

B1:持続発展型

B2：地域共存型

DB

DB

DB

DB

DB0

50

100

150

200

250

300

350

1992

199 4

199 6

1 998

2 000

2002

2004

200 6

200 8

2010

2012

2014

2016

201 8

202 0

A1:高度成長社会

A2：多元化型

B1:持続発展型

B2：地域共存型

DB

DB0

50

100

150

200

250

300

350

1992

1994

1996

199 8

2000

2002

2004

2006

2008

2010

201 2

201 4

2016

2018

2020

A 1:高度成長社会

A 2：多元化型

B 1:持続発展型

B 2：地域共存型

DB

DB

DB

Meteorology

Ecosystem

AgricultureHydrology

Land Use

Climatology

Health

农业

농업

Where is data?How to access?

Technical Terms among

Different Disciplines

Quality?Reliability?

DIAS, tackling a large increase in diversityof the earth observation data

For improving data interoperability, DIAS is developing a system for identifying the relationship between data by using ontology on technical terms and ideas, and geography.DIAS also is acquiring data base information from various sources by developing a cross-sectoral search engine for various data bases.

Dictionary to Ontology Geographical Ontology

Making Connection among Disciplines through Ontology

1 2 3 4 5 6 7 8 9 10 11 12Year2010Year2020Year2030Year2040Year20500

51015202530354045

単位:0.1度

月

0

50

100

150

200

250

300

350

1992

1994

1996

1998

2000

2002

2004

2006

2008

2010

2012

2014

2016

2018

2020

A 1:高度成長社会

A2：多元化型

B1:持続発展型

B2：地域共存型

Socio-economic Data SchemaClimate Data Schema

Water Cycle Data Schema

Land cover data schema

Agricultural Economy Model

Crop Growth Model

Land Use Model

MetadataElement

CoreElement(MD_Metadata) ProfessionalElement OrganizationalElement

+professionalElement0..*0..*+coreElement

+organizationalElement

0..*

Population Data SchemaSanitation Data Schema

Applications

Agricultural Production Management

DIAS develops an information system for agricultural production management by integrating the real-time monitoring data of farmland, the growing condition of each crop cultivar, meteorological data, numerical weather predictions, and climate model predictions. This system will be usable by the farming community, enabling them to make improved management decisions especially in regions which are susceptible to global warming impacts.

Applications

Ecosystem Conservation and Participatory Monitoring Program

DIAS compiles data bases of a number of important indices of biodiversity, including invasive alien species and endangered species through participatory monitoring programs, integrates to analyze the data with other earth observation data, and disseminates the products in a formto be easily used for decision making related to biodiversityconservation..

Applications

Ocean Circulation and Fishery Resources Management

DIAS provides usable information for a sustainable fishery resources management by constructing an oceanography- fishery cooperative platform that enables resource managers to investigate relationships between fluctuations in the fishery resources and the seasonal to decadal ocean variations derived froman ocean re-analysis based on the data assimilation by applying thefour dimensional variational assimilation methods..

Applications

Integrated Water Resources Management

The Asian countries cooperatively integrate data from earth observation satellites and in-situ networks with other types of data, including numerical weather prediction model outputs, geographical information, and socio-economic data, to generate information for making sound water resources management decisions.

Drought and Water Scarcity

Floods and Land Slides

Water Pollution

and Ecosystem

Degradation

Climate Change Impacts

on Water Cycle

GEOSSAsian Water Cycle Initiative

(AWCI)

1st GEOSS AP Symposium, Tokyo, Jan. 2007

To promote integrated water resources management by making usable information

from GEOSS, for addressing the common water-related

problems in Asia.

・A River Basin of Each Country・Observation Convergence

・Data Integration

Uniqueness

・Interoperability Arrangement

・Capacity Building・Open Data & Source Policies

・Early Achievements

1st Asian Water Cycle Symposium, Tokyo, Nov. 2005

1st Capacity Building Workshop, Sep. 2006

1st Task Team Meeting, Bangkok, Sep. 2006

2nd Asian Water Cycle Symposium, Tokyo, Jan. 2007

1st International Coordination Group Meeting, Bali, Sep. 2007

3rd Asian Water Cycle Symposium, Beppu, Dec. 2007

Myanmar

Malaysia

China

GEOSS Asian Water Cycle Initiative (AWCI)19 Member Countries

Mongolia

KoreaJapan

Pakistan Nepal

India

Bhutan

Myanmar Laos

Bangladesh

Sri Lanka

Thailand

Cambodia

Philippines

Indonesia

China

Vietnam

Uzbekistan

Malaysia

Myanmar

Malaysia

China

GEOSS Asian Water Cycle Initiative (AWCI)19 Member Countries

18 River Basins for Initial Demonstration

Mongolia

KoreaJapan

Pakistan Nepal

India

Bhutan

Myanmar Laos

Bangladesh

Sri Lanka

Thailand

Cambodia

Philippines

Indonesia

China

Vietnam

Uzbekistan

Malaysia

Upload

Meta Data

Meta Data Meta Data

Data Integration and Analysis System (DA-09-02a)

Quality Control

Data Provider (Observer)

User

Meta Data Registration

•Search with Metadata•Data Download

•Document Generation from Meta Data•Data Visualization

・・

ObservationData Meta Data

Data Upload＋(part of )Meta Data

ObservationData

Meta Data

Data Quality Control Process

Meta Data

Post-QCObservation Data

InputMeta Data

Data Download Search IF Document Generator Visualization System

Data A

rchivingD

ata IntegrationWeb-based Data Archiving & Integration System

Basic Information

Observation Point Inf.,Contact Person Inf.,……

Data status as of 2009/04/10

Myanmar

Malaysia

China

GEOSS Asian Water Cycle Initiative (AWCI)19 Member Countries

18 River Basins for Initial Demonstration

Mongolia

Korea Japan

Pakistan Nepal

India

Bhutan

Myanmar Laos

Bangladesh

Sri Lanka

Thailand

Cambodia

Philippines

Indonesia

China

Vietnam

Uzbekistan

Malaysia

0

1000

2000

3000

4000

5000

6000

7000

23 24 25 26 27 28 29 30

November 2004

[m3 /s

]

Observed

QPF

Land-use

Flood Prediction

Topography and River Channel

Network

Socio-Economic Data

Prediction of the inundation Area and Depth

GEOSS/AWCI Flood Evacuation Instruction System in Vietnam

Heavy Rainfall Prediction

Numerical Weather

Prediction

Atmospheric and Land Surface Water

Observation

Heavy Rainfall Prediction

Coupled with Satellite Data Assimilation

Operational Data

Obs..

Pred.

EvacuationInstruction

252006

simulation

The outflow from LaoCai is taken as inflow from its upper region.

26

Global Rainfall Map in near-real-time (GSMaP_NRT)• GSMaP (Global Satellite Mapping for

Precipitation) is originally funded by JST/CREST during 2002-2007. – Development of reliable MWR

algorithm consistent with TRMM/PR and precipitation physical model developed by using PR (Aonashi et al., 2009).

– Combination of microwave radiometer retrievals with GEO IR by the moving vector (like CMORPH) and new Kalman filtering method(Ushio et al., 2009).

• JAXA/EORC began to provide near-real-time version data of GSMaP (GSMaP_NRT) about 4-hour after observation via password protected ftp site since October 2008.

• Hourly browse images, kmz files for GoogleEarth, and 24-hour movies are also available from Web server.

Global Rainfall Map in near-real-time -- http://sharaku.eorc.jaxa.jp/GSMaP/

Cyclone "NARGIS" attacked Myanmar

27

Optimized reservoir operation (2006)

Ben Ngoc is in the 1 km downstream of the Hoa Binh reservoir.

Glacier Lake Outburst Floods (GLOFs)

Source: Google Earth

Daichi/PRISM: three independent optical systems for digital elevation data

Moraine topography

Daichi/PALSARSynthetic apertureRadar for identifying water surface

Lake area expantion

Water and energy budget distributed hydrological model (WEB-DHM)coupled with climate models

Lake area expantion

A1B B12046-2065 2081-2100 2046-2065 2081-2100

Number of models which show more severe distribution than

now82%14(/17)

94%16(/17)

76%13(/17)

53%9(/17)

5-year probable rainfall 1.18 1.31 1.14 1.1810-year probable rainfall 1.20 1.35 1.15 1.2100-year probable rainfall 1.20 1.36 1.17 1.18

Climate Change Impacts on Heavy Rainfallin Indonesia

Discharge(m^3/s)

T

Current Design Rainfall

Future Design Rainfall underClimate Change

Design HydrographDesign Rainfall

1 1.2

HydrologicalModel

Current Climate

Climate Change

Climate Change Impacts on Flood Control Planin Indonesia

Probable flood (10year)Current Climate

Probable flood (10year)50 years later

Quantifying uncertainty 

Climate models Multi‐model 

ensemble (MME)

Down‐scaling

Basin‐scale prediction of  

quantity & quality 

Water quantity and quality prediction

flood

ordinary water

drought

ground water

Information

Storage

Treatment

Current facility, plan, management

Flood control system

Water allocation &

 cost

Environment

Human life 

Industry

Logic model

Inter‐industry analysis

FloodDisaster potential

DroughtDisaster  potential

Impact assessm

ent

Filed survey

Early warning

Allocation policy

Land use

Adaptation  options

etc.

Innovative  technology‐ Flood control‐ quality control

Decision m

aking

Monitoring evaluation

implem

entationEnd to End Approach on Climate Change Adaptation

Scientific approach Engineering Approach Socio‐economical approach

System Generalization through Intercomparison Studies