Upload
rosamund-jeremiah
View
26
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Hydrologic Information System for the Nation. Ilya Zaslavsky Spatial Information Systems Lab San Diego Supercomputer Center UCSD. http://his.cuahsi.org http://hiscentral.cuahsi.org http://hydroseek.net http://river.sdsc.edu/ucsddash http://wron.net.au/DemosII/Modules/ODMKMLGatway.aspx - PowerPoint PPT Presentation
Citation preview
Hydrologic Information System
for the NationIlya Zaslavsky
Spatial Information Systems LabSan Diego Supercomputer Center
UCSD
Alexandria Library talk, December 4, 2009
http://his.cuahsi.orghttp://hiscentral.cuahsi.orghttp://hydroseek.nethttp://river.sdsc.edu/ucsddashhttp://wron.net.au/DemosII/Modules/ODMKMLGatway.aspxhttp://maxim.ucsd.edu/mapmaker/storet.aspxhttp://water.sdsc.edu:7788/demo/CUAHSI/index.html
San Diego Supercomputer Center• Founded in 1985, as one of the five
original supercomputer centers, funded by the National Science Foundation
• 400 employees• Advanced research in high-
performance computing and networking
• R&D and cyberinfrastructure projects: in neuroscience, geology, astronomy, environmental sciences, molecular biology, hydrology
SDSC building on UCSD campus
SDSC Spatial Information Systems Lab
Research and system development• Services-based spatial information
integration infrastructure, CI projects• Mediation services for spatial data, query
processing, map assembly services• Long-term spatial data preservation• Spatial data standards and technologies for
online GIS (SVG, WMS/WFS)• Support of spatial data projects at SDSC
and beyond
Mediator
LegendGenerator
MapAssembler
Ontology
…
GRID SERVICESFOR MAP INTEGRATION
Mediator
LegendGenerator
MapAssembler
Ontology
…
GRID SERVICESFOR MAP INTEGRATION
services
In Geosciences (GEON, CUAHSI, CBEO,…)
Spatial web services
FederalAgencies
Figure 1.26 The Geography Network.
ESRICounty spatial data and toxicant information
Telesis, other localNon-profits
CA state
WSDL
WSWSDL
WSWSDL
WSWSDL
WSWSDL
WSWSDL
WS
Student projects
The CHI ME Model
In regional development (NIEHS SBRP, CRN…)
In Neurosciences (BIRN, CCDB)
http://spatial.sdsc.edu/lab/
Contact: [email protected]
Consortium of Universities for the Advancement of Hydrologic Science, Inc.
An organization representing more than one hundred United States universities, receives support from the
National Science Foundation to develop infrastructure and services for the advancement of hydrologic
science and education in the U.S.http://www.cuahsi.org/
122 US Universities as
of July 2008
CUAHSI HIS: NSF support through 2012 (GEO)
Partners:Academic: 11 NSF hydrologic observatories, CEO:P projects,
LTER…Government: USGS, EPA,
NCDC, NWS, state and localCommercial: Microsoft, ESRI,
KistersInternational: Australia, UK
Standardization: OGC, WMO (Hydrology Domain WG, CHy);
adopted by USGS, NCDC
An online distributed system to support the sharing of hydrologic data from multiple repositories and databases via standard water data service protocols;
software for data publication, discovery, access and integration.
What is the CUAHSI HIS?
Rainfall & SnowWater quantity
and quality
Remote sensing
Water Data
Modeling Meteorology
Soil water
Sources of Observations Data
Observation Stations
Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System
USGS National Water Information System NOAA Climate Reference Network
Map for the US
Build a common window on water data using web services
9
Getting Water Data (the old way)Different Query Pages Different Query Responses
10
Web Pages versus Web Serviceshttp://www.safl.umn.edu/ http://his.safl.umn.edu/SAFLMC/cuahsi_1_0.asmx
Uses Hypertext Markup Language (HTML) Uses WaterML (a Markup Language for water data)
http://his.cuahsi.org/odmdatabases.html
CUAHSI Observations Data Model
Information communication
• Water web pages• Water web services
HyperText Markup Language (HTML)
Water Markup Language (WaterML)
Standard Water Data Services• Set of query
functions
• Returns data in WaterML
NWIS Daily Values (discharge), NWIS Ground Water, NWIS Unit Values (real time), NWIS Instantaneous Irregular Data, EPA STORET, NCDC ASOS, DAYMET, MODIS, NAM12K, USGS SNOTEL, ODM (multiple sites)
Next Step: WaterML 2.0; OGC Hydrology Domain Working Grouphttps://lists.opengeospatial.org/mailman/listinfo/hydro.dwg
http://external.opengis.org/twiki_public/bin/view/HydrologyDWG/WebHome
International Standardization of WaterML
14
OGC/WMO Hydrology Domain Working Group
Test bed HISServers
Central HIS servers
ArcGIS
Matlab
IDL, R
MapWindow
Excel
Programming (C#, VB..)
Desktop clients
Customizable web interface
(DASH)
HTML - XMLW
SD
L - S
OA
P
Modeling (OpenMI)
Global search (Hydroseek)
Water Data Web Services, WaterML
Con
trol
led
voca
bula
ries
Met
adat
aca
talo
gs
Ont
olog
y
ET
L
serv
ices
HIS LiteServers
External data providers
Deployment to test beds
Other popular online clients
ODM DataLoader
Streaming Data Loading
Ontology tagging (Hydrotagger)
WSDL and ODM registration
Data publishing
ODMTools
Server config tools
HIS CentralRegistry & Harvester
Hydrologic Information System Service Oriented Architecture
HIS
Des
kto
p
16
• Built for data– Storage– Loading– Analysis– Publication
• HIS Software free of charge
HIS Server Real-time Sensors
WaterOneFlowWeb Service
Data Archives
Outside Users, HIS Central, HydroDesktop
Local Users
ODM Tools
HIS
Ser
ver
SQL Server
Observations DataModel Database
HIS Central – Catalog and Search
17
Managing Varying Semantics
Nitrogen: e.g. NWIS parameter # 625 is labeled ‘ammonia + organic
nitrogen‘, Kjeldahl method is used for determination but not mentioned in
parameter description. In STORET this parameter is referred to as Kjeldahl
Nitrogen.
And: Dissloved oxygen
acre feet acre-feet
micrograms per kilogram
micrograms per kilgram
FTU NTU
mho Siemens
ppm mg/kg
In measurement units…
In parameter names…
Semantic Tagging of Harvested Variables
21
• Service registry and metadata catalog– Networks– Sites– Variables– Search Keywords
• Does not store actual observation data
• Example: GetSitesInBox query function
HIS Central ServicesHICentral
Web Service
GetValues Requests Per Day from HIS Central
All Services NWIS0
200
400
600
800
1000
1200
1400
1600
February - De-cember 2008January - October 2009
23
HydroDesktop Capabilities
• Add shapefiles to map• Change symbology and
labels• Print and export map
• GIS toolbox
GIS
• Search for data• Download data
• Display time series• Export data
Hydrology
Hydroseekhttp://www.hydroseek.net
Supports search by location and type of data across multiple observation networks including NWIS, Storet, and academic data
Visualization and Analysis of Large Datasets► Tiled wall
► OLAP cubes for repositories► OLAP cubes for catalogs
EPA STORET water quality repository
USGS NWIS catalog:
measurement totals
for selected nutrients over
decades
How we work with agencies on web service access to observational data
1. Establish an agreement with the agency on joint development of water data services, identify agency partners (ideally, with time to support joint work)
2. Identify the scope of the service, databases to be exposed, and access control, assign network and vocabulary codes
3. Map semantics of the service to WaterML semantics, and verify with the agency
4. Include discussion of agency data and interoperability issues in the context of OGC Hydrology Domain WG (if needed)
5. Develop a first draft of the web service
6. Unit testing, over a series of validation cases developed jointly with the agency
7. Harvest an observations metadata catalog for agency data, to be housed either at SDSC or at the agency
8. Develop a procedure for catalog updates
9. Register the water data service at HISCentral (including mapping of variables to ontology terms), and test it using HydroSeek and HydroExcel. Document the service
10. Review and test the service together with the agency, for possible approval as “operational”
Federal Agency Water Data Services at HISCentral (10/09)
Network Name Site Count Value Count Earliest Observation Notes
NWISDV 29894 274762525 1/1/1900 WaterML-compliant GetValues service from NWIS, catalog ingested
EPA 362645 94349967 1/1/1900 SOAP wrapper over WQX services, catalog harvested
NWISUV 11185 77935488 60 DAYS WaterML-compliant GetValues Service, catalog ingested
NCDC ISH 11555 3000000* 1/1/2005 WaterML-compliant GetValues service from NCDC, catalog harvested
NCDC ISD 24770 18165478 1/1/1892 WaterML-compliant GetValues service from NCDC, catalog harvested
NWISIID 369148 15501245 1/9/1867 SOAP wrapper over NWIS web site, catalog harvested
NWISGW 827200 8325633 1/1/1900 SOAP wrapper over NWIS web site, catalog harvested
RIVERGAGES 2206 263101295 1/1/2000 WaterML compliant REST services from Army Corps of Engineers
Integration with other infrastructures
• Technical: with real time data management middleware:– OpenSource DataTurbine:
http://geo.sdsc.edu/jnlp/usgs.jnlp
• Communities:– Superfund Basic Research
Program (NIEHS)– CZOs (Boulder Creek,
Stroud Water Research Center)– South East Asia/Malaysia(http://seahydro.um.edu.my/search)
The International Workshop on Hydrologic Data Management and
Modeling in South East AsiaJuly 20-24
University of Malaya
Learning how the system worksPublishing hydrologic dataSetting up a server for SEA
Already published: sample data from JPS (Malaysia) and from Indonesia
Looking for COD measurementsIn HydroSeek
CUAHSI Water Data Services
32
43 services15,000 variables1.8 million sites9 million series4.3 billion data
Summary• CUAHSI HIS = Cyberinfrastructure for managing and publishing
observational data– Supports many types of point observational data– Overcomes syntactic and semantic heterogeneity using a standard data model
and controlled vocabularies– Supports a national network of observatory test beds– Maintains national registry of services (1.75 million stations – the largest in
the world)
• WaterML is a standard language for consistently communicating water observations data from academic and government sources using web services; already adopted by several federal agencies. Joint WMO and OGC activity to enhance it.
• The system is already deployed at multiple locations• It is free and open source