Upload
caitlin-knighton
View
215
Download
2
Tags:
Embed Size (px)
Citation preview
Waters Observatory Network
• Observe water systems
• Observe in many different contexts and for different purposes
• Have a Waters Network Information System for sharing data– Common language for data– Geographic federation of dispersed data
sources through web services
Project co-PI in Phase 2
Collaborator in Phase I
CUAHSI HIS Partner Institutions
10 WATERS Testbed Sites
HISTeam
WATERSTestbed
WATERS Network Information System
SuperComputerCenters:NCSA,TACC
Domain Sciences:
Unidata, NCARLTER, GEON
Government:USGS, EPA,
NCDC, USDA
Industry:ESRI, Kisters,
MicrosoftCUAHSI HIS
HIS Team and its Cyberinfrastructure Partners
HIS Team:Texas, SDSC,Utah, Drexel,
Duke
SuperComputerCenters:NCSATACC
Domain Sciences:
Unidata, NCARLTER, GEON
Government:USGS, EPA,
NCDC, USDA
Industry:ESRI, Kisters,
Microsoft
HISTeam
WATERSTestbed
WATERS Network Information System
CUAHSI HIS
HIS and WATERS
CUAHSI Member Institutions
116 Universities as of November 2006
SupercomputerCenters:NCSA,TACC
Domain Sciences:
Unidata, NCARLTER, GEON
Government:USGS, EPA,
NCDC, USDA
Industry:ESRI, Kisters,
OpenMI
HISTeam
WATERSTestbed
WATERS Network Information System
CUAHSI HIS
HIS, WATERS and the CUAHSI Community
KISTERS Water Resources Management Data System
KISTERS is building CUAHSI web services onto their database
Stefan Fuest
SupercomputerCenters:NCSA,TACC
Domain Sciences:
Unidata, NCARLTER, GEON
Government:USGS, EPA,
NCDC, USDA
Industry:ESRI, Kisters,
OpenMI
HISTeam
WATERS Testbed
WATERS Network Information System
CUAHSI HIS
International Partners
CSIRO Land and Water ResourcesWater Resources Observations Network
(WRON)
European CommissionWater database design and model integration
(HarmonIT and OpenMI)
HIS Goals
• Hydrologic Data Access System – better access to a large volume of high quality hydrologic data
• Support for Observatories – synthesizing hydrologic data for a region
• Advancement of Hydrologic Science – data modeling and advanced analysis
• Hydrologic Education – better data in the classroom, basin-focused teaching
Rainfall & SnowWater quantity
and quality
Remote sensing
Water Data
Modeling Meteorology
Soil water
Water Data Web Sites
NWISWeb site output# agency_cd Agency Code# site_no USGS station number# dv_dt date of daily mean streamflow# dv_va daily mean streamflow value, in cubic-feet per-second# dv_cd daily mean streamflow value qualification code## Sites in this file include:# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC#agency_cd site_no dv_dt dv_va dv_cdUSGS 02087500 2003-09-01 1190USGS 02087500 2003-09-02 649USGS 02087500 2003-09-03 525USGS 02087500 2003-09-04 486USGS 02087500 2003-09-05 733USGS 02087500 2003-09-06 585USGS 02087500 2003-09-07 485USGS 02087500 2003-09-08 463USGS 02087500 2003-09-09 673USGS 02087500 2003-09-10 517USGS 02087500 2003-09-11 454
Time series of streamflow at a gaging station
USGS has committedto supporting CUAHSI’sGetValues function
Observation Stations
Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System
USGS National Water Information System NOAA Climate Reference Network
Map for the US
Water Quality Measurement Sites in EPA Storet
Substantial variation in data availability from states
Data from Bora Beran, Drexel University
Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)
Geographic Integration of Storet and TCEQ Data in HIS
Observations CatalogSpecifies what variables are measured at each site, over what time interval,
and how many observations of each variable are available
Point Observations Information Model
Data Source
Network
Sites
Variables
Values
{Value, Time, Qualifier}
USGS
Streamflow gages
Neuse River near Clayton, NC
Discharge, stage (Daily or instantaneous)
206 cfs, 13 August 2006
• A data source operates an observation network• A network is a set of observation sites• A site is a point location where one or more variables are measured• A variable is a property describing the flow or quality of water• A value is an observation of a variable at a particular time• A qualifier is a symbol that provides additional information about the value
Data Discovery and Delivery
Data Source
Network
Sites
Variables
Values
Observations metadata
Observations data
HIS ServerObservations
Catalog
Web services
Data Discovery
Data Delivery
• HIS facilitates data discovery by building and maintaining observations catalogs• Data delivery occurs through web services from remote data archives or local observations databases. Water resource agencies support data delivery services.
NWISNWIS
ArcGISArcGIS
ExcelExcel
NCARNCAR
UnidataUnidata
NASANASAStoretStoret
NCDCNCDC
AmerifluxAmeriflux
MatlabMatlab
AccessAccess JavaJava
FortranFortran
Visual BasicVisual Basic
C/C++C/C++
Some operational services
CUAHSI Web ServicesCUAHSI Web Services
Data SourcesData Sources
ApplicationsApplications
Extract
Transform
Load
http://www.cuahsi.org/his/
WaterOneFlow Web Services
Data access through web
services
Data storage through web
services
Dow
nlo
ads
Upl
oa
ds
Observatory servers
Workgroup HIS
SDSC HIS servers
3rd party servers
e.g. USGS, NCDC
GIS
Matlab
IDL
Splus, R
D2K, I2K
Programming (Fortran, C, VB)
Web services interface
Web portal Interface (HDAS)
Information input, display, query and output services
Preliminary data exploration and discovery. See what is available and perform exploratory analyses
HTML -XML WS
DL
- SO
AP
Hydrologic Information System Service Oriented Architecture
NWISNWIS
ArcGISArcGIS
ExcelExcel
NCARNCAR
UnidataUnidata
NASANASAStoretStoret
NCDCNCDC
AmerifluxAmeriflux
MatlabMatlabAccessAccess SASSAS
FortranFortran
Visual BasicVisual Basic
C/C++C/C++
CUAHSI Web ServicesCUAHSI Web ServicesIlya Zaslavsky
My server
Web services: key terms
Web ServiceCode
WSDL Web service description language Your Web application
Reference a web service, and call a WS method
Your desktop application
Reference a web service, and call a WS method
SOAP Messages
SOAP: Simple Object Access Protocol
- From different database structures, data collection procedures, quality control, access mechanisms to uniform signatures … Water Markup Language- Tested in different environments- Standards-based- Can support advanced interfaces via harvested catalogs- Accessible to community- Templates for development of new services- Optimized, error handling, memory management, versioning, run from fast serversAnd: working with agencies on setting up services!
Web service testing pages
http://water.sdsc.edu/webservicetestpage/
For Developers and Information Managers
• Code developers– HIS Team System: code repository, web services– Design Documents– Workbook for using web services in different environments– Web service testing and versioning guidelines and modules– Support
• Site managers– Step-by-step instructions for setting up the server (eventually, will be an
“HIS appliance”)– Configuration and data management tools
What it takes to setup and maintain HIS server, and how to get there from your current setup – THIS AFTERNOON
CUAHSI Hydrologic Information System Levels
National HIS – San Diego Supercomputer Center
Workgroup HIS – research center or academic department
Personal HIS – an individual hydrologic scientist
HIS Server
HIS Analyst
Map interface, observations catalogs and web services for national data sources
Map interface, observations catalogs and web services for regional data sources; observations databases and web services for individual investigator data
Application templates and HydroObjects for direct ingestion of data into analysis environments: Excel, ArcGIS, Matlab, programming languages; MyDB for storage of analysis data
Data Ingestion
MatlabExcelArcGIS Java .Net
HydroObjects
HIS Analyst
HIS Server
WaterOneFlow Services
Observations Data
Tim Whiteaker
Tutorials and Templates that show how to use Web services in various applications
ftp://ftp.crwr.utexas.edu/pub/outgoing/CUAHSI/HIS_workbook/20061115/
IncludesHydroObjects
Library
HIS Analyst Demo (Tim Whiteaker)
• Ingestion of data using web services into Excel and VB.Net
CUAHSI Hydrologic Data Access System
A common data window for accessing, viewing and downloading hydrologic information
USGSUSGS
NASANASANCDCNCDCEPAEPA NWSNWS
Observatory DataObservatory Data
http://river.sdsc.edu/HDAS
HIS Server
• Supports data discovery, delivery and publication– Data discovery – how do I
find the data I want?• Map interface and
observations catalogs
– Data delivery – how do I acquire the data I want?
• Use web services or retrieve from local database
– Data Publication – how do I publish my observation data?
• Use Observations Data Model
HIS Server Architecture
• Map front end – ArcGIS Server 9.2 (being programmed by ESRI Water Resources for CUAHSI – led by Dean Djokic, an IHE graduate)
• Relational database – SQL/Server 2005 or Express
• Web services library – VB.Net programs accessed as a Web Service Description Language (WSDL)
HIS Server Demo (Dean Djokic)
ESRI Water Resources Applications Group, Redlands CA
(a voluntary contribution to CUAHSI)
Digital WatershedHow can hydrologists integrate observed and
modeled data from various sources into a single description of the environment?
Digital WatershedHydrologic Observation
Data
GeospatialData
Weather and ClimateData
Remote SensingData
(NetCDF)
(GIS)(Relational database)
(EOS-HDF)
Digital Watershed
A digital watershed is a synthesis of hydrologic observation data, geospatial data, remote sensing data and weather
and climate data into a connected database for a hydrologic region
HIS ServersHydrologic
ObservationsServer
GIS Data Server
Weather and ClimateServer
Remote SensingServer
Digital Watershed
HIS Servers provide hydrologic observations, weather and climate, GIS and remote sensing data. For HIS version 1.0, the focus is a hydrologic observations server for data from gages and monitoring sites at point locations.
Water Resource Regions and HUC’s
NHDPlus for Region 17E
NHDPlus Reach Catchments ~ 3km2
About 1000 reach catchments in each 8-digit HUC
Average reach length = 2km 2.3 million reaches for continental US
National Land Cover Dataset
http://landcover.usgs.gov/nationallandcover.html
http://seamless.usgs.gov/Get the data:
Reach Attributes
• Slope• Elevation• Mean annual flow
– Corresponding velocity
• Drainage area• % of upstream
drainage area in different land uses
• Stream order
http://www.ncgc.nrcs.usda.gov/products/datasets/statsgo/
1:250,000 Scale Soil Information
Hydrologic Landscape Regions
USGS Characterization of nature of soil and groundwater systems
http://water.usgs.gov/lookup/getspatial?hlrus
Arc Hydro: GIS for Water Resources
• Arc Hydro– An ArcGIS data model
for water resources– Arc Hydro toolset for
implementation– Framework for linking
hydrologic simulation models
The Arc Hydro data model andapplication tools are in the publicdomain
Published in 2002
Arc Hydro Links Space and TimeIntegrate geospatial data for
land and water systems
You store this informationin a relational database orget it from web services
Link space and timewith HydroID
All Hydrofeatures of a testbed area will have a unique HydroID
Land base from NHDPlus + added data layers
Open Hydro adopts the Arc Hydro data model and exposes it using open source GIS and database tools. Open Hydro also includes a suite of tools for working with hydrologic data and managing the associated hydro geodatabase.
Dan Ames, Idaho State University
• Based on MapWindow GIS: Free open source GIS alternative for data visualization and analysis.
• MapWindow GIS follows OGC standards and works with ESRI data formats.
• MapWindow GIS is extensible using VB.NET, VB6, C#, and ActiveX.
• Sponsored by NASA, NOAA, DOE, and EPA (new platform for BASINS).
• Nearly 30,000 downloaders so far at: www.MapWindow.org
Open Hydro is still under development and ISU is seeking partners to work together on various functions.
Current completed functions include geodatabase import/export, and automatic watershed delineation and support for ESRI’s geography network and online data sources.
http://www.daymet.org/
Demo of NHDPlus and Weather Downloader for the San Marcos
Basin
By Ernest To
Center for Research in Water Resources
• Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them
Objective
NWIS
NARR
NAWQANAM-12
request
request
request
request
request
requestrequest
request
request
return
return
return
return
return
returnreturn
return
return
What we are doing now …..
Michael PiaseckiDrexel University
Semantic MediatorWhat we would like to do …..
NWIS
NAWQA
NARR
generic
request
GetValues
GetValues
GetValues
GetValues
GetValues
GetValuesGetValues
GetValues
GetValues HODM
Michael PiaseckiDrexel University
Demo on Semantic Mediator
By Bora Beran and Michael Piasecki, Duke University
• Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive
• Software standards for model linking• Uses model core as an “engine”• http://www.openMI.org
OpenMI – Links Data and Simulation Models
CUAHSI Observations Data Model as an OpenMI component
Simple River Model
Trigger (identifies what value should be calculated)
Typical model architectureApplication
User interface + engineEngine
Simulates a process – flow in a channelAccepts inputProvides output
ModelAn engine set up to represent a particular location e.g. a reach of the Thames
Engine
Output data
Input data
Model application
Run
Write
Write
Read
User interface
Accepts Provides
Rainfall
(mm)
Runoff
(m3/s)
Temperature
(Deg C)
Evaporation
(mm)
Accepts Provides
Upstream Inflow
(m3/s)
Outflow
(m3/s)
Lateral inflow
(m3/s)
Abstractions
(m3/s)
Discharges
(m3/s)
River Model
Linking modelled quantities
Rainfall Runoff Model
Data transfer at run time
Rainfall runoff
Output data
Input data
User interface
River
Output data
Input data
User interface
GetValues(..)
Models for the processes
River(InfoWorks RS)
Rainfall(database)
Sewer(Mouse)
RR(Sobek-Rainfall
-Runoff)
Data exchange3 Rainfall.GetValues
River(InfoWorks-RS)
Rainfall(database)
Sewer(Mouse)
2 RR.GetValues
7 RR.GetValues
RR(Sobek-Rainfall
-Runoff)
1 Trigger.GetValues
6 Sewer.GetValues
call
data
4
5 8
9
OpenMIOpenMI = Open Modeling Interface
It is a Standard for Model Linking
It is part of HarmonIT, a European Commission funded research project with 14 project partners (universities & software companies)
www.openmi.org www.harmonit.org
OpenMI Conceptual Framework
VALUES
All values are referenced in a what-where-when framework, allowing different data resources or models to communicate data
HIS as OpenMI Components
water balance model
NWISStreamflow
DaymetPrecipitation
To calculate storage, the model needs inflow and outflow
To calculate storage, the model needs precipitation
Goal: Link the National HIS web services with a simple water balance model using OpenMI as the mediator
Trigger:
Calculate storage
Demo on Intregating Web Services and OpenMI to make a
water balance model
By Jon Goodall, Duke University
Watershed Hydrovolumes
Geovolume is theportion of a hydrovolumethat contains solidearth materials
USGS Gaging stations
Hydrovolume
Stream channel Hydrovolumes
Need the capacity to represent Acoustic Doppler Current Profiler (ADCP) data (Iowa)
Residence time distributions(Minnesota)
Integration of surface water and groundwater data
• Describe the relationship between surface water features ( e.g. streams
and waterbodies) with groundwater features (aquifers, wells).
• Enable the connection with the surface water data model
Hydro network Aquifers
In the future go to 3D...
Water OneFlow• Like Geospatial OneStop, we need a “Water
OneFlow” – a common window for water data and models
• Advancement of water science is critically dependent on integration of water information
Federal
AcademicLocal
State