Hydrologic Modeling in 2011David R. Maidment

Center for Research in Water ResourcesUniversity of Texas at Austin

Leader of the CUAHSI Hydrologic Information System Project

With acknowledgements to Rick Hooper, David Tarboton & Barbara Minsker

Hydrologic Modeling in 2011

• The charge and challenges

• Hydrologic information system – web services

• Integrating models and data using scientific workflows

• Hydrologic Observing System

Hydrologic Modeling in 2011

• The charge and challenges

• Hydrologic information system – web services

• Integrating models and data using scientific workflows

• Hydrologic Observing System

Workshop Charge

• What new technologies for observing, simulating, and tele-communicating will emerge over the next 5-10 years?

• how will they change the grand challenges for modeling, what will those challenges be?

• Challenge for this session:– How all the new devices/opportunities emerging in the

realm of “cyber-infrastructure”— including, perhaps especially, visualization schemes — might change the way models are developed and applied, including the new kinds of scientific questions to be asked in association with modeling.

Hydrologic Modeling

• We want to trace the movement of water, chemical and biological constituents through atmospheric, surface and subsurface water

• We want to do water, mass and energy balances

Hydrologic Information System

• A system is a connected set of components e.g. University of Texas System

• A web-based system is a set of components connected using the internet

• A hydrologic information system (HIS) is a web-based system linking hydrologic databases, tools and models CUAHSI HIS partner institutions

USGS Water Watch System

A national hydrologic observing system already exists – CUAHSI adds to it

Real-time Water Quality Estimates

Estimated total nitrogen

Stream discharge

mg/L cfs

CUAHSI Member Institutions

105 Universities as of May 2006

Challenges

• How to use test-beds to design real WATERS Observatories?

• How to share data from the test-beds with the whole community?

• How to include CUAHSI/CLEANER data not collected in the test-beds?

• How to empower individual scientists?

• How to make use of petascale computing?

Hydrologic Modeling in 2011

• The charge and challenges

• Hydrologic information system – web services

• Integrating models and data using scientific workflows

• Hydrologic Observing System

CUAHSI Web Services

CUAHSIWeb Services

Library

Web Application: Data Portal

Your application• Excel, ArcGIS, Matlab• Fortran, C/C++, Visual Basic• Hydrologic model• …………….

Your operating system• Windows, Unix, Linux, Mac

Internet Simple Object Access Protocol

http://www.cuahsi.org/HIS/

Rainfall & SnowWater quantity

and quality

Remote sensing

Water Data

Modeling Meteorology

Soil water

Water Data Web Sites

NWISWeb site output# agency_cd Agency Code# site_no USGS station number# dv_dt date of daily mean streamflow# dv_va daily mean streamflow value, in cubic-feet per-second# dv_cd daily mean streamflow value qualification code## Sites in this file include:# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC#agency_cd site_no dv_dt dv_va dv_cdUSGS 02087500 2003-09-01 1190USGS 02087500 2003-09-02 649USGS 02087500 2003-09-03 525USGS 02087500 2003-09-04 486USGS 02087500 2003-09-05 733USGS 02087500 2003-09-06 585USGS 02087500 2003-09-07 485USGS 02087500 2003-09-08 463USGS 02087500 2003-09-09 673USGS 02087500 2003-09-10 517USGS 02087500 2003-09-11 454

Time series of streamflow at a gaging station

CUAHSI Hydrologic Data Access System

A common data window for accessing, viewing and downloading hydrologic information

USGSUSGS

NASANASANCDCNCDCEPAEPA NWSNWS

ObservatoriesObservatories

http://river.sdsc.edu/HDAS

Observation Stations

Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System

USGS National Water Information System NOAA Climate Reference Network

Map for the US

NWIS Station Observation Metadata

Describe what has been measured at this station

Web Page Scraping

ProgrammaticallyProgrammatically construct a construct a URL string as produced by URL string as produced by manual usemanual use of the web page of the web page

http://nwis.waterdata.usgs.gov/nwis/discharge?site_no=02087500&agency_cd=USGS&....http://nwis.waterdata.usgs.gov/nwis/discharge?site_no=02087500&agency_cd=USGS&....

ParseParse the resulting ASCII file the resulting ASCII file

NWISNWIS

ArcGISArcGIS

ExcelExcel

NCARNCAR

UnidataUnidata

NASANASAStoretStoret

CUAHSI CUAHSI

AmerifluxAmeriflux

MatlabMatlab

AccessAccess SASSAS

FortranFortran

Visual BasicVisual Basic

C/C++C/C++

Some operational services

CUAHSI Web ServicesCUAHSI Web Services

Data SourcesData Sources

ApplicationsApplications

Extract

Transform

Load

http://www.cuahsi.org/his/

Core Web Methods

Method Input Output

GetSites Obs Network All station codes in network

GetSiteInfo Station Code Lat/long, station name

GetVariables Obs Network or data source

All variable codes

GetVariableInfo Variable code Description of variable

GetValues Station code or lat/long point, variable code, begin date, end date

A time series of values

GetChart As for GetValue A chart plotting the values

Operational Services

Service Ameriflux Daymet MODIS NWIS NAM HODM

Bear Creek

GetSites Yes Yes

GetSiteInfo Yes Yes Yes

GetVariables Yes Yes

GetVariableInfo Yes Yes Yes

GetValues Yes Yes Yes Yes Yes Yes

GetChart Yes Yes

XML Output from GetValues

NWIS

DayMet

MODIS

What is a Data Model

• A data model is a model that describes in an abstract way how data is represented

• Data models describe structured data for storage in data management systems such as relational databases.

• Early phases of many software development projects emphasize the design of a conceptual data model.

Lets see what Wikipedia says

CUAHSI Point Hydrologic Observations Data Model

• A relational database stored in Access, PostgreSQL, SQL/Server, ….

• Stores observation data made at points

• Consistent format for storage of observations from many different sources and of many different types.

Streamflow

Flux towerdata

Precipitation& Climate

Groundwaterlevels

Water Quality

Soil moisture

data

Hydrologic Observations Data Model (HODM)

Serving investigator data

• Several choices– You build CUAHSI

compatible services from your database

– You copy data into the HODM and use CUAHSI services

– You copy your data to an HODM and it is served from SDSC

Your database

Your implementation ofCUAHSI services

HODM

StandardCUAHSI services

Modeling Services

• Simulation models can be packaged as web services

• They can be queried and provide responses just like data archives

• We have an integrated network of data sources and models

A big challenge to integrate all the data streams!

• Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them

Objective

NWIS

NARR

NAWQANAM-12

request

request

request

request

request

requestrequest

request

request

return

return

return

return

return

returnreturn

return

return

What we don’t want …..

Michael PiaseckiDrexel University

Semantic MediatorWhat we do want …..

NWIS

NAWQA

NARR

generic

request

request

request

request

request

request

requestrequest

request

request HODM

Michael PiaseckiDrexel University

Hydrologic Modeling in 2011

• The charge and challenges

• Hydrologic information system – web services

• Integrating models and data using scientific workflows

• Hydrologic Observing System

Regional Storm Water Regional Storm Water Modeling Program and Modeling Program and

Master Plan for San Master Plan for San AntonioAntonio

City ofSan Antonio

Modeling System

Rainfall Data:Rain gagesNexrad

Calibration Data:FlowsWater Quality

Geospatial Data:City, CountySARA, other

FloodplainManagement

IntegratedRegional Water

Resources planning

CapitalImprovemen

tPlanning

FloodForecasting

Water qualityplanning

San Antonio Regional Watershed Modeling System

“Bring the models together”

Database

Geo-HMS

Geo-RAS

GIS-Gflow

Interface

HEC-HMS

HEC-RAS

Gflow

GIS

GIS Preprocessors for Hydrologic Models

Interfacedata models

HMS

RAS

Gflow

GIS

GeoDatabase

Arc Hydrodata model

Connecting Arc Hydro and Hydrologic Models

Digital Rain Maps from National Weather Service (03/04/2004)

FEMA 100-year flood plain map in Bexar County

RC1 RU

09R CO

08R CO

07R CO06R CO

04R CO03R CO

01R CO

02R CO

05R CO

Regional Watershed Modeling System Case Study

Rosillo Creekwatershed

• Arc Hydro Geodatabasefor whole watershed• HEC-HMS hydrology modelfor whole watershed• HEC-RAS hydraulic model for Rosillo Creek

Salado Creek watershed

Components:

Bexar County

Arc Hydro and HEC-HMS

Arc HydroSchematic Network

HEC-HMSHydrologic

Model

Calculates Flows

Arc Hydro and HEC-RAS

Arc HydroChannel

Cross Sections

HEC-RASHydraulic

Model

Calculates Water Surface

Elevations

Flow Change PointsModels communicate with

one another through Arc Hydro at designated points

Nexrad Map to Flood Map in Arc 9 Model Builder FLO

ODPLAIN MAP

Flood map as output

Model for flood flow

Model for flood

depth

HMS

Nexrad rainfall map as input

Web-Accessible Regional Watershed Modeling System

Complete storage of simulationmodels and workflows in geodatabases

Hydrologic Modeling in 2011

• The charge and challenges

• Hydrologic information system – web services

• Integrating models and data using scientific workflows

• Hydrologic Observing System

CUAHSI Hydrologic Observing System

Continental US Scale (coast to coast data coverage, HIS-USA)

1:500,000 scale

Regional Scale (e.g. Neuse basin)

1:100,000 scale

Watershed Scale (e.g. Eno watershed )

1:24,000 scale

Site Scale (experimental site level)

Site scale

Mul

tisca

le in

form

atio

n de

liver

y

A multiscale web portal system for observing and interpreting hydrologic phenomena by integrating data and models for any location or region in the United States

Point Point Observation Scale (gage, sampling location)

North American Scale (e.g. North American

Regional Reanalysis of climate)

1:1,000,000 scale

GeoTemporal Reference Frame

• A defined geospatial coordinate system for (x,y,z)

• A defined time coordinate system (UTC, Eastern Standard Time, ….)

• A set of variables, V• Data values v(x,y,z,t)

Space (x,y,z)

Time, t

Variables, V

v – data values

Data Cube

Series and FieldsFeatures

Point, line, area, volumeDiscrete space representation

Series – ordered sequence of numbersTime series – indexed by time

Frequency series – indexed by frequency

Surfaces Fields – multidimensional arrays

Scalar fields – single value at each locationVector fields – magnitude and direction Random fields – probability distribution

Continuous space representation

mm / 3 hours

Precipitation Evaporation

North American Regional Reanalysis of Climate

Variation during the day, July 2003

NetCDF format

Continuous Space-Time Model – NetCDF (Unidata)

Space, L

Time, T

Variables, V

D

Coordinate dimensions

{X}

Variable dimensions{Y}

Space, FeatureID

Time, TSDateTime

Variables, TSTypeID

TSValue

Discrete Space-Time Data ModelArcHydro

Hydrologic Flux Coupler

Precipitation

Evaporation

Streamflow

Define the fluxes and flows associated with each hydrovolume

Groundwater recharge

See Chapter 9 of Status Report for Details

ArcGIS ModelBuilder Application for Automated Water Balancing

Fields Series

Geospatial

Water Resource Regions and HUC’s

NHDPlus for Region 17E

NHDPlus Reach Catchments ~ 3km2

Reach Attributes

• Slope• Elevation• Mean annual flow

– Corresponding velocity

• Drainage area• % of upstream

drainage area in different land uses

• Stream order

Ingestion of real-time streamflow data

A national hydrologic observing system already exists – CUAHSI adds to it

Continental Water Dynamics Model

Hydrologic Information System

Hydrologic Observing

System

Hydrologic Modeling System

Petascale Computing

• 2.6 million river reaches on a 1:100,000 scale map of continental US

• Solve continuity and momentum equations once on each reach (~ 5.2 million equations) takes ~ 200 parallel processors

• Pittsburgh Supercomputer Center has 3000 parallel processors

• It is within reach to simulate flows on all reaches continuously through time with data assimilation from gaging stations

Conclusions

• Web services support a web-based hydrologic information system connnecting data, tools and models

• Models can be configured as web services

• Scientific workflows automate the integration of components

• A continental water dynamics model is feasible