Presented by: Steve S. Di Lonardo & James C. Alair September 15 th , 2009 West Point, New York

Presented by: Steve S. Di Lonardo & James C. Alair September 15th, 2009 West Point, New York

New York City Department of Environmental ProtectionBureau of Water Supply

Water Quality

Water Information System for the Management and Storage of Time Series

Data

Outline Challenges associated with time series data WISKI software as a solution Improvements to NYC DEP time series data

quality and access? Case studies Other potential

applications


Water Quality

Challenges of time-series data

Government & environmental agencies produce large amounts of data • Requires the establishment of information

systems to acquire and store data• Limited or no built-in QA/QC capabilities• Often results in data silos • Limited data sharing / access between groups


Water Quality

Time series data As part of DEP efforts to improve and

facilitate the availability of time series data:• Pilot study using WISKI software• Database manager software• Designed by Kisters North America Inc.


Water Quality

WISKI database manager• Infrastructure to automatically IMPORT, PROCESS,

COMPUTE, STORE, and REPORT all time series data

• Unlimited sites, stations, parameters and data storage (only limited by hardware)

• Based on MS-SQL, Oracle, etc….• Scalable from a single user to hundreds of users


Water Quality

WISKI database manager Data file importing (current and

archival)• All major data loggers formats• Telemetry data acquisition systems• SCADA systems• XML, ZRXP, & Comma delimited files • User defined time-series data


Water Quality

Who uses WISKI?


Water Quality

• Surface water (temperature)• Ground water (salinity) • Meteorological (precipitation)• Water quality (O2)• Stream discharge (flood management)• Any time-series data!

Applications – Types of Data


Water Quality

How/Where does WISKI fit at DEP?

LIMS – lab/field data SAS – historical lab/field data SCADA – real-time data

(operations) WISKI

• Compatible with LIMS, SCADA, SAS, ARCGIS

• Organization of high frequency data files into a manageable user-friendly format


Water Quality

WISKI software at DEP• Purchased by NYC DEP (2005)

– Used primarily by EOH Hydrology - Kensico streams

• Advancements 2007 - Present– Kensico Streams– USGS gauge stations– Weather stations – Keypoints – Reservoir Elevation/Capacity

• Water Security Initiative (EPA Grant)– Integral part of keypoint data acquisition and storage


Water Quality

• Access via BWS intranet to all DEP employees

• Automated data imports (excel, html, asci)• Reduce need for data requests• Data available in a working format

WISKI as a software solution


Water Quality

NYC Watershed & Distribution


Water Quality


Water Quality

Data ComputationsWISKI Data Management System

• Data Conversions – By Formula– Through Rating

Tables• Creates Summary

Data – Min, Max, Mean, Total – Daily, Monthly, &

Yearly


Water Quality

WISKI Data Reports

Standard Reports• Year End Reports• Monthly Reports• Rating Curve Reports• Discharge Measurement Summary• User-defined Reports


Water Quality

• Time and user stamped data QA/QC• Original imported time-series data cannot be

edited• Altered data maintains history• User-defined plausibility flags (eg. Turbidity)• Restrict sensitive data to certain user groups

WISKI Data Management System


Water Quality

Tabular presentation of the Flags

Comments and Remarks

Graphical presentation of the Flags

WISKI Data Management System


Water Quality

Case Studies & Potential Applications at NYC DEP


Water Quality

Case Study: Sewage Spill Warning & Prevention


Water Quality

Water Budgets / Storm Monitoring

Site NameAll Flow

(cfs)Base Flow

(cfs)Storm flow

(cfs)Base Flow

%Storm Flow

%MB-1 10,284,699 3,109,881 7,465,052 30.2% 72.6%E9 49,745,086 14,065,627 35,679,459 28.3% 71.7%

E10 11,986,940 2,595,844 9,272,129 21.7% 77.4%

E11 12,840,648 4,407,686 8,432,962 34.3% 65.7%

Sum 221,367,437 76,834,411 144,752,056 34.7% 65.4%


Water Quality

Event Detection


Water Quality

Operational Shutdown Monitoring• Reservoir elevation

• Discharge and turbidity

• Plausibility checks for out of range values


Water Quality

Flood forecasting•Create a flood forecasting system using

existing gage data•Establish a web-based warning network for

towns

Flood watch

Flood warning


Water Quality

• Nodes: represent catchments, junctions or recording sites

• Links: define how the information flows in the model

Data Modeling


Water Quality

Data analysis - Trends


Water Quality

Water Meter Information• Efforts have been made by NYC to

automate water meter data acquisition

• WISKI could interface with the data stream and auto-generate reports

• Provide a historical database for data analysis


Water Quality

Summary• WISKI Software represents a state of the

art tool for dealing with any time series data• Does not require special infrastructure –

can interface with any data source• Significant improvement in QA/QC & data

access• Success requires collaboration


Water Quality

Acknowledgements

Kisters North America Kerri Alderisio Lorraine Janus DEP Management


Water Quality

Documents

Presented by: Steve S. Di Lonardo & James C. Alair September 15 th , 2009 West Point, New York