Upload
haliem
View
217
Download
0
Embed Size (px)
Citation preview
1
University of Massachusetts, Amherst, © 20041
Evaluating Stormwater Technology PerformanceModule II
Guidance for the Technology Acceptance Reciprocity Partnership (TARP) Stormwater Protocol: Stormwater Best Management Practice Demonstrations
October 2004
Prepared byEric Winkler Ph.D. and Nicholas Bouthilette
Center for Energy Efficiency and Renewable EnergyUniversity of Massachusetts – Amherst
Stormwater pollution, especially in developed urban areas is a leading cause of water quality degradation in U.S. rivers, lakes, streams, and other surface waters. Water quality problems associated with nonpoint sources of pollution, particularly stormwater, are being addressed by federal mandates that affect all states. The National Pollutant Discharge Elimination System (NPDES) Phase II, Storm Water Regulations requires stormwater plans from thousands of municipalities nationwide, and there is a renewed focus nationally on the total maximum daily load provisions (TMDL) in the Clean Water Act. These programs are predicted to have a significant influence on the rate at which new technologies enter the marketplace, as they bring more attention and increased resources to stormwater control issues.
As new products are added to the marketplace list of stormwater best management practices, there is a growing demand for good data and information on these technologies. To address this issue, the Technology Acceptance and Reciprocity Partnership (TARP) states of CA, MA, MD, NJ, PA, VA, and WA formed the Stormwater Work Group to layout a cost-effective path for evaluating innovative stormwater technologies by producing scientifically credible data, as well as useful information, which demonstrates how to apply the technology in the field to manage stormwater runoff. To support responsible use of stormwater technologies, the TARP Stormwater Work Group developed two Internet-based training modules. The two modules cover aspects of stormwater technology evaluation and are supported by the “TARP Protocol for Stormwater Best Management Practice Demonstrations.”Module I: Planning for a Stormwater BMP DemonstrationFactors Affecting Stormwater SamplingData Quality Objectives and the Test QA Plan Module II: Collecting and Analyzing Stormwater BMP Data Statistical AnalysesSampling DesignData Adequacy: Case Study
The purpose of the Protocol is to provide a uniform method for demonstrating stormwater technologies and developing test quality assurance (QA) plans for certification or verification of performance claims. In this training course you will learn:
to use the Protocol to identify gaps and inconsistencies in a test plan for evaluation of a stormwater technology and understand differences in state requirements;to recognize data deficiencies;to develop, implement, and review a test plan;to understand, evaluate, and implement statistical methods.
The goal of this training course is to encourage collaboration, data sharing and reciprocal review amongst states and practitioners, while providing a uniform method for demonstrating stormwater technologies and developing Test Quality Assurance (QA) Plans. As a result of these efforts there will be an increase in expertise for assessment of stormwater technology performance and a reduction decision-making timeframes for technology implementation.For access to the training modules:(1) http://www.clu-in.org/conf/tarp/stormwater/ (2) For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/. EPA Office of Superfund Remediation and Technology Innovation http://clu-in.org/studio/ECOS – Environmental Council of States http://www.sso.org/ecos/CEIT - Center for Environmental Industry & Technology, U S EPA New England http://www.epa.gov/ne/assistance/ceit/CEERE – Center for Energy Efficiency and Renewable Energy, University of Massachusetts http://www.ceere.org/
2
University of Massachusetts, Amherst, © 20042
Meet the Instructors
Nancy BakerMassachusetts Dept. of Environmental Protection1 Winter StreetBoston, MA 02108(617) 654-6524 (Voice)(617)292-5850 (Fax)[email protected]
Eric Winkler, Ph.D.University of Massachusetts160 Governors DriveAmherst, MA 01003(413) 545-2853 (Voice)(413) 545-1027 (Fax)[email protected]
Instructor Biographies:Nancy Baker is an environmental analyst in the Massachusetts Department of Environmental Protection, Northeast Regional Office, where she provides oversight and commentary on stormwater issues and coordinates the region’s reviews of development projects that require permits from the Department. Nancy has chaired the Technology Acceptance Reciprocity Partnership Stormwater Work Group since it’s inception in 1999 and has assisted with the Massachusetts Strategic Envirotechnology Partnership (STEP) evaluation of stormwater technologies, including the preparation of fact sheets, which complement the STEP verification reports.Previously, Nancy supervised the Department’s staff in Outreach Services, which produced publications and training materials, including the Massachusetts Stormwater Management Handbooks. She formerly was an analyst with the Massachusetts DEP Drinking Water Program and the Massachusetts Environmental Policy Act (MEPA) Unit in the Executive Office of Environmental Affairs. Nancy has a B.S. degree in biology, an M.S. degree in marine biology, and enjoys landscape gardening and gourmet cooking.
Eric Winkler, Ph.D. is Director of Technical Services in the Center for Energy Efficiency and Renewable Energy (CEERE) at the University of Massachusetts Amherst. He is primarily responsible for outreach and technical support services on energy and environmental systems to public and private sector clients. He is also Director of the Massachusetts Energy Efficiency Partnership leading state and region wide initiatives with industry, public utilities and state agencies on energy and resource conservation. Dr. Winkler is part of the implementation team for two other DOE funded programs at CEERE, the Industrial Assessment Center and the Northeast CHP Application Center.
Dr. Winkler also directs a university based research and development program for innovative energy and environmental technologies. He was technical lead for the Massachusetts Executive Office of Environmental Affairs/UMass Strategic Envirotechnology Partnership focusing on technology validation of environmental and energy technologies. Dr. Winkler has served on state and federal environmental advisory review panels, including the MA DEP Storm Water Advisory Panel and the EPA ETV Washington DOT Project Review Panel. Dr. Winkler has been Principal Investigator on a number of US EPA S319 projects, most recently leading a project to develop a clearing house of information on proprietary stormwater technologies and a prioritization process for investment in technology demonstrations and testing. He has published numerous reports and journal articles on innovative wastewater and stormwater technologies. Dr. Winkler received his Doctorate in Environmental Chemistry from the University of Massachusetts Amherst in 1995 and holds Masters Degrees in Public Health and Soil Science.
3
University of Massachusetts, Amherst, © 20043
Meet the Sponsors
TARP Stormwater Work Group
CaliforniaMarylandMassachusettsNew JerseyPennsylvaniaVirginia
State of Washington, Illinois, New York, and ETV are collaborating with TARP
TARP Member State
TARP – Technology Acceptance Reciprocity Partnershiphttp://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/A state tool to promote scientifically sound, cost-effective environmental decision making. Information on the performance of new technologies is critical to state environmental protection efforts. Regulatory standards and permits often rely on technology performance data. Unfortunately, few, if any, standardized methods have been established to guide the collection and evaluation of technology performance across the states. And, while states look to new environmental technologies as cost-effective opportunities to achieve better environmental performance, reliable performance information is hard to find. As a result, new technologies often face unnecessary and financially burdensome regulatory and permit hurdles that slow down or prevent their use. That is why TARP - The Technology Acceptance and Reciprocity Partnership - was formed by the states of California, Illinois, Maryland, Massachusetts, New Jersey, New York, Pennsylvania, and Virginia
4
University of Massachusetts, Amherst, © 20044
Goals of TARP Stormwater Work Group
Use protocol to test new BMPsApprove effective new stormwater BMPsGet credible data on BMP effectivenessShare information and dataIncrease expertise on new BMPsUse protocol for appropriate state initiatives
TARP Member State
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
5
University of Massachusetts, Amherst, © 20045
Evaluating Stormwater Technology Performance
Learning ObjectivesUse the TARP Stormwater Demonstration Protocol to review a test plan and a technology evaluationRecognize data gaps and deficienciesDevelop, implement, and review a test planUnderstand, evaluate, and use statistical methods
Logistical RemindersPhone audience
Keep phone on mute* 6 to mute your phone and again to un-muteDo NOT put call on hold
Simulcast audience
Use at top of each slide to
submit questions
Course time = 2 hours
3 question & answer periods
Links to additional resources
Your feedback
For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
To access training materials on-line: http://www.clu-in.org/conf/tarp/stormwater/
6
University of Massachusetts, Amherst, © 20046
Project NoticePrepared by The Center for Energy Efficiency and Renewable Energy, University of Massachusetts Amherst for submission under Agreement with the Environmental Council of States. The preparation of this training material was financed in part by funds provided by the Environmental Council of States (ECOS).
This product may be duplicated for personal and government use and is protected under copyright laws for the purpose of author attribution.
“Publication of this document shall not be construed as endorsement of the views expressed therein by the Environmental Council of States/ITRC or any federal agency."
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
7
University of Massachusetts, Amherst, © 20047
DISCLAIMER
The contents and views expressed are those of the authors and do not necessarily reflect the views and policies of the Commonwealth of Massachusetts its agencies or the University of Massachusetts. The contents of this training are offered as guidance. The University of Massachusetts and all technical sources referenced herein do not (a) make any warranty or representation, expressed or implied, with respect to accuracy, completeness, or usefulness of the information contained in this training, or that the use of any information, apparatus, method, or process disclosed in this report may not infringe on privately owned rights; (b) assume any liabilities with respect to the use of, or for damages resulting from the use of, any information, apparatus, method or process disclosed in this report. Mention or images of trade names or commercial products does not constitute endorsement or recommendation of use.
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
8
University of Massachusetts, Amherst, © 20048
Module I: Planning for A Stormwater BMP Demonstration
1. Factors Affecting Stormwater Sampling2. Data Quality Objectives and the Test QA Plan
3. Sampling Design4. Statistical Analyses5. Data Adequacy: Case Study
Module II: Collecting and AnalyzingStormwater BMP Data
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
9
University of Massachusetts, Amherst, © 20049
3. Sampling Design
Stormwater data collection guidance
Locating samples and stations
Selecting water quality parameters
QA/QCSample handling and record keepingField measures
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
10
University of Massachusetts, Amherst, © 200410
Sampling Plan Elements TARP Data Collection Criteria
(Section 3.3, TARP Protocol)
Any relevant historic dataMonthly mean rainfall and snowfall data (12 months over the period of record)Rainfall intensity over 15 minute increments
These elements are specified in the TARP as required components of the sampling plan.
11
University of Massachusetts, Amherst, © 200411
Protocol Minimum Criteria Identifying Qualifying Storm Event
(Section 3.3.1.2 and Section 3.3.1.3, TARP Protocol)
Minimum rainfall event depth is 0.1 inchMinimum inter-event duration of 6 hours (duration beginning a cessation of flow to unit)Base flow should not be sampled
Identification of qualifying event needs to verify flow to the unit and rainfall concurrently
Protocol specifies a minimum of a 6 hour inter-event period. Other national protocols (WASHDOT) may specify longer periods, up to 72 hours.Inter-event period is a value is based on a reasonable time in which pollutant buildup may occur.Sampling first flush, may be more important in capturing certain pollutants. Sampling long duration storms or storms that are closely link may not produce useful information. Also, consider concentration effects on treatment potential and sampling long durations storms.
12
University of Massachusetts, Amherst, © 200412
Analysis for Determining Number of Samples
Paired sampling approach-Significant difference in mean concentration
Coefficient of variationPower of 80%
95% confidence(Urban Stormwater BMP Performance Monitoring. USEPA, 2002.)
Coefficient of Variation
Diff
eren
ce in
sam
ple
set
mea
ns (
%)
Example: 80% difference in means, 100% CV = 20 sample pairs
Higher variability between sample means (CV) will require more sample pairs to get the same level of confidence for the difference between means.Smaller differences between paired samples will require more sample pairs to get the same level of confidence for significant differences.
13
University of Massachusetts, Amherst, © 200413
Qualifying Event Sample TARP Protocol Criteria
10 water quality samples per event10 influent and 10 effluentIf composite - 2 composites, 5 sub-samples
Data for flow rate and flow volumeAt least 50% of the total annual rainfall
CA – monitor 80-90% of rainfall
Sampling criteria.
14
University of Massachusetts, Amherst, © 200414
Qualifying Event Sample (continued)
Preferably 20 storms, 15 minimumSampling over the course of a full year of sampling to account for seasonal variationCompositing flow-weighted samples cover at least 70% of storm flow (and as much of the first 20% as possible)Examples of variation within TARP community
PA - Temporary BMPs sized using 2 year eventNJ – Water Quality design based on volume from a 1.25 inch event
These requirements may vary from region to region to rainfall characteristics.
15
University of Massachusetts, Amherst, © 200415
Sampling Locations
Location in close proximity to BMP technology inlet and outletConsider field personnel safety during equipment accessSecure equipment to avoid vandalismProvide a scaled plan view of site that indicates
All buildingsLand usesStorm drain inletsOther control devices
•Sampling can occur in catch basin, which offers a well mixed environment.•Also can sample within BMP structure, when sampling equipment will not interfere with operation of unit and will not be biased by location to close to treatment section.•This is often a good choice due to the convenience of building in testing structures or placement of test equipment.
16
University of Massachusetts, Amherst, © 200416
Automated Sampling Equipment
Water surface elevation should be less than 15 feet below the elevation of the pump in the sampler
Access to the equipment should take into consideration confined space safety
Flow measurement equipment should be located to avoid measuring turbulent flow
Area velocity meter placement is typically position in a location 10 times the diameter of the pipe from any point in which turbulence occurs (a bend, inlet, tee). Grab sampling or manual sampling may be needed for certain pollutants including:pH, Temperature, Cyanide, Total phenols, Residual chlorine, Oil and grease, E. coli, Total coliform, Fecal coliform, Fecal streptococci, Enterococci, Total petroleum hydrocarbons
17
University of Massachusetts, Amherst, © 200417
Adequacy of Sampling and Flow Monitoring Procedures
Primary and secondary flow measurement devices are requiredProgrammable automatic flow samplers with continuous flow measurements are recommendedTime-weighted composite samples are not acceptable unless flow is monitored and the event mean concentration can be calculated
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
18
University of Massachusetts, Amherst, © 200418
Monitoring Location Recommendations
Monitoring location design should consider whether the upgradient catchment system is served by a separate storm drain systemPay attention to potential combined sewer system or illicit connections which may contaminate stormwater systemThe storm drain system should be well understood to allow reliable delineation and description of catchment areaFlow-measuring monitoring stations in open channels should have suitable hydraulic control and where possible the ability to install primary flow measurement devices
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
19
University of Massachusetts, Amherst, © 200419
Monitoring Location Recommendations (continued)
Avoid steep slopes, pipe diameter changes, junctions, and irregular channel shaping (due to breaks, roots, debris)Avoid locations likely to be affected by backwater and tidal conditionsPipe, culvert, or tunnel stationing should be located to avoid surcharging (pressure flow) over the normal range of precipitationUse of reference watershed and remote rainfall data are discouraged
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
20
University of Massachusetts, Amherst, © 200420
Selecting Applicable Water Quality Parameters
Designated uses of the receiving water – consider stormwater discharge constituentsOverall program objectives and resources – adjust parameter list according to resources (test method capability, personnel, funds, and time)Use of “Keystone” pollutant may vary from state to state
It should be noted that some State to State variability exists in this area.Reciprocity between states may be compromised if the study design does not include consideration of other state’s water quality criteria.VA uses P as Keystone pollutant parameter.
21
University of Massachusetts, Amherst, © 200421
Resources for Standardized Test Methods (Section 3.1, TARP Protocol)
EPA Test Methods – pollutant analysis www.epa.gov/epahome/Standards.htmlASME Standards and Practices – pressure flow measurementsASCE Standards – hydraulic flow estimationASTM Standards – precision open-channel flow measurements for water constituent analysisNational Field Manual for Collection of Water Quality Data, Wilde et al., USGS water.usgs.gov/owq/FieldManual/“Guidance Manual: Stormwater Monitoring Protocols” –Caltrans. http://www.dot.ca.gov/hq/env/stormwater/special/newsetup/index.htm#monitoring
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
22
University of Massachusetts, Amherst, © 200422
Examples of Analytical Laboratory Methods
Total Phosphorus – SM4500-PENitrate and Nitrite – EPA353.1Ammonia – EPA350.1Total Kjeldahl Nitrogen – EPA351.2TSS – SM2540DSSC – ASTM D3977-97Enterococci – SM9230CFecal Coliform – SM9222DChronic Microtox Toxicity Test – Azur Environmental
Reference: ASTM, EPA, American Public Health Association, (Other non-standard methods or tests approved through acceptable regulatory process, EPA or state authority)
The process of choosing a method is dependent upon the method sensitivity and the data quality objectives. In addition field conditions including predicted pollutant concentrations may impact the choice of sample procedures, including interferences from other dissolved or suspended constituents.Furthermore, as is generally a part of “good” study design, consider the capability of available resources to provide testing methods (e.g. some labs cannot meet sensitivity requirements specified by EPA even if they physically can perform the test procedure).
23
University of Massachusetts, Amherst, © 200423
Constituent Listing with Detection Limits
Urban Stormwater BMP Performance Monitoring. US EPA, 2002.
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
24
University of Massachusetts, Amherst, © 200424
Representative Sampling
Sample CollectionUniformity varies between staff and techniqueAutomated sampling allows for reproducible samplesStandard Operating Procedures (SOPs) and Quality Assurance help track variability
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
25
University of Massachusetts, Amherst, © 200425
Required Elements in Lab QA/QC (Section 3.3.5, TARP Protocol)
The QAPP Test QA Plan and Sampling and Analysis Plan should include
Laboratory and sampling equipment decontaminationSample preservation and holding timeSample volumesQC samples (spikes, blanks, splits, field and blank lab duplicates)QA on sampling equipment (calibration)Packaging and shippingIdentification and labelingChain of custodyLab certification
QA on equipment needs to include: (Calibration of Automatic Samplers and Flow Measurement Devices)
Certified by a state or federal agency regulating laboratory certification or accreditation programs.
NELAC or ELAP Program used to perform standardized test methods and procedures
26
University of Massachusetts, Amherst, © 200426
Quality Control Tests and Acceptance Limits for Physical/Chemical Parameters Analyzed in the Laboratory
Each sample batch of ≤ 20
samples (≥5%)≤ 25Duplicates
Each sample batch of ≤ 20
samples (≥5%)75-125% rec.QCS
Total Suspended Solids
Each sample batch of ≤ 20
samples (≥5%)≤ 25Duplicates
Each sample batch of ≤ 20
samples (≥5%)75-125% rec.QCS
Suspended Sediment
Concentration
80-120% rec.LFM
85-115% rec.LFBEach sample batch of ≤ 20
samples (≥5%)≤28Duplicates
Each sample batch of ≤ 20
samples (≥5%)
90-110% rec.QCS
Total Kjeldahl N
80-120% rec.LFM
85-115% rec.LFBEach sample batch of ≤ 20
samples (≥5%)≤ 52Duplicates
Each sample batch of ≤ 20
samples (≥5%)
90-110% rec.QCS
Ammonia-N
80-120% rec.LFM
85-115% rec.LFB
Each sample batch of ≤20 samples
(≥ 5%)
≤45DuplicatesEach sample batch of ≤ 20
samples (≥5%)
90-110% rec.QCSNitrate and Nitrite
N
80-120% rec.LFMd
85-115% rec.LFBcEach sample batch of ≤20
samples (≥5%)≤20Duplicates
Each sample batch of ≤ 20
samples (≥5%)
90-110% rec.QCSb
Total Phosphorus
FrequencyAcceptance
Limits(RPD)
QC TestFrequencyAcceptance
Limits (RPD)a
QC Test
PrecisionAccuracy
Parameter
aRPD = relative percent difference among duplicates cLFB= laboratory fortified blank samplebQCS = quality control sample from source outside of laboratory dLFM = laboratory fortified matrix sample
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
27
University of Massachusetts, Amherst, © 200427
Laboratory Records
Sample dataNumber of samples, holding times, location, deviation from SOP’s, time of day, and dateCorrective procedures for samples inconsistent with the protocol
Management of recordsConsider electronic filingDocument data validation, calculations and analysis, and data presentationReview data reports for completeness, including requested analyses and all required QA
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
28
University of Massachusetts, Amherst, © 200428
Field Sample QC
Field matrix spikeA sample prepared at the sampling point by adding a known mass of the target analyte to a specified amount of sampleUsed to determine the effect of sample preservation, shipment, storage, and preparation on analyte recovery efficiency
Field split sampleThe split of a sample into two representative portions to be sent off to different laboratoriesEstimates inter-laboratory precision
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
29
University of Massachusetts, Amherst, © 200429
Quality Control
Field blanksA clean sample carried to the sampling site, exposed to sampling conditions, and returned to the laboratoryProvides useful information about pollutants and error that may be introduced during the sampling process
Field duplicatesIdentical samples taken from the same sampling location and time (not split)Identical sampling and analytical procedures to assess variance of sampling and analysis
Some issues regarding applicability of field duplicates with respect to realistically being able to perform such a task.Can you really get a valid duplicate?
30
University of Massachusetts, Amherst, © 200430
Quality Assurance / Quality ControlDocumentation and Records
DocumentationInclude all QC dataDefine critical records and information, as well as the data reporting format and document control procedures
ReportingField operation recordsLaboratory recordsData handling records
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
31
University of Massachusetts, Amherst, © 200431
Field Operation Records
Sample collection recordsShow that the proper sampling protocol was used in the field by indicating persons names, sample numbers, collection points, maps, equipment/methods, climatic conditions, and unusual observations
Chain of custody recordsDocument the progression of samples as they travel from samplinglocation to the lab to disposal area
QC sample recordsDocument QC samples such as field, trip, blanks, and duplicate samplesProvide information on frequency, conditions, level of standards, and instrument calibration history
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
32
University of Massachusetts, Amherst, © 200432
Field Operation Records (continued)
General field proceduresRecord field procedures and areas of difficulty in gathering samples
Corrective action reportsWhere deviations of standard operating procedures occurs, report methods used and/or details of procedure, and a plan to resolve noncompliance issues
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
33
University of Massachusetts, Amherst, © 200433
Questions and Answers (1 of 3)
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
34
University of Massachusetts, Amherst, © 200434
Module I: Planning for A Stormwater BMP Demonstration
Module II: Collecting and AnalyzingStormwater BMP Data
1. Factors Affecting Stormwater Sampling2. Data Quality Objectives and the Test QA Plan
3. Sampling Design4. Statistical Analyses5. Data Adequacy: Case Study
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
35
University of Massachusetts, Amherst, © 200435
4. Statistical Analyses
Data reporting and presentationStatistical method reviewAppropriate forms of analysesResults interpretation
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
36
University of Massachusetts, Amherst, © 200436
Presenting Statistical Data and Applicability Efficiency Calculation Implications
Determine the category of BMPBMPs with well-defined inlets and outlets whose primary treatment depends upon extended detention storage of stormwaterBMPs with well-defined inlets and outlets that do not depend upon significant storage of waterBMPs that do not have a well defined inlet and/or outletWidely distributed BMPs that use reference watersheds to evaluate effectiveness
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
37
University of Massachusetts, Amherst, © 200437
Analysis to Address Data Quality Objectives
What degree of pollution control does the BMP provide under typical operating conditions?How does efficiency vary from pollutant to pollutant?How does efficiency vary with input concentrations?How does efficiency vary with storm characteristics?How do design variables affect performance?How does efficiency vary with different operational and/or maintenance approaches?
Idealized questions one may ask.
38
University of Massachusetts, Amherst, © 200438
Analysis to Address Data Quality Objectives (Continued)
Does efficiency improve, decay, or remain stable over time?How does efficiency, performance, and effectiveness compare to other BMPs?Does the BMP reduce toxicity?Does the BMP cause an improvement or protect downstream biotic communities?Does the BMP have potential downstream negative impacts?
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
39
University of Massachusetts, Amherst, © 200439
Recommended Statistical Methods for Performance Evaluation
Efficiency ratioSummation of loadsRegression of loadsMean concentrationEfficiency of individual storm loads Lines of Comparative Performance Method
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
40
University of Massachusetts, Amherst, © 200440
Efficiency Ratio (ER)TARP Protocol Recommended Method
Where Event Mean Concentration (EMC):
V=volume of flow during period in=total number of events C=average concentration associated
with period jm=number of events measured
ER = 1 - = Average Outlet EMCAverage Inlet EMC
Average Inlet EMC - Average Outlet EMCAverage Inlet EMC
EMC =∑ CjVjj=1
n
∑ Vjj=1
n
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
41
University of Massachusetts, Amherst, © 200441
Efficiency Ratio Interpretation
EMCs weight all storms equallyMost useful when loads are directly proportional to the relative magnitude of the storm – accuracy varies with BMP typeMinimizes impacts of smaller/cleaner stormsAllows for use of data where portions of data are missing –would not significantly effect the average EMC
Can apply log normalization to avoid equal weighting of events
Mean of the Log EMCs =∑ Log(EMCj)j=1
m
m
Can also apply log normalization to avoid equal weighting of events.
42
University of Massachusetts, Amherst, © 200442
Summation of Loads
The sum of loads can be calculated using concentration and flow volume, as follows:
Removal efficiencies calculated by the summation of loads tend to be dominated by larger storm events
The summation of loads method uses a mass balance approach
SOL = 1 -Sum or outlet loadsSum of inlet loads
Sum of Loads = ∑(∑ CiVi) = ∑ EMCj*Vjj=1
m
j=1
m
i=1
n
Removal efficiencies calculated by the summation of loads tend to be dominated by larger storm events. The summation of loads method uses a mass balance approach.
43
University of Massachusetts, Amherst, © 200443
Regression of Loads
The Regression of Loads is defined as
Percent reduction in loads is approximated as
β - slope term in regression analysis
Data can be dominated by large storm eventsIt is recommended that the line not be forced through originMay require polynomial fit to achieve higher R2
Loads out = ß * Loads in = ß -Loads outLoads in
Percent Removal = 1- ß = 1 -Loads outLoads in
Data can be dominated by large storm events.It is recommended that the line not be forced through origin.May require polynomial fit to achieve higher R2
44
University of Massachusetts, Amherst, © 200444
Mean Concentration
The Mean Concentration is defined as:
May be useful for threshold level pollutants, e.g., bacterial or toxicsWeights samples equally and may result in bias due to variances in
sampling protocolsNot amenable to mass balance approachFlow measure must represent total event characteristics
MC = 1 -Average of outlet concentrationAverage of inlet concentration
May be useful for threshold level pollutants, e.g., bacterial or toxics.Weights samples equally and may result in bias due to variances in sampling protocols.Not amenable to mass balance approach.Flow measure must represent total event characteristics
45
University of Massachusetts, Amherst, © 200445
Efficiency of Individual Storm Loads
The efficiency of the BMP for a single storm is given by:
Average efficiency can be calculated as follows:
Weights all storms equallyMust have paired data for inflow and outflowDoes not account for interrelationship between storm events
Storm Efficiency = 1 -Loadout
Loadin
Average Efficiency =m
∑ Storm Efficiencyjj=1
m
Weights all storms equally.Must have paired data for inflow and outflow.Does not account for interrelationship between storm events.
46
University of Massachusetts, Amherst, © 200446
Interpreting Results and Inappropriate Analyses
Efficiency Ratio – Most useful when loads are directly proportional to the relative magnitude of the storm – accuracy varies with BMP typeSummation of Loads – A small number of large storms can significantly influence resultsRegression of Loads – Assumes removal efficiency is uniform over a range of operating conditions and concentrationsMean Concentration – Not appropriate where flow-weighted sampling is performed – weighs all events equally
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
47
University of Massachusetts, Amherst, © 200447
Using Statistics Inappropriately
Each method is likely to produce a different efficiencyMethod should be chosen by its relevance and applicability to each scenario, not by the efficiency value it producesBe aware of statistics being misused to support claimsReporting of ranges may be more appropriate under certain test conditions, e.g., determination that data is qualitative versus quantitative
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
48
University of Massachusetts, Amherst, © 200448
Questions and Answers (2 of 3)
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
49
University of Massachusetts, Amherst, © 200449
Module I: Planning for A Stormwater BMP Demonstration
Module II: Collecting and AnalyzingStormwater BMP Data
1. Factors Affecting Stormwater Sampling2. Data Quality Objectives and the Test QA Plan
3. Sampling Design4. Statistical Analyses5. Data Adequacy: Case Study
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
50
University of Massachusetts, Amherst, © 200450
5. Case Study I: Test Plan and Data Adequacy
Review of dataTest QA Plan and Data Quality Assurance Project PlanField and lab data adequacyData reporting and depiction of performance claims
Potential problemsField and lab violationsData reduction issues
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
51
University of Massachusetts, Amherst, © 200451
Background
Study of structural BMPPerformance claim made – 80% TSS removalStudy commenced prior to TARP Protocol as well as other published, public domain stormwater BMP monitoring protocolsResources for conducting study borne by single private entityStudy design included limited information on site, sampling equipment, sampler programming, calibration, sample collection and analysisTSS/SSC primary water quality parameter of interestFlow measurement and rain gauge equipment to be installedAnalytical testing to be performed by outside laboratorySample collection by technology developer
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
52
University of Massachusetts, Amherst, © 200452
Sample Plan Adequacy – Issues Identified
Provide complete engineering drawings of system includingEntire drainage area connected to systemPipe sizing and inlet locationsDescription of pervious and impervious surfacesDesign calculations used to size unitClimatic data used to design structuresAny additional structures or site details that may have bearing on system
Provide use characteristics of site includingVehicle and industrial usageMaintenance of site relative to sweeping, gutter maintenance, spill containment, and snow stockpiling/disposal
Indicate condition and maintenance of system prior to commencing test, e.g., was system cleaned out during or before initiation of this phase of the study?
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
53
University of Massachusetts, Amherst, © 200453
Sample Plan Adequacy –Test Equipment and Sampling Issues
Provide a description and/or reference to manufacturer’s documentation showing how flow meter and sampling equipment is calibrated and their location in the system (cross-section and plan view)Detailed description of sampling program by design and storm eventIndicate when and how much sample to be takenProvide explanation and methodology for discreet sampling. Indicate who, when, and where in the system these samples were takenIdentify equipment used and calibrations used for samplingProvide discussion/explanation for sampling without detention delay in the system between inlet and outlet
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
54
University of Massachusetts, Amherst, © 200454
Sample Plan Adequacy –Lab and Data Analysis Issues
Identify laboratory and controls for sampling handling, QA/QC on sample dataIdentify methodology used for PSD, including sieve sizes and sample sizesAnalytical method for TSS stated in protocol - EPA 160.2Need explanation and equations used to calculate EMC, including calculation worksheets
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
55
University of Massachusetts, Amherst, © 200455
Field Test Issues
Flow meters not calibrated for the first 10 eventsTruncated sampling protocolNo documented instrument calibrationOnce flow measurement error was identified, an adjusted flow factor is applied to the first 10 events, based on the outcome of the second 10 eventsMeasured and calculated storm volumes vary by up to a factor of 10 resulting in adjustment to flow volume
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
56
University of Massachusetts, Amherst, © 200456
Data Analysis Issue
Values in italics are untransformed flow values in ft3
63 %70.3188.4377,7530.302063 %2157.4326,6140.2519
73 %25.593.3336,8580.2718
56 %102.4233.68029,3781.0217
43 %93.2164.14071,6071.9116
62 %12.633.1409,5430.4315
79 %14.769.2122,9080.1314
57 %19.846.27013,2100.5313
58 %47.3111.6401,2840.4812
33 %59.188.8123147,5865.4511
38 %3861124,559 (608)0.1710
10 %33.637.262,183 (291)0.109
95 %521088.84018,045 (2406)0.758
28 %31433014,441 (1925)0.557
89 %57.8533.2186,831 (911)0.466
60 %145.9367.6119,630 (1284)0.535
94 %49.4857.61313,203 (1760)0.324
95 %63.61364.9129,117 (1216)0.323
85 %149.21010.71712,051 (1607)0.522
24 %50.365.92821,600 (2880)0.31
Event Removal Efficiency1
Effluent EMC(mg/L)
Influent EMC(mg/L)
Number of Sub SamplesFlow Volume M (ft3)1
Rainfall Depth(in)Storm
What is notable about this data set is that flows and rainfall depth do not correlate. See next slide.The entity analyzing this data modified the flows based on rainfall depths as they are correlated with events 11 thru 20.While this modification was noted, it may not be justified and may impact EMC values used to calculate performance efficiency.
57
University of Massachusetts, Amherst, © 200457
Predicted / Measured Runoff Values
0
5,000
10,000
15,000
20,000
25,000
30,000
0 0.5 1 1.5 2
Adjusted Runoff
Measured Runoff
95% Conf +
95% Conf -
Predicted Runoff
Linear (95% Conf +)
Linear (95% Conf -)
The values highlighted in red indicate that the relationship between rainfall and total runoff is inconsistent
Applying the rational method (Q=CiA) leads reviewers to believe that measured runoff is inaccurate when compared to predicted values
Runo
ff (
ft3 )
Precipitation (inches)
Illustrates an analysis of data which demonstrates that flow measurement equipment was not properly calibrated.
58
University of Massachusetts, Amherst, © 200458
Variation in Performance Values
59%Removal Efficiency by Efficiency of Individual Storm Events:
40%Removal Efficiency by Regression of Loads:
44%Removal Efficiency by Summation of Loads:
57%Removal Efficiency by Efficiency Ratio:
Removal Efficiencies for Events 11-20:
60%Removal Efficiency by Efficiency of Individual Storm Events:
77%Removal Efficiency by Regression of Loads:
73%Removal Efficiency by Summation of Loads:
83%Removal Efficiency by Efficiency Ratio:
Removal Efficiencies for all Events:
Using various statistical methods, a comparison of TSS removal efficiencies between all events and those where the flow measures are more representative of rainfall, demonstrates significant divergence of performance values.Analysis also demonstrates that efficiency measures by storm is not affected by errors in flow measurement.
59
University of Massachusetts, Amherst, © 200459
Data Analysis and Presentation Issues
Impact of adjusted flow on average net TSS/SSC removal resulted in positive bias in performance efficiencyMissing raw data including documented deviations from sampling plan, lab analysis and data managementMissing laboratory and data management QA/QCNo independent validation of calibrations, sampling and analysisMissing statistical analysis including relative percent differences (precision) and percent recovery (accuracy) as well as number of sample (n), standard deviations, means (specify arithmetic or geometric) and standard errorsMissing documentation of chain of custody protocol
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
60
University of Massachusetts, Amherst, © 200460
Ensuring Adequate Data
Provide QC activities including blanks, duplicates, matrix spikes, lab control samples, surrogates, or second column confirmationState the frequency of analysis and the spike compound sources and levelsState required control limits for each QC activity and specify corrective actions and effectiveness if limits exceeded
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
61
University of Massachusetts, Amherst, © 200461
Summary of Data Quality Assessment
5 Steps1. Review the DQOs and sampling design2. Conduct a preliminary data review3. Select the statistical test4. Verify the assumptions of the statistical test5. Draw conclusions from the data
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
62
University of Massachusetts, Amherst, © 200462
Summary: Sampling Design
Use of TARP Protocol data collection criteriaConsideration of issues relating to sampling locationsSampling water quality parametersLab analysis of water quality parameters
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
63
University of Massachusetts, Amherst, © 200463
Summary: Statistical Analysis
Protocol recommends the efficiency ratioSummation of loads, regression of loads, and mean concentration may be applicable Each method gives somewhat different resultsCheck statistics to be sure they support claims
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
64
University of Massachusetts, Amherst, © 200464
Summary: Case Study
Learn to recognize limitations in test plans, equipment data and documentation deficiencies, and problems with the statistical analysisMaking decisions on the usability of the dataSharing the data among TARP states and others
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
65
University of Massachusetts, Amherst, © 200465
Module II: Retrospective
Sampling designPlanning, contingency planning, and flexibility
Statistical analysisMethods show different results --- use caution interpreting results
Data adequacy: case studyLearning to deal with sampling and data deficiencies
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
66
University of Massachusetts, Amherst, © 200466
What Have You Accomplished?
Guidance for using the TARP protocolExposure to key issues in a technology demonstration field testKnowledge of TARP stormwater work group and others evaluating technologies
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
67
University of Massachusetts, Amherst, © 200467
Questions and Answers (3 of 3)
No associated notes. For more information on TARP and to download a copy of the TARP Protocol for Stormwater Best Management Practice Demonstrations, please visit: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/.
68
University of Massachusetts, Amherst, © 200468
Evaluating Stormwater Technology Performance
TARP: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/
Links to Additional Resources
For more information on TARP go to: http://www.dep.state.pa.us/dep/deputate/pollprev/techservices/tarp/For links to additional resources for this presentation go to: http://www.clu-in.org/conf/tarp/stormwater/resource.cfm