Embed Size (px)
Citation preview
United States Science Advisory Board EPA-SAB-EC-00-011Environmental Protection (1400A) August 2000Agency Washington, DC www.epa.gov/sab
The Science Advisory Board (SAB) of the U.S. Environmental Protection Agency is a body ofindependent experts who provide advice to the EPA Administrator on scientific and engi-neering issues. The SAB was established in its present form by the Congress in 1978. The
SAB’s approximately 100 members and more than 300 consultants include scientists, engineers, andother specialists drawn from a broad range of disciplines—physics, chemistry, biology, mathematics,engineering, ecology, economics, medicine, and other fields. Members are appointed by the Administrator to two-year terms. The SAB meets in public session, and its committees and reviewpanels are designed to include a diverse and technically balanced range of views, as required by theFederal Advisory Committee Act (FACA).
The Board’s principal mission is to review the quality and relevance of the scientific informationbeing used to support Agency decisions, review research programs and strategies, and provide broadstrategic advice on scientific and technological matters. In addition, the Board occasionally conductsspecial studies at the request of the Administrator to examine comprehensive issues, such as antici-pating future environmental problems and developing new approaches to analyze and compare risksto human health and the environment.
Toward Integrated Environmental Decision-making
U.S. Environmental Protection Agency
Science Advisory Board
Integrated Risk Project
August 2000
ii
UNITED STATES ENVIRONMENTAL PROTECTION AGENCYWASHINGTON, DC 20460
August 18, 2000
OFFICE OFTHE ADMINISTRATOR
SCIENCE ADVISORY BOARD
EPA-SAB-EC-00-011
Honorable Carol M. Browner Administrator U.S. Environmental Protection Agency Washington, DC 20460
Subject: Final SAB Report, Toward Integrated Environmental Decision-making
Dear Ms. Browner:
It is our pleasure to send you the accompanying final report, Toward Integrated EnvironmentalDecision-making, from the Science Advisory Board’s (SAB) Integrated Risk Project. This report marks thecompletion of the most complex and challenging activity ever undertaken by the Board, involving morethan 50 natural scientists, economists, and social scientists working together over more than three years.The report has been subjected to a public peer review process in accordance with the Agency’s PeerReview Policy, and we acknowledge the inputs and insights of the peer reviewers.
Originally conceived by some as a simple “updating” of the SAB’s Reducing Risk report, the IntegratedRisk Project (IRP) challenged the SAB to extend far beyond that 1990 product. Reducing Risk effectivelylegitimized the notion that larger environmental risks can be distinguished from smaller risks on the basisof scientific criteria. Some advocates of the IRP on Capitol Hill and in the Agency envisioned that the SABwould go further in today’s report and develop a ranked list of environmental risks, based not only onconsensus scientific insights but on consensus social values as well! Suffice it to say that this grand viewwas neither the SAB’s intention nor its product.
Instead, the SAB has developed a conceptual framework or vision for “the next step” in environmentalprotection in this country. The first step was taken in the 1970s when there was broad public consensusregarding environmental problems and their sources. The second step was taken when the Agency adoptedthe risk assessment/risk management paradigm in the 1980s to support risk-based decision-making.Building on Agency actions and several external reports generated in the 1990s, the SAB now proposes a
Framework for Integrated Environmental Decision-making that describes how a broader array of consid-erations and participants can and should affect environmental decision-making in the future.
One of the principle features of “the next step” is the involvement of a wider range of people — andtheir perspectives/values — in the decision-making process. The Framework also emphasizes use of thebest science (both natural and social sciences) to assess cumulative, aggregate risks; to consider a broaderrange of options for managing or preventing risks; to make clear the role of societal (public) values indeciding what to protect; to clarify the trade-offs (including costs and benefits) associated with choosingsome management scenarios and not others; and to evaluate progress toward desired environmentaloutcomes.
To be sure, the Framework is not a “turn-key” operation that needs only to be implemented. In fact,much work remains to be done. The IRP took the SAB, and the Agency, into unfamiliar territory, involv-ing research literatures in behavioral decision science and decision theory with which we have had limitedpast experience. The effort has emphasized the importance of expanding the scope of expertise—both inthe SAB and the Agency—into these important domains. The project also has emphasized the importanceof adopting an interdisciplinary approach that combines deep understanding of environmental science withtheory and empirical methods in behavioral and decision science.
If the journey toward more integrated environmental decision-making is to be successful, the Agencywill need to undertake a significantly expanded effort in developing improved tools and guidance that havebeen vetted with real problems. Specific focused research is needed on problems that range from improvingmethods for informed synthesis and elicitation of public environmental values, to tools and procedures thatsupport improve characterization and treatment of uncertainty, reasoned science-based deliberativeprocesses, and the evaluation of multi-dimensional risks.
We note that the seeds of integrated environmental decision-making as described in the report alreadyhave been planted in the Agency. New technologies and systems are being instituted at EPA that can worktogether to nourish and encourage those seeds to sprout and grow. Prototypic tools have been devised as apart of this SAB exercise that should be explored and, as appropriate, further developed through researchand applications to real problems in environmental decision-making. While the report charts a valuablefuture direction for the Agency, a number of specific paths can lead to the desired destination. There can beno substitute for a thoughtful strategy of experimentation worked out in the specific settings of differentenvironmental problems.
As noted in the final section of the report, “In some ways, the Board has been true to the old adage thatone’s goal should exceed one’s grasp. That is, the goal of this project — to articulate a complete and rationalmethod for including all aspects of integrated environmental decision-making in a single process — hasexceeded the SAB’s grasp. In fact, it is likely that that goal will never be reached to everyone’s satisfaction.
iii
And yet, the Framework that was developed during the course of the project, clearly points to the direc-tion in which “the next step” of environmental decision-making should go. The efforts of the individualSubcommittees can be examined for further insights about how the Agency might — or might not — make additional progress in that direction. In any event, there is enough direction and more than enoughchallenge in this report to keep the Agency, and others interested in the next step of environmental deci-sion-making, productively active for some time to come.” The challenges of improving and better integrat-ing environmental decision-making are considerable, but the end result should be worth the effort.
We look forward to discussing this report and your reaction to it at an upcoming SAB ExecutiveCommittee meeting.
Sincerely,
Dr. M. Granger Morgan, Acting Chair Dr. Genevieve M. Matanoski, ChairScience Advisory Board Integrated Risk Project Steering Committee
Science Advisory Board
iv
This report has been written as part of the activities of the Science Advisory Board, a public advisorygroup providing extramural scientific information and advice to the Administrator and otherofficials of the Environmental Protection Agency. The Board is structured to provide balanced,
expert assessment of scientific matters related to problems facing the Agency. This report has not beenreviewed for approval by the Agency and, hence, the contents of this report do not necessarily representthe views and policies of the Environmental Protection Agency, nor of other agencies in the ExecutiveBranch of the Federal government, nor does mention of trade names or commercial products constitute arecommendation for use. The SAB members and consultants who participated in the activities that resultedin this report did so as individuals, rather than as representatives of their employing organizations.
This particular project was conducted at the request of the EPA Administrator and addresses a broaderrange of issues and concerns than most SAB reports. Consequently, many of the recommendations in thisreport have more of a policy orientation than is usually the case.
Distribution and Availability: This Science Advisory Board report is provided to the EPA Administrator,senior Agency management, appropriate program staff, interested members of the public, and is posted onthe SAB website (www.epa.gov/sab). Information on its availability is also provided in the SAB’s monthlynewsletter (Happenings at the Science Advisory Board). Additional copies and further information areavailable from the SAB staff.
v
NOTICE
vi
STEERING COMMITTEE
Dr. Genevieve Matanoski (Chair) School of Hygiene and Public Health The Johns Hopkins University Baltimore, MD
Dr. Joan M. Daisey1
Lawrence Berkeley National Laboratory Berkeley, CA
Dr. Paul Deisler Austin, TX
Dr. Mark A. Harwell University of Miami Miami, FL
Dr. Wayne Kachel MELE Associates Brook Air Force Base, TX
Dr. Alan Maki Exxon Company, USA Houston, TX
Dr. Paul R. Portney Resources for the Future Washington, DC
Dr. Milton Russell Joint Institute for Energy & Environmentand University of Tennessee Knoxville, TN
Dr. Ellen K. Silbergeld University of Maryland Baltimore, MD
Dr. Paul H. Templet Louisiana State University Baton Rouge, LA
Dr. Valerie Thomas Princeton University Princeton, NJ
Dr. Bernard Weiss University of Rochester Medical Center Rochester, NY
Dr. Marcia Williams Putman, Hayes, & Bartlett, Inc. Los Angeles, CA
Dr. Terry F. Yosie2
Ruder and Finn, Inc. Washington, DC
Dr. Terry F. Young Environmental Defense Oakland, CA
Science Advisory Board Staff
Mr. Thomas O. MillerDesignated Federal Officer EPA Science Advisory Board Washington, DC
Ms. Stephanie Sanzone Designated Federal Officer EPA Science Advisory Board Washington, DC
Ms. Wanda Fields Management Assistant EPA Science Advisory Board Washington, DC
Mr. Thomas Super EPA Science Advisory Board Washington, DC
Dr. Donald G. Barnes Director EPA Science Advisory Board Washington, DC
ECOLOGICAL RISKS SUBCOMMITTEE
Dr. Mark Harwell (Chair) University of Miami Miami, FL
Dr. William Adams Kennecott Utah Copper Corp Magna, UT
Dr. Steven M. Bartell3
SENES Oak Ridge, Inc Oak Ridge, TN
Dr. Kenneth W. Cummins4
South Florida Water Management District Sanibel, FL
Dr. Virginia Dale Oak Ridge National Laboratory Oak Ridge, TN
Dr. Carol Johnston University of Minnesota Duluth, MN
Dr. Frederick K. Pfaender University of North Carolina Chapel Hill, NC
Dr. William H. Smith Yale University New Haven, CT
Dr. Terry F. Young Environmental Defense Oakland, CA
vii
INTEGRATED RISK PROJECT COMMITTEES
1 Deceased.2 Now with Chemical Manufacturers
Association.3 Now with Cadmus Group, TN.4 Now with Humboldt State Univ., CA.
Science Advisory Board Staff
Ms. Stephanie Sanzone Designated Federal Officer EPA Science Advisory Board Washington, DC
Ms. Wanda R. Fields Management Assistant EPA Science Advisory Board Washington, DC
HUMAN EXPOSURE ANDHEALTH SUBCOMMITTEE
Dr. Joan Daisey5 (Co-Chair) Lawrence Berkeley Laboratory Berkeley, CA
Dr. Bernie Weiss (Co-Chair) University of Rochester Medical Center Rochester, NY
Dr. Stephen Ayres School of Medicine VirginiaCommonwealth University Richmond, VA
Dr. Paul Bailey Mobil Business Resources Corp. Paulsboro, NJ
Dr. George Daston Procter and Gamble Co. Ross, OH
Dr. Curtis Klaussen University of KS Medical Center Kansas City, KS
Dr. Paul Lioy Rutgers University Piscataway, NJ
Dr. William Pease Environmental Defense Oakland, CA
Dr. Henry Pitot McArdle Laboratory for Cancer Research University of Wisconsin Madison, WI
Dr. Jonathan Samet Department of Epidemiology Johns Hopkins University Baltimore, MD
Dr. Valerie Thomas Princeton University Princeton, NJ
Dr. Lauren Zeise California EPA Berkeley, CA
Science Advisory Board Staff
Mr. Samuel Rondberg Designated Federal Officer EPA Science Advisory Board Washington, DC
Ms. Mary L. Winston Management Assistant EPA Science Advisory Board Washington, DC
ECONOMIC ANALYSIS SUBCOMMITTEE
Dr. Paul R. Portney (Chair) Resources for the Future Washington, DC
Dr. Nancy E. Bockstael Dept. of Agricultural and ResourceEconomics University of Maryland College Park, MD
Dr. Trudy Ann Cameron Dept. of Economics University of California Los Angeles, CA
Dr. Maureen L. Cropper The World Bank Washington, DC
Dr. A. Myrick Freeman Dept. of Economics Bowdoin College Brunswick, ME
Dr. Charles D. Kolstad Dept. of Economics University of California Santa Barbara, CA
Dr. Robert Repetto6
World Resources Institute Washington, DC
Dr. Robert Stavins JFK School of Government Harvard University Cambridge, MA
Dr. Thomas H. Tietenberg Dept. of Economics Colby College , Waterville, ME
Dr. W. Kip Viscusi Harvard Law School Cambridge, MA
Science Advisory Board Staff
Mr. Thomas Miller Designated Federal Officer EPA Science Advisory Board Washington, DC
Ms. Diana L. Pozun Management Assistant EPA Science Advisory Board Washington, DC
VALUATION SUBCOMMITTEE
Dr. Alan W. Maki (Co-Chair) Exxon Company, USA Houston, TX
Dr. Milton Russell (Co-Chair) Joint Institute for Energy & Environmentand University of Tennessee Knoxville, TN
Dr. Stephen M. Ayres Medical College of Virginia Virginia Commonwealth University Richmond, VA
viii
5 Deceased.6 Now with Stratus Consulting, Inc.,
Boulder, CO.
Dr. Caron Chess Center for EnvironmentalCommunications Rutgers University New Brunswick, NJ
Dr. Virginia Dale Oak Ridge National Laboratory Oak Ridge, TN
Dr. William H. Desvousges Triangle Economic Research Durham, NC
Dr. Thomas Dietz Dept. of Sociology and Anthropology George Mason University Fairfax, VA
Dr. A. Myrick Freeman Dept. of Economics Bowdoin College Brunswick, ME
Dr. Mark A. Harwell Center for Marine and EnvironmentalAssessment University of Miami Miami, FL
Professor Jerry A. Hausman Dept. of Economics Massachusetts Institute of Technology Cambridge, MA
Dr. Douglas E. MacLean Dept. of Philosophy University of Maryland Baltimore, MD
Dr. John W. Payne Fuqua School of Business Duke University Durham, NC
Dr. Edella Schlager School of Public Administration andPolicy University of Arizona Tucson, AZ
Dr. Margaret Shannon Center for Envir. Policy andAdministration Syracuse University Syracuse, NY
Dr. Paul Templet Institute for Environmental Studies Louisiana State University Baton Rouge, LA
Dr. Terry F. Young Environmental Defense Oakland, CA
Dr. James Wilson Dept. of Resource Economics and Policy University of Maine Orono, ME
Science Advisory Board Staff
Mr. Thomas Miller Designated Federal Officer EPA Science Advisory Board Washington, DC
Ms. Diana Pozun Management Assistant EPA Science Advisory Board Washington, DC
RISK REDUCTION OPTIONSSUBCOMMITTEE
Dr. Wayne M. Kachel (Co- Chair) MELE Associates Brooks AFB, TX
Ms. Marcia Williams (Co-Chair) Putman, Hayes & Bartlett, Inc Los Angeles, CA
Dr. Ann Bostrom School of Public Policy Georgia Institute of Technology Atlanta, GA
Ms. Dorothy Bowers Merck and Co., Inc Whitehouse Station, NJ
Mr. Robert Frantz General Electric Company Cincinnati, OH
Dr. Nina Bergan French SKY+ Oakland, CA
Ms. Mary A. Gade Illinois EPA Springfield, IL
Mr. Bradford Gentry Center for Environmental Law & Policy Yale University New Haven, CT
Dr. Ricardo R. Gonzalez Dept. of Radiological Sciences School ofMedicine University of Puerto Rico San Juan, PR
Dr. Michael Greenberg Dept. of Urban Studies & Community Health State University of New Jersey Rutgers, NJ
Dr. Linda E. Greer Natural Resources Defense Council Washington, DC
Dr. Hilary I. Inyang Center for Envir. Engineering & SciencesTechnologies University of Massachusetts Lowell, MA
Dr. Charles D. Kolstad Dept. of Economics University of California Santa Barbara, CA
Mr. Terrence J. McManus Intel Corporation Chandler, AZ
Dr. Wm. Randall Seeker Energy & Environmental ResearchCorporation Irvine, CA
Science Advisory Board Staff
Ms. Kathleen White Conway Designated Federal Officer EPA Science Advisory Board Washington, DC
Ms. Dorothy M. Clark Management Assistant EPA Science Advisory Board Washington, DC
ix
IRP PEER REVIEW COMMITTEE
Dr. M. Granger Morgan (Chair) Dept. of Engineering and Public Policy Carnegie Mellon University Pittsburgh, PA
Dr. Henry A. Anderson Wisconsin Bureau of Public Health Madison, WI
Dr. William E. Bishop Procter and Gamble Co. Cincinnati, OH
Dr. Donald F. Boesch Univ. of Maryland Center forEnvironmental Science Cambridge, MD
Dr. Richard J. Bull MoBull Consulting, Inc. Kennewich, WA
Dr. Terry Davies Resources for the Future Washington, DC
Dr. John D. Graham Harvard Center for Risk Analysis Harvard University Boston, MA
Dr. Catherine Kling Dept. of Economics Iowa State University Ames, IA
Dr. Debra Knopman Progressive Policy Institute Washington, DC
Dr. Morton Lippmann New York University School ofMedicine Tuxedo, NY
Dr. Warner North NorthWorks Inc. Belmont, CA
Dr. Richard Revesz New York University School of Law New York, NY
Dr. Bruce Tonn Oak Ridge National Laboratory Oak Ridge, TN
Science Advisory Board Staff
Dr. John R. Fowle, III Deputy Staff Director and DesignatedFederal Officer EPA Science Advisory Board Washington, DC
Ms. Wanda R. Fields Management Assistant EPA Science Advisory Board Washington, DC
INTEGRATED RISK PROJECT PEER REVIEWERS
x
1. Integrated Environmental Decision-making 1
1.1 Environmental Integration: The Next Step 11.2 The Call for Integrated Decision-Making 21.3 Signs of Progress 41.4 Scope of the Project 5
2. A Proposed Conceptual Framework 8
2.1 Overview of the Framework 82.2 Problem Formulation (Phase I) 112.3 Analysis and Decision-Making (Phase II) 122.4 Implementation and Performance
Evaluation (Phase III) 142.5 Building on Previous Frameworks 15
2.5.1 Integrated Aspects of Single Stressors/Risks 15
2.5.2 Integrated Aspects of MultipleStressors/Risks 16
2.5.3 Considering Environmental Values 17
3. Working Toward Implementation 18
3.1 Comparative Risk for Problem Formulation (Phase I) 183.1.1 What We Have 183.1.2 What We Need 20
3.2 Risk Assessment for Analysis and Decision-Making 223.2.1 What We Have 223.2.2 What We Need 22
3.3 The Analysis of Benefits and Costs in Decision-making (Phases I and II) 233.3.1 What We Have 233.3.2 What We Need 24
3.4 Forming, Eliciting, and Considering Public Values (Phases I and II) 253.4.1 What We Have 253.4.2 What We Need 27
3.5 Options Analysis (Phases I and II) 283.5.1 What We Have 283.5.2 What We Need 30
3.6 Performance Measures 313.6.1 What We Have 313.6.2 What We Need 35
4. Recommendations to the Agency 37
5. Lessons Learned 43
6. References Cited 45
xi
TABLE OF CONTENTS
xii
1.1 Environmental Integration: The Next Step
Environmental protection in the United Stateshas progressed through at least two major steps.The first step characterized the early days of the1970s when the Environmental Protection Agency(Agency) enjoyed both a public consensus andCongressional mandates about what needed to bedone; e.g., clean up the air and make the watersfishable and swimable. A second, more sophisti-cated step was taken in the mid-1970s and honedin the 1980s and 1990s when the “risk assess-ment/risk management (RA/RM) paradigm,” first proposed by the National Research Council(NRC, 1983), was adopted and implemented bythe Agency. The RA/RM approach has been especially useful in those instances in which thedamage and/or danger is not immediately evidentand some data and analysis are necessary aroundwhich to form a consensus for action.
In this document, the Science Advisory Board(SAB) describes the outlines of “the next step” inenvironmental decision-making. The SAB presentshere a broad conceptual way of thinking aboutenvironmental problems that responds to criti-cisms of the fragmented approaches that dominatecurrent national strategies for protecting humanhealth and the environment. At the core of thisconcept is integrated thinking about complexenvironmental problems, integrated resources andanalyses to address the problems as they occur inthe real world, and integrated input from thepublic and interested and affected parties.Integrated decision-making is a natural next step
from both prior SAB and Agency projects thatconsidered individual risks in relation to eachother. In addition, it builds upon the significantsuccess that has grown from many of the reinven-tion and reorganization activities in the Agencyduring the 1990s.
As a result of conducting the Integrated RiskProject (IRP), the SAB is proposing a conceptualFramework for Integrated EnvironmentalDecision-making (IED) to guide the Agency inthe continuing evolution of environmental deci-sion-making. In addition, IRP subcommitteeshave described a number of tools and approachesthat, with further development and application,may provide a means of developing some of theinformation and analyses suggested by theFramework. However, the SAB is not presenting a “turn-key” procedure that will solve all of theproblems associated with the current decision-making environment. Rather, the report points tonew directions that will broaden the decision-making process in terms of both the factors thatshould be considered and the parties who shouldbe involved in making and evaluating environmen-tal decisions. It is left to the Agency and others to apply the conceptual Framework to specificproblem sets, and thus to develop and implementpractical strategies and operations that will realizethe potential benefits of integrated environmentalthinking.
The evolution in environmental decision-making called for in this report is not meant todetract from past environmental accomplishmentsnor to replace existing regulatory processes andrequirements. The concept of an integrated
1
1. INTEGRATED ENVIRONMENTAL DECISION-MAKING
Framework is intended to be further developed,tested, and used in conjunction with existing envi-ronmental management approaches to strengthenthe risk reduction programs now being imple-mented and improve environmental managementin the future.
1.2 The Call for Integrated Decision-Making
Environmental decision-makers currentlydraw upon an eclectic mixture of data, tools, andanalyses to inform their decisions: ecological andhuman health risk assessment; benefit-cost andcost-effectiveness models; expanded risk commu-nication and public participation; and measures formonitoring the results of the decisions themselves.Although these contributions are essential inputsfor decision-making, they have often been appliedunevenly and to relatively narrow issues. In thearea of human health risks, for example, assess-ments often have been framed around single stressors or classes of stressors in relatively specificexposure situations. The deficiencies of suchhighly focused assessments are increasingly appar-ent; they sidestep the complexities, interrelation-ships, and subtleties of environmental problems as they actually confront society. Thus, a numberof recent studies have urged the Agency to beginto address environmental issues in a more inte-grated way (e.g., NAPA, 1995; Presidential andCongressional Commission on Risk Assessmentand Risk Management, 1997).
Much of the fragmentation in EPA’s approachto the control of environmental problems has itsroots in the statutory framework that guides thework of the Agency. From its formation in 1970,the EPA has been given responsibility for imple-menting a number of environmental statutes thatmandate targeted actions to control specific pollu-tants in specific media (cf., Clean Air Act languageregarding particulates in air) or specific routes ofexposure (cf., Safe Drinking Water Act language
regarding priority pollutants in drinking water).The focus on assessing and controlling chemicalcontaminants pollutant by pollutant in singlemedia has resulted in a regulatory system that is neither systematic nor comprehensive.Nonetheless, the system has been largely success-ful in controlling many of the targeted pollutantsand has provided a strong national underpinningfor an effective environmental protection programcomprised of federal, state, and local controls.
Despite these successes, there is a growingconsensus, both within and outside the Agency,that a more integrated approach to environmentalmanagement is needed. Prioritizing and managingrisks pollutant by pollutant and medium bymedium can be inefficient both in reducing themajor burdens of environmental impacts onhuman health and ecosystems and in allocatingsociety’s resources in the face of multiple demandson limited budgets. Further, that piecemealapproach ignores the integrated manner in whichhazards usually occur. Of still greater concern isthe possibility that such a fragmented approachmay cause us to overlook significant environmen-tal problems while we busy ourselves withcomparatively minor improvements thatcontribute little to the overall protection of humanhealth and ecosystems.
In some instances, current statutes and regula-tions prevent the Agency from considering allrelevant risk, benefit-cost, or other information.Further, as pointed out in a recent NAPA report,there are “no established criteria that the Agencymight use to set priorities that cut across statutorylines” (NAPA, 1995). The SAB views the issue ofstatutory integration as a policy discussion andoutside the bounds of the present study. However,the Board believes that even within the currentstatutory framework, there are numerous oppor-tunities for a more holistic assessment of risks andrisk management options. There are also opportu-nities for more inclusive decision-makingapproaches, both in terms of participation in the
2
Prioritizing andmanaging risks
pollutant bypollutant andmedium by
medium can beinefficient both
in reducing the majorburdens of
environmentalimpacts on
human healthand
ecosystemsand in
allocatingsociety’s
resources…
process by groups with diverse perspectives on theissues and in terms of the focus of such reviews,e.g., sectors, communities, industries, ecosystems,and special groups within the population. In addi-tion, there is a growing sense that a broader rangeof factors should influence a given environmentaldecision and that such factors should be moresystematically considered and articulated. Thesefactors include economic consequences (oftenexpressed in terms of benefits and costs) andconcerns of the public as expressed in their underlying values.
Answers now are needed to questions that werenot asked when the predominantly pollutant-by-pollutant regulatory system was first established.These new questions focus on, for example, cumulative risks to sub-populations or particulargeographic areas, the role of public involvement inenvironmental decision-making, and the balancebetween present and future consequences of deci-sions. The proposed Framework for making inte-grated environmental decisions should help directresearch and thinking about these issues in a waythat will generate answers to such questions.
New questions require new approaches to technical analyses and public policy-making.
What are the most serious environmental risks facing children, or the elderly, across the nation?
What are the factors that pose an aggregate set of risks to everyone living in a particular geographic area?
What are the interrelated risks to human and ecological health associated with a particular industrialsector?
Of all the risks affecting a geographic area or subset of the population, which are most serious, andwhich are we most capable — economically, technologically, and politically — of limiting?
How can we achieve the right balance between protecting the health and the welfare of present andfuture generations and assure ecological security for the long term?
What combinations of risk management tools — regulatory and non-regulatory, technological and non- technological — can be used in concert to achieve environmental goals in particular communities or ecosystems?
How can we measure our progress in solving integrated environmental problems and reaching our long-term goals: protection of ecological integrity, human health, and quality of life?
How can worthy, but competing, individual and group goals be accommodated?
What is the distribution across the population of benefits and costs associated with environmental risksand their possible management?
What is the appropriate role for public involvement in the assessment and management ofenvironmental risks?
3
…even withinthe currentstatutory
framework,there arenumerous
opportunitiesfor a more
holisticassessment ofrisks and riskmanagement
options.
1.3 Signs of Progress
In a 1983 publication entitled Risk Assessmentin the Federal Government: Managing the Process(NRC, 1983), commonly referred to as the “RedBook,” an NRC panel laid out the elements ofRA/RM using terminology that came to be thestandard. These concepts were adopted by theAgency in 1984 and have formed the basis for much of the Agency’s action to this day. Insummary, the NRC panel described the four stepsof risk assessment as hazard identification, dose-response assessment, exposure assessment, andrisk characterization, where the latter was definedas “the estimated incidence of the adverse effect ina given population.” In addition, the panel stressedthe scientific basis for risk assessment and the needfor both quantitative and qualitative expressions ofrisk. Risk management was viewed as “a decision-making process that entails consideration of politi-cal, social, economic, and engineering informationwith risk-related information to develop, analyze,and compare regulatory options and to select theappropriate regulatory response.”
The Red Book was extremely useful in articu-lating the risk assessment process and its relation-ship to risk management. The paradigm wasexpressed, however, in terms of single agents andsingle health effects in humans. Since that time, theAgency has developed risk assessment guidelinesto address a number of human health endpoints(i.e., cancer, reproductive and developmental toxi-city, and neurotoxicity), as well as guidelines forexposure assessment. In addition, the Agency hastaken steps to consider more integrated exposurescenarios; e.g., multi-route exposures to a)mixtures of chemical agents associated withSuperfund sites (U.S. EPA, 1989), b) a fuller rangeof combustion emissions (EPA, 1990; 1993a), andc) multiple pesticides, in response to the FoodQuality Protection Act.
The Agency also has made significant progressin adapting the Red Book paradigm to ecological
risk assessment. In 1992, the Agency released itsFramework for Ecological Risk Assessment (U.S.EPA, 1992) which used the term “characterizationof ecological effects” to include both hazard identi-fication and exposure assessment. The Frameworkalso added a) an explicit Problem Formulationphase prior to the analysis of exposure and effectsin order to emphasize the importance of articulat-ing the problem and b) a plan for analyzing andcharacterizing risk prior to conducting specific risk analyses. The resulting framework containedthree phases: Problem Formulation, Analysis, andRisk Characterization. An expanded discussion of ecological risk assessment principles andapproaches was subsequently provided by theAgency in final Guidelines for Ecological RiskAssessment (U.S. EPA, 1998). The guidelines notethat “although ecological risk assessments providecritical information to risk managers, they are only part of the environmental decision-makingprocess” (U.S. EPA, 1998). In addition to assessingthe relationship between a particular stressor and aparticular effect, the ecorisk guidelines set the stagefor considering multiple effects (including cascad-ing effects) associated with a single stressor orsource, as well as multiple causes of an observedeffect or change in ecological condition. TheAgency has already applied the ecological riskassessment paradigm to five watershed cases,which included developing a conceptual model for each that relates various stressors and effects(for discussion, see SAB, 1997).
Four additional Agency developments, thoughnot reviewed by the SAB for scientific content,merit mention here: a) guidance and support forcomparative risk analysis; b) extra-statutoryapproaches to environmental protection; c) guid-ance on planning and scoping for cumulative riskassessment; and d) inclusion of interested andaffected parties in decision-making.
First, Comparative Risk Analysis (CRA) hasbeen defined by the Agency as “both an analyticalprocess and a set of methods used to systemati-
4
cally measure, compare, and rank environmentalproblems” (U.S. EPA, 1993b). The Agency, in itsUnfinished Business report (U.S. EPA, 1987), andthe SAB, in Reducing Risk (SAB, 1990), engagedin comparative risk analyses. In its 1990 report,the Board concluded that it was possible, on ascientific basis, to distinguish between large risksand small risks using a set of technical criteria. Inthe years that followed, the Agency promoted thewide use of CRA as a process for setting prioritiesby considering multiple stressors and multipletypes of risks within specific regions, such as citiesor states, or for the nation as a whole.Comparative risk analysis is intended principallyas a policy-development and broad resource-allo-cation tool. In contrast to Unfinished Businessand Reducing Risk, however, state and local-levelcomparative risk analyses have highlighted therole of the public and stakeholder groups, in addition to the scientific/technical community, indefining risk priorities. Support for broader inclusion of public values in decision-making is a theme that has been echoed by a number ofrecent reports (e.g., NRC, 1996; Presidential/Congressional Commission on Risk Assessmentand Risk Management, 1997).
Second, during the 1990s, the Agency experimented with a number of approaches to re-inventing environmental protection, includinggreater use of community-based decision-making,voluntary cross-media emissions reductions, inte-grated environmental agreements with states, andvoluntary regulatory reform efforts with an arrayof stakeholders (U.S. EPA, 1999).
Third, the Agency has issued cumulative riskguidance that directs program offices to “considera broader scope that integrates multiple sources,effects, pathways, stressors, and populations forcumulative risk analyses in all cases for which rele-vant data are available” (U.S. EPA, 1997a). Thecumulative risk guidance also notes that on-goingAgency efforts to involve stakeholders “willprovide the solid basis for engaging interested and
affected parties in risk assessment and riskmanagement issues.”
Fourth, the Agency has made progress inrecent years in including interested and affectedparties in the decision-making process. A recentdescription of one of these processes— regulatorynegotiation in the microbial disinfection and disinfection byproducts rules—can be found inPontius (1999).
Although the SAB has not reviewed the roleand adequacy of the science being used in theplanning and evaluation of these activities, theseendeavors signal movement in the Agency towardmore integrated and inclusive methods of environ-mental decision-making. These experiments,however, do not yet represent the mainstream ofEPA’s efforts.
1.4 Scope of the Project
It is in this atmosphere that the SAB under-took the task of revisiting its 1990 report,Reducing Risk, to update and extend the thinkingabout how science can best inform the decision-making process. The Charge to the SAB from theAgency included requests to update the risk rank-ings in Reducing Risk using explicit scientificcriteria and the judgments of SAB panel members;identify risk reduction opportunities and strate-gies; identify uncertainties and data quality issuesassociated with the risk rankings; assess costs andbenefits of risk reduction options; and propose anew framework for assessing the value of naturalresources to society.
The initial charge also included a request that the SAB explore techniques and criteria foridentifying emerging risks. However, the SABconcluded that its report, Beyond the Horizon:Using Foresight to Protect the EnvironmentalFuture (SAB, 1995) provided criteria and sugges-tions germane to this charge question and so didnot elaborate further on future risks as part of theproject.
5
…the SABconcluded that
it could bestassist the
Agency not bydeveloping listsof risk prioritieswith associatedrisk reductionoptions, but byinvestigatingapproaches
that the Agencymight use fordeveloping
such priorities.
After careful consideration of the Charge anddiscussion with Deputy Administrator FredHansen, the SAB concluded that it could bestassist the Agency not by developing lists of riskpriorities with associated risk reduction options,but by investigating approaches that the Agencymight use for developing such priorities. In addi-tion, the SAB hoped to place the various technicalanalyses within the broader decision-makingcontext. Subcommittee Charges
The project, known as the Integrated RiskProject (IRP), was guided by a Steering Com-mittee (SC) and five specialized Subcommitteeswho began work in Fiscal Year 1996. The Sub-committees and their respective charges were asfollows:
a) The Steering Committee (SC), chaired by Dr. Genevieve Matanoski, set the overall direc-tion for the project by defining scope andtimetables. The SC met periodically over thecourse of the project to: (1) assess the progressand direction of the subcommittees; (2) ensurethat the results could be integrated into acomprehensive decision process for identifyingcurrent and future environmental risks; and (3)review options for reducing risks in a holisticcontext. The SC’s efforts were designed specifi-cally to illustrate the relationship among thevarious factors influencing risk managementdecisions; e.g., technical assessment of the risksand risk reduction options, economic consider-ations, equity considerations, and so forth.
b) The Ecological Risks Subcommittee (ERS),chaired by Dr. Mark Harwell, was chargedwith assessing and ranking risks to ecosystemsat the national scale, and suggesting ways inwhich the risk ranking methodology could beapplied at smaller geographical scales; e.g.,regional, state, or local. The group was alsoasked to explore commonalities and differenceswith the Human Exposure and Health
Subcommittee (HEHS) methodology, with theaim of integrating the two ranking schemes.7
c) The Human Exposure and Health Sub-committee (HEHS), co-chaired by Drs. JoanDaisey and Bernard Weiss, was charged withdeveloping a methodology for assessing andranking risks to human health, consideringways in which an integrated risk ranking couldbe produced that includes both cancer andnon-cancer risks, and test the methodology fora limited set of environmentally mediatedhealth issues. The Subcommittee was alsoasked to explore commonalities and differenceswith the ERS methodology.8
d) The Risk Reduction Options Subcommittee(RROS), co-chaired by Dr. Wayne Kachel andMs. Marcia Williams, was charged with devel-oping a methodology for selecting an optimalset of risk reduction options with due regardfor the human health and ecological risks(defined in terms of risks associated with environmental stressors, locations, or expo-sure/transport media). The Subcommitteeillustrated the methodology by applying it to a small set of example problems.9
e) The Economic Analysis Subcommittee(EAS), chaired by Dr. Paul Portney, wascharged with assessing current methods forestimating costs and benefits associated bothwith the implementation of risk reduction
6
7 The work of the ERS will result in a separate SAB reportfrom the Board’s Ecological Processes and EffectsCommittee.
8 The work of the HEHS is summarized in a Working Paper.In addition, the Agency is exploring further development ofthe prototypic Internet-based tool created in the course of theproject.
9 The work of the RROS will result in a separate SAB reportfrom the Board’s Environmental Engineering Committee.
strategies and with allowing risks to go unad-dressed. The EAS was also asked to considerthose aspects of the “net benefits” equationthat cannot easily be monetized.10
f) The Valuation Subcommittee (VS), co-chaired by Drs. Alan Maki and Milton Russell,was charged to consider a new framework forassessing the value of ecosystems to humans,including ecological services and environmen-tally mediated health and quality of life values.The work of the VS was intended to provide awider societal view of risk and risk reductionoptions than that derived from science-basedrisk assessments and current methods ofeconomic analysis.11
Over the course of the project, the SC andsubcommittees held over 25 public meetings and teleconference calls. Although most of these
meetings were held in Washington, D.C., publicsessions were also held in Berkeley and SanFrancisco, CA; Atlanta, GA; New Orleans, LA;and Baltimore, MD. While the bulk of theSubcommittee work was completed in FY 1998,the SC has continued its work of producing asynthesis report, subjecting it to peer review, andresponding to comments from the reviewers.
This report contains several sections. Section 2synthesizes much of the SAB’s deliberations into aconceptual Framework for making integratedenvironmental decisions. Section 3 describes six of the major elements that contribute to theFramework, identifying specifically the informa-tion/tools that are already available and the areasin which additional work is particularly needed.The major recommendations of the SAB are laidout in Section 4. In the closing Section 5, the SABbriefly reviews some of the “lessons learned” inthe course of carrying out the IRP.
7
10 The work of the EAS will result in a separate SABBackgrounder that provides an introduction to benefit-costanalysis, its strengths, and its limitations.
11 The work of the VS is summarized in a Working Paper thatcould serve as input to an SAB/Agency Workshop toexplore the issue further with a broader range of participants.
2.1 Overview of the Framework
The approach to integrated environmentaldecision-making that emerged from the SAB’sdeliberations is captured in the conceptualFramework for Integrated EnvironmentalDecision-making (IED) in Figure 1. The IEDFramework recognizes that risks often are experi-enced simultaneously and are cumulative (i.e.,additive, synergistic, and/or antagonistic); thatefforts to manage one risk may have impacts —positive or negative — on other risks; that benefit-cost scenarios may be affected by the scope of theproblem definition; and that values deliberation isessential to the development of environmentaldecisions.
As illustrated in Figure 1, the Framework laysout a series of straightforward questions. What arethe most important environmental risks? What areour environmental goals? What are the best riskreduction opportunities? How can we achieve ourgoals and objectives? How will we know whetheror not we are meeting our goals? What modifica-tions in our approach are needed to improve envi-ronmental results? Are we meeting our objectives?Is the nature of the problem changing? Findinganswers to these fundamental questions requiresapplication of scientific and technical assessmentand analysis techniques, as well as political, policy,and values-driven choices. By fostering a moreintegrated look at environmental goals and priori-ties, the Framework should help us to makechoices not only about which actions to take, butalso about which actions are not worth taking.The explicit acknowledgment of the trade-offs
required to achieve multiple, often competing,goals is an important part of setting priorities.
The proposed Framework consists of threeinteracting phases:
Phase I—Problem Formulation, which involvespreliminary analysis and comparison of risks,establishment of goals, and preliminary analysis ofrisk reduction options. The “problem” may bedefined in terms of either a single risk (e.g., onestressor, but associated with multiple sources withmultiple routes of exposure) or multiple risks (e.g.,environmental stressors associated with a particulargeographic location, possibly including multiplesources, pathways, types of receptors, and effects).
Phase II—Analysis and Decision-Making, whichincludes in-depth analysis of risks and projectedrisk reduction under possible management scenar-ios and selection of a preferred option or set ofoptions, based on criteria such as feasibility, cost-effectiveness, seriousness of the risks addressed,and equity.
Phase III—Implementation and PerformanceEvaluation, in which preferred management optionsare implemented and the environmental results aremonitored and evaluated, thereby providing impor-tant feedback so that management approaches canbe modified (“adapted”), as necessary.
The various activities that take place during thethree phases of decision-making are discussedfurther below, and in the outputs from the IRPsubcommittees.
8
2. A PROPOSED CONCEPTUAL FRAMEWORK
9
Figure 1. Framework for Integrated Environmental Decision-making
PHASE IPROBLEM FORMULATION
(What are the most important environmental risks?What are our environmental goals?)
Risk Comparisons Goal Setting
Preliminary Options Analysis
PHASE IIANALYSIS AND DECISION-MAKING
(What are the best risk reduction opportunities?How can we achieve our goals and objectives?)
Risk Assessment Screening/Selection
Options Analysis Performance Measures
PHASE IIIIMPLEMENTATION and
PERFORMANCE EVALUATION
(How are we doing?)
Implementation Monitoring and Reporting
Information Evaluation
– Information– Expert Judgment– Values
– Information– Expert Judgment– Values– Legal and – Institutional Milieu
REPORTCARD
(Are we meetingour objectives?)
REPORTCARD
(Is the natureof the problem
changing?)
10
The straightforward structure of the Frameworkdiagram belies the complexities involved in puttingthe concept of integrated decision-making into prac-tice. For example, the Framework envisions itera-tive, analytic/deliberative interactions (NRC, 1996)involving and eliciting values from the Agency, thepublic, and interested and affected parties. There hasbeen an evolution in the thinking and practice at the
Agency about who should be involved in the assess-ment and management of risks, with a recognitionthat public values underlie the selection of specificgoals and risk reduction options. Although it is notalways clear how best to include a broader range ofparticipants in decision-making, the Agency hasbegun to implement various aspects of integrateddecision-making and is “learning by doing.”
Characteristics of Integrated Environmental Decision-making
Transparency All parties should be able to follow and understand how the decision was reached.
Flexibility Integrated decision-making approaches should be applied in a flexible manner depending on thespecific circumstance; i.e., where appropriate, to permit valid short-cuts, to eliminate unnecessaryprocedures, and so to expedite the process of decision-making and implementation.
Dynamic process design The technical analyses required to implement integrated decision-making should be iterative andinterconnected. For example, during the Problem Formulation Phase, problem scoping and definitionand preliminary analysis of options will affect the development of goals, and vice versa. Some iterationis also required between Problem Formulation and Analysis, since preliminary analyses will oftenpoint out missing elements in the problem definition or inconsistencies in goals.
Explicit feedback, interaction, and cooperation Integrated decision-making approaches should involve cooperation and open, continuingcommunication among scientists, managers, members of the public, and others involved in thedifferent phases of the Framework.
The use of information from many sources Integrated decision-making approaches should draw upon concepts and methods originating in manydifferent scientific, technical, and scholarly fields (e.g., physical and biological sciences, public health,environmental engineering, political science, social science, philosophy, and economics), as appropriatefor any given case.
A way of thinking about environmental problems Integrated environmental decision-making is not just a series of methodologies, but rather is a way ofthinking, in a whole and complete way, about any environmental problem in order to maximize theefficient reduction of aggregate risk to populations or ecological systems.
There are also new technical tools available(e.g., improved computer hardware, increasinglypowerful computer models, and greater availabil-ity of geographic information system (GIS) soft-ware) that have improved the Agency’s ability tocollect, analyze, and disseminate information inways that will enhance integrated decision-making. In addition, advances in the socialsciences are providing improved techniques forhelping people develop considered values, for elic-iting and using public values in decision-making,for communicating technical information in waysthat are helpful to non-expert participants, and forunderstanding people’s choices and preferences.
Although integrated decision-making requiresthe involvement of a broad spectrum of partici-pants (e.g., scientists, engineers, economists, deci-sion-makers, and the public), the different groupshave unique roles to play. In other words, theFramework does not imply that “everyone mustbe involved in everything all the time.” For exam-ple, just as scientists are not expected to providethe perspective of the general public, members ofthe general public are not expected to conduct thetechnical analyses of scientists. Decision-makers,after considering the various sorts of information(data, views, and judgments) generated in ProblemFormulation (Phase I) and Analysis and Decision-making (Phase II), must ultimately make the deci-sion. At the same time, it is important that thevarious groups maintain effective communicationwith each other, informing one another of theirperspectives and insights, so that the collectivewisdom of all of the participants is brought tobear on the problem.
2.2 Problem Formulation (Phase I)
Integrated decision-making should begin withProblem Formulation, in which risk assessors, riskmanagers, and interested and affected parties seekagreement through extensive dialogue and discus-sion on what analytical and deliberative steps need
to be taken by whom, by when, and why — if nothow. This initial Phase includes three related tasks:
a) Risk Comparisons, in which sets of risks—including risks to ecological systems, humanhealth, and/or quality of life—are ranked orrated, and a set of risks selected for detailedconsideration in the second phase;
b) Goal Setting, in which the participants agreeon goals relating to the risks of concern andmeasures that will be used to evaluate progresstowards those goals; and
c) Preliminary Options Analysis, in which aninitial range of risk reduction options is identi-fied and considered in terms of the estimatedtotal reduction of risk and likely benefit-costof each.
11
Risks to Quality of Life
Although “quality of life (QOL) risks” is animprecise term that means different things todifferent people, examples of risks oftenincluded in this category are aesthetic,economic, and equity impacts, as well aseffects on peace of mind, cultural orcommunity identity, and recreationalopportunities. In most of the statecomparative risk projects, risks to humanwelfare or quality of life were consideredseparately from human health and ecologicalrisks by a non-technical subcommittee thatdeveloped criteria and produced a ranked listof QOL risks. The EPA (1993) developed aguidebook for assessing quality of life risksthat provides a starting point for suchassessments.
The dialogue among the participants in Phase I isdesigned to focus the problem and to ensure thatthere is a logical consistency between the stressorsand risk being considered, on the one hand, andthe risk reduction opportunities that may be avail-able, on the other. In this initial Phase, the discus-sions are at the level of planning, scoping, andscreening, rather than the detailed analysesconducted in Phase II.
Scientists play an important role in Phase I bycollecting, analyzing, and presenting data in such away that all parties can appreciate the type andmagnitude of the problem(s) under discussion. Thisactivity will generally involve all four parts of riskassessment, including assessment of exposures expe-rienced by special populations and/or ecologicalresources. Planning, scoping, and screening —including selection of endpoints of concern — alsorequires explicit input of societal values and stake-holder participation. For instance, while some of the ecological endpoints may be chosen because oftheir role in a valued ecosystem, there may also beecological endpoints chosen because of their directsignificance to society. Examples of the latter includeboth economically important species and “charis-matic” species. Similarly, in integrated decision-making, judgments may have to be made aboutdiverse health endpoints, such as cancer risks in thegeneral population and the risk of reproductive/developmental risks in children. While scientists can help to characterize such risks, they are notuniquely qualified to set priorities among them andbroader deliberation is essential. Finally, decision-makers also play an important role during ProblemFormulation; in addition to bringing the scientificand other resources of the Agency to bear on theproblem, they also should help to identify the rangeof potential decisions and viable managementoptions, while examining economic, political, orother constraints on those options. Decision-makersalso serve as managers of the overall process.
During the development of the IED frame-work, the SAB emphasized that, although infor-
mation on the nature and extent of risks is criticalto Problem Formulation, the ranking of riskreduction opportunities is equally important when establishing risk priorities. One means ofdescribing the relationship between risk rankingsand possible priorities is pictured in Figure 2.
In summary, participants in the initial Phase —scientists, decision-makers, and interested andaffected parties ñshould seek agreement throughan open, yet structured, exchange of information,concerns, opinions, and values (i.e., iterative,deliberative-analytic dialogue as described inNRC, 1996) on a series of issues that will definethe problem so that more in-depth analyses can be conducted in the subsequent phase of decision-making.
2.3 Analysis and Decision-Making(Phase II)
In Phase II of the IED Framework, the analyststake the information and general directions gainedin Phase I and generate more detailed, more fullysupported assessments of risks and risk reductionoptions. For integrated decision-making, optionsanalysis should include consideration of riskreduction opportunities with regard to their tech-nical feasibility, aggregate risk reduction to beobtained (e.g., reductions in “target” risks andcollateral reduction in all affected risks), fulleconomic consequences of various risk reductionscenarios, and so forth. Decision-makers alsoshould request analysis of potential options withregard to sustainability, equity, and other potentialdecision criteria.
Options analysis generally is more “analytic”than “deliberative” (NRC, 1996) although acontinued level of interaction between the participants in the overall process (scientists, riskmanagers, and interested and affected parties) isimportant. Options Analysis is also moreresource-intensive than Problem Formulation.
In the decision-making portion of Phase II, the
12
While scientistscan help tocharacterize
risks, they arenot uniquely
qualified to setpriorities
among themand broader
deliberation isessential.
Agency or other decision-makers should a) utilizeoutputs from the analyses of risk and risk reduc-tion options, b) consider widely-held publicvalues, as well as the views of participating stake-holders, c) consider the legal, economic, and insti-tutional constraints, and d) ultimately, make thedecision. Clearly, this process is not totally scien-tific. However, the best science should inform andcontribute to decision-making. Developments inthe social and decision sciences, for example, areproviding improved methods for value elicitationand multi-attribute decision-making. The docu-mentation supporting the decision should makeexplicit a) the implications of the chosen manage-ment option(s) to the health of ecological orhuman systems, b) the economic costs and bene-fits associated with the selected option, and c) thesocietal values that both influence and are affectedby the decision, including values relating to
economic efficiency, sustainability, equity, andquality of life. Integrated decision-making requiresexplicit consideration of the trade-offs involved inpursuing multiple environmental goals and/or insimultaneously pursuing environmental and non-environmental goals. In some cases, analysis mayindicate that a particular management option isnot worth doing because of the greater good thatmight be achieved by investing those resourcestoward the achievement of another goal.
It is important that the scientific and technicalanalyses prepared during Phase II articulate clearlythe uncertainties associated with the estimates ofrisk, the estimates of risk reduction that may beachieved by different management options, andthe economic assessments of various risk manage-ment scenarios. Integrated decision-making doesnot eliminate the uncertainties associated withmaking decisions. However, by encouraging an
13
Figure 2. The Relationship Between Risk Rankings and Possible Priorities
Integrateddecision-making
requires explicit
considerationof the
trade-offsinvolved inpursuingmultiplegoals…
no
Research orEducationPriority
ManagementPriority
Are RisksHigh?
Is RiskReductionPotentialHigh?
yes
no no
yes yesIs Risk ReductionEconomicallyviable? equitable?supported by thepublic?
Monitor forChange
MonitorPerformance
uncertain
open and comprehensive examination of environ-mental problems, integrated decision-makingshould lead to a clearer identification of thenature, extent, and consequences of the uncertain-ties associated with the available information. In any event, environmental decision-makingmust proceed in the presence of uncertainties, andnothing in the proposed Framework should beconstrued as precluding environmental decisionssimply because uncertainties remain.
2.4 Implementation and PerformanceEvaluation (Phase III)
In the third phase of integrated decision-making,the chosen risk reduction option or set of optionsis implemented and evaluated over time to ascer-tain the extent to which it is achieving the desiredenvironmental outcomes. The specific activitiesrequired to implement an environmental decision
will depend on the suite of management optionsselected for any particular problem or set of prob-lems. The Agency has considerable experiencewith many risk reduction options (e.g., adoptingbest available technology, imposing permit limits,and regulatory negotiation), and is gaining valu-able new experiences with others (e.g., NationalEnvironmental Performance Partnership System).
In contrast to implementation, the perform-ance evaluation process is fundamentally rooted inscience because it is science that can translate thepublic’s overarching environmental goals (e.g.,improved health, sustainable ecosystems) intodiscrete, measurable components. Accordingly,science is essential in deciding what to monitor,i.e., specifying the endpoints of concern for thesystems at risk and identifying the specific meas-ures that need to be monitored in order to charac-terize the status and trends for those endpointswith respect to the environmental goals. Further,
14
Desired Outputs from Problem Formulation:
The initial goals for the decision-making exercise, including environmental goals to be achieved;
Which environmental problems/stressors/systems will be included and which will not, and the reasonsfor these decisions;
The health, ecological, and quality-of-life effects of concern;
The spatial, temporal, and organizational dimensions of the problem;
Relevant data and models, and possible approaches to data analysis;
Scoping of the uncertainties involved and research needed to significantly reduce critical uncertainties;
Initial review of the range of options available to reduce risks, considering likely economic, political, orother constraints;
The endpoints upon which the condition of the ecological, human health, or societal systems ultimatelywill be judged; and
The types of factors that will be considered when reaching a decision.
The SABadvocates the
use ofEnvironmentalReport Cardsto documentperformance
and outcomesof risk
reductionactivities…
issues such as spatial and temporal variability,measurement error, and time lags must beaddressed explicitly in order to demonstrate envi-ronmental conditions and to separate signal fromnoise. And finally, reference conditions andbenchmarks or milestones along the way to thedesired system conditions must be defined scien-tifically so that meaningful and measurableperformance criteria for success or failure can bedefined.
The SAB advocates the use of EnvironmentalReport Cards to document performance andoutcomes of risk reduction activities at severallevels and for different audiences. As noted inFigure 1, a successfully implemented performanceevaluation system will generate several importantfeedbacks affecting the first and second phases.One feedback loop is to the Analysis andDecision-Making Phase, reporting on how wellthe selected risk reduction options and strategiesare achieving the environmental goals. This feed-back loop allows for adaptive management andchanges in implementation activities, including thepossible need to identify and analyze additionaloptions to further reduce risks. Information alsofeeds back into Phase I allowing re-examination,as needed, of the initial goals, risk rankings orother aspects of Problem Formulation. As riskreduction options are put into place, for example,particular risks should be reduced, and a newcomparison of risks may be appropriate. In addi-tion, there may be a shift in or redefinition of soci-etal values over time, requiring different sets ofenvironmental goals and, therefore, different envi-ronmental decisions. In short, integrated decision-making emphasizes the need for performanceinformation at several points in the process andfor systematic review of environmental decisionsin light of new scientific understanding, shifts insocietal values, changes in stakeholder preferencesand available resources, and/or responses of theenvironment to previous decisions.
2.5 Building on Previous Frameworks
The IED Framework builds upon severalprevious efforts, in particular the risk assessment/risk management model described by the NationalResearch Council (NRC, 1983), the update to thatreport (NRC, 1994), the ecological risk assessmentframework (U.S. EPA, 1992), the report of thePresidential/Congressional Commission on RiskAssessment/Risk Management (1997), and the riskcharacterization process described by the NRC(1996), which focused on the interaction betweenanalytic and deliberative processes in decision-making.
Building on this base of information, the IEDFramework is intended to integrate a range offactors that are important in modern risk assess-ment and risk management (see box). In particular,the process envisioned in the Framework movesbeyond these earlier efforts in the three areasdiscussed below.
2.5.1 Integrated Aspects of SingleStressors/Risks
Although the SAB emphasizes that the Agencyshould consider multiple stressors in an integratedmanner, it recognizes that integrated thinking alsocan enhance the decision-making process forsingle stressors, which have typified many Agencydecisions in the past. When a single stressor isconsidered, integrated thinking should expand theprevious approaches by:
a) Characterizing stress-effects relationshipsacross all systems and populations;
b) Exploring a broader range of risk reductionoptions, some of which may be qualitatively new;
c) Assessing the benefits and costs of each option,including explicit consideration of non-mone-tary benefits and costs;
15
d) Assessing the magnitude and nature of theaggregate risk reduction associated with eachoption;
e) Involving scientists, options analysts, stake-holders, and risk managers collectively at vari-ous points throughout the process; and
f) Establishing a performance evaluation “reportcard” to characterize the efficacy of the imple-mented risk reduction option and to signalboth the need and opportunity for adaptingthe original management decision.
2.5.2 Integrated Aspects of MultipleStressors/Risks
One of the most important extensions of inte-grated decision-making is its focus on multiplestressors. The SAB suggests prototypic structuredapproaches that can be used to begin to exploremultiple ecological risks and multiple humanhealth risks. In some cases these approaches willlead to consideration of combinations or groupsof stressors; e.g., all organophosphate pesticides,all automobile emissions, or all factors leading tolocal loss of biodiversity.
16
Types of Integration in the IED Framework
Risk Comparisons Consider a wide range of environmental risks simultaneously so that the seriousness of risks can becharacterized relative to one another.
Integrated Risk Assessment Develop scientific data and analytical methods for assessing risks from multiple stressors, from multiplesources, by multiple exposures, resulting in multiple effects/outcomes, in order to represent real worldsituations more accurately.
Integrated Analysis of Management Options Investigate options to reduce subsets of ranked risks, rather than considering single risks in isolation, inorder to achieve greater aggregate risk reduction.
Integrated Analysis of Economic Consequences Identify the full range of benefits and costs, both monetized and non-monetized, associated withreduction of multiple risks.
Integrated Performance Information Use performance evaluation measures to better characterize conditions and to adapt implementedactions appropriately.
Integrating Multiple Disciplines and Points of View Understand and utilize information from all concerned parties in decision-making.
Initial steps in integrated decision-makingshould produce relative rankings of risks tohuman health and ecosystems, independently. In the context of integrated decision-making,however, it is also important to go further anddevelop integrated health and ecological risk rank-ings. A merger of such qualitatively different risksinto a single scheme requires even larger valuejudgments than the ranking of similar risks andshould be informed by a broader societal valua-tion process.
Qualitative differences in human health andecological risks should be considered duringProblem Formulation. For example, the focus ofhealth and ecological risk assessment is different;that is, the focus of health risk assessment is anindividual within a single species, while the major-ity of ecological risk assessments focus on entirepopulations of one or many species. More gener-ally, ecological risk assessments often address theintegrated risks to a prescribed region, such as awatershed, versus health risk assessments that aregenerally more national in scope. This difference isreflected in the different types of stressors ofconcern; cf., carcinogens for humans vs. habitatfragmentation. However, in those instances inwhich there are common stressors of concern (e.g.,chlorinated pesticides or climate change) or whereeffects of a stressor on an ecological systemproduce effects on human health or quality of life(e.g., habitat alteration that affects the range andactivity of disease vectors and infective parasites,or changes in the abundance of commerciallyimportant or endangered species), there is anopportunity for some merging of concerns to takeplace. Integrated decision-making should combinerisks into logical groupings, e.g., those with acommon source or pathway, in order to identifyrisk reduction opportunities across stressors. Inorder to be successful, this analysis requires open,publicly-accessible, and frequent dialogue amongthose who assess and compare risks, those whodetermine methods for reducing risks, and those
who make the final decisions. In summary, as inte-grated thinking is developed to address multiplestressors, it should:
a) Lead to a more realistic comparison of risks tohumans and to ecosystems, where some ofthose risks may be posed by combinations ofrelated stressors;
b) Lead the Agency to consider in a systematicfashion all of the appropriate factors related torisks in a given circumstance, including aggre-gate risk, economic factors, and societal values;and
c) Lead to action that will increase the reductionin aggregate risk posed by a combination ofstressors in a given circumstance.
2.5.3 Considering Environmental Values
The SAB’s integrated environmental decision-making Framework highlights the analytic/delib-erative process (NRC,1996). It is through such aprocess that societal values intersect with thescientific risk characterization and risk reductionanalyses. The integrated decision-making conceptemphasizes the role and timing of stakeholder anddecision-maker inputs to the analytic processes. Itexplores more deeply the valuation of environ-mental outcomes and risks and the need to includenot only the concepts of economic efficiency andwillingness-to-pay, but also issues such as environ-mental sustainability and equity. Societal valuesconstitute the milieu in which integrated environ-mental decision-making occurs, forming the basisof goals for improved social welfare, improvedecological/health conditions, and long-termsustainability and equity. Ultimately, it is in therealm of social values that the success or failure ofenvironmental decisions will be judged.
17
Integrateddecision-
making shouldcombine risks
into logicalgroupings,
e.g., those witha commonsource or
pathway, inorder to identifyrisk reductionopportunities
acrossstressors.
The elements within the Phases of theFramework and how they interact with each otherare at the heart of an integrated approach to deci-sion-making. This section of the report elaboratesupon many of those elements and points the wayfor further development of each in order to meetthe needs of integrated decision-making.
3.1 Comparative Risk for ProblemFormulation (Phase I)
The term Risk Comparisons is used in thisreport to denote the characterization and ranking ofrisks posed by environmental stressors, where anenvironmental stressor is any physical, chemical, orbiological change or agent that could affect ecologi-cal or human health systems. The objective of riskcomparisons is to determine how members of anarray of risks compare in magnitude and scope toeach other, so that a subset of risks can be identifiedfor simultaneous evaluation and decision-making.As depicted in the Framework, the resulting infor-mation on comparative risk should be integratedwith other relevant technical information (includingexpected costs and benefits), expert judgment, andvalues brought into the process through delibera-tion with interested and affected parties. This inter-action in Phase I should lead to agreement on goals,limitations on analysis, and a tentative identificationof possible risk reduction options.
3.1.1 What We Have
One approach to Risk Comparisons, devel-oped by the Ecological Risks Subcommittee (ERS)
and applied to the comparison of ecological risks,utilizes an expert group to develop and weightrisk ranking factors in order to produce thegroup’s consensus judgment of the relative risksassociated with various environmental stressors.The ERS approach considers the two fundamentalcomponents of ecological risk—the stress or expo-sure regime and the response or ecological effectsregime— for each stressor of concern. Becauseeach stressor may result in multiple effects ondifferent entities in an ecosystem, the goal of therisk comparisons is to characterize the dominantrelationships between the environmental stressorsand ecological effects. Ecosystem-specific stress-effect relationships are then transformed into arelative ranking of risk at a specific spatial scale(e.g., regional, national, or global) by applying aseries of numerical factors that, in the Subcom-mittee’s view, reflect the relative impact of theeffects on the ecosystem(s) at risk.
The ERS categorized ecological risks fromchemical, biological, and physical stressors intofive narrative categories: Highest, High, Medium,and Low Ecological Risks, and Unknown ButPotentially Important Risks. Stressors ranked bythe ERS as posing Very High ecological risks arelisted in Table 1. These conclusions represent theconsensus expert judgment of the Subcommittee,based on general ecological principles. The trans-parency of the ERS methodology allows others toevaluate the scientific bases for the risk rankingsand to reach their own conclusions. A differentgroup of experts, using a different set of riskfactors or weightings, might come to different riskrankings. Nevertheless, the conclusions of the
18
3. WORKING TOWARD IMPLEMENTATION
The SABsuggestsprototypicstructured
approaches for ratingmultiple
ecological risks or
multiple humanhealth risks.
ERS are consistent with other previous nationalrankings of ecological risks (e.g., EPA, 1987; SAB,1990) and suggest that the present greatest risks tothe nation’s environment result from physical andbiological, rather than chemical, stressors. Furtherdetail on the ERS method and findings will beprovided in a separate SAB report.
A second approach to Risk Comparisons, lessfully developed by the Human Exposure andHealth Subcommittee (HEHS) and illustratedusing human health risk issues, employs Internet-based polling of individuals (experts) for their(professional) judgment of the relative degree ofrisk associated with various stressors, and solicitsinformation on which factors most influenced thisjudgment and the degree of confidence each indi-vidual (expert) assigns to his/her assessments (seeFigure 3). Thus, this method for developing riskcomparisons also provides information on therange of recorded opinions.
The HEHS developed a prototype for anInternet-based system for polling and characteriz-ing expert judgments about the risks posed byvarious environmental stressors. The prototyperestricted-access Internet site includes an EntryPage, where survey participants register, followed
by a Ranking Page (Figure 3), where participantsare asked to rate a series of stressors as Very High,High, Medium, Low, Very Low, or UnknownRisk. The Ranking Page also asks for the factorsthat influenced the rating and a confidence ratingfor each risk ranking. The Ranking Page containsan Information Window, which allows partici-pants to access summary information for eachstressor. The HEHS recommends that informationbe provided on relevant exposure routes and path-ways, population exposed, average dose, toxico-logical and health effects data, and so forth. Anexample of the output data that would result fromthe polling approach is shown in Figure 4.
While there were some differences, the twosubcommittees proposed quite similar sets offactors that they felt were important in evaluatingthe comparative risks associated with various envi-ronmental stressors (see box). Both subcommit-tees also recommended that information on eachof the factors be assembled in a summaryformat—a risk data sheet— to assist non-expertparticipants in Problem Formulation to assess therelative priority that should be accorded eachstressor.
The consensus approach or the individualpolling approach could be applied to either humanhealth or ecological risk comparisons using stres-sors and rating factors specific to each group. Themethodologies also could be used to elicit stake-holder or public views on risk priorities byexpanding the composition of the surveyed orempaneled group. Clearly, there will be variationamong the results of the various groups — evenamong groups of experts — because of differencesin level of expertise and knowledge about a partic-ular stressor or because of differences in thedegree of concern that individual participantsattach to various health or environmentaloutcomes associated with a stressor.
While the examples developed by the ERS andthe HEHS provide illustrations of the type of newtools we believe need to be developed, these
19
Table 1. Environmental Stressors Posingthe Highest Risk at the National Scale:Conclusions of the ERS*
Hydrologic Alterations Harvesting Living Marine Resources Habitat Conversion Climate Change Introduction of Exotic Species
*Given current efforts to manage risks
specific examples were created with limitedconsideration of the literature in modern socialand decision science. As the Agency develops andrefines such tools for future use, it will be impor-tant to involve experts from these fields.
3.1.2 What We Need
During the design of the Framework, the SABparticipants acknowledged that technical riskrankings, in isolation, offered insufficient guidancefor policy decisions. Given the multitude of prob-
lems and issues to be addressed, a more compre-hensive and systematic framework for analyzingand reducing environmental health, ecological, andquality of life risks appeared necessary. DuringProblem Formulation, the Agency needs methodsfor comparing risks that are robust, transparent,effective, and inexpensive. As noted, some initialsteps have been taken for ranking risks withincategories; e.g., human health, quality of life, orecosystem risks.
Additional work, however, needs to be doneon methods to incorporate non-expert values in
20
Figure 3. Internet-based risk rating form. The respondent selects a stressor, thenaccesses the data sheet by selecting the Information Window.
risk comparisons and priority-setting, includinginputs from the general public as well as interestedand affected parties, and to integrate the analysisof multiple risks.
The assessment and comparison of risks andthe definition of environmental goals are inter-related. In other words, scientific information onthe nature and extent of various risks influencesthe relative priority that society places on thoserisks. In addition, other factors (e.g., dread, previ-ous experiences, the degree to which exposure isvoluntary, and underlying values) also influencethe relative priority assigned by the public to envi-ronmental risks. For this reason, the ultimatepriorities for action that emerge during ProblemFormulation should be a product of the interac-tion between risk comparisons by experts and themore inclusive goal-setting processes, supple-
mented by a preliminary assessment of risk reduc-tion potential of the various options. The purposeof ranking risks and ranking risk reduction oppor-tunities is to assist decision-makers to allocateresources in Phase II.
The interaction between risk comparisons andgoal-setting could be enhanced if the risk compar-isons are structured so as to identify linkagesbetween health, ecological, and quality of liferisks; e.g., common stressors or root causes, indi-rect affects on health of an ecological impact.Defining risk problems using a common dimen-sion (e.g., in terms of stressors) will facilitate theidentification of both direct and indirect effects ofstressors, and of those stressors that affect bothhumans and natural systems. Approaches forachieving this sort of integrated analysis willrequire further development.
21
Figure 4. Synthetic plots of risk versus confidence (12 respondents): A) highvariability; B) low risk, mid to high confidence; C) high risk, low confidence; D) high risk, high confidence.
VL
Ris
k R
atin
g
L M H VH
VH
H
M
L
VL
Confidence
A VL
Ris
k R
atin
g
L M H VH
VH
H
M
L
VL
Confidence
B
VL
Ris
k R
atin
g
L M H VH
VH
H
M
L
VL
Confidence
C VL
Ris
k R
atin
g
L M H VH
VH
H
M
L
VL
Confidence
D
3.2 Risk Assessment for Analysis andDecision-Making (Phase II)
3.2.1 What We Have
The Agency has been a leader in developingguidelines for risk assessment. Beginning with thefirst version of cancer risk assessment guidelines in1976, the Agency now has risk assessment guide-lines in place for six human health endpoints. Inaddition, the Agency broke new ground with thepublication of ecological risk assessment guide-lines in 1998. These guidance documents — and their updates — provide state-of-the-artinformation on how to perform detailed analyticalassessments of risk that can inform the decision-making process.
In 1997, the Administrator signed aCumulative Risk Policy that committed theAgency to move in the direction of assessing themultiple effects of multiple stressors throughmultiple routes of exposure, which is certainly
consistent with the Framework. This Policy,however, promises a level of sophistication in riskanalysis that currently does not exist.
3.2.2 What We Need
Most of the risk assessment guidelines that arein place generally address single stressors resultingin single endpoints; e.g., the cancer risk fromarsenic. However, the public is increasingly askingmore holistic questions; e.g., “What is my totalrisk of living in this environment?” Implicit in thequestion is consideration of the total risks fromvarying, complex mixtures of stressors, as well asthe cumulative impacts of multiple exposures,experienced over time and space, to populationswho might have differing degrees of susceptibilityand resources to address the consequences ofthose risks. In addition, current guidelines implythat the goal of reducing risk to a specified level(e.g., 10-4 or 10-6 lifetime cancer risk) is to bepursued regardless of the associated benefits and
22
Correspondence of Human Health and Ecological Risk Comparison Factors
Human Health Risk Factors Ecological Risk Factors
Size of population affected Proportion of resource at risk
Particular subpopulations at risk Distribution of “hot spots”
Severity and persistence of effects Recovery potential, species depletion
Persistence in environment and/or human body Duration of stress-effects co-occurrence
Percent of attributable incidence
Potential future risk
Secondary stress induction
Special ecological significance
costs, including opportunity costs. When benefit-cost and other information is considered, however,the public may set a different priority amongmultiple public welfare goals. The scientificcommunity has taken the initial steps in scientificrisk assessment by applying simplifying assump-tions; cf., a generic receptor exposed to a singlestressor under one set of conditions. This is analo-gous to the first approximation methods used inthe physical sciences, such as assuming no atmos-phere in gravity experiments. But now the scien-tists appear to be more willing — even insistent —to ask the more “real world” questions associatedwith complex mixtures and diverse populations.However, the need for such information and thewillingness to work on the problem have yet toyield suitable and effective methods. Much workremains to be done and much experience is yet tobe gained before we will able to assess cumulativerisks posed by multiple stressors and/or multipleexposures over time.
3.3 The Analysis of Benefits and Costs inDecision-making (Phases I and II)
In deciding what to do about a particular envi-ronmental problem, it is impossible to escape thequestion of values. At its heart environmental deci-sion-making is about how people, acting as indi-viduals or through their government, can makethemselves best off through actions that affect theenvironment directly and indirectly. Judging whichchoice, among alternative actions, will produce themost “well being” requires information not onlyon risks, but also on how “well being” (i.e., goalsand the relationships among goals and amongalternative choices or actions) is defined. Integrateddecisions on whether and how to address certainenvironmental risks or sets of risks requires theconsideration of both a full range of risks and thefull economic consequences associated with possi-ble decisions. An understanding of the tradeoffsimplied by these choices is crucial both when
problems are formulated for consideration (inPhase I) and when they are being analyzed in detailto support decision-making (in Phase II).
3.3.1 What We Have
The work of the Economic AnalysisSubcommittee (EAS) is presented in a separate SAB document that describes the strengths andlimitations of benefit-cost analysis. In short, riskinformation, by itself, is inadequate to guide deci-sions because there are a near-infinite number ofsubstances and conditions that can impose harm,and it is impossible to act to eliminate them all. Theability to reduce risks is limited by the resources oflabor, capital, knowledge (technology), and physicalendowment available to devote to the task. The taskof environmental decision-makers is to provide —for now and for the future — the healthiest, safest,most ecologically secure set of conditions for whichthe American people are willing to pay, in the faceof other competing goals. Given these conditions,the decision-maker can use a number of approaches(within the limits of enabling legislation) to deter-mine whether and by how much to reduce risk.Prominent among these approaches is benefit-costanalysis that seeks to strike a balance between thebenefits associated with reducing risks and the costsassociated with taking a particular action. In idealterms, benefit-cost analysis measures the good andbad effects of a proposed action in some commonterm so that decision-makers can more directlydetermine whether the associated gains to societywill outweigh the losses.
In real terms, the application of benefit-costanalysis to environmental decision-making is chal-lenging. On the one hand, assessing benefit isdifficult. Often the goods and services that comefrom ecosystems are not traded in markets.Further, even if real markets exist for the goodsand services, the changes in human health andecosystem functions associated with a particularaction are difficult to predict or measure. Where
23
The task ofenvironmental
decision-makers is to
provide — fornow and for the
future — thehealthiest,
safest, mostecologically
secure set ofconditions for
which theAmericanpeople are
willing to pay…
markets exist, economists have well-establishedmethods for determining peoples’ willingness totrade one resource use for another. Where marketsdo not exist, however, economists must use tech-niques that are more controversial in order to esti-mate the values people place on an ecosystemgood or service.
Cost is also difficult to assess, especially whenviewed as economists do as opportunity costs,government administration costs, transactioncosts, general equilibrium effects, and socialimpacts, in addition to direct compliance costs.(Opportunity costs are the foregone social bene-fits associated with applying scarce resources toenvironmental protection instead of other possibleuses of those resources.)
Other challenges for the benefit-cost analystinclude consideration of time and equity. Costsand benefits associated with an action may not berealized over the same time period (e.g., benefitsmay accrue to present generations, but costs to afuture generation, or vice versa). This difference inthe time horizon often raises equity concerns. Inaddition, future costs and benefits cannot becompared directly with current costs and benefitswithout taking into account the potential earningsthat may be associated with deferred expendituresand consumer preferences for consumption in thepresent vs. in the future. Thus, economists havedeveloped a system of discounting the stream ofbenefits and costs as they are realized over time sothat future costs and benefits can be comparedwith present costs and benefits. Although thediscounting process can be controversial, manyeconomists believe that discounting of the streamsof benefits and costs in a consistent way is essentialto show current decision-makers the full implica-tions (present and future) of alternative choices.
In the matter of equity, while benefit-costanalysis generally provides a societal level view ofthe benefits and costs associated with a proposedaction, the technique is less adept at accounting forthe uneven distribution of those benefits and costs
across different individuals and groups in society.Another complicating factor is that individualsaffected by the environmental action are oftenunequally endowed in wealth and income. Thisdifference influences both the benefits and costs asthey enter the evaluation framework, and thesense of fairness that pervades the results.Although benefit-cost analysis can shed light onthese distributional issues, such analysis cannotanswer the societal question regarding the desireddistribution of benefits and costs. The definitionof what is “equitable” is left to the decision-maker,in conjunction with the other participants in thedecision process.
3.3.2 What We Need
As noted above, more work needs to be doneto enable decision-makers to assess the impact ofpotential decisions on the distribution of benefitsand costs across the population and on society atdifferent points in the future. In addition, theFramework suggests a role for benefit-cost analy-ses on the impacts of potential managementoptions to help formulate risk reduction priorities.“Rough-cut” benefit-cost analyses should be suffi-cient as first approximations of whether theoptions are economically feasible or not.
In summary, benefit-cost analysis is a consis-tent, coherent, and transparent tool for looking attrade-offs among competing goals. When welldone, the products of specific benefit-cost analysescan generally be relied upon as useful inputs fordecision-making purposes, but are always limitedby data and sometimes by methodology. In somecases and for some questions (e.g., equity), theinherent uncertainties limit benefit-cost analysis toproviding only indicative information, which,nonetheless, can be useful. As with risk assessment,risk comparison, and other elements in theFramework, there is a recognition that even thebest and most complete benefit-cost analysis isinadequate, by itself, to yield “the answer” in a
24
particular case. Additional work is needed to refineexisting approaches for forming and eliciting thevalues that individuals place on possible environ-mental or health outcomes and on their preferencesamong possible risk reduction options. The use ofdeliberation, as part of or as a complement to moretraditional benefit-cost analyses, holds promise forcharacterizing difficult-to-monetize values andthus bringing those values to the table. The nextsection discusses these issues in greater detail.
3.4 Forming, Eliciting, and ConsideringPublic Values (Phases I and II)
The term “values” takes on two meanings inintegrated decision-making, and it is important todistinguish between the two in order to avoidconfusion and mis-communication. In the firstinstance, “values” refers to the set of underlyingfactors that, taken together, cause people to holdthe opinions that they hold and to make thechoices that they make when presented with realsituations. There are many such value systems thatcontribute to the normative makeup of people in adiverse society. The diverse nature of peoples’values contributes to the complexity and difficultyresponsible authorities face when making deci-sions on whether or not to act to reduce risks.These values must be considered, along with thestatutory framework and other factors, whengoals are identified during Problem Formulation.
In the second instance, “values” is used in anoperational sense as a measure of what oneoutcome is worth in comparison with alternativeoutcomes. This is the sense in which values arereflected in benefit-cost analysis and is the sense inwhich the term in used in this section.
3.4.1 What We HaveThe Valuation Subcommittee (VS) engaged in a
series of intensive discussions that explored thenature of natural resource valuation and the basisbehind the public construct of values. The general
themes that emerged during the Subcommittee’sdiscussions are listed in the box on page 26.
The VS discussions affirmed the notion thatintegrated environmental decision-makingrequires a process within which the decision-maker can meld the results of science and the goalsof the people served and formulate acceptabledecisions. In principle, economic analysis providesone such approach to valuing the human healthand ecosystem benefits of proposed actions interms of the change in economic well-being asso-ciated with the action. Economic values from thisapproach are said to be derived from the vector ofall underlying abstract values held by individualsin concert when a situation is presented.
By their nature, changes in human health andecological conditions often are not easily observedin quantifiable terms that can be rendered in themonetary units that most often are used tocompare possible outcomes in benefit-cost analy-ses. For this reason, some people assert that thebenefits received from environmental protectionare systematically under-estimated and, conse-quently, that ecological systems and human healthare under-protected as compared to other goalsand desires of people.
An example of the difficulty faced by decision-makers can be seen in the case of determiningecosystem values. In principle ecosystem valuesare not different from other values, and they neednot enter into the decision process in any uniqueway. At the same time, however, measuring andincorporating the values ascribed to anthropogenicchanges in ecological conditions does present seri-ous difficulties that require special care.12
Ecosystem value can come from direct use,from indirect use, or simply from the value that thepublic places on keeping the ecosystem viable
25
…integratedenvironmental
decision-making
requires aprocess within
which thedecision-makercan meld the
results ofscience andthe goals of the peopleserved…
12 Although the VS focused on ecosystem valuation, valuationof human health conditions presents similar challenges inenvironmental decision-making.
(“existence value”). Eliciting these values is difficultfor several reasons. For example, people doing thevaluing often have insufficient knowledge of howchanges in ecological factors affect the things theycare about. They need expert scientific assistance inmaking the important connections. Also, many ofthe benefits from ecological systems are subtle anddo not enter into conscious consideration in thenormal course of events, in contrast to marketgoods and services which are much more evident.To account for these benefits, special techniquesfor eliciting preferences are required. In addition,some of the values ascribed to ecological outcomes(e.g., equity, sustainability, and stewardship) arisein a social context and may not surface fromcommonly used individual preference measures.
Most importantly, because ecological servicesoften do not enter into markets or enter themincompletely, it is difficult — if not impossible —to provide robust and dependable monetizedmeasures of the benefits they deliver. Often it is
not possible to even determine quantitative meas-ures for differences in outcomes. Qualitativemeasures of benefits and costs, therefore, must bearrived at and then incorporated into decisionprocesses. In short, the valuation of ecologicalcosts and benefits is prone to error becauseelements valued by people may be omitted orincorrectly specified and because measurement isinherently more difficult than with goods andservices for which market and market-like meas-ures are available.
Because of these limitations, there is a need toimprove existing methods for estimating the valuethat society places on various aspects of humanhealth and ecological condition. One approach toassure that all relevant elements are included indecisions, and that they are valued properly, is toexpand the use of deliberative processes.Deliberative processes play an important role ineliciting values of people, and in obtaining stake-holder participation in decision-making. If such
26
Environmental Valuation Themes
Ecological valuation is an anthropocentric exercise; i.e., people’s wishes count; there is no external setof values waiting to be discovered for application to decision-making.
The value of anything reflects its contribution toward the achievement of some goal. The process ofvaluation cannot be separated from the need to reach agreement on goals.
Environmental valuation requires interaction and deliberation among scientists, decision-makers, andother stakeholders to identify goals and to define endpoints to characterize those goals.
Existing economic approaches, broadly considered, are consistent and coherent frameworks forvaluation because they organize a system of trade-offs. However, they are not mechanisms forproducing “the answer” because they may omit trans-economic values that may be important, mayinclude some elements that are difficult or impossible to estimate, and may employ preferenceelicitation processes that are incomplete.
An expanded, rich, and complex process using multiple approaches is required to fully encompassecological valuation.
Deliberativeprocesses playan important
role in elicitingvalues of
people, and inobtaining
stakeholderparticipation in decision-
making.
processes are used discriminately and tailored tothe situation, they should lead not only to moresatisfactory results, but also should not delay —and indeed may even speed the delivery of— envi-ronmental protection because they lessen post-decision controversy.
However, incorporating people’s values andpreferences into decision-making is not an easytask. For instance, not all values, and therefore thepreferences they reflect and the choices theydirect, are equal within or across people. Somevalues are held more tightly than others, and thuswould be expected to be traded-off differently —if at all — from those of lesser importance to indi-viduals. Further, stated preferences may beconstructed during the elicitation process itself.Knowledge of decision theory, cognitiveprocesses, and how people react to complex situa-tions in reaching decisions can help inform delib-erative processes that are intended not only toelicit values information but also to obtain stake-holder participation in the decision-makingprocess. Finally, the deliberative processes must beopen and transparent so that people can under-stand how decisions are made. Through suchopenness and the use of procedures that arecommon to facilitation and mediation, institutionscan build trust and thereby help to legitimize theprocesses used in integrated decision-making.
The VS identified four classes of decisions andthe type of deliberative process that would beappropriate for each (Figure 5).
When agreement about values (economic andnon-economic) is high and the state of knowledge(relevant science, economics, and social science) issufficient and/or non-controversial, Agency deci-sion-making is likely to be routine and delibera-tion will only be needed periodically, if at all, foroversight (oversight deliberation). At the otherextreme, when agreement about values is low andthe state of knowledge is insufficient and/orcontroversial, decision-making is likely to requiremulti-dimensional tradeoffs based on insufficient
knowledge. In this situation, integrated delibera-tion, involving both experts and outside stake-holders, is needed throughout the decision-making process. Between these two extremes lieintermediate intensities of deliberation to fit inter-mediate states of knowledge and agreement onunderlying values.
3.4.2 What We Need
The Working Paper that resulted from the VSeffort offers recommendations for a future work-shop to explore the topic of natural resource valuation more fully with a larger group of partici-pants, including interested and affected parties.The SAB experience testifies to both the impor-tance and the difficulty of such open discussion,frank exchange of views, and search for commonground. A workshop that reaches out to a largeraudience could be beneficial. While it is not certain
27
Figure 5. Typology of DeliberationProcesses with Stakeholders andExperts (adapted from Chess et al., 1998)
StakeholderDeliberation
OversightDeliberation
IntegratedDeliberation
ExpertDeliberation
High Low
Insufficient
Sufficient
STATE OF VALUEAGREEMENT
STATE OFKNOWLEDGE
that such a workshop will settle the remainingquestions regarding natural resource valuation,such a workshop should further the Agency’sunderstanding of recent valuation developments,additional methods for eliciting public values, andthe role of valuation in integrated environmentaldecision-making.
In addition, there is a need for further researchon and experimentation with approaches to valu-ing benefits of environmental systems. Existingapproaches are inadequate in their treatment ofsuch values as fairness and sustainability. Theyalso have difficulty in incorporating the systemicbenefits of ecosystem attributes such as biodiver-sity and are incomplete in their treatment ofdynamic responses of ecosystems to change. Moreholistic approaches that take into account the webof interactions among decisions surroundingecological systems and related production andconsumption activities would be helpful.
3.5 Options Analysis (Phases I and II)
In both Phase I and Phase II, after iterativelyassessing and comparing environmental risks andconsidering the implications of various risk reduc-tion possibilities on the grounds of economics orother measures of welfare, the IED Frameworkcalls for an overall design and selection of riskmanagement scenarios to address the environmen-tal problem at hand. Such Options Analysisduring Problem Formulation (Phase I) informsgoal-setting, thereby ensuring that goals are notdefined solely on the basis of risk, but also takeinto account possibilities to reduce risks withinlikely constraints and trade-offs that will berequired among goals. During Phase II, when the environmental problem(s) and associated envi-ronmental goals have been defined, OptionsAnalysis becomes a more in-depth considerationof possible risk reduction options. In both Phases,the possible options should be analyzed withregard to their potential to reduce single and
multiple risks of concern, associated costs, sustain-ability, equity, and other criteria specified by theparticipants.
3.5.1 What We Have
The Risk Reduction Options Subcommittee(RROS) developed a process for identifying themost effective risk management approaches for avariety of types of risk problems that mightconfront a decision-maker. Specifically, theyconsidered options analysis from three differentperspectives: a) for a stressor-based problem (e.g.,ozone); b) for a geographically based problem(e.g., risks associated with an urban area); and c)for a media-based problem (e.g., contaminatedgroundwater). Their efforts, which are outlinedhere, will be detailed in a separate SAB report.
In short, the RROS found that approaches fordeveloping and evaluating options to reduce singlerisks are fairly well developed. However, the IEDFramework emphasizes the importance of examin-ing a broader array of potential options than mighttypically have been done in the past. Criteriashould be developed to screen potential options,aggregate or disaggregate options, and, through aniterative process, converge on a set of options thatanalysis indicates would best meet the goals. Asnoted in the previous sections, analysis of theeconomic and societal consequences of variousoptions is an important aspect of options analysis.
The results of the Subcommittee’s deliberationson identifying, screening, and selecting risk reduc-tion options, and the relationship of these steps tothe phases of the IED Framework, are captured inthe 10-step process depicted in Figure 6. The firststep in the method is to define the problem.Articulation of the environmental problem(s),including specific goals for what/whose risk is tobe reduced and by how much, is critical to subse-quent steps. Clear environmental goals, withexplicit statement of the relationships and thepotential tradeoffs among goals, are the founda-
28
tion from which objectives for the risk reductionprogram can be derived. In addition to informa-tion on the nature of the risks, other aspects ofProblem Formulation that are particularly criticalfor the design of a risk reduction program include:a) statement of the desired state or goal in measur-able terms; b) specification of methods for meas-uring improvements toward the goal; and c)identification of likely constraints on possiblesolutions. Constraints may include such things as
budget, time, jurisdictional authority, and legalityof proposed options.
The remaining nine steps in the methodologycover development of background information,identification and screening of potential riskreduction options, selection of option(s) that bestmeet the screening and selection criteria, anddocumentation of the decision. There is an exten-sive decision analytic literature that presents meth-ods (e.g., benefit-cost analysis, matrix qualitative
29
Figure 6. Risk Reduction Options Selection Methodology
Quantify Effects
Document Process
Select Option
Yes
No
Optimize Options
Evaluate Remaining Options
Screen and Prioritize Options
Establish Screening Criteria
Indentify All Risk Reduction Options
Develop Background Information
Define Problem Insights fromLater Steps
ExternalInformation
Experience
Stop GoalsMet?
Ph
ase
III
Ph
ase
IIP
has
e I
ranking, and multi-attribute decision procedures)that may be useful in the screening and selectionprocess.
The RROS also analyzed seven different cate-gories of options (as well as subcategories) (Table2) and described situations in which they felt eachcategory of options would be most effective andleast effective.
3.5.2 What We Need
The task of developing and selecting riskreduction options is most easily envisioned interms of reduction of a single risk, e.g., one associ-ated with a particular stressor or source.Integrated environmental decision-making,
however, requires a more complex analysis ofoptions. Although it is often best to address riskswith a combination of risk reduction tools, thisoften is not done because of inadequate informa-tion on the multiple sources of a stressor and theirrelative contribution to total risk.
Extending the methodology to an integratedproblem set containing risks from multiple stres-sors (e.g., those experienced by a particularcommunity) will further increase the complexityof the analysis. Thus, it will be important to aggre-gate or disaggregate the problem set (e.g., using“root cause” or “common source/common path-way” analysis) so that analysis of risk reductionoptions is more manageable. Although it may notbe possible to group all risks of concern on thebasis of their technical attributes, a scientificanalysis of the risks may well reveal commonali-ties that indicate which risks will be affected bythe same risk reduction option.
The likely effect of this integrated view is thatthe option(s) selected to reduce a group of risksmight differ from that which would be selected toreduce the top ranked risk, if it were to be consid-ered in isolation. The complexity of the analysisalso increases greatly as the number of screeningand selection criteria increase.
One objective of options analysis in an inte-grated decision-making context might be to iden-tify those options that may simultaneously reduce,directly or indirectly, risks posed by more thanone stressor. Such analysis would be important ifthe management goal were to maximize the reduc-tion of the total aggregate risk from multiple stres-sors, rather than to maximize the reduction ofrisks posed by any single stressor. This approach,requiring as it may the simultaneous considerationof risks from quite different types of stressors, hasnot yet been fully developed; and it will not betrivial to implement. Nevertheless, the SABbelieves that its development and implementationoffer tremendous potential for improving environ-mental health overall.
30
Table 2. Risk Reduction OptionCategories Evaluated by the RROS
1. Communication/Education a. Information Reporting b. Technical Assistance
2. Enforcement 3. Engineering 4. International and Intergovernmental
Cooperation 5. Management Systems 6. Market Incentives
a. Tradable Emissions b. Pollution Charges c. Subsidies
7. Regulation a. Standards
(i) Harm-based (ii) Technology-forcing (iii) Design
b. Product Bans c. Challenge Regulations
Examples of challenges inherent in applyingoptions analysis to an integrated problem set,consisting of multiple stressors, sources, orendpoints, include the following:
a) Aggregation is helpful in identifying multiplestressors that may have a common set of riskreduction options with the objective of select-ing a set of options that will provide the mostrisk reduction for the set of risks beinganalyzed. Practically speaking, it will not bepossible to optimize risk reduction over allstressors of concern considered at once. Forthis reason, screening and aggregation of stres-sors into manageable subsets should be drivenby analysis of common aspects of stressors,root causes, and/or activities.
b) In order to compare risk reduction across setsof options and sets of risks, and to evaluate riskreduction per unit cost, it is critical to have acommon measurement of risk or commondenominator for all risks. In many cases,comparisons of risks and risk reduction underdifferent scenarios will involve unlike risks(e.g., cancer risk in humans vs. chronic healtheffects vs. effects on wildlife populations), evenwhere those risks have a common “root cause”(e.g., a single stressor or source). While modelscan be developed to weight and combine thedifferent attributes in disparate types of risk,getting wide-spread social agreement on suchweightings may be quite difficult.
c) Uncertainty associated with options analysis is likely to increase as a broader set of optionsand an array of stressors are considered simultaneously. Sources of this uncertaintyinclude the relative contribution of differentstressors/ sources to the total aggregate risk; the effectiveness of combined options for reduc-ing aggregate risk; and the benefit-cost, equity,or other tradeoffs involved in addressing groups
of risks. The extent of uncertainty associatedwith a decision may affect the balance ofoptions selected. When uncertainty is high, forexample, it may be easier to gain support foreducation/communication or managementsystem approaches rather than for regulation.
3.6 Performance Measures
In recent years there has been increased inter-est in tracking, measuring, and reporting on theperformance of governmental actions. Mostnotably, the Government Performance andResults Act (GPRA) requires that federal agenciesreport annually on measures of the results of theirvarious programs and activities. In addition, therehas been discussion and activity in the area of“environmental report cards” as a mechanism forsuccinctly summarizing the state of the environ-ment in easily comprehensible terms.
3.6.1 What We Have
To evaluate performance, one should have agood set of measures and a system for evaluatingand reporting data on those measures.
In the area of measures, the SAB identifiedfour types of performance measures that can beused in concert to evaluate progress toward environmental goals, whether broad goals thatrelate to a number of management and regulatoryprograms or specific goals defined for a particularintegrated environmental decision:
a) Process Measures are measures of administra-tive effort or program actions that arepresumed to result in environmental or healthimprovements (e.g., number of permits issued,number of enforcement cases pursued, numberof contaminated sites cleaned up to standards).
b) Stressor Measures (levels) are measures(levels) of stressors in the environment used to
31
GPRA requiresthat federal
agencies reportannually onmeasures ofthe results oftheir various
programs andactivities.
determine attainment or non-attainment ofdesired reductions in stressor levels; e.g., totalemissions of a pollutant, concentrations ofparticulates in ambient air, levels of dissolvedoxygen or turbidity in a stream, and density ofroads in a watershed.
c) Exposure Measures are measures of the co-occurrence or contact between an individual orpopulation and environmental stressor(s) overa defined time period. The term “exposure” istraditionally associated with chemical stressors(e.g., contaminant levels in food, concentra-tions of contaminants in tissues, time-activitymeasures, and total exposure to a contaminantvia all routes), whereas the term co-occurrenceis often used as a broader term applicable tochemical, physical, and biological stressors.Exposure measures are more directly related toeffects than ambient levels of stressors in envi-ronmental media because they address directcontact with the stressor. These measures mayalso be more readily linked to risk manage-ment decisions than effects measures sincecauses of adverse effects are often multi-facto-rial and difficult to relate to a single environ-mental variable. The use of biological markersof exposure as a measure of environmentalinsults is a rapidly expanding field.
d) Effects Measures are measures of humanand/or ecological effects (e.g., asthma rates,deaths from acute poisoning from householdproducts or pesticides, deaths from cancer,acres of wetlands gained or lost, local extinc-tions of important species) the changes inwhich can be used in one of two ways: (1) toassess the impact of an environmental riskreduction program; and/or (2) for conditionassessment, in which a suite of effects meas-ures are evaluated and reported in combinationto characterize the health or condition of anentire population or ecosystem. Condition
assessment provides a baseline against which toevaluate the success of broad policies or multi-ple decisions impacting a population orgeographic region.
Although it is difficult to relate changes inhuman health condition to a single environmentalfactor, recent studies are improving our ability toidentify precursors to chronic diseases so that thedetection of early effects occurs closer in time tothe environmental exposure(s). Examples ofprecursor measures include changes in the P53gene associated with exposure to sunlight (as anearly marker of damage that may develop intoskin cancer), changes in lung function associatedwith exposures to air pollutants, and changes inIQ associated with exposure to contaminants such as lead.
The Committee considers measures of effectsor condition to be environmental outcomemeasures. We note that this definition differs from the Agency’s definition of “environmentaloutcomes,” which also includes measures of stressor levels (EPA, 1997b). We recommend,however, that stressor measures be kept distinctfrom environmental outcome measures because a)changes (increases or decreases) in stressor levelsdo not necessarily translate into changes (increasesor decreases) in risk, and b) the public’s environ-mental goals are typically in terms of desired states of health or condition, rather than desiredstressor levels.
The different types of measures can be thoughtof in terms of a spectrum of measures rangingfrom least directly to most directly related toactual environmental outcomes, which are ofprimary interest to the public. In many cases, thespectrum of measures will also correspond to aspectrum of response times, ranging from shorterterm to longer term measures. For example, it isrelatively quick and easy to document the numberof permits issued in a year, but it is more difficultand time-consuming to determine the actual
32
33
Evaluating the Success of an Integrated Environmental Decision: A Watershed Example
To illustrate the use of each of the types of performance measures to report on a specificIntegrated Environmental Decision, consider a watershed management or restoration program. Inthis example, the ultimate goal of the program is to maintain or re-establish an ecosystem thatsupports a diversity of habitat types along with their resident communities of plants and animals,supports essential ecological functions, and is self-sustaining. Here, the Integrated EnvironmentalDecision will consist of a set of interrelated actions, many of which will be designed to addressmultiple stressors in order to achieve a reduction in the aggregate risk from those stressors. OtherIED program actions will focus on restoring damage from past stressors (such as the restoration ofriparian zones damaged by livestock operations in order to decrease sedimentation downstream,provide shade and cooler temperatures for aquatic species, and provide additional nutrients to thesystem from dropping leaves).
The evaluation criteria used to judge overall IED program results will include measures of habitatquality (such as length of intact corridors of natural riparian vegetation), water quality and tempera-ture, hydrology that mimics natural variations, the extent of connectivity between floodplains andthe river, nutrient balance, presence of sustainable native populations, and the like. Taken together,these effects measures effectively describe the condition of the watershed, i.e., whether the water-shed, in fact, can sustainably support native populations and their habitats and maintain ecologicalfunctions — and they therefore report on the aggregate results of the IED program.
Each of these effects measures can also be used as to evaluate the success of individual actionswithin the IED program. Following the example above, measures of length of intact riparian corri-dor, water temperature, and nutrient balance would be used to assess the success of specific actionstaken to restore riparian zones.
The time frame required to see changes in the effects measures will vary. Some, such as populationlevels of short-lived species of interest, may be detected after only one year of a management regimethat alters pollutant inputs and water releases to the system. Other environmental responses maytake more than ten years to be detectable.
In addition to the effects measures and condition assessment, direct reporting on the decreasedpressure from various stressors will be useful. In the example above, such stressor measures mightinclude the decrease in the rate of new riparian damage and increases in the release of cool waterfrom dams in the summer. Stressor measures relating to other actions that are also part of the IEDprogram might include reductions in ambient pollution levels, decreases in the number of unscreenedpumps that injure fish, and the restoration of periodic flood flows.
Finally, process measures will provide insight into the level of effort expended and provide ashorter-term indicator that the program is proceeding as planned. Examples of process measuresmight include conservation easement contracts signed for the management of riparian corridors,changes in the regulations governing water releases from dams, and numbers of water pollutionpermits updated with new effluent limits.
impact of that process measure on the conditionof the environment. Figure 7 illustrates this spec-trum and relates the Committee’s terminology toother commonly used terms for environmentalperformance measures.
In the area of reporting systems, the SABrecommends that feedback be provided on theextent and distribution of environmental risks todetermine whether the relative seriousness of riskswas accurately characterized in the first place andwhether specific risks have changed as a result ofan implemented risk reduction program. Reportcards also should provide the basis for evaluatingthe performance of specific risk reductionprograms or decisions, as judged against decisioncriteria such as efficacy at reducing aggregate risk,cost, equity, and time required to achieve riskreduction goals. In summary, environmentalreport cards should provide the information
needed to a) identify opportunities for coursecorrection and adaptive management, i.e., modifi-cation of risk reduction approaches in light ofperformance information or new information onrisks; and b) assign specific accountability—toindividuals, programs, or organizations—for envi-ronmental results.
Report cards for evaluating integrated environ-mental decisions should contain specific mile-stones that can be used to measure progresstowards achieving the environmental goal(s)agreed upon by the participants. Each of theselected endpoints defined during ProblemFormulation (Phase I) should be a part of thereport card, as well as the specific measures orindicators that are monitored to characterize thoseendpoints. The frequency of the reporting shouldbe commensurate with the nature of the risk andthe expected time frame for system response.
34
Anenvironmental
resultsreporting
system shouldinclude a mixof process,stressor,
exposure, andoutcome
measures.
Figure 7. Spectrum of Performance Measures
Environmental Outcomes
Process Stressor Exposure Effects/Condition Measures Measures Measures Measures
activity measures1 pressure indicators3 co-occurrence4 adverse effects5
output measures2 release measures health outcomes6
response indicators3 emission measures state indicators3
“beans”
Least directly related Most directly related to Environmental Outcomes to Environmental Outcomes
1EPA, 1997b; 2GPRA; 3OECD , 1998; 4ERS Method; 5EPA, 1998; 6NRC, 1996
3.6.2 What We Need
Progress is being made in the area of environ-mental report cards. For example, there is an effortunderway, initiated by the Office of Science andTechnology Policy and jointly funded by eightfederal agencies, to design a national environmen-tal report card. This multi-year project, which isbeing conducted by the Heinz Center, has alreadyproduced a prototype design for a report card onthe state of the nation’s ecosystems and specificindicators of ecosystem condition for 3 types ofecological systems: coastal/ocean, croplands, andforest systems (Heinz Center, 1999). Over thenext several years, the Heinz Center plans todevelop a list of indicators for an additional 3types of systems: freshwater, arid lands/range-lands, and cities/suburbs. Although the SAB hasnot reviewed the appropriateness of the reportcard design or the specific proposed measures, theeffort is a clear indication of growing federalagency interest in environmental report cards. Asomewhat analogous reporting effort that focuseson human health is Healthy People 2000—andnow Healthy People 2010—which defined a seriesof health objectives and established a series ofhealth indicators that are monitored and reportedannually (e.g., see National Center for HealthStatistics, 1999).
An environmental results reporting systemshould include a mix of process, stressor, exposure,and outcome measures. Historically, the Agencyhas focused primarily on process and stressormeasures. Therefore, more emphasis should begiven to outcome measures, developing new oneswhere required. Environmental outcome meas-ures—whether of health, ecological condition, orquality of life—may not exist, may be subtle ordifficult to measure, or may be observable onlyover long time frames. Evaluating the performanceof risk prevention programs may be particularlyproblematic. Thus, process, stressor, and exposuremeasures will continue to be important in environ-
mental decision-making and management.Outcome measures, however, are an importantmeans of verifying the theoretical basis for thecontrol or abatement options chosen. In otherwords, if the postulated relationship between stres-sors and effects is inaccurate, then stressor or expo-sure reductions might not produce the expectedimprovements in adverse effects or condition.
Decisions on the design of a report cardsystem have implications for information collec-tion systems, both those that track administrativeprocesses and those that collect environmentaldata (e.g., ambient monitoring programs). Inselecting performance measures, there will oftenbe a gap between what it would be desirable toknow and what it may be feasible to know. Thisgap may exist as a result of several factors, includ-ing the limited state of knowledge about the rela-tionship between stressors and effects, the costsinvolved in obtaining certain types of information,and the willingness of affected people to provideinformation. Nonetheless, it is important thatcareful consideration of the most desirableperformance measures, including those based onan established chain from process and stressormeasures to exposure and outcome measures,influence the types of information that are actuallycollected.
The need for data to assess performance ofintegrated environmental decisions emphasizesagain the importance of monitoring programs thatcan measure both ecological and human healthexposure and outcomes. Implementation of areporting system for human health risk reductionwill be hampered in many cases by a lack of datain key areas; e.g., exposures, differential suscepti-bilities among subpopulations, and incidence ofnon-fatal diseases. In the ecological arena, assess-ments of ecological integrity require informationnot only on water quality, air quality, soil quality,water flow, and populations of certain species,which are commonly monitored today, but alsomeasurements of biological community structure,
35
…the design of a report card
system hasimplications for
informationcollection
systems, boththose that trackadministrative
processes and those that collect
environmentaldata…
the presence of successional states, diversity ofhabitat types across the landscape, connectivity,altered topography, hydrodynamic patterns, andso forth. This latter set of parameters is not typi-cally monitored by EPA, but has received greaterattention by other federal agencies (e.g., USGS,USFS). This fact emphasizes the importance ofstrengthening and maintaining the collaborationamong the many agencies that conduct monitor-ing, an effort begun several years ago under theauspices of the Committee on Environment andNatural Resources (CENR, 1997).
Even the Agency’s traditional list of processmeasures will need to be expanded in order toevaluate the performance of some of the newapproaches to environmental management. Inrecent years, the Agency has added more tools tothe environmental toolbox, bringing to bear suchnew approaches as economic incentives, negoti-ated agreements, and the like. Therefore, thecollection of data to evaluate these new
approaches should evolve as well. There areseveral important inputs currently missing frommost reporting systems that would provide valuable information for assessing how well environmental programs have worked and whatchanges or adjustments might be made. In the case of a marketable permit system, for example,systematic reporting about the number of transac-tions, the average price per unit of the tradedcommodity, the number of market participants,the net reduction in pollution output, and average marginal cost per unit of pollutionreduced should be compared with initial predic-tions. To the extent that a risk managementprogram involves more than one tool, the entirepanoply of actions should be reviewed for efficacy,cost, and other measures of performance. Separateinformation should also be provided on the “non-quantitative” inputs to decision-making, includingthe effects of the risk reduction program onsustainability and equity.
36
37
As a result of conducting the Integrated RiskProject, the SAB has developed a set of 10 recom-mendations that should help the Agency take “thenext step” in improving environmental decision-making in this country.
Recommendation 1: EPA should continuedevelopment of integrated, outcomes-based environmental protection, whilemaintaining the safeguards afforded bythe current system.
Previous environmental managementapproaches, both regulatory and non-regulatory,have resulted in substantial improvements inhuman health and ecological condition, particu-larly in regard to chemical risks. While maintain-ing the current regulatory and managementstructure, the Agency should continue its develop-ment of more broadly integrated approaches toenvironmental decision-making. In particular, theAgency should use the concepts inherent in theFramework laid out in this report to seek oppor-tunities for going beyond current protections bytaking a truly integrated look at risks, opportuni-ties for risk reduction, and the consequences ofaddressing (or failing to address) those risks. Tothe degree possible, the focus of this next step ofenvironmental decision-making should be onenvironmental results; that is, on demonstrableoutcomes (improvements) in the environmentresulting from integrated action. As a part of thiseffort, the Agency will need to develop additionaltools to implement the approach and to monitorthe results of actions taken.
Recommendation 2: Because science playsa critical role in protecting theenvironment, EPA should commit theresources necessary to expand thescientific foundation for integrateddecision-making and outcomes-basedenvironmental management.
An important theme of the IED Framework isthe need to integrate scientific and technical infor-mation on risks and risk reduction opportunitieswith information on societal values, aspirations,and goals. The call for greater inclusion of multi-ple disciplines and points of view, however, mustnot obscure the fact that science and scientificmethods are critical to sound environmental deci-sion-making. Science has a unique and critical roleto play in protection of the environment. It isthrough scientific investigations that many envi-ronmental problems are first discovered; e.g., thedepletion of stratospheric ozone, hypothesesabout environmental endocrine disruptors. Sciencealso is instrumental in developing, testing, andevaluating risk reduction options, and socialsciences offer techniques for assessing societalpreferences and wants. Implementation of theIED Framework, particularly its application tomultiple-stressor problems, will in many casesrequire new science, both empirical and theoreti-cal. In order to gain the greatest benefit from this next step in environmental decision-making,the Agency will need to invest in researchdesigned to support integrated risk assessment and management.
4. RECOMMENDATIONS TO THE AGENCY
38
Recommendations to Foster Integrated Environmental Decision-making
1. EPA should continue development of integrated, outcomes-based environmental protection,while maintaining the safeguards afforded by the current system.
2. Because science plays a critical role in protecting the environment, EPA should commit theresources necessary to expand the scientific foundation for integrated decision-making andoutcomes-based environmental management.
3. EPA should apply and encourage the broader use of risk comparison methodologies that clearlyidentify how scientific information and judgment are incorporated into risk comparisons.
4. EPA should explore a broader range of risk reduction options in combination to manageenvironmental risks.
5. When evaluating risk reduction options, EPA should strive to weigh the full range of advantagesand disadvantages, both those measured in dollars as costs and benefits and those for which theremay not be a comprehensive dollar measure, such as sustainability and equity.
6. EPA should seek and develop methods to characterize public values and incorporate those valuesinto goal-setting and decision-making.
7. EPA, by itself and in concert with others, should identify, collect, and disseminate scientifically-based environmental metrics organized in new ways to support a more integrated approach tomanaging environmental risk.
8. EPA, by itself and in concert with others, should develop a system of “report cards” to organizeand disseminate information on the status of ecological and human health and the quality of lifein order to assess the effectiveness of its environmental decisions and to guide futureenvironmental management.
9. EPA should expand and develop new collaborative working relationships with other federal andnon-federal governmental agencies and others who also will be involved in integratedenvironmental decision-making.
10. EPA should explore options for reducing risks from significant stressors that currently areaddressed inadequately by the nation’s environmental institutions.
39
Recommendation 3: EPA should applyand encourage the broader use of riskcomparison methodologies that clearlyidentify how scientific information andjudgment are incorporated into riskcomparisons.
Scientific information on risk, such as quanti-fied risk assessments and scientifically demon-strated relationships between stressors and effects,provides an essential basis for making objectiverisk comparisons. However, desired scientificinformation often is incomplete or absent, andscientists have to use their best professional judg-ment to bridge important gaps in the data.Further, scientific information and analyses, bythemselves, are not sufficient for comparing unlikerisks; e.g., when comparing human healths risksvs. ecosystem risks or cancer risk in adults vs.neurologic risk to children. Thus, public valuesalso must come into play in making the compar-isons. For these reasons, methodologies used tocompare environmental risks—whether humanhealth risks, ecosystem risks, or both—shouldincorporate not only the most up-to-date scien-tific information, but also should identify explic-itly where professional judgment and values haveinfluenced the results. In the course of this projectthe SAB developed, to varying extents, two proto-type methodologies for risk ratings by technicalexperts. These and similar approaches should befurther explored as possible points of departurefor the Agency as it seeks to develop and usemethods for comparing relative environmentalrisks that incorporate public values.
Recommendation 4: EPA should explore abroader range of risk reduction options incombination to manage environmental risks.
In 1990, the SAB recommended in ReducingRisk that the nation make greater use of all the
tools, including market forces, information, andproduct specifications, available to reduce risk.Now, almost a decade later, many of those toolsare being used to a greater extent than ever before.The challenge for EPA is not only to expand theuse of those various tools, but also to use them increative, coordinated ways to reduce multiple risksto multiple receptors in communities and ecosys-tems across the country. In the course of this proj-ect, the SAB described a prototype methodologyfor identifying and selecting risk reduction optionsto address single or multiple stressors. Thismethodology should be evaluated by the Agencyas a possible approach for better identifying multi-dimensional strategies to control complex envi-ronmental problems that involve many sources,stressors, and receptors.
Recommendation 5: When evaluating riskreduction options, EPA should strive toweigh the full range of advantages anddisadvantages, both those measured indollars as costs and benefits and those forwhich there may not be a comprehensivedollar measure, such as sustainability and equity.
The benefit-cost paradigm that underlies manyenvironmental decisions is the simple formulationof whether the gains that accrue from protectiveactions are worth what is given up to attain them.There is a subsidiary question: Would other possi-ble actions be preferable? This question highlightsthe importance of taking all effects, includinglong-term effects, into account. It also raises thesometimes hidden issue of how people valuedifferent aspects of the environment. Some ofsociety’s environmental values can be measureddirectly in monetizable terms, and others can beinferred and translated into monetizable termswith some confidence. But other things thatpeople value, such as sustainability and equity,often may be expressed only qualitatively, yet are
of no less importance. As a part of this project, theSAB has described the applicability and limita-tions of the benefit-cost approach, and suggestedareas where new approaches to characterizingvalues are essential. The SAB also has attempted todefine better the full range of relevant questionsthat must be considered in environmental valuation.
Recommendation 6: EPA should seek anddevelop methods to characterize publicvalues and incorporate those values intogoal-setting and decision-making.
Community and national values have been andwill continue to be a primary driver of commu-nity-level and national efforts to protect the envi-ronment. However, values usually are not weighedtransparently in the decision-making process.Rather, they usually are implicit in the judgmentsmade by decision-makers. Thus, they often influ-ence decisions in ways that are not clear to, orreviewable by, the public.
Because public values undoubtedly help shapeenvironmental decisions, it is important to under-stand and document their role in and influence ondecision-making. It is also important to elicitpublic values systematically, differentiate valuesfrom technical information as a part of decision-making, and include their effects on decisions aspart of the public record. In this way, value judg-ments will be neither disregarded nor disguised.Individual and community values should besolicited systematically by social scientists andother appropriately trained individuals. The delib-erative processes that are used in arriving at deci-sions should involve professionals trained in fieldslike consensus-building and dispute resolution.EPA should make more extensive use of experts inthe areas of behavioral and decision science so thata more complete representation of communityvalues is incorporated into the Agency’s decisions.Research likely will be necessary to developimproved methods for helping people develop
considered values and for eliciting and usingpublic values in decision-making.
Recommendation 7: EPA, by itself and in concert with others, should collect and disseminate scientifically basedenvironmental metrics to support a moreintegrated approach to managingenvironmental risks.
The transition to and effectiveness of integratedenvironmental protection will depend to a largeextent on the availability and utilization of appro-priate information in the areas of exposure, humanhealth, ecological health, and quality of life.Current information collection efforts, however,often are insufficient, inadequately organized, orfocused on inappropriate endpoints. In the area ofhuman health, for example, data on non-fataloutcomes of environmental exposures, such asasthma, are not being collected except in a rudi-mentary manner or at certain sites. Only mortalityinformation is collected systematically at all loca-tions and can be related to some limited informa-tion on selected exposures. With regard toecological data, more comprehensive informationis needed on the current status of ecosystems suchas wetlands, lakes, forests, and grasslands. Thisinformation should include the extent to whicheach ecosystem is exposed to and affected by non-chemical stressors such as habitat conversion, habi-tat fragmentation, and invasions of exotic species.
The challenge facing EPA and the nation is notonly one of collecting the right environmentaldata, but of finding new ways to manage and usethose data. Working with other federal and stateagencies, EPA should take a leadership role inidentifying critical environmental data gaps,including data on exposures and health andecological outcomes; integrating the largely frag-mented data collection efforts already underway;and disseminating integrated environmental infor-mation to decision-makers and the public.
40
41
Recommendation 8: EPA, by itself and inconcert with others, should develop asystem of “report cards” to organize anddisseminate information on the status ofecological and human health and thequality of life in order to assess theeffectiveness of its environmental decisionsand to guide future environmentalmanagement.
One of the most valuable uses of environmen-tal data is to measure the results of the actionssociety takes to reduce environmental risk.However, even if such data did exist, through avigorous implementation of Recommendation 7,there would still be a need to develop widelyaccepted and commonly used methods for evalu-ating a) the state of our environment and b) thesuccess of national environmental protectionefforts — where, over the long term, success ismeasured in terms of demonstrable outcomes inthe health of humans and ecosystems.
Reporting on measures of progress towardsecological, human health, and quality of life goalswill provide a number of benefits. First, the exer-cise of defining performance measures and report-ing on them will bring more focus and disciplineto the Agency by expressing the relationshipbetween investments (measured in time, money, orinformation) and environmental results. Second,shorter term measures of progress (includingprocess measures or measures of stressor reduc-tions) will be useful for accountability and coursecorrection. Third, longer term measures ofprogress (such as improvements in overall humanand ecosystem health and quality of life) will bemost helpful in determining whether goals arebeing met, whether further actions are needed tocontrol well-recognized stressors, or whether newactions are needed to control new stressors.
To strengthen their credibility and utility, envi-ronmental report cards should contain informa-
tion derived from objective measurements, betransparent and clearly documented, and provideintegrated information on progress towards multi-ple inter-related environmental goals.
Recommendation 9: EPA should expandand develop new collaborative workingrelationships with other federal and non-federal governmental agencies and otherswho also will be involved in integratedenvironmental decision-making.
Inherent in integrated decision-making is theidea that problem formulation and decision-making must match in scale and location. In somecases, decisions may be most effective when localor state governments play the primary role. Inothers, coordinated action across several levels ofgovernment or among a number of state and/orlocal governments will be required. In still others,national decision-making may be the preferredapproach. In any event, integrated thinking aboutenvironmental problems will tend to drive deci-sion-making to the agency or level of governmentwhere decisions are most appropriately made. Asa consequence, EPA’s role will evolve to one inwhich the depth of control gives way to broaderinvolvement in partnership with other agencies.EPA will continue to be responsible for imple-menting and enforcing federal environmental lawsand statutes, conducting environmental researchand development, and conducting stressor-specificrisk assessments. At the same time, the Agencycan exert national leadership to bring togetherappropriate agencies and stakeholders to exploreintegrated approaches to environmental problems.
Recommendation 10: EPA should exploreoptions for reducing risks from significantstressors that currently are addressedinadequately by the nation’senvironmental institutions.
Over the course of the Integrated Risk Project,it became clear to the SAB that a number ofimportant human health and ecological risks arenot being addressed adequately by the nation’senvironmental institutions. In many cases this isbecause risk management responsibility is notclearly assigned to any one government entity, oris scattered over many agencies and/or levels orgovernment. This fragmented approach results inuncoordinated and incomplete efforts to identifycause-and-effect linkages and to manage risks.With regard to ecological risks, the SAB hasconcluded that many of the highest ranking risks(e.g., hydrologic alterations, harvesting of livingmarine resources, habitat conversion, climatechange, and introduction of exotic species) areassociated with physical and biological, ratherthan chemical, stressors, which do not fall clearly
within the purview of any single federal Agency.Important human health risks that remain unad-dressed include those for which the environmentalexposure link is suspected but not certain (e.g.,asthma, brain cancer, and non-Hodgkinslymphoma) and risks associated with susceptibleand/or compromised human populations.
To control many of these inadequatelyaddressed risks will require the kind of integrateddecision-making envisioned in this report. It willalso require a new kind of integrated institutionalleadership. When EPA determines that seriousrisks are not being addressed effectively by exist-ing environmental institutions or decision-makingsystems, the Agency has a responsibility to informthe public about those risks and bring together theappropriate federal, state, and local agencies toaddress them.
42
43
The Integrated Risk Project (IRP) has beenunlike any other project undertaken by the SAB.Conceived by some as a simple updating to the1990 Reducing Risk report, it soon became evidentthat the IRP was far more complex and far moredifficult than earlier undertakings. Issues of science(e.g., risk assessment, cumulative risk, and econom-ics) merged with even less tractable — butarguably even more important — issues of socialsciences and ethics (e.g., personal and group choice,trans-generational equity, and the role of govern-ment vis a vis the public). More than 50 partici-pants from various fields of science and socialscience worked together to share their respectiveexperiences and insights in a collaborative effort toilluminate a path that would lead to “the answer”.In the process, the participants gained a new appre-ciation for the benefits from and the barriers tointerdisciplinary efforts of this sort. Differences inbackground, nomenclature, and academic cultureof the participants both enhanced and hinderedprogress.
Recent advances in science and technologylegitimize asking some of the “tough questions”that were necessarily simplified just a few yearsago. Questions of total cumulative risk of multipleendpoints, from multiple pollutants, from multiplesources, by multiple routes of exposure are nowbeing confronted directly. More powerful methodsof data collection and analysis, more comprehen-sive models, and broader consideration of manage-ment options are evidence of the growing scientificsupport for a more integrated approach to environ-mental decision-making. Inclusion of interestedand affected parties in addressing environmental
problems, the emphasis on an iterative analytical/deliberative process, and continual grappling withthe ethical aspects of decision-making (e.g., themeanings of “values”, the uneven distribution ofbenefits and costs, and explicit consideration ofeffects on future generations) point to a recogni-tion of the fact that today’s problems includeimportant aspects that must be addressed by socialsciences and ethics.
It is important to recognize that successfulapplication of the IED Framework will requiresome adjustments in the manner and degree towhich the many participants interact over thecourse of the decision-making process. Integratedenvironmental decision-making requires the shar-ing of information, ideas, approaches, and manage-ment deliberations to a degree now seldompracticed among individuals of very different back-grounds. Although this sharing is a positive aspectof the Framework, it may require significant adap-tation on the part of individual policy-makers andinstitutions. For example, decision-makers willneed to interact more extensively with scientificand technical analysts and the public in the courseof developing integrated approaches to environ-mental risks. Likewise, scientific and technicalexperts will need to recognize the role (and limita-tions) of science in decision-making, and also torecognize the legitimate role of values in establish-ing environmental goals and selecting managementapproaches. This culture change will be assisted byfamiliarity and experience with integrated environ-mental decision-making. Experience, combinedwith discipline, will also be needed to apply theFramework with discretion to the depth of detail
5. LESSONS LEARNED
44
necessary for a particular problem, and no more;i.e., the Framework should not become a barrier todecision-making.
Additional challenges likely to be encounteredwhen applying the Framework include the follow-ing:
a) Problems of understanding arising from differ-ences in terminology and outlook imbedded inthe different disciplines and backgrounds ofthe participants in the process;
b) Difficulties of using both qualitative and quan-titative measures concurrently in the decisionprocess;
c) The need to compare different types of risks(e.g., health, ecological and quality of life risks)within a common decision framework and todiscern and define the inter-relationshipsamong risks so as to define common goalsacross the different risk types; and
d) Time-lags between implementation of riskreduction plans and the detection of results
and effects, which make the selection of appro-priate performance measures particularlyimportant.
In some ways, the Board has been true to theold adage that “One’s goal should exceed one’sgrasp.” That is, the goal of this project — to artic-ulate a complete and rational method for includingall aspects of integrated environmental decision-making in a single process — has exceeded theSAB’s grasp. In fact, it is likely that that goal willnever be reached to everyone’s satisfaction. Andyet, the Framework for Integrated EnvironmentalDecision-making that was developed during thecourse of the project clearly points to the directionin which “the next step” of environmental deci-sion-making should go. The efforts of the individ-ual Subcommittees can be examined for furtherinsights about how the Agency might — or mightnot — make additional progress in that direction.In any event, there is enough direction and morethan enough challenge in this report to keep theAgency — and others interested in the next stepof environmental decision-making — productivelyactive for some time to come.
45
Chess, C., T. Dietz, and M. Shannon. 1998. “WhoShould Deliberate When?” Human EcologyReview 5(1): 45-48.
Morgenstern, R. D. (Ed.). 1997. EconomicAnalyses at EPA: Assessing Regulatory Impact.Resources for the Future, Washington, D.C.
National Academy of Public Administration.1995. Setting Priorities, Getting Results: ANew Direction for EPA.
National Center for Health Statistics. 1999.Healthy People 2000 Review, 1998-1999.Hyattsville, Maryland: Public Health Service.
National Research Council. 1983. RiskAssessment in the Federal Government:Managing the Process. National AcademyPress, Washington, DC.
National Research Council. 1994. Science andJudgment in Risk Assessment. NationalAcademy Press, Washington, DC.
National Research Council. 1996. Stern, P.C. andH.V. Fineberg (Ed.s). Understanding Risk:Informing Decisions in a Democratic Society.National Academy Press, Washington, DC.
Organization for Economic Co-operation andDevelopment (OECD). 1998. TowardsSustainable Development -EnvironmentalIndicators. 130p.
Pontius,W. 1999. Negotiating Regulations forMicroorganisms and Disinfection By-Products.Journal of American Water Works Association91:14.
Presidential/Congressional Commission on RiskAssessment and Risk Management. 1997. RiskAssessment and Risk Management inRegulatory Decision-Making.
Sandman, Peter M. 1990. Getting to Maybe: SomeCommunications Aspects of Siting HazardousWaste Facilities. In: Readings in Risk. T.S.Glickman and M. Gough, (Ed.s). Resources forthe Future. Washington, DC. Pp. 233-245.
Science Advisory Board. 1990. Reducing Risk:Setting Priorities and Strategies forEnvironmental Protection (EPA-SAB-EC-90-021).
Science Advisory Board. 1995. Beyond theHorizon: Using Foresight to Protect theEnvironmental Future (EPA-SAB-EC-95-007).
Science Advisory Board. 1997. Advisory on theProblem Formulation Phase of EPA’sWatershed Ecological Risk Assessment CaseStudies (EPA-SAB-EPEC-ADV-97-001).
U.S. Environmental Protection Agency. 1987.Unfinished Business: A ComparativeAssessment of Environmental Problems.Washington, DC.
6. REFERENCES CITED
46
U.S. Environmental Protection Agency. 1982.Framework for Ecological Risk Assessment(EPA/630/R-92/001). Office of Research andDevelopment, Washington, DC.
U.S. Environmental Protection Agency. 1989.Risk Assessment Guidance for Superfund, Vols.1 and 2 , Interim Final (EPA/540/1-89/001 and002).
U.S. Environmental Protection Agency. 1990.Methodology for Assessing Health RisksAssociated with Indirect Exposure toCombustor Emissions—Interim Final(EPA/600/6-90/003).
U.S. Environmental Protection Agency. 1993a.Draft Addendum to the Indirect ExposureDocument.
U.S. Environmental Protection Agency. 1993b. AGuidebook to Comparing Risks and SettingEnvironmental Priorities (EPA 230-B-93-003).Office of Policy, Planning, and Evaluation,Washington, DC.
U.S. Environmental Protection Agency. 1997a.Guidance on Cumulative Risk Assessment,Part 1: Planning and Scoping. July 3, 1997.
U.S. Environmental Protection Agency. 1997b.EPA Strategic Plan (EPA/190-R-97-002).Office of the Chief Financial Officer,Washington, DC.
U.S. Environmental Protection Agency. 1998.Guidelines for Ecological Risk Assessment(EPA/630/R-95/002Fa).
U.S. Environmental Protection Agency. 1999.Reinventing Environmental Protection: 1998Annual Report (EPA100-R-99-002).
47
48
C4
United StatesEnvironmental ProtectionAgency(1400A)Washington, DC 20460-0001
Official BusinessPenalty for Private Use $300
