Enhanced Geospatial Data for Meta-Analysis and Environmental Benefit Transfer:

An Application to Water Quality Improvements

Meta-Analysis of Economics Research Network, MAER-Net 2014 Athens Colloquium. Athens, Greece, September 11-13.

The findings, conclusions, and views expressed here are those of the author[s] and do not necessarily reflect those of the U.S. EPA. No official Agency endorsement should be inferred.

Robert J. JohnstonClark University, USA

Elena BesedinAbt Associates, Inc.

Ryan StaplerAbt Associates, Inc.

Benefit Transfer and Meta-Regression Analysis

Benefit Transfer (BT) uses economic value estimates from existing primary research to approximate the value of a similar but separate change in a different resource or site.

Benefit function transfer (BFT) allows transferred estimates to be adjusted for attributes of the policy context and can improve transfer performance.

Meta-regression analysis is increasingly used to estimate umbrella benefit functions that synthesize information from multiple primary studies.

The goal is to predict a benefit measure for unstudied policy sites, not to estimate a treatment effect.

Benefit Transfer and Meta-Regression Analysis

Dependent variable in a Bayesian or classical meta- regression model (MRM) is a summary welfare statistic drawn from comparable primary studies—often a willingness to pay (WTP) estimate.

Independent variables characterize policy, site and population attributes hypothesized to explain variation across observations.

Estimated functions are used to predict WTP values for unstudied policy sites, in principle improving transfer accuracy relative to unit value BT and single site BFTs.

Core Geospatial Factors in Meta-Analysis

Valuation MRMs often give significant attention to the measurement of resource type and quality (e.g., the magnitude of water quality change).

Theory suggests that mean WTP estimates should be also be sensitive to other core factors including: geospatial scale (the geographical size of affected

environmental resources or areas), market extent (the size of the market area over which WTP

is estimated), and substitute availability (the availability of proximate,

unaffected substitutes). No MRMs in the current valuation literature enable

simultaneous adjustments for these factors.

Geospatial Scale (Size of Affected Areas)

MRMs almost universally apply simplistic approaches to characterize value patterns associated with spatial scale.

Common examples include the use of categorical variables to distinguish size categories of affected resources or areas (e.g., “large” versus “small” wetlands; “national” versus “local” improvements).

A few MRMs incorporate explicit measures of site area (Brander et al. 2007; Londoño and Johnston 2012; Ghermandi et al. 2010).

No published MRMs incorporate explicit, quantitative measures of both a change in scope (the magnitude of a resource quality change) and geospatial scale (the size of the area over which the quality change occurs).

Market Area and Substitutes

All else equal, larger sampled market areas (e.g., states versus communities) are associated with smaller mean per household WTP estimates (Johnston and Duke 2009). This is consistent with commonly observed WTP

distance-decay (Bateman et al. 2006). WTP is also expected to vary (inversely) with the quantity

of unaffected substitute resources in close proximity (Schaafsma et al. 2012).

Moderator variables quantifying these factors are almost universally omitted from valuation MRMs.

At most, models include broad proxies for substitute availability (e.g., Ghermandi et al. 2010).

The Present Analysis

These limitations hamper the ability of MRMs to support BT without strong, ad hoc assumptions.

The present analysis develops a MRM for US water quality benefit transfer that incorporates quantitative measures of spatially explicit factors predicted by theory to influence WTP.

The metadata combine primary study information with extensive geospatial data from geographic information system (GIS) data layers and other external sources.

The result is the first meta-analytic benefit function able to adjust for simultaneous variations in geospatial scale, market extent and spatially differentiated substitutes.

The Metadata

Extend and update metadata of Johnston et al. (2005). Observations were identified and added to the metadata

following the guidelines of Stanley et al. (2013) for research identification and coding.

Drawn from studies that estimate WTP for water quality changes in US water bodies that support ecosystem services including aquatic life, recreational use, and nonuse values.

Include studies that estimate total (use & nonuse) values, use generally accepted stated preference methods, and estimate comparable Hicksian welfare measures.

Include 143 observations from 52 stated preference studies conducted between 1981 and 2011.

The Metadata

Primary changes to Johnston et al. (2005) metadata: deletion of studies conducted prior to 1980 and others

that did not meet updated screening criteria; the addition of 21 new studies; development of new spatially-explicit variables.

All monetary values adjusted to 2007 US dollars. Comparable standard errors or t-statistics on WTP

estimates not available (common in valuation metadata). Supplementary geospatial data drawn from sources

including National Hydrography Dataset, Hydrologic Unit Code Watershed Boundary Dataset, and National Land Cover Database.

Core Geospatial Variables

Index of geospatial scale and market extent: ln_rel_size = log of (total affected shoreline [km] divided by total sampled market area [km2]).

Additional scale for rivers: riv_flow = water flow through affected river reaches, in cubic feet per second, based on data from the National Hydrography Dataset.

sub_frac = The proportion of water bodies of the same hydrological type affected by the water quality change, within affected state(s). For rivers and bays measured as proportional shoreline. For lakes measured as proportional surface area.

A Few Other Key Variables

lnquality_ch: natural log of the change in mean water quality valued by the study, specified on the 100-point water quality index

ln_ar_agr = log of (proportion of affected area designated as agricultural)

ln_income = log of (median per household income in the sampled area).

nonusers = binary variable identifying surveys targeting only nonusers or analyses over nonuser subsamples.

non_reviewed = Binary (dummy) variable indicating that the study was not published in a peer-reviewed journal (default is studies published in peer reviewed journals).

The Meta-Regression Model

Dependent variable: natural log of household WTP for water quality improvements measured on standard 100 point water quality index (McClelland 1974).

27 independent variables characterizing: (1) study methodology, (2) populations, (3) market areas and study sites, (4) water bodies and (5) water quality change.

Multilevel regression model, robust variance estimation, non-weighted, translog functional form.

Wald χ2 = 8112.91, p<0.0001; R2=0.63. 20 coefficients statistically significant at p<0.10 or better. Outperforms restricted model that omits core geospatial

variables (χ2 = 22.15, df 3, p<0.0001).

Results for Selected Variables

Variable Coefficient Estimates(Standard Errors)

ln_rel_size 0.050  (0.022)**sub_frac 0.545  (0.218)**riv_flow 4.28e-06   (1.66e-06)**lnquality_ch 0.284  (0.107)***lnbase -0.078  (0.153)

Results for Selected Variables

VariableUnrestricted Model

Coefficient Estimates(Standard Errors)

lnyear -0.458  (0.094)***non_reviewed -0.817  (0.196)***nonusers -0.455  (0.115)***lnincome 0.774  (0.404)*ln_ar_agr -0.402  (0.097)***

Findings

In addition to the core effects of water quality change on WTP, per household WTP increases: With the size of affected water bodies relative to the

size of the surveyed market area. When a larger proportion of regional water bodies (of

the same type) are affected by the proposed policy. With the flow of water through affected river reaches.

All results are intuitive and supported by theory. No published MRM enables adjustments for these

simultaneous effects (e.g., for BT). Associated WTP variations are inelastic.

Implications for Benefit Transfer

Past BTs using MRM results have had to apply strong and often ad hoc assumptions when calculating and aggregating WTP over policy and study areas.

Assumptions were required to account for counter-intuitive invariance of WTP to core geospatial factors.

What type of errors might result? To illustrate BT implications, we project per household

WTP for illustrative water quality improvements, within policy sites that differ in geospatial scale, market extent and substitute availability.

BT results forecast using both restricted and unrestricted MRM results.

Illustrative Benefit Transfer Scenario

Water quality improvement equal to the mean over the metadata (lnquality_ch=2.909); equivalent to a change of 18.335 on 100-point WQI.

Beginning from a mean baseline of lnbase=3.596 (36.440 on the WQI).

Assume a forecast of annual mean WTP per household (lump_sum=0; wtp_median=0; volunt=0), over a general population sample (nonusers=0) in the mid-Atlantic region (ma=1; mp=se=sw=mw=0), for a water quality improvement in a single river (river=1; mult_bod=0).

All other variables are held at mean values from the metadata.

Illustrative Benefit Transfer Scenario

WTP forecast for three scenarios: (1) mean, minimum and maximum values of three core geospatial variables (ln_rel_size, sub_frac, riv_flow).

Water quality improvement equal to the mean over the metadata (lnquality_ch=2.909); equivalent to a change of 18.335 on 100-point WQI.

Baseline of lnbase=3.596 (36.440 on the WQI). Assume forecast of annual mean WTP per household

(lump_sum=0; wtp_median=0; volunt=0), over general population sample (nonusers=0) in mid-Atlantic region (ma=1; mp=se=sw=mw=0), for improvement to a single river (river=1; mult_bod=0).

All other variables are held at mean values.

Illustrative Benefit Transfer ScenarioVariable Assigned Variable Values

Scenario 1(Mean of Sensitivity Analysis Variables)

Scenario 2(Min. of Sensitivity Analysis Variables)

Scenario 3(Max. of Sensitivity Analysis Variables)

ln_rel_size -1.292 -11.135 7.811sub_frac 0.188 0.0003 1.000riv_flow 4328.008 17.6976 152198.700       WTP Est.: Unrestricted Model

$50.76 

$27.49 $234.50

WTP Est.: Restricted Model

$56.31 $56.31 $56.31

Effects on Benefit Transfer

Results suggest that researchers should exercise caution when conducting BT that does not account for geospatial scale, extent of the market, and substitute availability.

From the unrestricted model (holding all else constant): Compared to Scenario 1 (mean), WTP under Scenario 2

(min) declines by 46% ($50.76 versus $27.49). Scenario 3 (max) projects a WTP estimate of $234.50—a

362% increase over Scenario 1. Comparing WTP under Scenarios 2 and 3 reveals an even

larger difference of 753% ($27.49 vs $234.50). The restricted model generates an identical WTP estimate

of $56.31 for all three scenarios, illustrating an invariance to geospatial factors common in published MRMs.

Conclusions

To the knowledge of the authors, this is the first MRM that accounts for simultaneous effects of geospatial scale, market extent and substitute availability on WTP.

Results indicate statistically significant and policy relevant impacts of these geospatial factors.

Effects of these variables alone can lead to a nearly eight-fold difference in BT estimates.

Results highlight the potential hazards in valuation MRMs that overlook the potential influence of (often study-invariant and unreported) geospatial factors.

Additional research is required to evaluate whether similar MRM specifications and findings are applicable to other types of environmental changes and policy contexts.

Questions or Comments?

Robert J. JohnstonDirector, George Perkins Marsh InstituteProfessor, Department of EconomicsClark University950 Main St.Worcester, MA 01610Phone: (508) 751-4619Email: rjohnston@clarku.edu