Correction

ENVIRONMENTAL SCIENCESCorrection for “Assessment of the Legionnaires’ disease out-break in Flint, Michigan,” by Sammy Zahran, Shawn P. McElmurry,Paul E. Kilgore, David Mushinski, Jack Press, Nancy G. Love,Richard C. Sadler, and Michele S. Swanson, which was firstpublished February 5, 2018; 10.1073/pnas.1718679115 (Proc NatlAcad Sci USA 115:E1730–E1739).The authors wish to note, “While we declare no conflicts of

interest, we would like to amend our original disclosure state-ment in the interest of full transparency. We wish to disclose thatDrs. McElmurry and Kilgore were served subpoenas by the FlintSpecial Prosecutor, Todd Flood, to testify, under oath, at inves-tigatory proceedings and at preliminary examinations for Mr. LyonsandDr.Wells. Mr. Lyons is the Director of theMichigan Departmentof Health and Human Services (MDHHS), and Dr. Wells isChief Medical Executive for the MDHHS. MDHHS providedfunding for this work.”

Published under the PNAS license.

Published online June 11, 2018.

www.pnas.org/cgi/doi/10.1073/pnas.1808389115

www.pnas.org PNAS | June 19, 2018 | vol. 115 | no. 25 | E5835

CORR

ECTION

Dow

nloa

ded

by g

uest

on

June

6, 2

021

Dow

nloa

ded

by g

uest

on

June

6, 2

021

Dow

nloa

ded

by g

uest

on

June

6, 2

021

Dow

nloa

ded

by g

uest

on

June

6, 2

021

Dow

nloa

ded

by g

uest

on

June

6, 2

021

Dow

nloa

ded

by g

uest

on

June

6, 2

021

Dow

nloa

ded

by g

uest

on

June

6, 2

021

Dow

nloa

ded

by g

uest

on

June

6, 2

021

Dow

nloa

ded

by g

uest

on

June

6, 2

021

Dow

nloa

ded

by g

uest

on

June

6, 2

021

Dow

nloa

ded

by g

uest

on

June

6, 2

021

Dow

nloa

ded

by g

uest

on

June

6, 2

021

http://www.pnas.org/site/aboutpnas/licenses.xhtmlwww.pnas.org/cgi/doi/10.1073/pnas.1808389115

Assessment of the Legionnaires’ disease outbreak inFlint, MichiganSammy Zahrana,b, Shawn P. McElmurryc, Paul E. Kilgored, David Mushinskia, Jack Pressc, Nancy G. Lovee,Richard C. Sadlerf, and Michele S. Swansong,1

aDepartment of Economics, Colorado State University, Fort Collins, CO 80523; bDepartment of Epidemiology, Colorado School of Public Health, Fort Collins,CO 80523; cDepartment of Civil & Environmental Engineering, Wayne State University, Detroit, MI 48202; dDepartment of Pharmacy Practice, Wayne StateUniversity, Detroit, MI 48201; eDepartment of Civil & Environmental Engineering, University of Michigan, Ann Arbor, MI 48109; fDepartment of FamilyMedicine, Michigan State University, Flint, MI 48502; and gDepartment of Microbiology and Immunology, University of Michigan, Ann Arbor, MI 48109

Edited by Andrea Rinaldo, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, and approved January 5, 2018 (received for review October27, 2017)

The 2014–2015 Legionnaires’ disease (LD) outbreak in GeneseeCounty, MI, and the outbreak resolution in 2016 coincided withchanges in the source of drinking water to Flint’s municipal watersystem. Following the switch in water supply from Detroit to FlintRiver water, the odds of a Flint resident presenting with LD in-creased 6.3-fold (95% CI: 2.5, 14.0). This risk subsided followingboil water advisories, likely due to residents avoiding water, andreturned to historically normal levels with the switch back in watersupply. During the crisis, as the concentration of free chlorine inwater delivered to Flint residents decreased, their risk of acquiringLD increased. When the average weekly chlorine level in a censustract was

water temperature, corrosion rate(s), and pipe wall effects. Ultimately,the loss of free chlorine in a distribution system is complex, andthe myriad of factors leading to it are commonly referred to,collectively, as chlorine demand. With limited historical dataavailable, it is impossible to identify which constituents causedchlorine demand in the Flint system or had a direct effect onbiofilm and legionellae growth. Because some of these poten-tially confounding water quality and water system variablespromote the growth of legionellae, the influence of free chlorinealone on LD would likely be underestimated. Therefore, althoughthe literature indicates insufficient chlorine is a contributing factorto LD outbreaks (6–11), we do not attempt to link the lack ofchlorine residual as the sole mechanistic cause of LD. Instead, herewe use free chlorine concentration as an indicator of the potentialfor legionellae growth. Utilities are required by law to measure

disinfectant residual, which is an easily measured value; there-fore, we posit that disinfectant residual concentration may be auseful surrogate for indicating LD risk.Before the switch in Flint’s water source, the concentration of

free chlorine (mg/L as Cl2) across eight water monitoring loca-tions in the city was similar (Fig. 1B), as demonstrated by thestrong between-monitor correlation in free chlorine (r > 0.70; TableS1). Within weeks of the switch, significant fluctuations in theconcentration of free chlorine were observed both between moni-tors (spatial variation) and at each monitor over time (temporalvariation), with the mean between-monitor correlation falling by∼30% and the average within monitor SD increasing from 0.200 to0.416 mg/L as Cl2. For example, throughout the postswitch period, asustained collapse of free chlorine below 0.5 mg/L was observed atmonitoring location 6 for all but a few weeks, the chlorine residual

Fig. 1. Spike in LD cases coincident with switch in water supply and increased variation observed in the Flint water distribution system. (A) Quarterly LDincidence in Genesee County, MI, 2010 through 2016. The count of LD cases in Genesee County as compiled in the Michigan Disease and Surveillance System atthe quarterly time step. Bars in gray correspond to the preswitch period, bars in maroon correspond to the postswitch period, and bars in navy correspond tothe switch back period. (B) Free chlorine at eight monitoring locations in Flint’s water distribution system, 2013–2016. Free chlorine (mg/L as Cl2) was reportedweekly during the three water regime phases defined above (vertical lines) and the periods and dates (year/week) shown at eight locations in Flint.

Zahran et al. PNAS | Published online February 5, 2018 | E1731

ENVIRONMEN

TAL

SCIENCE

SPN

ASPL

US

http://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplemental

measured at monitoring location 8 was persistently >1.5 mg/L, andthat at monitoring location 7 varied greatly. With the switch back toDWSD-supplied water, the extreme variation in free chlorine be-tween locations subsided, with the exception of location 6.During the period that treated Flint River water was distrib-

uted to Flint residents, poor water quality and extended periodsof low chlorine residual may have enabled legionellae growth inthe distribution system (4, 12). Residual chlorine is maintained inwater distribution systems to inhibit the growth of pathogens,including L. pneumophila (6, 7). Free-living L. pneumophila areinactivated within 15 min of exposure to 0.4 mg Cl2/L (8).However, this pathogen also resides in biofilms attached to pipewalls and replicates within predatory free-living protozoa (13,14), two habitats that require higher doses of chlorine to killlegionellae (9–11). To reduce the risk associated with bacterialgrowth in water distribution systems, regulatory agencies rec-ommend a minimum free chlorine residual of 0.2–0.5 mg/L (15–17). The effectiveness of chlorine disinfection depends on systemconditions and chemistry; for example, iron and assimilable or-ganic carbon can both consume chlorine and support L. pneu-mophila growth (18). However, because chlorine residual is oneof the most common measurements of water conditions withindistribution systems, here we exploit the chlorine residual valuesreported in Flint from 2013 to 2016 to investigate how theselevels associated with the occurrence of LD.Analytically, the timing of changes in Flint’s source water and

treatment, the accompanying spatiotemporal variations in freechlorine, and the enhanced level of monitoring allow us to sta-

tistically calculate the effect of water disinfection on LD risk atthe scale of a municipal water system. To evaluate the hypothesisthat changes in Flint’s source water and treatment resulted in theGenesee County LD outbreak, we develop a series of statisticaltests that exploit spatiotemporal details for the complete in-ventory of LD cases that occurred from 2010 to 2016 in Geneseeand neighboring Wayne and Oakland Counties. LD case dataobtained from the Michigan Department of Health and HumanServices (MDHHS) included relevant epidemiological informa-tion on dates of symptom onset and referral to the MichiganDisease Surveillance System, as well as residence of LD cases bycensus tract. In analyses that follow, we construct a series of re-gression models that capture the variation in LD risk attributableto four distinct phases of exposure to water regimes in Flint’smunicipal water system. Using these models, we derive the in-cidence of human LD as a function of residual chlorine concen-tration in a full-scale municipal water distribution system. Weassess the robustness of our models by excluding all likely hospital-acquired LD cases and by ascertaining if the risk of LD in censustracts adjacent to Flint increased as a function of commuter flowinto Flint. Results of this analysis can inform the management ofwater systems dependent on chlorine disinfection.

Water Exposure RiskTo capture the effects of changes in source water and treatment,we constructed a series of difference-in-differences regressionmodels that exploit spatial variation (in Flint versus outside ofFlint) and four distinct phases of water-related LD exposure

Fig. 2. Probability of observing a case of LD in Genesee County during four phases of the Flint water crisis. (A) The four phases of water exposure risk aredefined. Phase A is the period before switch with water supplied by the DWSD from Lake Huron. Phase B is the period after switch to Flint River water treatedby the City of Flint and before water boil advisories. Phase C is the period after switch to treated Flint River water and after boil advisories. Phase D is theperiod after switch back to water derived from Lake Huron. Start and end dates for each phase are indicated. (B) The probability of observing a case of LD inFlint and non-Flint census tracts by phases in the Flint water crisis. The estimated probability (with 95% confidence intervals) of observing a case of LD in acensus tract in each of the four phases of water regime exposure risk in Oakland and Wayne census tracts (control group, non-Flint tracts, navy) and in Flintcensus tracts (treatment group, Flint, maroon) are shown. Estimated probabilities are derived with all other model covariates (i.e., meteorological and de-mographic) fixed at sample means.

E1732 | www.pnas.org/cgi/doi/10.1073/pnas.1718679115 Zahran et al.

www.pnas.org/cgi/doi/10.1073/pnas.1718679115

risk during the Flint water crisis (Fig. 2A and Table S2). Thedifference-in-differences method infers an exposure risk effect bycomparing the difference between periods on the outcome ofinterest (the presence of an LD case in a census tract in a givenweek) for treated census tracts relative to not treated censustracts (Table S2). In the preswitch period, LD risk in Flint andnon-Flint census tracts was similar (Fig. S1). Thus, the difference-in-differences method posits that the LD risk in non-Flint census tractsrepresents what would have occurred in Flint census tracts if not forthe switch in source water and treatment.The association between a census tract in Flint presenting with

an LD case and the switch in water supply from DWSD to theFlint River is measured in weekly periods as an odds ratio (OR).Throughout, the estimated treatment effect is the coefficient ofthe interaction between space and time. In Table 1, models 1, 2,and 3 contrast Flint × postswitch, phase B vs. C vs. A, and phaseD vs. A, respectively, as defined in Fig. 2. In model 1, thepostswitch period combines phases B and C. Other factors heldequal, model 1 of Table 1 shows that the switch in source waterand treatment increased the odds of a census tract in Flint havinga case of LD by factor 7.3 [95% confidence interval (CI): 3.5–15.0]. When the non-Flint Genesee County census tracts areincluded in our control group (Table S3, model 1), the switch inwater regime increased the risk of LD incidence in Flint by 440%(OR = 5.4, 95% CI: 2.7–11.2).Next, we test whether boil water advisories issued by author-

ities in Flint attenuated the risk of LD. After positive tests forEscherichia coli contamination, public boil water advisories in-creased water avoidance by residents across the city (19, 20).Indeed, the odds of an LD case in a Flint census tract increasedby a factor of 10 (OR = 10.0, 95% CI: 4.2–23.8) in the postswitchpreadvisory period compared with a 6 factor increase (OR = 5.9,95% CI: 2.6–13.2) in the postswitch postadvisory period (Table1, model 2). Although the difference in LD risk in Flint between

the preadvisory versus postadvisory periods is epidemiologicallysubstantive, it is not statistically significant.In October 2015, the MDHHS and the Genesee County Health

Department jointly announced a state of emergency and instruc-ted residents to avoid drinking the water. On October 16, Flintreconnected to the DWSD water system. This switch back in watersource and treatment provides another test of the water systemhypothesis. In particular, we compare LD risk in the switch backphase D versus phase A both in Flint (Table 1, model 3) and incontrol neighborhoods of Oakland and Wayne Counties (Table S3,model 3). In both models, the risk of an LD case appearing in aFlint census tract during the switch back period is indistinguishablefrom the preswitch period, indicating that the switch back in watersupply ended the LD outbreak in Flint.The estimated probabilities of observing an LD case in a

census tract inside or outside Flint through the four phases ofwater exposure risk (Fig. 2A) is plotted in Fig. 2B. Before theswitch to the Flint River water source, there is only negligibledifference in the estimated probabilities of LD incidence be-tween Flint and non-Flint census tracts. In contrast, the LD riskin Flint increases significantly in the postswitch water regimeperiod and then lessens somewhat following water advisories.After the switch back to the DWSD water source, the LD risk inFlint returns to the level before the regime switch. These distinctshifts in estimated probabilities in Flint versus non-Flint neighbor-hoods between each water exposure phase support the hypothesisthat changes in Flint’s municipal water system were responsible forthe outbreak and subsidence of LD incidence.

Free ChlorineThe extreme temporal and spatial variation in free chlorine in-duced by the switch in Flint’s water supply (Fig. 1B and Table S1)provides an unprecedented opportunity to analyze the relation-ship between a water quality parameter and LD incidence in a

Table 1. Odds ratios of tract presenting with case of Legionnaires’ disease: water regime exposure effects

Variables

Model 1: phases B and C vs.A non-Flint Geneseecensus tracts excludedfrom control group

Model 2: phase B vs. C vs.A non-Flint Geneseecensus tracts excludedfrom control group

Model 3: phase D vs.A non-Flint Geneseecensus tracts excludedfrom control group

Flint 0.603 0.603 0.584*[0.325, 1.118] [0.325, 1.118] [0.314, 1.085]

Postswitch 0.822*[0.674, 1.002]

Flint × postswitch 7.245***[3.504, 14.979]

Postswitch/preadvise 0.779[0.567, 1.071]

Postswitch/postadvise 0.844[0.670, 1.063]

Flint × postswitch/preadvise 10.007***[4.211, 23.782]

Flint × after switch/postadvise 5.854***[2.595, 13.204]

Switch back 1.381***[1.135, 1.680]

Flint × switch back 0.990[0.310, 3.165]

N 309,192 309,192 277,480Ntracts 991 991 991Log likelihood −3,930.58 −3,929.78 −3,784.70Wald χ2 286.79 293.15 279.08

Notes: 95% confidence intervals in braces, ***P < 0.01, **P < 0.05, *P < 0.1. Models 1 through 3 control for average temperature, average humidity,average precipitation, percent of households in a census tract receiving public assistance, and percent of population ≥50 y of age and include a census tractrandom effect.

Zahran et al. PNAS | Published online February 5, 2018 | E1733

ENVIRONMEN

TAL

SCIENCE

SPN

ASPL

US

http://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplemental

full-scale municipal water distribution system. For this purpose,we develop a monitor-to-parcel assignment algorithm that le-verages best available information on parcel occupancy/vacancy,residence time of water (i.e., water age), and the Flint waterdistribution system pipe network (Fig. S2).Table 2 reports ORs of a census tract in Flint presenting with

an LD case by the estimated chlorine residual in water deliveredto residents. In addition to controlling for demographic andmeteorological factors that influence LD outcomes, this modelcaptures other factors that may contribute to neighborhoodvariation in LD risk, including socioeconomic status and age >50 y.Model 1 shows results where free chlorine is measured as a con-tinuous variable. We find that a unit increase in free chlorine(1 mg/L) reduced the odds (OR = 0.21, 95% CI: 0.07–0.62) of anLD case being reported by about 80%. Models 2 and 3 show resultswhere the concentration of free chlorine is measured categoricallyas

Flint after the city switched to the Flint River as its municipal watersource but not before (Fig. 4). This statistical relationship indicatesthat exposure to Flint water partially accounts for the observed in-crease in LD in neighboring municipalities.

DiscussionThat a sustained and widespread inability to maintain adequatefree chlorine residuals in Flint’s municipal water system wasresponsible for the LD outbreak in Genesee County in 2014 and2015 is supported by this ensemble of causal inference tests,integration of multiple datasets, and repeated substantiation ofhypotheses. The odds of a neighborhood (i.e., census tract) inFlint reporting a case of LD increased by a factor of 7.3 in theperiod after the switch to the Flint River water source (Table 1,model 1a). The relative risk between Flint and non-Flint censustracts in this postswitch period was over 6–1, with an estimated80% of LD cases in Flint attributable to the change in watersource and treatment. When boil water advisories increasedwater avoidance by residents, the odds of an LD case reportingfrom a Flint neighborhood subsided from an OR of 10.0–5.9(Table 1, model 2). The advisories, along with General MotorsCorporation’s statement that the water was too corrosive to useat their engine plant, likely confirmed residents’ suspicions thatthe water was unsafe and resulted in behavior change that re-duced their exposure. Furthermore, the risk of LD returned topre-Flint water crisis levels after the switch back to the LakeHuron water supply (Table 1, model 3).During the period when water was drawn from the Flint River,

the free chlorine residual that associated with mitigation of LDrisk was nearly five times greater than it was before the switch inwater supply (1.4 versus 0.3 mg/L; Fig. 3). This response in ourmodel is indicative of an increase in free chlorine demand that isconsistent with reports during this period of enhanced levels ofiron and assimilable organic matter, both of which promotelegionellae growth and react chemically with free chlorine,thereby reducing its availability for disinfection reactions.Exploiting the extraordinary variation in water quality in the

Flint distribution system, we developed an analysis of human LD

incidence as a function of free chlorine residual in a community-scale distribution system. When water was supplied by the FlintRiver, a 1 mg/L increase in free chlorine reduced the risk of anLD case in a neighborhood by about 80% (Table 2, model 1).Conversely, the odds of an LD case increased by factors of2.9 and 3.9 when the average weekly chlorine levels in a censustract were

municipal water systems similar to that in Flint, increasing theamount of free chlorine residual above trace levels at all points inthe distribution network is likely to reduce LD risk. The optimallevel of chlorine residuals must take into account potentially det-rimental effects, such as formation of disinfection by-products andincreased rates of corrosion. However, our analyses establish thatother things held equal, maintaining disinfectant residual at allpoints within water distribution systems can substantially minimizethe risk of Legionnaires’ disease.

Materials and MethodsData. Deidentified data on LD cases from 2010 to 2016 were obtained fromthe MDHHS by Data Use and Confidentiality Agreements following approvalfrom the Institutional Review Board for the Protection of Human ResearchSubjects (MDHHS IRB 201608-01-EA, Wayne State University IRB 067016B3E).Data represent a complete inventory of LD cases in Genesee, Oakland, andWayne Counties over this 6-y period. Each case is time-stamped with dates ofsymptom onset, patient diagnosis, and referral to the Michigan DiseaseSurveillance System (MDSS). The precise date of referral to the MDSS isavailable for all LD cases. Only 623 of the 833 (25.2% missing) LD cases inGenesee, Oakland, and Wayne Counties have a recorded diagnosis date.Although MDHHS staff generously provided enhanced onset date datacollected from case investigations, only 694 cases (16.7% missing) had averified date of symptom onset. In consultation with scientific personnel atMDHHS, the date of the referral is the most reliable and valid indication oftiming. Median difference in elapsed time between referral and diagnosisdates is 1 d, and 5 d between referral and symptom onset dates. These lagsinform our use of referral timing data in chlorine analyses. Given the veryhigh correlation between referral and onset date (R2 = 0.9973), using referraldate with a time lag adjustment in chlorine models resolves the timing errorand preserves maximum information (limiting themissing information bias thatarises with use of onset date). MDHHS data are also referenced geographicallyby the residence of the LD case at the census tract scale. Of the 833 LD casesobserved over this time period, all but 27 cases included a verifiable address(or census tract residential indicator), including 3 in Genesee, 2 in Oakland, and22 in Wayne County.

To test the water regime hypotheses, we exploited the temporal andspatial properties of MDHHS case data to develop an outcome variable, LDincidence, that is observable in time (before and after the switch in waterregime) and space (in and outside regime and chlorine-treated neighbor-hoods). LD incidence is a binary variable equal to 1 if a confirmed case of LD isobserved in census tract i in week t and 0 otherwise.

To estimate LD effects from the switch in water source and treatment, andthe ensuing variability in chlorine residuals, we also collected a suite ofdemographic and meteorological control variables. With respect to neigh-borhood (census tract) demography, two variables from the US Census Bu-reau are used: percent of population receiving public assistance and percentof population ≥50 y of age. Both socioeconomic status and age (≥50 y) areknown correlates of LD risk. Three county-level meteorological variables areused: average weekly temperature, average weekly humidity, and averageweekly precipitation. The thermal forces of temperature, humidity, andprecipitation are known to govern the observed seasonality of LD incidencerates through growth effects on legionellae bacteria (25–27).

Capturing Water Regime Effects. To capture effects of changes to source waterand treatment, or water regime, we deployed a quasi-experimental methodcalled difference-in-differences. An illustration of the method is summarizedin Table S2. Our first difference is spatial, corresponding to whether a censustract is located inside Flint (F), and therefore treated by the shift in waterregime, or not in Flint (NF). Our second difference is period-based, corre-sponding to whether a census tract is observed before (A) or after (B) theswitch in water regime. The difference-in-differences method infers a causaleffect by comparing the difference between A and B on the outcome ofinterest (LD incidence) for treated census tracts (F) relative to not treatedcensus tracts (NF).

A key assumption of the difference-in-differences method, known as theparallel paths requirement, posits that the average period difference (A − B)in control group (NF) census tracts constitutes the counterfactual averagedifference between A and B in F census tracts if not for the treatment orswitch in water regime. In this analysis, the preperiod parallel paths re-quirement is satisfied (Supporting Information). The differential behavior ofLD incidence rates in post-water regime switch period of Fig. S1 is indicativeof a powerful place-specific period effect. Note in Fig. S1 the extraordinaryincrease in the LD incidence rate in Genesee County. Although it is analyt-ically tempting to conclude that the switch in water regime governs the LDspike in Genesee, our analysis plan aimed to rule out forces coincidental withthe regime switch, evaluate plausible alternative explanations, and identifya potential causal mechanism for the observed increase in LD in GeneseeCounty.

Our analysis begins by identifying regime effects. In the regime effectequations detailed below, our first difference is always geographic, corre-sponding to whether a census tract is located in Flint (and is therefore arecipient of Flint water) or is not in Flint (either located in Oakland,Wayne, orparts of Genesee County not in Flint). Our second difference is a period in-dicator that is variously defined by whether or not parameters from a given

Fig. 4. LD incident risk in non-Flint census tracts by commuter flow to Flint. LD incident risk is shown as a function of the number of commuters from GeneseeCounty locations other than Flint either before (navy) or after (maroon) the switch to the Flint River as the Flint municipal water source. Probabilities areestimated with all other model covariates fixed at their sample means, and bars indicate 95% confidence intervals.

E1736 | www.pnas.org/cgi/doi/10.1073/pnas.1718679115 Zahran et al.

http://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalwww.pnas.org/cgi/doi/10.1073/pnas.1718679115

census tract are observed before the first switch in water source (phase A),after the switch to the Flint River but before the issuance of water qualityalerts (phase B), after the switch and after the issuance of water qualityalerts (phase C), and after the switch back to the DSWDwater supply (phaseD). Fig. 2A summarizes the precise timing of each phase. Expectations ofparameter behavior involving various statistical comparisons by phase aredescribed below.

Water Regime Effects: Preswitch vs. Postswitch. We begin with a more globaltest of a water regime effect, estimating a baseline census tract randomeffects logistic equation for the probability of census i in week t presentingwith a case of LD (1 = yes, and 0 = no):

ProbðLDit = 1jFi , Pt ,Rit ,XiÞ

=Λ ½ β0 + β1Fi + β2Pt + δðFi × PtÞ+Γ1Rit +Γ2Xit + ζi �, [1]

where Λ½ · � is the cumulative distribution function (CDF) of the logistic dis-tribution; Rit is a vector of temperature, precipitation, and humidity mea-sures (from Weather Underground); Xit is a vector of census tract controlvariables including percentage of population ≥50 y of age and percent ofhouseholds receiving public assistance (from US Census Bureau); ζi is therandom effect of census tract i; Fi is an indicator variable = 1 if the censustract is in Flint; and Pt = 1 if the census tract is observed in the postswitchperiod (combining phases B and C detailed in Fig. 2A), with the treatmenteffect of the regime switch captured by the estimated coefficient ðδÞ, con-stituting our difference-in-differences of F and P. In the presentation of logitmodel results below we exponentiate the estimated coefficient δ to derivean odds ratio, with the expectation that expδ > 1 indicating that the switchfrom Detroit to Flint River water caused an increase the risk of a census tractin Flint presenting with an LD case.

Water Regime Effects: Division of the Postswitch Period. Next, we test whetherboil water advisories issued by authorities in Flint attenuated the risk of LD,providing an additional probe of whether the switch in water regime causedthe observed spike in LD incidence in Flint. We divide the postswitch periodinto two phases, B and C (as detailed in Fig. 2A); compare LD outcomes incensus tracts in phase B (after switch, before advisories) and phase C (afterswitch, after advisories) to phase A (before the switch in water regime); andthen compare outcomes in phase C to B. These comparisons assume that theissuance of boil water advisories induced a meaningful reduction in wateruse by residents. Boil advisories were not issued to address the presence oflegionellae in the Flint water supply: authorities issued advisories afterpositive tests for E. coli contamination. However, the advisories may haveconfirmed suspicion among residents that the drinking water was unsafe,thereby unintentionally increasing water avoidance by residents. Twosources indicate that the advisories meaningfully affected residential waterexposure. First, Google Trend search interest data on water contaminationin the Flint–Saginaw–Bay City metropolitan area increased measurablyaround boil advisory dates, indicating awareness of the official warningsamong the local population (see ref. 19). Second, a large, statistically sig-nificant, and sustained increase in sales of bottled water in Genesee Countycorresponded with the issuance of advisories (18).

To examine whether advisories reduced LD risk (through a water avoid-ance pathway), we estimate a census tract random effects logistic equationfor the probability of census i in week t presenting with an incidence of LD(1 = yes, and 0 = no):

ProbðLDit = 1jGCi , PBt , PCt ,Rit ,XiÞ

=Λ ½ β0 + β1Fi + β2PBt + β3PCt + δ1ðFi ×PBtÞ+ δ2ðFi ×PCtÞ+Γ1Rit +Γ2Xit + ζi �,[2]

where all terms carry from Eq. 1, with the exception of PBt, which is equal to1 if the census tract is observed in the postperiod but before boil wateradvisories, and PCt, which assumes a value of 1 if the census tract is observedin the postperiod and after boil water advisories. The comparison of esti-mated coefficients δ1 and δ2 indicates whether water avoidance behavior ofresidents in Flint helped attenuate the LD outbreak and provides support forthe water regime hypothesis. Insofar as waterborne exposure to legionellaein Flint is linked to LD risk in a given census tract in time, and boil wateradvisories helped to reduce LD risk in Flint by inducing water avoidance inresident population, it is expected that expδ1 > 1, expδ2 > 1, and expδ1 > expδ2 .

Water Regime Effects: Switch Back in Water Supply. We analyze whether theswitch back in water supply on October 16, 2015, caused a reduction in LD

incidence by estimating a census tract randomeffects logistic equation for theprobability of census i in week t presenting with an incidence of LD (1 = yes,and 0 = no):

ProbðLDit = 1jFi , PDt ,Rit ,XiÞ

=Λ ½ β0 + β1Fi + β2PDt + δ1ðFi × PDtÞ+Γ1Rit +Γ2Xit + ζi �,[3]

where all terms carry from Eq. 1, with the exception of PDt which is equal to1 if the census tract is observed in the switch back period (phase D in Fig. 2A)and 0 if observed in the preswitch period (phase A in Fig. 2A). The causaleffect of the switch back in water regime is captured by the estimated co-efficient ðδÞ. Insofar as the rise and fall of LD incidence in Flint was caused bya city-wide failure in water treatment, this test is expected to yield anexpδ ≈ 1, indicating that the switch back to Detroit water returned the risk ofa census tract in Flint presenting with an LD case to precrisis levels.

Chlorine Residual. Free chlorine (mg/L as Cl2) measurements at eight moni-toring locations in Flint from 2013 to 2016 were obtained from the MonthlyOperating Reports provided by the Flint Water Department (SupportingInformation). Chlorine was measured at each location two to three times perweek. Fig. 1B illustrates the behavior of average free chlorine at the weeklytime step at each monitor site. Vertical lines bisecting the space correspondto water regime switch moments. Note the high between-monitor agreementin the level of free chlorine in the preswitch period, indicating negligiblespatial variation in water quality across the City of Flint. In the postswitchperiod, we observe extraordinary temporal (or within monitor) and spatial(or between monitor) variation in the level of free chlorine.

Table S1 summarizes the statistical behavior of free chlorine within andbetween monitors in time illustrated in Fig. 1B. Analytically, the un-precedented exogenous variation in free chlorine levels observed in Fig. 1Band Table S1 is what we exploit to identify statistically the effect of changesin water quality on LD incidence in Flint, MI.

Free Chlorine Data Assignment. To test the chlorine residual hypothesis, weneeded to select a method for associating a location in the City of Flint mapwith chlorine residual monitoring points. Although a number of approachescould be used, we choose to develop a physically relevant monitor-to-parcelassignment algorithm that leverages best available information on parceloccupancy/vacancy, residence time of water, and the Flint water distributionsystem pipe network. We tried a number of common approaches, includingvarious proximity and Thiessen polygon-based methods that deliveredquantitatively similar results. A hydraulic-based chemical transport model ofthe water distribution system could be used to estimate free chlorine re-sidual. However, it is unclear if this will have a meaningful effect on theresults given the spatial and epidemiologic limitations of surveillance data.The algorithm begins by finding the shortest path (or spine) from the cen-troid of each parcel to the Flint Water Plant (FWP) via the pipe networkobeying the water age gradient. The water age gradient used is correlatedwith spatial variation in blood lead levels during the switch in water supply(28), demonstrating the utility of this metric to account for physical vari-ability within the water distribution system. Water age is dynamic and likelyvaried spatially during the study period. Utilizing the water age gradienthelps to incorporate major hydraulic constraints that proximity-basedmethods fail to accommodate (e.g., flow restrictions due to pipe size). Thisresults in 41,286 parcel to FWP spines. Next, the algorithm finds the shortestpath of each monitor to each spine, again obeying the water age gradient.Each parcel is then assigned the (weekly average) chlorine value of themonitor with a spine juncture nearest to the parcel. Because LD incidencedata are organized at the census tract scale, we average parcel chlorine tothe census tract to generate 8,000 fully observed census tract (i) by week (t)observations in Flint from 2013 to 2016. The outcome of the monitor-to-parcel assignment algorithm is provided in Fig. S2.

Effect of Free Chlorine. At adequate levels (commonly assumed to be con-centrations ≥0.2 mg/L as Cl2), chlorine effectively suppresses legionellaegrowth. Under normal circumstances, it is near impossible to identify sta-tistically a chlorine residual → legionellae → LD incidence pathway becauseof insufficient time and space variation in chlorine within water distributionsystems. As observed in Fig. 1B, the switch in water regime in Flint inducedstriking temporal and spatial variation of free chlorine in the Flint waterdistribution system. Our estimation strategy analytically leverages this quasi-random behavior in free chlorine throughout the city. We estimate a census

Zahran et al. PNAS | Published online February 5, 2018 | E1737

ENVIRONMEN

TAL

SCIENCE

SPN

ASPL

US

http://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplementalhttp://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1718679115/-/DCSupplemental

tract random effects logistic equation for the probability of a census i inweek t presenting with an incidence of LD (1 = yes, and 0 = no):

ProbðLDit = 1jCit−1, Pt , PDt ,Rit ,XiÞ

=Λ ½ β0 + β1Cit−1 +Γ1Pt +Γ2PDt +Γ3Rit +Γ4Xit + ζi �, [4]

where all terms carry from Eqs. 1 and 3 with the exception Cit−1 denotingthe average weekly free chlorine (mg Cl2/L) at census tract i in time t − 1. The1-wk lag in free chlorine is included to account for the difference betweensymptom onset and referral date information in MDHSS case data. Recall theuse of referral date information was necessitated due to missing and im-precise data for symptom onset date. In addition to a continuous measure offree chlorine, we examine threshold effects of free chlorine, with Cit−1 =1 if < 0.5 mg/L or 1 in threshold models.

The census tract-specific residual, ζi, in the random effects model is meantto capture the combined effect of all omitted census tract-specific covariatesthat cause neighborhood variation in LD susceptibility. Omitted variablesmay include the underlying health frailty of residents or other sources ofwater chemistry parameters that affect legionellae growth, such as iron, pH,water temperature, or assimilable organic carbon. The census tract-specificrandom effect measures the difference in LD risk in a given census tractversus LD risk across all tracts in the City of Flint. Results from a Hausmanspecification test (χ2 = 2.54,p= 0.96) indicate that model coefficients areefficiently estimated by random as opposed to census tract fixed effects.

Robustness Test: Hospital Hypothesis. To test the plausibility of the hospital-based outbreak hypothesis, we recapitulate Eq. 1 through Eq. 4 but limitanalysis to non-hospital A-related LD cases. Potential hospital A-related casesare identified by screening (i) all cases in the MDHSS case file indicatingadmission to hospital A and LD between 2014 and 2016 and/or whetherMDHHS staff, on the basis of case analysis, override the reported admissionindication and assign the case as a hospital A admission (n = 64) and (ii) allnon-hospital A hospitalized cases with epidemiological investigation notesindicating a prior admission to hospital A between 2014 and 2016 (n = 19).All 19 cases in screen ii appear in screen i, giving a total of 64 cases poten-tially related to hospital A. Granting the hospital outbreak hypothesismaximum explanatory power, we assume that all 19 cases with a prior ad-mission to hospital A contracted LD at hospital A. Of the 45 cases remainingfrom screen i, all but 6 were returned to our analysis pool because the recordedsymptom onset date was before the hospital admission date. Adding 6 and 19gives 25 cases plausibly resulting from a hospital-based outbreak.

By limiting the analysis to non-hospital A-related cases, we test whetherestimated water regime and free chlorine suppression effects appreciablychange. If hospital A-related cases fully govern LD outcomes in Flint, then ourstatistical results pertaining to water regime and loss of free chlorine effectsought to disappear. However, if water regime and chlorine coefficients donot appreciably change with the exclusion of hospital A-related LD cases,

then it is highly unlikely that the LD spike in Flint resulted from a hospital-based outbreak only. Although the hospital exposure thesis is not incom-patible with our water regime/chlorine hypothesis—the hospital is similarlydrawing water from a portion of Flint’s water distribution system wherechlorine residual was often very low—our case exclusion tests allow one torule out an exclusively hospital-based argument.

The Genesee County (Outside of Flint) Epidemic. To test the hypothesis that thesizeable increase in LD incidence in neighborhoods (or census tracts) adjacent toFlint, in Genesee County, were due to water exposure in Flint, we utilized theLongitudinal Employer-Household Dynamics Employment Statistics dataset(https://lehd.ces.census.gov/data/). This dataset estimates that 15,857 workersflow into Flint every day, with 12,843 of them residing in neighboring areas inGenesee County, constituting a remarkable 61.7% of all employed personsworking inside Flint. Although commuting data capture inflows for the pur-poses of work, it is reasonable to assume exposure to the Flint water distri-bution system through leisure and other activities as well.

Restricting to not-Flint Genesee County census tracts, we test the com-muter flow hypothesis by estimating the following census tract randomeffects logistic equation for the probability of census tract i in week t pre-senting with a case of LD (1 = yes, and 0 = no):

ProbðLDit = 1jGCi , Pt ,Rit ,XiÞ

=Λ ½ β0 + β1GCi + β2Pt + δðGCi ×PtÞ+Γ1Rit +Γ2Xit + ζi �,[5]

where all terms carry from Eq. 1, with the exception of GC which is equal tothe observed count of daily commuters to Flint originating in non-FlintGenesee County census tract i. Insofar as exposure to Flint water is thesource of the non-Flint outbreak, it is expected that δ increases mono-tonically in GC.

ACKNOWLEDGMENTS. We are grateful for the assistance of all members ofthe Flint Area Community Health and Environment Partnership whichhelped guide the development of this manuscript. Specifically, Marcus Zervos(Henry Ford) provided guidance on epidemiologic surveillance and LD patho-genesis. Lead investigators of this group not already identified include (listedalphabetically): Carol Miller [Wayne State University (WSU)], Jessica Robbins-Ruszkowski (WSU), Joanne Smith-Darden (WSU), Judith Moldenhauer (WSU),Lara Treemore-Spears (WSU), Ben Pauli (Kettering), Joanne Sobeck (WSU),Poco Kernsmith (WSU), Susan Lebold (WSU), Tam E. Perry (WSU), Yongli Zhang(WSU), Matt Seeger (WSU), and Laura Sullivan (Kettering). Mariana Runho andMohammed Dardona (WSU) assisted in compiling the chlorine dataset. Thework reported was supported by MDHHS under Contract 20163753-00 andNational Institute of Environmental Health Sciences of the National Institutesof Health (NIH) under Award R21 ES027199-01. As contractually mandated, themanuscript was submitted to the MDHHS for review more than 30 d in ad-vance of being submitted for publication. The content is solely the responsi-bility of the authors and does not necessarily represent the official views of theMDHHS or NIH.

1. Beer KD, et al. (2015) Surveillance for waterborne disease outbreaks associated withdrinking water—United States, 2011–2012. MMWR Morb Mortal Wkly Rep 64:842–848.

2. Collier SA, et al. (2012) Direct healthcare costs of selected diseases primarily or par-tially transmitted by water. Epidemiol Infect 140:2003–2013.

3. Liu G, et al. (2017) Potential impacts of changing supply-water quality on drinkingwater distribution: A review. Water Res 116:135–148.

4. Rosen MB, Pokhrel LR, Weir MH (2017) A discussion about public health, lead andLegionella pneumophila in drinking water supplies in the United States. Sci TotalEnviron 590–591:843–852.

5. Fonger R (October 13, 2014) General Motors shutting off Flint River water at engineplant over corrosion worries. MLive. Available at www.mlive.com/news/flint/index.ssf/2014/10/general_motors_wont_use_flint.html. Accessed December 6, 2017.

6. Muraca P, Stout JE, Yu VL (1987) Comparative assessment of chlorine, heat, ozone,and UV light for killing Legionella pneumophila within a model plumbing system.Appl Environ Microbiol 53:447–453.

7. Zhang Z, et al. (2009) Legionella control by chlorine dioxide in hospital water systems.J Am Water Works Assoc 101:117–127.

8. Yabuuchi E, et al. (1995) An outbreak of Pontiac fever due to Legionella pneumophilaserogroup 7. II. Epidemiological aspects. Kansenshogaku Zasshi 69:654–665.

9. Ashbolt NJ (2015) Environmental (saprozoic) pathogens of engineered watersystems: Understanding their ecology for risk assessment and management. Pathogens4:390–405.

10. Cervero-Aragó S, Rodríguez-Martínez S, Puertas-Bennasar A, Araujo RM (2015) Effectof common drinking water disinfectants, chlorine and heat, on free Legionella andamoebae-associated Legionella. PLoS One 10:e0134726.

11. Thomas V, et al. (2004) Amoebae in domestic water systems: Resistance to disin-fection treatments and implication in Legionella persistence. J Appl Microbiol 97:950–963.

12. Craun GF, et al. (2010) Causes of outbreaks associated with drinking water in theUnited States from 1971 to 2006. Clin Microbiol Rev 23:507–528.

13. Falkinham JO, 3rd, Hilborn ED, Arduino MJ, Pruden A, Edwards MA (2015) Epidemi-ology and ecology of opportunistic premise plumbing pathogens: Legionella pneu-mophila, Mycobacterium avium, and Pseudomonas aeruginosa. Environ HealthPerspect 123:749–758.

14. Taylor M, Ross K, Bentham R (2009) Legionella, protozoa, and biofilms: Interactionswithin complex microbial systems. Microb Ecol 58:538–547.

15. Davis ML (2010) Water and Wastewater Engineering: Design Principles and Practice.Professional Edition (McGraw-Hill, New York).

16. Great Lakes – Upper Mississippi River Board of State and Provincial Public Health andEnvironmental Managers (2012) Recommended Standards for Water Works (HealthResearch Inc, Albany, NY).

17. World Health Organization (2014) Water Safety in Distribution Systems (WHO, Geneva).18. Manske C, Hilbi H (2014) Metabolism of the vacuolar pathogen Legionella and im-

plications for virulence. Front Cell Infect Microbiol 4:125.19. Christensen P, Keiser DA, Lade GE (2017) Economic effects of environmental crisis:

Evidence from Flint, Michigan (Iowa State University, Ames, IA), Mimeo.20. Zahran S, McElmurry SP, Sadler RC (2017) Four phases of the Flint water crisis: Evi-

dence from blood lead levels in children. Environ Res 157:160–172.21. American Water Works Association Disinfection Systems Committee (2008) Commit-

tee report: Disinfection survey, Part 1-Recent changes, current practices, and waterquality. J Am Water Works Assoc 100:76–90.

E1738 | www.pnas.org/cgi/doi/10.1073/pnas.1718679115 Zahran et al.

https://lehd.ces.census.gov/data/http://www.mlive.com/news/flint/index.ssf/2014/10/general_motors_wont_use_flint.htmlhttp://www.mlive.com/news/flint/index.ssf/2014/10/general_motors_wont_use_flint.htmlwww.pnas.org/cgi/doi/10.1073/pnas.1718679115

22. Fonger R (February 16, 2017) CDC finds first genetic link between Legionnaires’outbreak, Flint water. M Live. Available at www.mlive.com/news/flint/index.ssf/2017/02/cdc_finds_first_genetic_link_b.html. Accessed January 18, 2018.

23. AWWA Disinfection Systems Committee (2008) Committee report: Disinfectionsurvey, Part 2-Alternatives, experiences, and future plans. J Am Water WorksAssoc 100:110–124.

24. US Environmental Protection Agency (2008) FACTOIDS: Drinking water and groundwater statistics for 2007 (US Environmental Protection Agency, Office of Water,Washington, DC), EPA 816-K-07-004.

25. Garcia-Vidal C, et al. (2013) Rainfall is a risk factor for sporadic cases of Legionellapneumophila pneumonia. PLoS One 8:e61036.

26. Garrison LE, et al. (2016) Vital signs: Deficiencies in environmental control identifiedin outbreaks of Legionnaires’ disease–North America, 2000–2014. MMWR MorbMortal Wkly Rep 65:576–584.

27. Hicks LA, et al. (2007) Increased rainfall is associated with increased risk for legion-ellosis. Epidemiol Infect 135:811–817.

28. Sadler RC, LaChance J, Hanna-Attisha M (2017) Social and built environmental correlatesof predicted blood lead levels in the Flint water crisis. Am J Public Health 107:763–769.

Zahran et al. PNAS | Published online February 5, 2018 | E1739

ENVIRONMEN

TAL

SCIENCE

SPN

ASPL

US

http://www.mlive.com/news/flint/index.ssf/2017/02/cdc_finds_first_genetic_link_b.htmlhttp://www.mlive.com/news/flint/index.ssf/2017/02/cdc_finds_first_genetic_link_b.html