15
Critical Perspective High-Throughput Screening and Environmental Risk Assessment: State of the Science and Emerging Applications Daniel L. Villeneuve, a, * Katie Coady, b Beate I. Escher, c Ellen Mihaich, d Cheryl A. Murphy, e Tamar Schlekat, f and Nat alia Garcia-Reyero g a US Environmental Protection Agency, Mid-Continent Ecology Division, Duluth, Minnesota b Toxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA c Helmholtz Centre for Environmental Research–UFZ, Leipzig, Germany d Environmental and Regulatory Resources (ER 2 ), Durham, North Carolina, USA e Michigan State University, Fisheries and Wildlife, East Lansing, Michigan, USA f Society of Environmental Toxicology and Chemistry, Durham, North Carolina, USA g Environmental Laboratory, US Army Engineer Research and Development Center, Vicksburg, Mississippi ABSTRACT: In 2007 the United States National Research Council (NRC) published a vision for toxicity testing in the 21st century that emphasized the use of in vitro high-throughput screening (HTS) methods and predictive models as an alternative to in vivo animal testing. In the present study we examine the state of the science of HTS and the progress that has been made in implementing and expanding on the NRC vision, as well as challenges to implementation that remain. Overall, significant progress has been made with regard to the availability of HTS data, aggregation of chemical property and toxicity information into online databases, and the development of various models and frameworks to support extrapolation of HTS data. However, HTS data and associated predictive models have not yet been widely applied in risk assessment. Major barriers include the disconnect between the endpoints measured in HTS assays and the assessment endpoints considered in risk assessments as well as the rapid pace at which new tools and models are evolving in contrast with the slow pace at which regulatory structures change. Nonetheless, there are opportunities for environmental scientists and policymakers alike to take an impactful role in the ongoing development and implementation of the NRC vision. Six specific areas for scientific coordination and/or policy engagement are identified. Environ Toxicol Chem 2019;38:12–26. Published 2018 Wiley Periodicals Inc. on behalf of SETAC. This article is a US government work and, as such, is in the public domain in the United States of America. Keywords: Computational toxicology; Adverse outcome pathway; Effects-based monitoring; In vitro to in vivo extrapolation; Risk assessment; Mixtures ENVIRONMENTAL RISK ASSESSMENT OF CHEMICALS Government authorities, industries, and the public they serve have a shared interest in protecting both human health and ecosystems from potential adverse effects associated with chemicals entering the environment as a result of human activities. The framework for environmental risk assessment (US Environ- mental Protection Agency Risk Assessment Forum 1998, 2014) provides a systematic process for evaluating the likelihood that adverse effects may occur as the result of exposure to chemicals in the environment. The primary elements of that process include 1) clear formulation of a problem statement in light of an appropriate protection goal or management decision; 2) assessment of the probable routes, severity, duration, and frequency of exposure; 3) assessment of the adverse or toxic effects that may result when exposure occurs; and 4) final risk characterization that brings the first 3 elements together. Risk assessments are conducted in a wide range of regulatory and commercial contexts but may be broadly categorized as prospective (i.e., trying to assess risks before any release or exposure takes place) or retrospective (i.e., evaluating risks associated with exposures that have already occurred and/or are still occurring). Regardless of context, quality risk assessments are dependent on the availability of data and/or models for estimating exposure and probable effects, and the lack of appropriate data and models can represent a significant impediment to environmental safety decision-making. The lack of available data for evaluating the safety of many chemicals in * Address correspondence to [email protected] Published online 23 November 2018 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/etc.4315 Environmental Toxicology and Chemistry—Volume 38, Number 1—pp. 12–26, 2019 12 Received: 25 July 2018 | Revised: 26 August 2018 | Accepted: 9 November 2018 Published 2018 SETAC wileyonlinelibrary.com/ETC

High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

* A

Pub

(wil

DO

Environmental Toxicology and Chemistry—Volume 38, Number 1—pp. 12–26, 2019

12 Received: 25 July 2018 | Revised: 26 August 2018 | Accepted: 9 November 2018

Pub

Critical Perspective

High-Throughput Screening and Environmental RiskAssessment: State of the Science and Emerging Applications

Daniel L. Villeneuve,a,* Katie Coady,b Beate I. Escher,c Ellen Mihaich,d Cheryl A. Murphy,e Tamar Schlekat,f andNat�alia Garcia-Reyerog

aUS Environmental Protection Agency, Mid-Continent Ecology Division, Duluth, MinnesotabToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USAcHelmholtz Centre for Environmental Research–UFZ, Leipzig, GermanydEnvironmental and Regulatory Resources (ER2), Durham, North Carolina, USAeMichigan State University, Fisheries and Wildlife, East Lansing, Michigan, USAfSociety of Environmental Toxicology and Chemistry, Durham, North Carolina, USAgEnvironmental Laboratory, US Army Engineer Research and Development Center, Vicksburg, Mississippi

ABSTRACT: In 2007 the United States National Research Council (NRC) published a vision for toxicity testing in the 21st centurythat emphasized the use of in vitro high-throughput screening (HTS) methods and predictive models as an alternative to in vivoanimal testing. In the present study we examine the state of the science of HTS and the progress that has been made inimplementing and expanding on the NRC vision, as well as challenges to implementation that remain. Overall, significantprogress has been made with regard to the availability of HTS data, aggregation of chemical property and toxicity informationinto online databases, and the development of variousmodels and frameworks to support extrapolation of HTS data. However,HTS data and associated predictive models have not yet been widely applied in risk assessment. Major barriers include thedisconnect between the endpoints measured in HTS assays and the assessment endpoints considered in risk assessments aswell as the rapid pace at which new tools and models are evolving in contrast with the slow pace at which regulatory structureschange. Nonetheless, there are opportunities for environmental scientists and policymakers alike to take an impactful role in theongoing development and implementation of the NRC vision. Six specific areas for scientific coordination and/or policyengagement are identified. Environ Toxicol Chem 2019;38:12–26. Published 2018 Wiley Periodicals Inc. on behalf of SETAC.This article is a US government work and, as such, is in the public domain in the United States of America.

Keywords: Computational toxicology; Adverse outcome pathway; Effects-based monitoring; In vitro to in vivo extrapolation;Risk assessment; Mixtures

ENVIRONMENTAL RISK ASSESSMENT OFCHEMICALS

Government authorities, industries, and the public they servehave a shared interest in protecting both human health andecosystems from potential adverse effects associated withchemicalsentering theenvironmentasa resultofhumanactivities.The framework for environmental risk assessment (US Environ-mental Protection Agency Risk Assessment Forum 1998, 2014)provides a systematic process for evaluating the likelihood thatadverseeffectsmayoccur as the result of exposure to chemicals inthe environment. The primary elements of that process include 1)

ddress correspondence to [email protected]

lished online 23 November 2018 in Wiley Online Library

eyonlinelibrary.com).

I: 10.1002/etc.4315

lished 2018 SETAC

clear formulationofaproblemstatement in lightofanappropriateprotection goal or management decision; 2) assessment of theprobable routes, severity, duration, and frequencyof exposure; 3)assessment of the adverse or toxic effects that may result whenexposure occurs; and 4) final risk characterization that brings thefirst 3 elements together. Risk assessments are conducted in awide range of regulatory and commercial contexts but may bebroadly categorized as prospective (i.e., trying to assess risksbefore any release or exposure takes place) or retrospective (i.e.,evaluating risks associated with exposures that have alreadyoccurred and/or are still occurring). Regardless of context, qualityrisk assessments are dependent on the availability of data and/ormodels for estimating exposure and probable effects, and thelack of appropriate data and models can represent a significantimpediment to environmental safety decision-making. The lackofavailable data for evaluating the safety of many chemicals in

wileyonlinelibrary.com/ETC

Page 2: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

High-throughput screening and environmental risk assessment—Environmental Toxicology and Chemistry, 2019;38:12–26 13

commerce was a significant driver for considering new ap-proaches (National Research Council 2007; Judson et al. 2009).

A VISION FOR MODERNIZING CHEMICALSAFETY EVALUATION, CA. 2007

In a landmark report published in 2007, the United StatesNational Research Council (NRC) clearly identified the lack oftoxicity data, for most chemicals in commerce, as a major barrierto efficient and cost-effective risk assessment (National ResearchCouncil 2007; Judson et al. 2009). The authors laid out a visionthat emphasized theuseofpredictive,high-throughput screening(HTS) assays to characterize the ability of chemicals to perturbbiological pathways critical to health. High-throughput assayswere defined as those that can be “automated and rapidlyperformed to measure the effect of substances on a biologicalprocess of interest” (National Research Council 2007). Broadlyspeaking, they referred to assays that could be run in 96-, 384-, or1536-well plate formats. This allows hundreds or thousands ofchemicals to be screened over a wide concentration range andfacilitates the use of many different assays that can encompassdifferent biological endpoints from different organs and differentsystems. They envisioned that data from these HTS assays wouldbe complemented by collection of chemical-specific informationrelated to physicochemical properties, important functionalgroups, key structural features, or commercial production anduse(s), among others, to identify exposure potential and criticalaspects of uptake, distribution, metabolism, elimination, and/or

TABLE 1: Initiatives, legislation, and online tools contributing to the sciencand human risk assessment, listed in chronological order and separated by

YearUnited States

Initiative Purpose

1998; 2014 USEPA Guidelines for EcologicalRisk Assessment, www.epa.gov/risk/guidelines-ecological-risk-assessment

Framework for environmentaassessment, systematicprocess

2000 Interagency CoordinatingCommittee on the Validationof Alternative Methods, ntp.niehs.nih.gov/go/iccvam

Founded by NIEHS to facilitinteragency and internatiocollaborations fordevelopment, regulatoryacceptance, and use ofalternative tests for toxicitesting

2005

2007 National Research Council,www.nap.edu/catalog/11970/toxicity-testing-in-the-21st-century-a-vision-and-a

Vision for toxicity testing in21st century

2008 Tox21, ntp.niehs.nih.gov/go/tox21

Research, develop, evaluatetranslate innovative testmethods, using HTS

2013

wileyonlinelibrary.com/ETC

accumulation and by fundamental understanding of biologicalpathways that could facilitate quantitative extrapolation model-ing. The NRC identified critical needs for implementation of theiroverall vision, which included 1) development of appropriatesuites of HTS assays, 2) availability of targeted in vivo data tocomplement and provide an interpretive context for the HTSresults, 3) computational extrapolation models that could predictwhich exposures may result in adverse changes, 4) infrastructureto support the basic and applied research to develop the assaysandmodels, 5) validation of the assays and data for incorporationinto guidance regarding their interpretation and use, and 6)evidence that the new approaches are adequately predictive ofadverse outcomes (National Research Council 2007). Althoughthat vision was focused entirely on human health risk assessmentand primarily prospective chemical assessments, it was expectedthat many aspects of the vision could translate to ecological andretrospective risk assessments as well (Villeneuve and Garcia-Reyero 2011). It was also understood that realizing the visionnecessitates thedevelopmentof biologically basedextrapolationtools ormodels that allowcell- or tissue-leveldata tobeapplied toindividual or population-level outcomes (Ankley et al. 2010).

HIGH-THROUGHPUT SCREENING, STATE OFTHE SCIENCE IN 2018

The science of HTS has progressed significantly sincepublication of the NRC report in 2007 (Table 1). In April 2018,over 100 international participants from academia, government,

e of high-throughput screening and implementation into ecologicalregion

Europe

Initiative Purpose

l risk

atenal

ty

NORMAN Network, www.norman-network.net

Enhances the exchange ofinformation on emergingenvironmental substances andencourages the validation andharmonization of commonmeasurement methods andmonitoring tools

the

, and

Solutions, www.solutions-project.eu

Produce consistent solutions forthe large number of legacy,

(Continued)

Published 2018 SETAC

Page 3: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

TABLE 1: (Continued )

YearUnited States Europe

Initiative Purpose Initiative Purpose

present, and future chemicalsposing a risk to Europeanwater resources with respect toecosystems and human health

2016 EU ToxRisk, www.eu-toxrisk.eu/

Drive the required paradigm shiftin toxicological testing awayfrom “black box” animaltesting toward a toxicologicalassessment based on humancell responses and acomprehensive mechanisticunderstanding of cause–consequence relationships ofchemical adverse effects

2016 SEURAT-1: Safety EvaluationUltimately ReplacingAnimal Testing, www.seurat-1.eu

Blueprint for future safety ofchemicals, without usinganimal testing

Legislation Purpose Legislation Purpose

2000 EU Water FrameworkDirective, http://ec.europa.eu/environment/water/water-framework/

Expands the scope of waterprotection to all waters,surface waters andgroundwater, achieving “goodstatus“ for all waters by a setdeadline

2007 REACH: Registration,Evaluation, Authorizationand Restriction ofChemicals, http://ec.europa.eu/environment/chemicals/reach/

Improves protection of humanhealth and environmentthrough better and earlieridentification of intrinsicproperties of chemicalsubstances

2013 EU Cosmetics Act, ec.europa.eu/growth/sectors/cosmetics/legislation_en

Bans animal testing, emphasizesproduct safety, while takinginto consideration the latesttechnological developments

2016 Frank R Lautenberg ChemicalSafety for the 21st CenturyAct, www.epa.gov/assessing-and-managing-chemicals-under-tsca/frank-r-lautenberg-chemical-safety-21st-century-act

Amends the Toxic SubstanceControl Act to includealternative tests (in vitro, insilico) and use risk-basedchemical assessments

Tool Purpose Tool Purpose

1996 ECOTOX Knowledgebase,cfpub.epa.gov/ecotox/

Knowledgebase that providessingle-chemical environmentaltoxicity

2002 USEPA Chemistry Dashboard,comptox.epa.gov/dashboard

Access to chemistry data forchemicals

2009 Add-my-Pet dynamic energybudget wiki, www.bio.vu.nl/thb/deb/deblab/add_my_pet/

Collection of dynamic energybudget model parameters for1400 species

2012 AOP knowledge base (AOPwiki,etc.), aopkb.oecd.org/index.html

Collection of AOPs and toolsrelated to AOPS

2015 ToxCast, www.epa.gov/chemical-research/toxcast-dashboard

Database of high-throughputassay data

2015 Toxcast Pipeline, www.epa.gov/chemical-research/toxcast-

R package for processing andmodeling chemical screening

(Continued)

14 Environmental Toxicology and Chemistry, 2019;38:12–26—D.L. Villeneuve et al.

Published 2018 SETAC wileyonlinelibrary.com/ETC

Page 4: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

TABLE 1: (Continued )

YearUnited States Europe

Initiative Purpose Initiative Purpose

data-generation-toxcast-pipeline-tcpl

data

2016 SeqAPASS: SequenceAlignment to Predict AcrossSpecies Sensitivity, seqapass.epa.gov/seqapass/

A Web-based tool that comparesprotein homology acrossspecies

2017 Integrated ChemicalEnvironment: IVIVE and PBPKmodels, ice.ntp.niehs.nih.gov

Provides high-quality curateddata and appropriate tools tosupport development andevaluation of new, revised, andalternative methods

AOP¼ adverse outcome pathway; HTS¼ high-throughput screening; IVIVE¼ in vitro to in vivo extrapolation; NIEHS¼National Institute of Environmental Health Sciences;NORMAN¼Network of reference laboratories, research centres and related organisations for monitoring of emerging environmental substances; PBPK ¼ physiologicallybased pharmacokinetic; USEPA ¼ US Environmental Protection Agency.

High-throughput screening and environmental risk assessment—Environmental Toxicology and Chemistry, 2019;38:12–26 15

industry, and other sectors came together to take stock of theprogress that has been made in HTS science and its applicationin environmental risk assessment (Society of EnvironmentalToxicology and Chemistry [SETAC] North America FocusedTopic Meeting on High-Throughput Screening and Environ-mental Risk Assessment, 16–18 April 2018, Durham, NC, USA).In addition, challenges that remain regarding implementation ofNRC’s vision and broadening its scope to encompass a full rangeof environmental risk-assessment contexts were identified.Building from the 6 critical needs for implementation thatwere identifiedby theNRC,wedetail the state of the science andongoing challenges in application of HTS in risk assessment.Although the topics, activities, and examples described in thepresent article are by no means comprehensive with respect toall progress in the field, they do provide a general view on howfar the science has come and how far it still needs to go for HTS toplay a routine role in environmental risk assessment.

Implementation need 1: Development of HTSbioassays and data

A first major need for implementation of an HTS assay–basedparadigm was the availability of the assays themselves andassociated data. The ToxCastTM and Tox21 initiatives (Kavlocket al. 2012; Tice et al. 2013) were landmark achievements in thatregard. Together, the ToxCast and Tox21 data sets representthe largest publicly accessible libraries of HTS data currentlyavailable. Most of the assays used by ToxCast were performedunder contract by commercial vendors (Kavlock et al. 2012),allowing a wide range of biological endpoints (hundreds) to beinterrogated in a relatively rapid manner using assays that hadalready been developed. Since 2007, over 4700 chemicals havebeen screened in at least a portion of the ToxCast assays(Richard et al. 2016; US Environmental Protection Agency2018a). The Tox21 program (Tice et al. 2013) has employed theinfrastructure of the National Toxicology Program to screen over8900 chemicals (US Environmental Protection Agency 2018b),although in a smaller number of assays (tens) compared to thenumber included in ToxCast. All data from these pioneeringHTS

wileyonlinelibrary.com/ETC

programs are accessible through the US Environmental Protec-tion Agency’s (USEPA’s) CompTox Chemistry dashboard(Williams et al. 2017; US Environmental Protection Agency2018c). In addition to these large-scale efforts, targeted HTSassay development, followed by screening of large chemicallibraries available through the ToxCast and Tox21 programs arebeing used to gradually expand the biological pathway spacecovered by the assays (e.g., Paul Friedman et al. 2016). However,apart from the incorporation of zebrafish embryo toxicity assays(e.g., Reif et al. 2016) and some research-scale attempts toincorporate cross-species testing platforms (e.g., Arini et al.2017), assay and data coverage remain strongly mammalian-biased, reflecting the human health-centric nature of theToxCast and Tox21 efforts, to date.

Implementation need 2: Availability of in vivodata to complement HTS results

The aggregation of targeted, in vivo data is another areawhere significant progress toward implementation has beenmade. For example, the USEPA’s CompTox Chemistry Dash-board (US Environmental Protection Agency 2018c) aggregatestoxicity data from a wide variety of sources and presents it inviewable and downloadable tables linked with chemicalstructures. Likewise, the USEPA’s ECOTOX knowledgebase(US Environmental Protection Agency 2018d) has been modern-ized to greatly improve search capabilities as well as datavisualization and interpretation. The ECOTOX knowledgebasehas also been dynamically linked to the Chemistry Dashboard sothat both ecological and human health-related effects data,where available, can be readily accessed.

Both the development of literature-derived toxicologicalknowledgebases and the ability of individual assessors to search,review, and evaluate the peer-reviewed literature have beenaccelerated by rapid advances in informatics and associatedtechnologies. Computational tools for searching the literatureand both automated and semiautomated extraction of data andinformation from various online sources are developing sorapidly that it is difficult to stay abreast of the strengths and

Published 2018 SETAC

Page 5: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

16 Environmental Toxicology and Chemistry, 2019;38:12–26—D.L. Villeneuve et al.

critical limitations of different approaches. When it comes tocomputer-assisted literature and data extraction, documenta-tion of an appropriate problem statement, search strategy,retrieval and extraction approach, and subsequent cleanup andformatting of the extracted data have become critical fortransparent application of literature-obtained data in riskassessments (Vandenberg et al. 2016).

Implementation need 3: Computationalextrapolation models

The proposed shift away from direct observation of apicaloutcomes in whole-organism toxicity tests necessitates devel-opment of a range of different extrapolation models (Figure 1).The ideal case is to be able to identify probable hazards directlyfrom chemical structure. However, chemical structure andproperty-driven interactions are more readily defined at themolecular level than at the highly integrated, apical levels ofbiological organization. Consequently, a variety of models,tools, and frameworks are being developed to bridge the gapbetween biological activities that can be predicted fromstructure or measured efficiently and cost-effectively and theoutcomes that are of concern to risk assessors (Table 1). Forexample, in vitro to in vivo extrapolation (IVIVE) models areneeded to translate relevant in vitro concentrations to equivalentblood or tissue doses in exposed organisms. Biologically basedextrapolation models are needed to understand how pathwayperturbations typically measured in a cell or even a cell-freesystem may translate to higher-level impacts in an intactorganism. The integrated impacts on multiple physiologicalprocesses in an intact organism, such as growth, maintenance,reproduction, and resilience to environmental change, need to

FIGURE 1: Examples of extrapolationmodels and frameworks being develophigh-throughput screening data and their integrated application.

Published 2018 SETAC

be translated into estimates of individual fitness and, at least inthe case of ecological risk assessment, relevant effects onpopulations or ecosystem services. Finally, for both human andecological risk assessments, cross-species extrapolation fromthe model systems represented in toxicity tests or HTS assays tothose that are of concern for a given risk assessment is ofteninvolved. Given the critical role of extrapolation under theproposed vision for toxicity testing, progress and the state of thescience related to development of several types of extrapolationmodels, tools, and frameworks are detailed in the presentsection.

Chemical property information and structure-based pre-diction models. Along with the rapid expansion and availabil-ity of HTS data sets and aggregation of available in vivo data,there has been a complementary explosion in the resourcesavailable for chemical property characterization. Moderncomputing power along with available high-speed Internethave brought access to chemical property information forhundreds of thousands of structures to our computers and evenour mobile phones. For example, the USEPA’s ChemistryDashboard (US Environmental Protection Agency 2018c) haslinked many external databases and prediction models andprovides extensive documentation on models via direct hyper-links, thereby providing one-stop access to a wide range ofchemical-specific information. The widespread availability ofchemical structure and property information as well as crystalstructures for proteins has made construction of in silico 3-dimensional docking models nearly routine. In addition, thecombination of chemical property information and HTS-basedbiological effects data, in many cases, now provides largeenough libraries of biologically active and inactive chemical

ed to predict which exposuresmay result in adverse outcomes, based on

wileyonlinelibrary.com/ETC

Page 6: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

High-throughput screening and environmental risk assessment—Environmental Toxicology and Chemistry, 2019;38:12–26 17

structures to employ machine learning approaches. Theseapproaches can identify structural alerts and define chemicalcategories likely to interact with specific biological targets and/or pathways. Such resources and tools are developing so rapidlythat it is extremely challenging for scientists and risk assessors tokeep pace with the development of new structure-basedprediction models and databases. Consequently, at present,deficiencies in structure-based prediction model documenta-tion, validation, training, and outreach needed to buildconfidence in the fit-for-purpose use of such models are,perhaps, the greatest barrier to application. Arguably, as withother types of computational models, there is a need for thescientific community to develop a set of rules and best practicesthat ensure the quality and applicability of the models todecrease uncertainty and increase acceptance. For instance, theInteragency Coordinating Committee on the Validation ofAlternative Methods (ICCVAM) sponsored a global project todevelop in silico models for acute oral systemic toxicity thatpredicted specific endpoints (US National Toxicology Program2018a). More than 130 models were submitted and discussedduring a workshop (US National Toxicology Program 2018b). Aconsensus model was then developed and used to generatepredictions that performed well when compared to results fromanimal tests (Kleinstreuer et al. 2018).

Translating in vitro concentration-response to equivalentin vivo effect or environmental exposure concentrations.Several tools and frameworks have been developed to supportIVIVE of HTS data for health risk assessment (Judson et al. 2014).Some generalized models for IVIVE/reverse toxicokinetics have

FIGURE 2: Extrapolating in vitro concentration-response to equivalent in vivovivo extrapolation/reverse toxicokinetic modeling has been applied to gener(B) Plasma concentrations in fish estimated based on the free fractionbioconcentration modeling can then be used to estimate the water concent

wileyonlinelibrary.com/ETC

been proposed (Wetmore et al. 2015). These allow at leastscreening-level estimation of human plasma doses equivalent toa nominal in vitro effect concentration. Likewise, dose–responsedata for many different endpoints for a given chemical can becombined into a distribution of “biological pathway activatingconcentrations” (BPACs), which can be converted into “biologi-cal pathway activating doses” (BPADs) for diverse humanpopulations by reverse toxicokinetic modeling (Judson et al.2011). From the resulting distribution of BPADs, a lowerconfidence limit can be selected as a conservative exposurelimit (Figure 2A). An approach like the BPAD approach in humanhealth could be used in screening-level ecological risk assess-ments as well.

At present, BPAC and BPAD estimation is based on nominalactivity concentrations in in vitro assays. However, it isrecognized that nominal concentrations do not account forfree versus bound chemical fractions in vitro (Armitage et al.2014; Fischer et al. 2017). Consequently, generalizable massbalance/partitioning models that can provide a more realisticestimate of the freely dissolved concentrations in an assay havebeen developed (Armitage et al. 2014; Fischer et al. 2017).However, to be widely employed, assay-specific parameters likelipid and protein content of the cells, volume and composition ofthe microplate wells, properties of the carrier solvent(s), as wellas medium/serum used (liposome–water and protein–waterpartition coefficients, etc.) need to be measured and madeavailable (e.g., Fischer et al. 2017). With appropriate availabilityof assay-specific parameters and in vitro toxicokinetic models,HTS activity data expressed as concentrations in wells could bemore readily equated to free concentrations in, for example, fish

effect (A) or environmental exposure (B) concentrations. (A) In vitro to inate screening-level estimates of equivalent human blood concentrations.of the active chemical concentration in the assay test well. Reverseration that would yield the equivalent internal dose.

Published 2018 SETAC

Page 7: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

18 Environmental Toxicology and Chemistry, 2019;38:12–26—D.L. Villeneuve et al.

plasma. Further, using reverse bioconcentration models, theycould be equated to critical water concentrations (Figure 2B) thatcould be used in establishing water quality benchmarks andother ecological risk-assessment applications. Parameter esti-mation tools and databases of physiological parameter valuesfor different species are being assembled to facilitate the use ofphysiologically based toxicokinetic models to aid IVIVE (Bes-sems et al. 2014). As with most extrapolation models, however,transparency and reporting standards are critical for implemen-tation (Bell et al. 2018).

Extrapolating from pathway-based biological activity topotential in vivo hazard(s). With regard to extrapolationacross levels of biological organization, the adverse outcomepathway (AOP) framework has been developed with the aim oforganizing scientifically credible support for extrapolationbetween pathway perturbations commonly measured in HTSassays and impacts on survival, growth, reproduction, orindividual health that are more widely considered relevant torisk assessment (Ankley et al. 2010). Since initial publication ofthe NRC’s vision for toxicity testing in the 21st century, the AOPframework has evolved from an informal concept to aninternationally coordinated program supporting developmentof a harmonized AOP knowledgebase (Society for Advancementof AOPs 2018; Organisation for Economic Co-operation andDevelopment 2018a) and accompanying guidance for AOPdocumentation and weight-of-evidence evaluation (Organisa-tion for Economic Co-operation and Development 2018b).Recognizing that chemical exposures in the real world are likelyto result in concurrent perturbation of multiple pathways in anorganism and that those pathways are highly interconnected,approaches for deriving and analyzing AOP networks are alsoemerging (Knapen et al. 2018; Villeneuve et al. 2018). However,those approaches are in early stages of development and havenot been sufficiently tested and refined for immediate applica-tion in risk assessment.

Integrating effects in the context of organism fitness.Leveraging concepts from the AOP framework, dynamic energybudget (DEB) theory is being used to help integrate specificpathway perturbations into overall estimates of organism fitness(Murphy et al. 2018). Dynamic energy budget theory describes amulticompartment dynamical systems model of the perfor-mance (growth, development, reproduction, and mortality) oforganisms and their metabolism throughout their life cycle(Kooijman 2010). Dynamic energy budget models are con-structed around assumptions of conservation of energy and ofelemental matter that are general enough to cover a widediversity of unicellular and multicellular organisms with diverselife histories and energetic systems. It has been envisioned thatoutcomes from high-throughput assays can be used as damagevariables (e.g., quantitative representation of damaged mem-branes, impaired protein function, oxidative stress) that can becorrelated or mechanistically linked to DEB fluxes (Murphy et al.2018;Muller et al. 2018). However, although several case studieshave been developed, the application of DEB models tointegrate the toll of multiple biological perturbations on the

Published 2018 SETAC

fitness of specific organisms has not moved from concept intoregulatory practice. In part, this is because, for parametrization,DEB models require specific time-resolved data for multipleendpoints collected under different temperature and foodregimes (Jager 2016).

Cross-species extrapolation. Cross-species extrapolationof pathway-based data is being facilitated by the USEPA’sSequence Alignment to Predict Across Species Susceptibility(SeqAPASS) tool (LaLone et al. 2016; US EnvironmentalProtection Agency 2018e). This tool leverages the rapidlygrowing database of protein sequence information to gener-ate quantitative comparison of protein conservation acrossspecies. Although this is by no means the only relevantdeterminant of a species’ probable susceptibility to a chemicalthat acts via a specific protein target or pathway, it provides animportant line of evidence that can help inform appropriatecross-species extrapolation of HTS as well as strategicdevelopment of new HTS assays intended to provide broadercoverage than the current human-centric assay space. Inaddition, embedding the HTS into an AOP framework anddeveloping concrete linkages to DEB facilitates cross-species extrapolations because DEB models have beenparameterized for over 1000 species (Murphy et al. 2018;Add-my-Pet 2018).

Overall, chemical structure–based prediction methods, IVIVEapproaches, the AOP framework, DEB models, and SeqAPASSeach provide examples of focused research activities that havebeen aimed at building the theory, tools, andmodels that will beneeded to extrapolate HTS data for use in risk assessment formultiple ecologically relevant species. However, althoughdevelopment of the scientific and technical foundations of anewapproach to toxicity testing is critical, there are also practicaland policy needs that must be addressed.

Implementation need 4: Infrastructure to supportbasic and applied research

Because, at least to date, technologies do not implementthemselves and obstacles are not resolved without effort,development of institutes, organizations, and consortia ofvarious kinds has been critical to coordinating and supportingthe basic and applied research needed to elicit the envisionedparadigm shift in toxicity testing. The ongoing support andgrowth of the USEPA’s National Center for ComputationalToxicology and continued engagement in Tox21 by multiplefederal agencies were foundational in North America. Likewise,similar efforts have been mounted worldwide. The SEURAT-1(SEURAT-1 2018) and Solutions (Solutions 2018) projects are just2 examples involving European Union and global partners in thescientific endeavor of transforming our approach to chemicalsafety assessment. Through consortia like the ICCVAM, strategicresearch efforts are being coordinated amongmultiple agenciesand organizations (e.g., US National Toxicology Program2018c). Notably for the SETAC community, the ICCVAM hasalso initiated an ecotoxicology working group, recognizing theneed to provide coordination and infrastructural support for

wileyonlinelibrary.com/ETC

Page 8: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

High-throughput screening and environmental risk assessment—Environmental Toxicology and Chemistry, 2019;38:12–26 19

development of transformative approaches in ecological toxicitytesting and risk assessment.

Implementation need 5: Validation of assays anddata to ensure fit for purpose

Validation of assays and resulting data has generally beenviewed as a prerequisite for regulatory application. Validationtypically involves establishing that an assay is specific, sensitive,and reliable for its intended purpose. It also entails theidentification of reference compounds (positive and negativecontrols) that can be used to establish that the assay is beingconducted and is functioning as intended and that the datagenerated are of high quality. A primary goal of validation,particularly mediated through the Organization for EconomicCo-operation and Development (OECD), is the mutual accep-tance of data among different regulatory domains.

High-throughput assays and data have presented somechallenges for traditional assay validation. First, becausemany ofthe early HTS data sets (e.g., ToxCast) relied on commercialassay providers and largely on assays intended for pharmaceu-tical candidate screening, many of the methods were at leastpartially proprietary and not intended to be transferred to orconducted in other laboratories (e.g., requiring specializedequipment, reagents, cell types, or infrastructure). Conse-quently, traditional round-robin testing, characteristic of manyvalidation processes, was not feasible. In addition, in the case ofmany multiplexed assays that measure multiple endpointsreflective of different pathway activation in a single assay, useof reference chemicals to establish performance-based stand-ards was also problematic. Furthermore, distinction of truepositives and negatives from false positives and negatives relieson the availability of orthogonal data or comparison of resultswith some widely accepted “gold standard” assay, which inmany cases was not available for HTS data sets. Consequently,although validation is viewed as critical for acceptance andapplication of HTS data, conventional approaches to assayvalidation do not always apply and, in these cases, newstandards for mutual acceptance of data need to be developed.

A major step forward with regard to HTS assay validation hasbeen improved documentation of the assay methods. FollowingOECD guidance document 211 (Organisation for Economic Co-operation andDevelopment 2017), testmethod descriptions arenow being developed for assays employed in ToxCast andTox21. This description includes the development of a bioassayontology that clearly defines the relationships between a givenassay, the number of different entities measured in that assay(termed “assay components”), and component endpoints (e.g.,increase or decrease in those entities measured) indicative ofeither activation or inhibition of a pathway/process. Whereappropriate to the assay, performance measures and referencechemicals are being identified. In addition, acceptance is beingfostered through full transparency with regard to the datathemselves and their processing. For example, the ToxCastPipeline, which is a data storage and statistical analysis platform,is organized into 7 levels of data processing/analysis. The assayannotations, raw and processed data, curve fits, and other

wileyonlinelibrary.com/ETC

summary information are publicly available (US EnvironmentalProtection Agency 2015; Filer et al. 2017). Additional statisticalmethods aimed at providing uncertainty estimates associatedwith values derived from curve-fitting, such as a confidenceinterval for the concentration at which 50% of activity isobserved, are under development (Watt and Judson 2018).Thus, although the kind of information needed to assure validityof HTS data and foster acceptance was often lacking when thedata were first released, that information (i.e., detailed assayinformation, structured data releases with transparent andreproducible analysis methods, and uncertainty estimates) isemerging, and appropriate guidance for the validation of HTSassays is taking shape.

Nonetheless, it is not clear that the current level of assaydocumentation goes far enough. Even if documentation defineshow the assays were performed, what reference chemical(s) wasused, and the performance criteria that were met, interpretationcan still be challenging. Information on data interpretation andprediction models for which the data are used is still lacking formost assays, despite calls for this information as part of OECDguidance document 211 (Organisation for Economic Co-operation and Development 2017). Additional details regardingassay specificity and relevance to whole-organism toxicity vialinkage to specific AOPs would also enhance use in riskassessment. Consequently, although strides have been madein terms of both assay validation and assay description,additional translational information is needed to more effec-tively utilize the data in practice.

STATE OF APPLICATION OF HTS TOENVIRONMENTAL RISK ASSESSMENT IN2018

A view of the state of the science in 2018 suggests thatconsiderable progress has been made in developing thescientific and technical foundations needed for implementationof the NRC’s vision. Risk assessors have expressed awareness ofthe HTS data themselves and conceptual recognition of themany ways in which those data might be used in the future(Textbox 1). However, adoption and acceptance into thechemical risk-assessment process have been limited. Thedisconnect between what is measured in most HTS assays(e.g., receptor binding, enzyme inhibition, gene expression,changes in cell growth or morphology) and the outcomes ofconcern from a risk-management perspective (e.g., humanhealth; survival, growth, and reproduction in wildlife; sustain-ability of populations; impacts on ecosystem services) remainsone of the greatest impediments to use. In concept, the AOPframework was developed to help address this disconnect(Ankley et al. 2010). However, well-developed AOPs to aidinterpretation of these data do not exist in most cases, and evenwhere they do exist, they provide only a qualitative connection,rather than quantitative extrapolation into terms that arerelevant for risk assessment.

The challenge posed by inadequate anchoring to AOPs iscompounded by the legacy nature ofmuch of the legislation andregulatory structures under which risk assessments are currently

Published 2018 SETAC

Page 9: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

20 Environmental Toxicology and Chemistry, 2019;38:12–26—D.L. Villeneuve et al.

conducted. Both the risk-assessment framework itself and muchof the major legislation under which chemical risk assessmentsare conducted predate the NRC vision. Consequently, statutesoften do not allow for the acceptance or use of data fromalternatives to traditional whole-animal toxicity tests. Updatingof these regulatory structures may be needed before the use ofHTS data in environmental risk assessment becomes common.Thus, although scientific institutes and consortia have taken upthe charge of coordinating research needed to develop thedata, tools, and models to support the vision for toxicity testing,there may be a need for parallel coordination by institutes andconsortia of risk assessment and policy professionals to adaptrisk assessment frameworks and approaches to better utilizethem.

A number of newer regulatory frameworks have alreadystarted to foster a transition toward alternatives to animal testingdata and methods. Some of the earliest drivers came fromEuropean initiatives like the Registration, Evaluation, Authorisa-tion and Restriction of Chemicals (REACH; EC No. 1907/2006),the European Cosmetics Act (EC No. 1223/2009), and theEuropean Union’s Water Framework Directive (Directive 2000/60/EC), with the first 2 in particular advocating or requiringreduction, refinement, and replacement of animal testing. Morerecently, the Frank R. Lautenberg Chemical Safety for the 21stCentury Act (2016) represents the first US chemical safetylegislation to promote the use of nonanimal alternative testingmethods (US Environmental Protection Agency 2018f). Conse-quently, regulatory frameworks are starting to create oppor-tunities to consider HTS data in risk assessments.

Nonetheless, it is perceived that more work is needed.Although HTS data sets are large and cover far more chemicalsthan conventional testing, gaps in both the biological andchemical spaces remain. Furthermore, the HTS data sets arecomplex. By and large, regulators do not feel confident in how tointerpret and extrapolate those data, and it remains unclear howto place the data into proper context for decision-making.

Published 2018 SETAC

Nearer-term opportunities for application fall in the realm ofscreening, prioritization, and mode of action identification,where knowing a chemical’s mode of action or “lead activity”could be used to inform subsequent testing and data gatheringactivities (Textbox 1). It is expected that programs accustomedto “data-poor” risk assessments that already rely heavily onstructure–activity relationships, read-across, and othermodelingapproaches (e.g., assessment of new chemicals under the ToxicSubstances Control Act, emergency management scenarios),would likely be earlier adopters than “data-rich” programs withauthority to require extensive whole-animal toxicity testing.Nonetheless, ongoing development of even broader data sets,application case studies, and training are expected to fostercomfort, confidence, and competence in using these dataappropriately. This can all be viewed as part of the sixth elementrequired for implementation of the NRC vision, developingevidence and case studies that the new approaches areadequately predictive of adverse outcomes and providescientific reliability and quality at least equivalent to thatachieved using traditional methods (US Environmental Protec-tion Agency 2018f).

BROADENING THE SCOPE

Although the vision of the NRC was focused on theprospective assessment of chemical risks to human health, thecurrent ambitions for modernizing chemical risk assessment arebroader. Given significant conservation of biological pathwaysacross organisms, many of the current mammalian-focused HTSdata can be reasonably extrapolated to a diversity of vertebratesand in some cases even invertebrates (LaLone et al. 2018).Likewise, AOPs available for translating HTS responses topotential in vivo hazards are as abundant for ecological toxicityoutcomes as for human health–focused pathways (Society forAdvancement of AOPs 2018). Indeed, some of the strongestapplication case studies for HTS data and tools, to date, have

wileyonlinelibrary.com/ETC

Page 10: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

High-throughput screening and environmental risk assessment—Environmental Toxicology and Chemistry, 2019;38:12–26 21

been in retrospective environmental monitoring. For example,HTS data available for individual chemicals are being used tobetter understand the toxicological significance of chemicalmonitoring data (e.g., in water quality monitoring). Building ontraditional risk-assessment concepts like hazard quotients andhazard indices, ratios of measured chemical concentrations andin vitro effect concentrations in pathway-based assays are beingsummed for all chemicals detected in a sample (e.g., Blackwellet al. 2017). In this way, both relative exposure concentrationsand the relative potency of chemicals tested individually in aspecific assay are accounted for, allowing exposure–activityratios of a mixture (EARmix) and the bioanalytical equivalentconcentration of a mixture (BEQmix) to be estimated. The termsof EARmix and BEQmix can be converted into each other(Figure 3), provided the effect levels (y) are similar. Aharmonization in terminology will allow an improved compara-bility of bioassay studies in the future.

In addition, the same types of assays used for single-chemicalevaluation in ToxCast and Tox21 are being increasingly used forenvironmental samples, including water, sediment, and eventissues and biofluids. The European Union’s Solutions project(Solutions 2018), for example, involving over 100 scientists fromnearly 40 institutions, conducted pioneering studies on theapplication of HTS assays for water quality monitoring andeffect-directed analysis. This included bioassay-based screeningof water samples for bioactive contaminants. Direct analysis ofenvironmental samples in HTS assays allows for determination ofBEQbio, which are based on concentration–response modelingcompared to an appropriate reference compound.

FIGURE 3: “Iceberg modeling” compares bioactivity detected by directly tpredicted by multiplying the concentrations of each chemical analytically dsumming across all detected chemicals in the mixture (BEQmix). Assay-specchemical that yields an equivalent magnitude of response as a reference cheknown composition of the mixture reasonably accounts for the biological acconstituents are likely contributing. The BEQ terminology is very similar to theto BEQmix. ACC¼ activity cutoff concentration (often close to EC20[i]); Ci¼ coyields an equivalent magnitude of response y; ref¼ reference; REPi¼ assay-

wileyonlinelibrary.com/ETC

The comparison of BEQbio with BEQmix has been termed“iceberg modeling” (Escher et al. 2013). The BEQmix constitutesthe visible tip of the iceberg, the calculated mixture effect ofquantified chemicals, whose concentrations (Ci) and relativeeffect potency (REPi) are known. The BEQbio relates to the entireiceberg, and BEQbio minus BEQmix is a measure of effecttriggered by (yet) unknown chemicals (Figure 3). Comparison ofbiological activities observed in these assays (BEQbio) with thosepredicted based on known chemical composition (BEQmix)reveals what was long anticipated, that unknown/undetectedconstituents of environmental samples often contribute largefractions of their overall biological activity (Neale et al. 2015,2017).

In cases where unknowns are contributing to observedbiological activities, high-throughput tools also offer distinctadvantages for effects-directed analyses (Brack et al. 2016;Muschket et al. 2018) aimed at identifying sources and drivers ofbiological activity. Specifically, large numbers of fractions/samples can be rapidly and efficiently analyzed using high-throughput platforms. Given the number of useful applicationsin environmental monitoring and retrospective mixture assess-ments, an even larger consortium, organized via the NORMANnetwork (Network of reference laboratories, research centersand related organizations for monitoring of emerging environ-mental substances; NORMAN2018), has been actively engagedin the development of performance-based criteria for bench-marking bioassays for use in water quality monitoring anddeveloping effect-based trigger values (Escher et al. 2018).Derivation of effect-based trigger values, along with guidance

esting a complex mixture in a screening assay (BEQbio) with bioactivityetected in the mixture by its assay-specific relative potency and then

ific relative potency is estimated as the effective concentration of a testmical. By comparing BEQbio with BEQmix one can evaluate whether thetivity observed, assuming additivity, or whether analytically undetectedexposure–activity ratio (EAR) terminology. The EARmix can be convertedncentration of chemical i; ECy¼effective concentration of chemical thatspecific relative potency.

Published 2018 SETAC

Page 11: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

22 Environmental Toxicology and Chemistry, 2019;38:12–26—D.L. Villeneuve et al.

on their interpretation and significance, offers real opportunitiesfor practical application of HTS assays and data in environmentalmonitoring. Similar efforts are under way in North America.Collectively, these efforts paint a picture of vigorous scientificdevelopment of HTS applications that lie well beyond the scopearticulated in the NRC’s vision for toxicity testing in the 21stcentury.

WHAT’S NEXT?

In their proposed timeline for implementation of the vision fortoxicity testing in the 21st century, theNRCenvisioned that “newknowledge and technology generated from the proposedresearch program will be translated to noticeable changes intoxicity-testing practices within 10 years” (National ResearchCouncil 2007). Certainly, significant progress, only a fraction ofwhich was noted here, has been made in the areas of need thatthe NRC identified. They envisioned that “within 20 years,testing approaches will more closely reflect the proposed visionthan current approaches,” assuming adequate and sustainedfunding. Taking stock of the state of the science in 2018, what arethe critical challenges that lie ahead?

At least at some level, the commitment, collaboration, andcommunication needed to support the establishment of newapproaches and methodologies, including HTS, are in place.Within the United States, 16 federal agencies have contributedand agreed to a strategic plan to move these efforts ahead (USNational Toxicology Program 2018c). The Tox21 collaborationhas been sustained for over a decade, and a road map forexpanding the chemical and biological space addressed by theHTS programs as well as known challenges, such as the lack ofmetabolic competence and unknown chemical disposition of invitro systems, is under way (Thomas et al. 2018). The OECDremains committed to an AOP development program, andEurope remains dedicated to the effort through activities like theNORMAN network (NORMAN 2018), ToxRisk (EU-ToxRiskProject 2018), and a variety of other programs. Consequently,global support for this ongoing transformation in chemical safetyassessment appears strong. Nonetheless, there are many andvaried challenges yet to be addressed. Each of these representsareas where SETAC and its members could take an impactfulrole in advancing the vision for 21st-century chemical safetyassessment.

Opportunities for impact: HTS assays fornonmammalian physiology

The human health research community has made it a priorityto continue to fill gaps in the toxicological space covered by themajor HTS programs. To that end, assays for underrepresentedmodes of action like neurotoxicity are being developed.Likewise, HTS assays that query the entire human transcriptomeare being employed to maximize pathway coverage. However,to date there has been no systematic or parallel effort to developHTS assay and testing infrastructure for pathways that areimportant in ecological risk assessment but are neitherconserved with mammalian physiology nor represented in

Published 2018 SETAC

current HTS programs. A few prominent examples includephotosynthetic pathways in plants and invertebrate endocrinesystems. Both plants and invertebrates often drive ecological riskassessment, and a variety of pesticide classes are specificallydesigned to interact with biological pathways and functionsunique to these taxa. Nonetheless, HTS assays for thosepathways are unavailable at present. In principle, developmentof HTS assays for important aspects of invertebrate and plantbiology should be “low hanging fruit” because the biology iswell understood and/or assays that could be adapted to highthroughput are already established (e.g., Organisation forEconomic Co-operation and Development 2011; Song et al.2017). A coordinated effort to map out the major gaps in thecurrent mammalian-centric HTS assay space relative to screen-ing for ecologically relevant hazards would allow for a strategic,coordinated, and efficient approach to ecological HTS assaydesign and development.

Opportunities for impact: Accessible testinginfrastructure

To date, most HTS data have been generated by largeconsortia with access to government funding and contractingmechanisms. At present, it is generally not practically feasible foran individual investigator, manufacturer, or regulatory body torapidly have a chemical or sample screened through a well-established and validated battery of HTS assays. Likewise, newHTS assays are often being developed in individual laboratories,whichmay screen a large library consisting of several hundred toa few thousand compounds, then move on to a new assay.Although such efforts have succeeded in placing a large amountof data into the public domain, most current HTS assays are notreadily accessed or applied for the assessment of new chemicalsor environmental samples. The lack of access to HTS testingplatforms limits both the amount and scope of the data that aregenerated and broader acceptance and uptake in risk assess-ment. If there are roughly 80 000 chemicals in commerce anddata are still only available for 1 in 10 of those, let alone theirrelevant transformation products, mixtures, and so on, HTS datamay still only be useful in a small minority of risk assessments.Creating and investing in an infrastructure in which researchers,regulators, and industry, including those from developingcountries, could submit chemicals/samples and have a stan-dardized and validated set of HTS assays available to test theircompound or sample would broaden the available data whiledistributing the costs of testing across the community.Establishment of one or more certified, accessible, HTS testingfacilities would also have substantial benefits for validation andmutual acceptance of data. The lack of accessibility of HTS couldbe addressed through creative public–private partnerships thatshould make it possible to expand the amount of data in thepublic domain while protecting confidential business informa-tion, for example, through substantially discounted pricing forusers who agree to make their data public. At the same timeadditional effort should be invested in development of guide-lines, performance-based measures, controlled vocabularies,and databases that could be used to establish the quality,

wileyonlinelibrary.com/ETC

Page 12: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

High-throughput screening and environmental risk assessment—Environmental Toxicology and Chemistry, 2019;38:12–26 23

validity, and comparability of low- tomedium-throughput assaysthat are viable for individual investigators to run in theirlaboratories and would be accepted as comparable to thoseavailable through commercial HTS services.

Opportunities for impact: Enhanced internationalcoordination

Stakeholders around the globe would stand to benefit fromgreater accessibility to HTS infrastructure. In particular, theenvironmental monitoring community has been one of the firstto acknowledge the potentially transformative nature of thesetesting platforms (Neale et al. 2015, 2017; Schroeder et al. 2016;Blackwell et al. 2017; Muschket et al. 2018). However, arecognized limitation, to date, has been a relatively limitedamount of interaction and awareness between various globalefforts to develop the approaches, models, and tools to supportthe application of HTS in environmentalmonitoring. This has led,to some extent, to the use of inconsistent terminologies andapproaches and less efficient use of limited research anddevelopment resources. Given its role as a global organization,SETAC is uniquely positioned to help coordinate a truly globalresearch strategy to tackle both scientific and policy hurdlesrelated to the application of HTS in this domain and to assist inproviding developing countries access and training related toHTS technology and its applications.

Opportunities for impact: Response–response,not just dose–response

Considering the state of the science, there is little skepticismthat HTS data can be generated. As noted, however, a majorbarrier to application is the disconnect between what ismeasured in an HTS assay and assessment endpoints consid-ered meaningful for decision-making. The AOP framework canaddress this to some extent, but quantitative extrapolationalong AOPs is currently limited by a lack of data that address thecritical question: how much perturbation of a key event is toomuch? Toxicologists have been trained to think in terms of dose-response (i.e., what magnitude of exposure is needed to elicit aneffect?). However, quantitative understanding needed toextrapolate along an AOP requires a more generalizedunderstanding of how much change in some upstream biologi-cal response (i.e., an early key event in an AOP) is needed toevoke a defined level of downstream biological effect (e.g.,eliciting a later key event in an AOP). This is captured as aresponse–response relationship, where, conceptually, the mag-nitude of change in an upstream biological event is plotted onthe x-axis and the magnitude or severity of a downstreambiological effect is plotted on the y-axis. The relationshipbetween the endpoints may be captured as a regressionequation or a more complicated mathematical model orprobabilistic function. Regardless of form, the aim is to developa quantitative relationship between one biological change andanother it is causally related to (e.g., Conolly et al. 2017).Ultimately, the goal is to be able to define at what point thosebiological changes become so severe that an organism’s health

wileyonlinelibrary.com/ETC

or ecological fitness is compromised. Although this may seemclosely aligned with traditional toxicology experiments,answering it often involves investigation of multiple endpointsat multiple time points to understand the progression ofdifferent key events along an AOP (e.g., Villeneuve et al. 2013;Villeneuve 2016; Conolly et al. 2017). Thus, a shift in mind-setis required with regard to the kinds of experimental designsneeded to support the AOP framework and its use for morequantitative applications of HTS in risk assessment. This shiftcould also coincide with a concerted effort to collaborate withthe broader biological research community (e.g., cell andmolecular biologists, physiologists, ecologists) because thestudy of both unperturbed biological systems and variousresponses to nonchemical stressors can both add to key eventinventories and augment understanding of response–responserelationships.

Opportunities for impact: Ecosystem relevance

However, even if AOPs can be successfully developed andapplied to translate HTS data to relevant endpoints at theindividual level, there remain deficiencies in our ability totranslate individual-based endpoints into predicted population-or ecosystem-level consequences (Forbes et al. 2017). Ecologi-cal risk is measured by focusing on sensitive individuals andrelies on a few well-studied model organisms. Nevertheless,unlike in human health risk assessment, the concern for hazard isat the population level, whichmakes the gap betweenmeasuredeffects and protection goals even larger, particularly whenfocusing on HTS and/or mechanistic data to inform decisions.Furthermore, the large number of species in ecological riskassessment with different life histories, strategies, and environ-ments significantly increases the challenge of extrapolation andthe level of uncertainty when using in vitro or in silico tools forhazard prediction. Predicted effects on survival, growth,development, and reproduction at the individual level stillneed to be integrated in the context of life-history traits tounderstand their overall significance to a population. Likewise,effects that may translate across trophic levels to haveecosystem-scale impacts should not be ignored. A simpleexample is the case of an herbicide affecting photosystem II ofplants. Although the chemical may be completely innocuous toanimal life, if it were to eliminate the base of the food chain,populations of a broad range of taxa could nonetheless beimpacted. Consequently, the ongoing effort to link ecologicalmodeling frameworks through dynamic energy budgets andother integrated approaches like the AOP framework is anotherarea where a strategic research effort could make a powerfulimpact (Forbes et al. 2017; Murphy et al. 2018).

Opportunities for impact: Aggregation ofecological exposure data sets

Finally, although the focus of the present article is primarily onwhat HTS could bring to the hazard or effects side of the risk-assessment equation, it is recognized that exposure is criticaland constitutes a large source of variability when extrapolating

Published 2018 SETAC

Page 13: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

24 Environmental Toxicology and Chemistry, 2019;38:12–26—D.L. Villeneuve et al.

from laboratory to field studies. The temporal and spatialvariability of exposure concentrations in the field and theirintersection with the different life histories, behaviors, andphysiological attributes of different species is an aspect ofecological risk assessment that cannot be ignored. Althoughinvestment in the HTS tools for biological effects assessment isimportant, there needs to be parallel investment in aggregationof exposure and toxicokinetic information. Human exposure andabsorption, distribution, metabolism, and elimination data arebeing aggregated through sources like the USEPA’s ChemistryDashboard (US Environmental Protection Agency 2018c).However, to date, there are no ongoing parallel efforts toaggregate ecological exposure data or relevant parametersneeded to develop robust toxicokinetic models for a wide rangeof vertebrate and invertebrate wildlife and plants.

CONCLUSIONS

It is apparent that the science needed to support implemen-tation of a broad vision of toxicity testing in the 21st century hasadvanced considerably over the decade since initial publicationof the NRC report. Many of the objectives defined by the NRChave been at least partially achieved. However, the uptake ofthese approaches into risk-assessment practices has beenlimited. The reasons for that limited uptake are multifactorial,but the challenges involved have been articulated and definedand are not insurmountable. At the same time, none are trivial,and well-coordinated strategic research, development, andinvestment will be required to address them in an effective andefficient manner. Although several cross-agency and globalpartnerships have already laid out strategic goals and roadmapsfor addressing some of the critical science challenges, thereremain several key areas where SETAC leadership and/oreffective coordination of the SETAC community with the effortsof other institutes and societies could make highly impactfulcontributions. The authors encourage SETAC and its member-ship to take up the charge.

Acknowledgment—The present article was prepared by thesteering committee for the SETAC Focused Topic Meeting onHigh Throughput Screening and Environmental Risk Assess-ment, Durham, NC, USA, April 2018. The authors acknowledgeall of the presenters and participants from the meeting for theirwork and insights. All of the content was inspired and informedby their contributions. The American Chemistry Council, theAmerican Petroleum Institute, B&C Consortia Management,Bayer, Colgate-Palmolive, Crop Life America, The Dow Chemi-cal Company, ER2, Exponent, Global Silicon Council, MichiganState University, Mississippi State University, Monsanto, Tes-senderlo Kerley, The Humane Society of the United States, theUS Army Engineer Research and Development Center, and theUS Environmental Protection Agency provided financial and/orin-kind support for the meeting. K. Paul-Friedman providedcomments on an earlier draft of the manuscript.

Disclaimer—The mention of products and/or trade names doesnot constitute endorsement or recommendation for use.

Published 2018 SETAC

Likewise, all views expressed are the personal views of theauthors and neither constitute nor necessarily reflect thepositions or policies of their organizations.

Data availability—Data, associated metadata, and calculationtools are available from the corresponding author([email protected]).

REFERENCES

Add-my-Pet. 2018. Add-my-Pet portal. [cited 2018 October 11]. Availablefrom: https://www.bio.vu.nl/thb/deb/deblab/add_my_pet/.

Ankley GT, Bennett RS, Erickson RJ, Hoff DJ, Hornung MW, Johnson RD,Mount DR, Nichols JW, Russom CL, Schmieder PK, Serrrano JA, TietgeJE, Villeneuve DL. 2010. Adverse outcome pathways: A conceptualframework to support ecotoxicology research and risk assessment.Environ Toxicol Chem 29:730–741.

Arini A, Mittal K, Dornbos P, Head J, Rutkiewicz J, Basu N. 2017. A cell-free testing platform to screen chemicals of potential neurotoxicconcern across twenty vertebrate species. Environ Toxicol Chem36:3081–3090.

Armitage JM, Wania F, Arnot JA. 2014. Application of mass balance modelsand the chemical activity concept to facilitate the use of in vitro toxicitydata for risk assessment. Environ Sci Technol 48:9770–9779.

Bell SM, Chang X, Wambaugh JF, Allen DG, Bartels M, Brouwer KLR, CaseyWM, Choksi N, Ferguson SS, Fraczkiewicz G, Jarabek AM, Ke A, LumenA,Lynn SG, Paini A, Price PS, Ring C, Simon TW, Sipes NS, Sprankle CS,Strickland J, Troutman J, Wetmore BA, Kleinstreuer NC. 2018. In vitro toin vivo extrapolation for high throughput prioritization and decisionmaking. Toxicol In Vitro 47:213–227.

Bessems JG, Loizou G, Krishnan K, Clewell HJ 3rd, Bernasconi C, Bois F,Coecke S, Collnot EM, DiembeckW, Farcal LR, Geraets L, Gundert-RemyU, KramerN, K€usters G, Leite SB, PelkonenOR, Schr€oder K, Testai E,Wilk-Zasadna I, Zald�var-Comenges JM. 2014. PBTK modelling platforms andparameter estimation tools to enable animal-free risk assessment:Recommendations from a joint EPAA–EURL ECVAM ADME workshop.Regul Toxicol Pharmacol 68:119–139.

Blackwell BR, Ankley GT, Corsi SR, DeCicco LA, Houck KA, Judson RS, Li S,Martin MT, Murphy E, Schroeder AL, Smith ER, Swintek J, Villeneuve DL.2017. An “EAR” on environmental surveillance and monitoring: A casestudy on the use of exposure-activity ratios (EARs) to prioritize sites,chemicals, and bioactivities of concern in Great Lakes waters. Environ SciTechnol 51:8713–8724.

BrackW, Ait-Aissa S, Burgess RM, BuschW, Creusot N, Di Paolo C, Escher BI,Mark Hewitt L, Hilscherova K, Hollender J, Hollert H, Jonker W, Kool J,Lamoree M, Muschket M, Neumann S, Rostkowski P, Ruttkies C, ScholleeJ, Schymanski EL, Schulze T, Seiler TB, Tindall AJ, De Arag~ao UmbuzeiroG, Vrana B, Krauss M. 2016. Effect-directed analysis supportingmonitoring of aquatic environments—An in-depth overview. Sci TotalEnviron 544:1073–1118.

Conolly RB, Ankley GT, ChengW,MayoML, Miller DH, Perkins EJ, VilleneuveDL, Watanabe KH. 2017. Quantitative adverse outcome pathways andtheir application to predictive toxicology. Environ Sci Technol51:4661–4672.

Escher BI, A€t-A€ssa S, Behnisch PA, Brack W, Brion F, Brouwer A, Buchinger S,Crawford SE, Du Pasquier D, Hamers T, Hettwer K, Hilscherov�a K, HollertH, Kase R, Kienle C, Tindall AJ, Tuerk J, van der Oost R, Vermeirssen E,Neale PA. 2018. Effect-based trigger values for in vitro and in vivobioassays performed on surface water extracts supporting the environ-mental quality standards (EQS) of the European Water FrameworkDirective. Sci Total Environ 628–629:748–765.

Escher BI, van Daele C, Dutt M, Tang JY, Altenburger R. 2013. Most oxidativestress response in water samples comes from unknown chemicals: Theneed for effect-based water quality trigger values. Environ Sci Technol47:7002–7011.

EuropeanUnion-ToxRisk Project. 2018. EUTOXRISK. [cited 2018October 11].Available from: http://www.eu-toxrisk.eu/.

Filer DL, Kothiya P, Setzer RW, Judson RS, Martin MT. 2017. Tcpl: TheToxCast pipeline for high-throughput screening data. Bioinformatics33:618–620.

wileyonlinelibrary.com/ETC

Page 14: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

High-throughput screening and environmental risk assessment—Environmental Toxicology and Chemistry, 2019;38:12–26 25

Fischer FC, Henneberger L, K€onig M, Bittermann K, Linden L, Goss KU,Escher BI. 2017.Modeling exposure in the Tox21 in vitro bioassays.ChemRes Toxicol 30:1197–1208.

Forbes VE, Salice CJ, Birnir B, Bruins RJ, Calow P, Ducrot V, Galic N, Garber K,Harvey BC, Jager H, Kanarek A, Pastorok R, Railsback SF, Rebarber R,Thorbek P. 2017. A framework for predicting impacts on ecosystemservices from (sub)organismal responses to chemicals. Environ ToxicolChem 36:845–859.

Jager T. 2016. Predicting environmental risk: A road map for the future.J Toxicol Environ Health A 79:572–584.

Judson R, Houck K, Martin M, Knudsen T, Thomas RS, Sipes N, Shah I,Wambaugh J, Crofton K. 2014. Invitro and modelling approaches to riskassessment from the US Environmental Protection Agency ToxCastprogramme. Basic Clin Pharmacol Toxicol 115:69–76.

Judson RS, Kavlock RJ, Setzer RW, Hubal EAC, Martin MT, Knudsen TB,Houck KA, Thomas RS, Wetmore BA, Dix DJ. 2011. Estimating toxicity-related biological pathway altering doses for high-throughput chemicalrisk assessment. Chem Res Toxicol 24:451–462.

Judson R, Richard A, Dix DJ, Houck K, Martin M, Kavlock R, Dellarco V, HenryT, Holderman T, Sayre P, Tan S, Carpenter T, Smith E. 2009. The toxicitylandscape for environmental chemicals. Environ Health Perspect117:685–695.

Kavlock R, Chandler K, Houck K, Hunter S, Judson R, Kleinstreuer N, KnudsenT, Martin M, Padilla S, Reif D, Richard A, Rotroff D, Sipes N, Dix D. 2012.Update on EPA’s ToxCast program: Providing high throughput decisionsupport tools for chemical risk management. Chem Res Toxicol25:1287–1302.

Kleinstreuer NC, Karmaus A, Mansouri K, Allen DG, Fitzpatrick JM,Patlewicz G. 2018. Predictive models for acute oral systemic toxicity: Aworkshop to bridge the gap from research to regulation. ComputToxicol 8:21–24.

Knapen D, Angrish MM, Fortin MC, Katsiadaki I, Leonard M, Margiotta-Casaluci L,Munn S,O’Brien JM, PolleschN, Smith LC, Zhang X, VilleneuveDL. 2018. Adverse outcome pathway networks I: Development andapplications. Environ Toxicol Chem 37:1723–1733.

Kooijman SALM. 2010. Dynamic Energy Budget Theory for MetabolicOrganization, 3rd ed. Cambridge University, Cambridge, UK.

LaLone CA, Villeneuve D, Doering JA, Blackwell BR, Transue TR,Simmons CW, Swintek JA, Degitz SJ, Williams AJ, Ankley GT.2018. Evidence for cross species extrapolation of mammalian-basedhigh-throughput screening assay results. Environ Sci Technol, inpress. DOI: 10.1021/acs.est.8b04587.

LaLone CA, Villeneuve DL, Lyons D, Helgen HW, Robinson SL, Swintek JA,Saari TW, Ankley GT. 2016. Editor’s highlight: Sequence Alignment toPredict Across Species Susceptibility (SeqAPASS): A web-based tool foraddressing the challenges of cross-species extrapolation of chemicaltoxicity. Toxicol Sci 153:228–245.

Muller EB, Klanjscek T, Nisbet RM. 2018. Inhibition and damage schemeswithin the synthesizing unit concept of dynamic energy budget theory.J Sea Res, in press. DOI: 10.1016/j.seareas.2018.05.006.

Murphy CA, Nisbet RM, Antczak P, Garcia-Reyero N, Gergs A, Lika K,Mathews T,Muller EB, Nacci D, Peace A, Remien CH, Shultz IR, StevensonLM, Watanabe K. 2018. Incorporating sub-organismal processes intodynamic energy budget models for ecological risk assessment. IntegrEnviron Assess Manag 14:615–624.

Muschket M, Di Paolo C, Tindall AJ, Touak G, Phan A, Krauss M, Kirchner K,Seiler TB, Hollert H, Brack W. 2018. Identification of unknown antiandro-genic compounds in surface waters by effect-directed analysis (EDA) usinga parallel fractionation approach. Environ Sci Technol 52:288–297.

National Research Council. 2007. Toxicity Testing in the 21st Century: AVision and a Strategy. National Academies, Washington, DC.

Neale PA, Ait-Aissa S, Brack W, Creusot N, Denison MS, Deutschmann B,Hilscherov�a K, Hollert H, Krauss M, Nov�ak J, Schulze T, Seiler TB, Serra H,Shao Y, Escher BI. 2015. Linking in vitro effects and detected organicmicropollutants in surface water using mixture-toxicity modeling. EnvironSci Technol 49:14614–14624.

Neale PA, Altenburger R, Ait-Aissa S, Brion F, BuschW, deArag~aoUmbuzeiroG, Denison MS, Du Pasquier D, Hilscherova K, Hollert H, Morales DA,Novac J, Schlichting R, Seiler T-B, Serra H, Shao Y, Tindall AJ, TollefsenKE, Williams TD, Escher BI. 2017. Development of a bioanalytical test

wileyonlinelibrary.com/ETC

battery for water quality monitoring: Fingerprinting identified micro-pollutants and their contribution to effects in surface water. Water Res123:734–750.

NORMAN. 2018. NORMAN—Network of reference laboratories, researchcentres, and related organizations for monitoring of emerging environ-mental substances. Verneuil-en-Halatte, France. [cited 2018 October 11].Available from: https://www.norman-network.net/.

Organisation for Economic Co-operation and Development. 2011. Test No.201: Freshwater alga and cyanobacteria, growth inhibition test. OECDGuidelines for the Testing of Chemicals, Paris, France.

Organisation for Economic Co-operation and Development. 2017. Guidancedocument for describing non-guideline in vitro test methods. Series onTesting and Assessment, No. 211. ENV/JM/MONO(2014)35. Paris,France.

Organisation for Economic Co-operation and Development. 2018a. AOPknowledge base. Paris, France. [cited 2018, October 11]. Available from:https://aopkb.oecd.org/.

Organisation for Economic Co-operation and Development. 2018b. User’shandbook supplement to the guidance document for developing andassessing adverse outcome pathways. ENV/JM/MONO(2016)12. Paris,France.

Paul Friedman K,Watt ED, HornungMW, Hedge JM, Judson RS, Crofton KM,Houck KA, Simmons SO. 2016. Tiered high-throughput screeningapproach to identify thyroperoxidase inhibitors within the ToxCast phaseI and II chemical libraries. Toxicol Sci 151:160–180.

Reif DM, Truong L, Mandrell D, Marvel S, Zhang G, Tanguay RL. 2016. High-throughput characterization of chemical-associated embryonic behav-ioral changes predicts teratogenic outcomes. Arch Toxicol 90:1459–1470.

Richard AM, Judson RS, Houck KA, Grulke CM, Volarath P, Thillainadarajah I,Yang C, Rathman J, Martin MT, Wambaugh JF, Knudsen TB, Kancherla J,Mansouri K, Patlewicz G, Williams AJ, Little SB, Crofton KM, Thomas RS.2016. ToxCast chemical landscape: Paving the road to 21st centurytoxicology. Chem Res Toxicol 29:1225–1251.

Schroeder AL, Ankley GT, Houck KA, Villeneuve DL. 2016. Environmentalsurveillance and monitoring—The next frontiers for high-throughputtoxicology. Environ Toxicol Chem 35:513–25.

SEURAT-1. 2018. SEURAT-1 towards the replacement of in vivo repeateddose systemic toxicity testing. [cited 2018 October 11]. Available from:http://www.seurat-1.eu/.

Society for Advancement of AOPs. 2018. AOP-Wiki. [cited 2018 October 11].Available from: http://aopwiki.org

Solutions. 2018. Who is Solutions. [cited 2018 October 11]. Leipzig,Germany. Available from: https://www.solutions-project.eu/solutions/.

Song Y, Villeneuve DL, Toyota K, Iguchi T, Tollefsen KE. 2017. Ecdysonereceptor agonism leading to lethal molting disruption in arthropods:Review and adverse outcome pathway development. Environ Sci Technol51:4142–4157.

Thomas RS, Paules RS, Simeonov A, Fitzpatrick SC, Crofton KM, Casey WM,Mendrick DL. 2018. The US Federal Tox 21 Program: A strategic andoperational plan for continued leadership. ALTEX 35:163–168.

Tice RR, Austin CP, Kavlock RJ, Bucher JR. 2013. Improving the human hazardcharacterization of chemicals: A Tox21 update. Environ Health Perspect121:756–765.

US Environmental Protection Agency. 2015. ToxCast & Tox21 summaryfiles for invitrodb_v2. [cited 2015 October 28]. Washington, DC.Available from: https://www.epa.gov/chemical-research/toxicity-forecaster-toxcasttm-data

US Environmental Protection Agency. 2018a. TOXCAST—EPA ToxCastscreening library. Washington, DC. [cited 2018 October 11]. Availablefrom: https://comptox.epa.gov/dashboard/chemical_lists/toxcast

US Environmental Protection Agency. 2018b. TOX21SL: Tox21 screeninglibrary. Washington DC. [cited 2018 October 11]. Available from https://comptox.epa.gov/dashboard/chemical_lists/tox21sl

US Environmental Protection Agency. 2018c. Chemistry dashboard. Wash-ington DC. [cited 2018 October 11]. Available from https://comptox.epa.gov/dashboard/.

US Environmental Protection Agency. 2018d. ECOTOX Knowledgebase.Washington DC. [cited 2018 October 11]. Available from https://cfpub.epa.gov/ecotox/.

Published 2018 SETAC

Page 15: High-throughput screening and environmental risk assessment: …€¦ · bToxicology and Environmental Research and Consulting, The Dow Chemical Company, Midland, Michigan, USA

26 Environmental Toxicology and Chemistry, 2019;38:12–26—D.L. Villeneuve et al.

US Environmental Protection Agency. 2018e. Sequence Alignment toPredict Across Species Susceptibility (SeqAPASS). Washington DC.[cited 2018 October 11]. Available from https://seqapass.epa.gov/seqapass/.

US Environmental Protection Agency. 2018f. Strategic plan to promote thedevelopment and implementation of alternative test methods within theTSCA program. EPA-740-R1-8004. Washington, DC.

US Environmental Protection Agency Risk Assessment Forum. 1998. Guide-lines for ecological risk assessment. EPA/630/R-95/002F. Washington,DC.

US Environmental Protection Agency Risk Assessment Forum. 2014.Framework for human health risk assessment to inform decision-making.EPA/100/R-14/001. Washington, DC.

US National Toxicology Program. 2018a. Predictive models for acute oralsystemic toxicity. US Department of Health and Human Services. [cited2018 October 11]. Washington DC. Available from: https://ntp.niehs.nih.gov/go/tox-models

US National Toxicology Program. 2018b. Workshop: Predictive models foracute oral systemic toxicity. US Department of Health and HumanServices. Washington DC. [cited 2018 October 11]. Available from:https://ntp.niehs.nih.gov/go/atwksp-2018

US National Toxicology Program. 2018c. A strategic roadmap for establish-ing new approaches to evaluate the safety of chemicals and medicalproducts in the United States. [cited 2018 October 11]. Bethesda, MD.Available from: https://ntp.niehs.nih.gov/pubhealth/evalatm/natl-strategy/index.html

Vandenberg LN, Agerstrand M, Beronius A, Beausoleil C, Bergman A, BeroLA, Bornehag CG, Boyer CS, Cooper GS, Cotgreave I, Gee D, GrandjeanP, Guyton KZ, Hass U, Heindel JJ, Jobling S, Kidd KA, Kortenkamp A,MacleodMR,MartinOV,Norinder U, ScheringerM, Thayer KA, Toppari J,

Published 2018 SETAC

Whaley P, Woodruff TJ, Rud�en C. 2016. A proposed framework for thesystematic review and integrated assessment (SYRINA) of endocrinedisrupting chemicals. Environ Health 15:74.

Villeneuve D. 2016. Adverse outcome pathway on aromatase inhibitionleading to reproductive dysfunction (in fish). OECD Series on AdverseOutcome Pathways 4. Organisation for Economic Co-operation andDevelopment, Paris, France. Available from: https://doi.org/10.1787/5jlsv05mx433-en

Villeneuve DL, Angrish MM, Fortin MC, Katsiadaki I, Leonard M, Margiotta-Casaluci L, Munn S, O’Brien JM, Pollesch NL, Smith LC, Zhang X, KnapenD. 2018. Adverse outcome pathway networks II: Network analytics.Environ Toxicol Chem 37:1734–1748.

Villeneuve DL, Breen M, Bencic DC, Cavallin JE, Jensen KM, Makynen EA,Thomas LM, Wehmas LC, Conolly RB, Ankley GT. 2013. Developingpredictive approaches to characterize adaptive responses of thereproductive endocrine axis to aromatase inhibition: I. Data generationin a small fish model. Toxicol Sci 133:225–233.

Villeneuve DL, Garcia-Reyero N. 2011. Vision and strategy: Predictiveecotoxicology in the 21st century. Environ Toxicol Chem 30:1–8.

Watt ED, Judson RS. 2018. Uncertainty quantification in ToxCast highthroughput screening. PLoS One 13:e0196963.

Wetmore BA, Wambaugh JF, Allen B, Ferguson SS, Sochaski MA, Setzer RW,Houck KA, Strope CL, Cantwell K, Judson RS, LeCluyse E, Clewell HJ,Thomas RS, Andersen ME. 2015. Incorporating high-throughput expo-sure predictions with dosimetry-adjusted in vitro bioactivity to informchemical toxicity testing. Toxicol Sci 148:121–136.

Williams AJ, Grulke CM, Edwards J, McEachran AD, Mansouri K, Baker NC,Patlewicz G, Shah I, Wambaugh JF, Judson RS, Richard AM. 2017. TheCompTox Chemistry Dashboard: A community data resource forenvironmental chemistry. J Cheminform 9:61.

wileyonlinelibrary.com/ETC