Ann. occup. Hyg., Vol. 45, No. 1001, pp. S49–S54, 2001 2001 British Occupational Hygiene Society

Published by Elsevier Science Ltd. All rights reservedPrinted in Great Britain.

0003–4878/01/$20.00PII: S0003-4878(01)00002-3

Challenges of Probabilistic Assessment of Operatorand Residential Non-Dietary ExposureSUSAN HUNTER YOUNGREN†*, MARK A. YOUNGREN‡ andLEILA BARRAJ††Novigen Sciences, Inc., 1730 Rhode Island Ave, NW, Suite 1100, Washington, DC 20036, USA;‡MITRE Corporation, 1820 Dolley Madison Blvd, McLean, VA 22102, USA

There are several challenges in the area of probabilistic exposure assessment of operator andresidential non-dietary exposure: the proper uses of the available data and how to adequatelyunderstand and characterize the uncertainty in the assessment. The adequacy of the datacan be assessed either qualitatively or quantitatively. If a qualitative assessment were deemedacceptable, then expert judgement would be the primary methodology. However, in mostinstances a quantitative assessment of adequacy will be possible and will provide more perti-nent information. In that case statistical techniques should be used. Understanding andcharacterizing uncertainty is a separate challenge, which is often ignored. Instead riskassessors present characterizing the variability in the assessment as a characterization ofboth uncertainty and variability. This can lead to misinterpretation of the outputs of theassessment. Interpretation of outputs can be discussed on two levels — familiarizing the riskmanagers and the policy makers with the types of outputs resulting from a probabilisticassessment (graphs and/or matrices of values) as well as explaining the meanings and impli-cations of the values. The burden is placed on the assessor to clearly reflect to the riskmanagers and policy makers the way on which variability and uncertainty were handled inthe assessment. In advising the risk managers and policy makers, it is important to remindthem of the purpose of conducting a probabilistic risk assessment: the probabilistic assess-ment allows a means to characterize the degrees of uncertainty and variability in terms ofprobabilities. 2001 British Occupational Hygiene Society. Published by Elsevier ScienceLtd. All rights reserved

Keywords: Monte Carlo assessment; probabilistic assessment; uncertainty; variability; exposure assessment

INTRODUCTION

The goal of non-dietary exposure assessments is tocharacterize the exposure of the population of con-cern (e.g., adults, toddlers, workers, etc.) and to ident-ify the variability associated with that exposure. Typi-cally, the primary objectives are to estimate the levelof exposure via ingestion, inhalation, or dermalabsorption of the substance and to identify the sourcesof both variability and uncertainty in the estimate. Inaddition, the exposure assessment can also be usefulin identifying the potential importance of a specificroute relative to other pathways of exposure. Thisneed to characterize both the variability and uncer-

Received 5 October 2000; in final form 11 December 2000.*Author to whom correspondence should be addressed. Tel.:+1-202-293-5374; Fax: +1-202-293-5377; E-mail:syoungren@novigensci.com

S49

tainty in the assessment has challenged the pest con-trol industry and the scientific discipline of riskassessment. New tools are needed to conduct theassessments. These analyses, if conducted usingdefault assumptions or simplistic methodologies, willgenerate ‘worst case’ estimates that are extreme. Thesimplified approach does not permit any discrimi-nation between risks that are likely to reflect risks thatreally exist and those that are truly hypothetical. Therisk assessment community has turned to probabilisticassessments of exposure so as to avoid generation ofestimates that if possible are highly improbable.

There are several challenges in the area of proba-bilistic assessment of operator and residential non-dietary exposure:

� the proper uses of the databases available to assessoperator or residential non-dietary exposure;

� understanding and characterizing both variabilityand uncertainty; and

S50 S. Hunter Youngren et al.

� aiding the risk manager in the interpretation of theprobabilistic exposure assessment output.

This paper will discuss these challenges and pro-vide suggestions for methods to properly assess andexplain a probabilistic assessment.

PROPER USE OF DATA INCORPORATINGUNCERTAINTY AND VARIABILITY

Exposure data are available from a variety of pub-lished sources [e.g., US EPA’s Exposure FactorsHandbook (US EPA, 1997), AIHC’s Sourcebook(Anon, 1994), US EPA’s THERdBase and Sociodem-ographic Data used for Identifying Potentially HighlyExposed Populations (US EPA, 1999), the PesticidesHandlers Exposure Database [PHED] (Versar, 1995),etc.] as well as unpublished, and sometimes pro-prietary data, such as the data collected by the USOutdoor Residential Exposure Task Force (ORETF),and others. These data sources characterize variousexposure factors using a variety of approaches; forexample, some sources provide estimates of centraltendency, some provide estimates of upper bounds,and some provide distributions that characterize theentire data set. Also, the various data sources can dif-fer in terms of their quality and quantity. Given thesedifferences, the data sets likely differ in their utilityfor characterizing exposure for given populations orsub-populations of interest.

The probability that an adverse health effect willresult from exposure to a given chemical is primarilydependent on a variety of exposure parameters —including the magnitude, duration, frequency androute (i.e. oral, dermal, inhalation) of exposure. Vari-ous sub-populations — identified by factors such asage or activity patterns, for example — will likelydiffer in terms of these exposure parameters. Forexample, children may differ from adults in terms ofmost probable route of exposure; while the elderlymay differ from younger adults in terms of the magni-tude of certain exposures. In addition, various sub-populations may differ in terms of their sensitivity orsusceptibility of response to a given level of exposure.As regulatory agencies become increasingly inter-ested in identifying specific sub-populations that are‘at increased risk’, there is a great need to evaluatethe adequacy of existing data for assessing exposurefor specific sub-populations.

Adequacy of dataA set of criteria can be established to determine if

a data set is ‘adequate for use’ and to ensure its properuse. There are three major components that must besatisfied to determine a ‘wholly adequate’ data set.The data must represent inter-individual variation,intra-individual variation (often termed temporalvariation), and uncertainty. Concerns about theimpacts of variability and uncertainty are not new. A

number of means for characterizing variability anduncertainty in exposure assessments have beenpresented. A paper by Whitmore (1985) focused onprocedures for estimating distributions depending onthe type of information available: (1) confidenceinterval estimates; (2) estimation of measurementaccuracy and precision; (3) use of goodness-of-fittechniques to model variation and uncertainties withparametric distributions; and (4) evaluation of alterna-tive parametric probability distribution models. Whileat the US EPA, White and Schaum (1989) focusedon the possible use of Monte Carlo simulation inexposure assessments, including a discussion of thecompleteness of uncertainty characterizations. Asearly as 1992, detailed discussions of uncertaintyanalysis have been included in US EPA guidelines(US EPA, 1992). However, there has been no defini-tive process described to assess the adequacy of dataused in exposure assessments.

The adequacy of the data can be assessed eitherqualitatively or quantitatively. If a qualitative assess-ment is deemed acceptable, then expert judgementcould be the primary methodology. That is, based onthe similar assessments conducted in the past with useof similar data, a decision will be made that the datafulfill or do not fulfill the standards of adequacy.However, in most instances a quantitative assessmentof adequacy will be possible and will provide morepertinent information. In that case statistical tech-niques should be used.

There are several methods to compensate for datainadequacy. The first, obviously, is the collection ofadditional data. However, there is seldom time and/ormoney to rely on this method. The second method ofcompensation for data inadequacy is the applicationof very large safety factors through the use of defaultassumptions. This is the method of choice in mostcases today. However, as well as being somewhatarbitrary, this commingles the issues of variabilityand uncertainty. This makes it difficult to describe theinputs and distorts any relevant interpretation of out-puts.

Another method is to use probabilistic risk assess-ment directly to quantitatively express uncertainty aswell as variability. For example, if a particular distri-bution is assumed to have a mean and variance basedon inadequate data, you can either increase the vari-ance or treat the mean and variance parameters asrandom variables themselves and place distributionson them to reflect the uncertainty about them. Forinstance, if a limited data set shows that the averageexposure of a child interacting with a dog is 2 h perday, the assessor can treat that two hours as a centralmoment of a distribution of mean interaction time,thus reflecting the uncertainty in that mean. This alsois a method to accommodate disagreements betweendifferent data sets or particular values. For example,if there are two sets of data with different means thatare supposed to reflect the same exposure, by using

S51Operator and residential non-dietary exposure

a distribution of the means that is based on the twodata means you can more accurately reflect the varia-bility of the mean and the variability of the data.1 Thisis also a means of combining expert opinions whenexperts differ.

A final way is to use data that are similar, but rel-evant to different populations or uses, are as prior dis-tributions for Bayesian analysis of limited data setsof the exposure of interest. In many cases classicalstatistical analysis of the data set will not yield a dis-tribution that can be used in a probabilistic assess-ment. For example, they may be skewed due to largesample-to-sample variability that is often present invery small data sets.

The adequacy and proper use of existing data isalso an issue when considering the general purposeof a risk assessment. That is, regulatory agencies userisk assessment for a variety of purposes, and the pur-pose of the risk assessment can define the types andprecision of exposure data that are needed. Forexample, risk assessments may be used for:

� Protection of a population and/or population sub-group (i.e., defining conditions of exposure thatcan be considered ‘safe’ or ‘unsafe’;

� Prediction of risk (i.e., developing estimates ofexposure that will occur if a specific action isundertaken or not taken);

� Priority setting (i.e., deciding between possibleoptions); and,

� Litigation (i.e., presenting information about levelsof exposure and resulting risk in toxic tort cases).

The different types of risk assessments require dif-fering levels of precision in risk estimates, which inturn have differing levels of data requirements.

Additionally, the proper use of existing exposuredata and characterizing uncertainty and variabilitymay be a concern, depending on whether the goal isto estimate ‘individual risk’ or ‘population risk’.Obtaining an estimate of ‘individual risk’ is often thegoal when the concern is exposures and/or risks forpersons at the upper end of the population distri-bution. On the other hand, an estimate of ‘populationrisk’ may be used to compare risks across varioussubgroups or to answer questions on how many per-sons are above a certain risk or exposure level. Thepurpose of an assessment (rule making, clean up, etc.)can affect whether ‘individual risk’ or ‘populationrisk’ estimates are needed, and again, this will affectthe data requirements for the assessment.

Thus, when assessing health risk it is necessary toconsider the purpose of the assessment as well as thevarying data requirements (i.e., ‘adequacy criteria’)

1 This would be applied when it is not possible to simplymathematically combine the two data sets into one com-bined data set. For example, the data may be from differentsurrogates, both of which are applicable but have been mea-sured differently.

for the assessment. That is, it is necessary to charac-terize the data set itself in order to assess its adequacyto support a specific type or level of risk assessment.To date, there has been little in the way of a system-atic evaluation of the adequacy of these various datasets for characterizing exposure.

Small or inadequate datasetsThere is another basic problem facing risk

assessors — that is, the need to estimate risk undercircumstances where (1) there are very little or norelevant exposure data, or (2) data were developed bysources that did not foresee the data’s utility inexposure assessments (i.e. data were not specificallydeveloped for risk assessment purposes and are con-sidered ‘inadequate’). In addition, data sets frequentlyvary considerably, possibly due to covariates notmeasured at the time the data were gathered.

Any assessment of exposure and risk based uponsuch data should be regarded as highly uncertain; thatis, there is a high degree of uncertainty surroundingthe accuracy of the assessment. A variety of statisticalapproaches are available for use in developing modelsbased on small, highly variable data sets such as thosecases just described. One approach uses Monte Carlosimulation, representing uncertainty in risk modelparameters through random sampling from underly-ing distributions. Another approach involves the useof prior distributions, explicitly incorporated into riskmodels using Bayesian techniques.

The best case would be large data sets of actualexposure to the actual population being assessed.Reality intrudes and the data usually have severalsources of inexactitude, e.g.,

� Small data sets;� Data sets from an inappropriate subpopulation

(e.g., adult activity pattern data used for children);� Data sets from a large diverse population, rather

than a specific subpopulation (e.g., data fromoccupational exposures to any pesticides whenassessing exposure to a specific pesticide);

� Data are derived from animals rather than humans;� Data were developed for a different purpose (e.g.,

exposure data targeted at occupational exposureextrapolated to homeowners near a facility);

� Data were developed for a different exposure, butare being used as surrogates (e.g., agricultural pes-ticide applicator data used for homeownerapplication).

The method chosen to quantitatively measure theuncertainty and variability in exposure assessmentwill influence the interpretation of the output.

ExamplesUncertainty in exposure assessments has been

classified into three categories: scenario uncertainty;parameter uncertainty; and model uncertainty. Listed

S52 S. Hunter Youngren et al.

below are examples of how these uncertainties mayarise in an aggregate exposure assessment and howmodels could be used to quantify these uncertainties.

Scenario uncertainty. Scenario uncertaintiesinclude descriptive errors, aggregation errors andincomplete analysis. For instance, for a flea productfor a pet scenario, uncertainty may result from incor-rect information regarding the regions in which theproduct is used. For example, the assessor may knowthat the compound is used monthly in the southernpart of the country, but does not have data regardingits use in the other regions. In this case, two runs maybe used; one, which assumes that pets in the entire USmay be treated monthly, and the other, which restrictsmonthly usage to the southern region. A comparisonof the results of the two analyses allows the user toevaluate the impact of the scenario uncertainty.Another example of scenario uncertainty refers toassigning the same concentration value to the entireindoor inhalation exposure, when concentrations mayvary across various rooms in the home. For instance,if an individual spends 40% of his/her ‘indoor’ timein one room, and 60% in another room, then aggregat-ing the time spent indoors in a total daily value andassigning that total daily time the same concentrationvalue may result in a biased estimate. The assessorshould assess the impact of this type of scenariouncertainty by comparing the results of the two analy-ses.

The impact of evaluating uncertainty and varia-bility can be seen in the following example:

An assessment can be run in three ways: (1) stan-dard default deterministic assumptions, (2)incorporating chemical-specific deterministic datain place of default assumptions, where available,and (3) incorporating chemical-specific distri-butions in place of default assumptions, whereavailable, including temporal and spatial speci-ficity.

For this example, it was assumed that Chemical A isapplied to turf, carpets and used for pet treatments.Table 1 presents the assumptions used in the analysis.

Table 2 presents the dermal exposure estimates forthis assessment and Table 3 presents a comparisonof the dermal risk estimates estimated as Margins ofExposure (MOEs). These exposure estimates arepresented graphically in Fig. 1.

Based on the data available, only small safety fac-tors would be needed for pet and turf while largerfactors may be needed for carpet as there is greateruncertainty in the data and more default assumptionswere used. This methodology described allows theanalyst to determine the main contributors toexposure by both route and by product use.

Parameter uncertainty. Parameter uncertaintyincludes measurement errors, sampling errors, varia-bility and use of surrogate data.

Two examples of measurement uncertainty in thedata used for assessing pesticides in food may be thepresumed tendency of some survey respondents tounderestimate their body weights or to under-reportfood consumption. In the first example, parameteruncertainty may result in potential overestimation ofthe exposures, while in the second example; it mayresult in potential underestimation of exposures.

Alternatively, an assessor may know that the distri-bution of air concentrations is likely to follow a log-normal distribution, but he or she may not be able toestimate the average of this distribution with enoughprecision. The assessor can thus conduct two assess-ments, one that uses the most likely estimate of themean and another that uses an upper bound on thatestimate. The assessor can then compare the resultsof both analyses for an evaluation of the impactuncertainty in the parameter estimate.

Model uncertainty. Model uncertainty arisesbecause models are simplified representations ofactual processes. The US EPA has developed Stan-dard Operating Procedures (SOPs) for AssessingResidential Exposure. These equations provide mod-els for estimating exposure from various types oftreatments. If the assessor is not certain of whichmodel (SOP equation) better represents the actualconditions; he/she can run several analyses using thevarious equations he/she thinks apply. A comparisonof the results of the various analyses provides theassessor with a measure of the impact of the uncer-tainty in the exposure model used.

Another example of model uncertainty may resultfrom using the wrong model to represent the degra-dation, over time, in air (or soil) concentrations. Theassessor then may want to evaluate degradation byassuming a first order decay in one case and a lineardegradation in another, and finally, use actualmeasurements for the various days. Running analysesusing the two models can assess the impact of theuncertainty in the degradation model used, or theobserved measured data, and comparing the results ofthese analyses.

INTERPRETATION OF THE PROBABILISTICEXPOSURE ASSESSMENT OUTPUT

The question of how to interpret the output of aprobabilistic exposure assessment can be discussed ontwo levels. The first level is a simplistic question ofwhat do the numbers or graphs mean that are createdthrough the use of probabilistic assessments. The pro-cess of informing the assessors — those actually con-ducting the probabilistic assessments — is still notcomplete, while it is also necessary to simultaneouslyfamiliarize the risk managers and the policy makersso that when they are faced with a graph or a matrixwith a variety of values they understand their mean-ing and implications. The second level is much more

S53Operator and residential non-dietary exposure

Table 1. Assumptions used in default and probabilistic analyses

Turf Carpet Pet

Default Probabilistic Default Probabilistic Default Probabilistic

Applicator Professional Professional Homeowner Homeowner Homeowner HomeownerFrequency NAa 2x, 6 mo apart NA 2×, 7 mo apart NA Monthly

with firstapplication in

springDay of Day 0 Amortized over Day 0 Amortized over Day 0 Amortized overexposure 7 days 7 days 7 daysDegradation NA Data NA 7 months until NA 28 days until

non-detect non-detectDuration of 2 h/day Distribution 8 h/day Distribution 2 h/day Distributionexposure from activity from activity from activity

pattern survey pattern survey pattern surveyDislodgeability 10% Data 5% 5% 20% Data

aNA: Not applicable (in the case of degradation, default assessment is based on all exposures occurring on the day ofapplication; Day 0: Day of application

Table 2. Toddlers — dermal exposure (µg/day)

Turf Pet Carpet Total

EPA defaults 9.76 2.72 7.10 19.58Point estimates 2.44 2.36 7.10 11.90Maximum

Monte Carlo — mean 0.015 1.21 4.20 4.46Monte Carlo — 95th percentile 0.57 2.31 6.32 8.47

AverageMonte Carlo — mean 0.0033 0.65 1.95 2.53Monte Carlo — 95th percentile 0.140 1.24 2.77 4.19

Table 3. Toddlers — dermal MOEs

Turf Pet Carpet Total

EPA defaults 90 322 123 45Point estimates 359 371 123 74Maximum

Monte Carlo — mean 57 859 721 208 196Monte Carlo — 95th percentile 1544 379 138 103

AverageMonte Carlo — mean 261 460 1346 449 346Monte Carlo — 95th percentile 6250 708 316 209

complex, yet relates to the same two groups. Thesecond level of interpretation is the questions that areposed regularly when the outputs of a probabilisticassessment are presented: “Where will the EuropeanUnion (EU) regulate? What percentile are the regu-lators concerned about? Does the Pesticide Manage-ment Regulatory Authority (PMRA) of HealthCan-ada agree?”

The burden is placed on the assessor to clearlyreflect to the risk managers and policy makers theway on which variability and uncertainty werehandled in the assessment. In advising the risk man-agers and policy makers, it is important to remind

them of the purpose of conducting a probabilistic riskassessment: the probabilistic assessment allows ameans to characterize the degrees of uncertainty andvariability in terms of probabilities. The mostimportant point is that the use of a probabilisticassessment reduces, if not eliminates, the need forsafety factors. The earlier discussion considered theproblems with limited data sets and the means toaccount for uncertainty and variability in risk assess-ments. Ultimately, if it is decided there is somedefinitive measure of acceptable risk (e.g., a bright-line such as the 1×10�6 criteria currently used in can-cer risk assessments) than a probabilistic assessment

S54 S. Hunter Youngren et al.

Fig. 1. Graphical representation of exposure from hypothetical chemical A applied to turf and carpets and used for pet treatments.

that incorporates all variability and has no uncertaintywould yield a precise point on the distribution thatwould correspond to this criteria. In practice, the mea-sures of variability are limited and uncertainty is notzero. Therefore, some measure of safety should beincorporated. This is often shown through the choiceof the percentile used for regulation. For example, theUS EPA regulates acute dietary exposures at the99.9th percentile. However, the acute dietary assess-ment is considered to be based on the largest data setavailable, particularly in contrast to the available datafor assessing residential exposures. Because, thesafety factors should reflect the degree to which thevariability and uncertainty were addressed in theexposure assessment, it should be deemed adequateto regulate at a lower percentile. It is not the purviewof the risk assessor to define the risk managementdecisions, rather to present a full characterization ofthe exposure including a description of the inherentuncertainty and variability in the assessment.

CONCLUSION

The ‘traditional’ Monte Carlo assessment typicallystrives to characterize the variability of the inputrather than assuming that all exposure is clusteredaround a specific point estimate. Historically, the useof safety factors has been the approach to compensatefor this assumption, but these are arbitrary and do notallow an understanding of either the variability oruncertainty in the actual exposures. Further, theresulting estimates are typically so unrealistic oncemultiple conservative assumptions and decisions arecombined that it is difficult to use the estimates toguide priority setting let alone for risk assessment.If there are data to support quantitative measures ofvariability, than there are means to identify the actualproportion or probability of exposure for the risk

being assessed. Safety factors may still be used, butthey should be regarded as compensating for uncer-tainty rather than for how well the variability has beenmeasured (or captured). One possibility would be toset levels of a safety factor based on the quality andquantity of the data used to characterize the varia-bility of the exposure factor(s). For example, if theassessment of risk is based entirely on a small dataset of limited applicability or entirely on expert judge-ment, a large safety factor might be applied (typically,by regulating at the tail of the distribution). Whereas,if there are large data sets directly applicable to theexposure and the population or sub-population(s) ofconcern, then acceptable risk levels can be basedfurther up the tail of the distribution.

REFERENCES

Anon. Exposure factors sourcebook. Washington, DC: AIHC,1994.

US Environmental Protection Agency (US EPA). Guidelinesfor exposure assessment. Federal Register 57(104):22888-22938 May 29, 1992.

US Environmental Protection Agency (US EPA). Exposurefactors handbook. US EPA, National Center for Environ-mental Assessment, Office of Research and Development.Washington, DC, EPA/600/P-95/002Fa,b,c, 1997.

US Environmental Protection Agency (US EPA). Sociodemo-graphic data used for identifying potentially highly exposedpopulations. US EPA, Office of Research and Development,National Center for Environmental Assessment-W. Wash-ington, DC, EPA/600/R-99/060. July, 1999.

Versar, Inc. PHED: the pesticide handlers exposure database.Reference manual, version 1.1. Prepared for the PHED TaskForce representing Health Canada, US EPA, and AmericanCrop Protection Association. February, 1995.

White P, Schaum J. Use of Monte Carlo analysis in exposureand risk assessment. US EPA, Exposure Assessment Group,Washington, DC, June 2, 1989.

Whitmore RW. Methodology for characterization of uncer-tainty in exposure assessments. Prepared by Research Tri-angle Institute for the US EPA, Exposure Assessment Group,Washington, DC, EPA/600/8-85/009, 1985.