MeasurementErrorinNutritionalEpidemiology:Impact,CurrentPracticeforAnalysis,and

OpportunitiesforImprovement

PamelaShaw

shawp@upenn.eduonbehalfofSTRATOSTG4

10May,2017

9th EMR-IBS,Thessaloniki

Outline

• Background

• MotivatingexamplesshowingimpactofMeasurementerror

• Regressioncalibration

• Literaturesurvey:methodologyandresults

• Conclusions

2

STRATOSTG4:MeasurementErrorandMisclassification

MEMBERSHIP• LaurenceFreedman,Gertner/IMS,Co-Chair• VictorKipnis,NCI,Co-Chair• RaymondCarroll,TexasA&MU• VeronikaDeffner,Munich,LMU• KevinDodd,NCI• PaulGustafson,U.BritishColumbia• RuthKeogh,LondonSchoolofHygiene• HelmutKuechenhoff,Munich,LMU• PamelaShaw,U.Pennsylvania• JanetTooze,WakeForestSchoolofMedicine

3

TG4Projects

1. LiteratureSurveyforhowmeasurementerrorisaddressedin4typesofepidemiologicalstudies

2. Guidancepaperfornutritionalepidemiologists

3.Guidancepaperforbiostatisticians

4

TG4LiteratureSurvey• Therehavebeenmanystatisticaladvancestoaddressinmeasurementerrorinthepastfewdecades

• TG4wasinterestedinassessingthecurrentpracticeforacknowledgingandaddressingmeasurementerrorinepidemiologic/observationalstudies– Wanttoidentifyknowledgegapsandopportunitiesforimprovement

• Weconductedaliteraturesurveyfocusedontypesofepidemiologicstudieswithexposuresthatarewellknowntobesubjecttomeasurementerror

5

Example1:ClassicalMeasurementerror

• Classicalmeasurementerror(CME)israndom,meanzeroerror

• CovariateX*withCMEcanbewrittenas:X*=X+u,whereuismean0errortermindependentofXandY

• SupposeY=𝛽0+𝛽1X+𝜀,thenregressingYonX*willestimateslope𝛽#∗≠𝛽1

• 𝛽#∗ willbeattenuatedtoward0

−4 −2 0 2 4

−1

0

1

2

3

4

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

−4 −2 0 2 4

−1

0

1

2

3

4

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

𝜷𝟏∗ = 𝝀𝜷 , where

𝜆 =var(𝑋)

var 𝑋 + var(𝑢)

So 0< 𝜆 <16

Y

X

Example2:MeasuringDietaryIntake

• Measuringdietaryintakeisofinterestinepidemiologyasthereareanumberofdiseasesforwhichdietaryfactorsarethoughttobeimportantriskfactors,includingcancer,heartdiseaseanddiabetes

• Dietaryintakeisacomplexexposuretomeasure– Madeupofmanynutrientsobtainedfromavarietyoffoods– Containsday-to-dayvariability,possiblyalsotemporalvariability

• Thereareseveralprevailingdietaryassessmentmethods– Self-report:Foodfrequencyquestionnaire,24hourrecall,daily

foodrecord– Objectivebiomarkers:recoveryorconcentrationmarkers

7

MeasuringEnergyIntake

0 500 1000 1500 2000 2500 3000

0500

1000

1500

2000

2500

3000

Biomarker Energy

Visit 1

Vis

it 2

0 500 1000 1500 2000 2500 3000

0500

1000

1500

2000

2500

3000

FFQ Energy

Visit 1

Vis

it 2

8

𝜌 =.72 𝜌 =.70

EnergyIntakevsBodyMassIndexNeuhouser etalAJE2008

9

RegressionCalibration:AsimpleapproachtoadjustforME

PrenticeBiometrika 1982

• Supposetrueintake:X

• Error-pronemeasure:X*(FFQintake)

• Objectivebiomarker:X**=X+u

• PredictedX=E(X**|X*,Z)=E(X+u|X*,Z)=E(X|X*,Z)

=a1+a2X*+a3Z+a4 ZX*

Regressioncalibration:RegressoutcomeYonpredictedintake,othercovariatesZ

10

HRforUncalibratedvsCalibratedEnergyIntakePrentice,ShawetalAJE2009

11

SurveyAreasEachoffourtopicareashaditsownliteraturesearch

• Nutritionalintakecohortstudies(PamelaShaw/RuthKeogh)

• Dietaryintakepopulationsurveys(KevinDodd)

• Physicalactivitycohortstudies(JanetTooze)

• Airpollutioncohortstudies(VeronikaDeffner/HelmutKuechenhoff)

12

OverallApproach• Focusedonerror-pronevariableasexposureinanalysis

• Forcohortstudies,literaturesearchdoneintwostages– SearchA:Surveyrecentarticlestoassesshowoftenarticlesacknowledgedand/oraddressedmeasurementerror

– SearchB:Surveyrecentarticlesthatadjustedformeasurementerrortodescribemethodsincurrentpractice

• Questionnairesfilledoutforeachreviewedarticle

• Excludedreviews,cross-sectionalstudies,case-controlstudiesandmeta-analyses

• Eachtopicareaconductedaqualitycontrolreview– 20%re-reviewedbyindependentreviewer

13

NutritionalEpidemiologyCohortStudies:SurveyMethodology

• DateRangeA:Feb2014-Jun2015;B:Jan 2001-Jul2015

• Limitedsearchtothreecommondiseaseswithdietaryriskfactors:cancer,heartdiseaseanddiabetes– Restricteddaterangetofindabout50articlesfromSearchAand30

articlesfromSearchB

• SearchB:added(measurementerrorORmisclassificationtoSearchA– Notmanyarticles,sodidadditionalkeywordsearchesincluding:

(measurementerrorORmisclassification)ANDnutritionalepidemiology

• Physicalactivityandpollutioncohortmethodologysimilar,exceptreliedondaterangeandrandomsamplingtoreducenumberofarticlesreviewed

14

NumberofArticlesReviewed*SearchA SearchB

Nutritional Epidemiologycohort studies

51 27

DietaryIntakePopulationSurvey

67 N/A

Physical Activitycohort studies

30 40

AirPollutioncohort studies

50 25

* Number in table excludes articles that were identified by search terms but upon closer examination did not meet inclusion criteria

15

SearchASurveyResultsNutritional Epi Cohort

N= 51

Phys activity CohortN=30

Diet IntakeSurvey N=67

PollutionCohortN=50

Mention ME as potential problem n(%)

48 (94%) 17 (57%) 53/67 (79%) 20 (40%)

Used a method to adjust forME N (%)

5 (10%) 0 (0%) 19/67 (28%) 3 (6%)

% categorizing exposure

Any 50/51(98%) Exclusively 27/51 (53%)

Primary exposure

21/30 (70%)

Statistic of main interestN (%)

HR 45 (88%)OR 3 (6%)RR 2 (4%)

Slope 5(10%)

HR 11 (37%)OR/RR 9(30%)GLM 5 (17%)Other 5 (17%)

Mean 51 (76%)Median 28(42%)%-tiles 21(31%)Quality 31(46%)

16

MethodstoAddressMeasurementErrorNutritional Epi CohortN= 27*

Phys Activity Cohort N=40

Dietary Intake Pop. SurveyN=67

PollutionCohortN = 25

RegressionCalib. 26 (96%)SIMEX 1 (4%)Other 1 (4%)-----------------Search A: None 90%

RegressionCalib. 1(50%)

Other 1 (50%)-------------------Search A: None 95%

NCI 10(53%)Means 7(37%)ISU 1 (5%)MSM 1 (5%)------------------Search A: None 72%

Sens Analysis4 (80%)

Instr Variables1 (20%)

------------------Search A:None 94%

• Number excludes articles that were identified by search terms but upon review did not use a method to correct for error.

• Row percents do not add to 100% due to use of multiple methods.

17

OtherObservationsfromDietandPhysicalActivityCohortSurveys

• Commoninthecohortstudiestohavemultiplecovariateswitherror:eg diet+physicalactivity,smoking,and/oralcoholintake– Manyadjustforbothdiet+PA,only1articleadjustedforerrorinboth

physicalactivity(Zhangetal,AJE2014)– Errorsinsmoking/alcoholnotaddressed

• Mostcategorizedthecontinuousexposures– Impactsofcategorizinganexposuresubjecttoerrorareignored– Commonbelief:categorizationwilllowerimpactofmeasurementerrorin

theanalysis

• Mostpeoplewhomentionederrorasaproblemmadeanincomplete/incorrectclaim– Manyonlymentionedattenuationinfoundassociations– Someclaimednobiasinassociationssinceprospectivesubjectrecall– Someclaimednobiassinceinstrumentwasvalidated

18

OtherobservationsfromDietaryIntakePopulationSurveys

• Moststudies(80%)used24HRasprimaryinstrument– 31/53usedonly124HR,resthadrepeatsonatleastasubsample

– 8/31(26%)reportedpercentilessubjecttobias

• 16/31paperswith124HRmentionedthatusualintakeoradjustmentforwithin-personvariationwasneeded

• 8/11(73%)ofpapersusingmultiple24HRstoreportmedians/percentiles,usedacomplexmethod(NCI/MSM)

19

OtherObservationsfromtheAirPollutionCohortSurvey

• Statementsaboutthemeasurementerrorareoftenvague– Theoriginofthemeasurementerrorisoftennotclearlyspecified– Thesizeandtheimpactofthemeasurementerrorisoftennotstated

• Measurementerrorisoftenmentionedbutrarelyaddressedindetailorcorrected– Themajorityofthestudiesusedailyandspatiallyaggregateddata– TheoftenprevailingBerkson error(throughtemporalandspatial

aggregation)isnotoronlyinsufficientlydescribedanditsimplicationsarenotdiscussed

– Errorsoriginatingfromstayingindifferentmicroenvironmentsareoftenneglectedoronlypoorlyconsidered

• Manydifferentexposuremeasuresareanalyzedseparatelyorjointly;ahomogeneousprocedureislacking

20

Conclusions• Incohortstudies:measurementerroracknowledged,but

implicationsnotfullyunderstoodandcommonlynotaddressedinstatisticalanalysis– Veryfewusedmethodstoadjustformeasurementerror– ForPAstudies,littlemotivationtoadjustforerrorsincethenaïve

associationsaregenerallyalignedwithapriorihypotheses– Manystudieshadmultiplevariablesmeasuredw/error

• Indietaryintakepopulationsurveys:minoritycorrectedformeasurementerror– Majorityofthosethatdidapplyacorrectionmethodweretaking

advantageofsoftware(e.g.NCImethod)

• RegressioncalibrationmostcommonmethodtoaddressmeasurementerrorindietandPAstudies

21

Moreworkisneeded....

• Identifythevarioussourcesofmeasurementerror• Disseminateideasofmeasurementerrorcorrection

– Discussionofsoftwareinguidancedocuments,tutorialsinclinicaljournals,talksatepiandclinicalconferences

• Correctmisconceptions,suchas:– Randomerrorwon’tcausebiasinassociations– Attenuationistheonlypossibledirectionofbias– Categorizationreducestheeffectofmeasurementerror– Validatedquestionnairesdon’thavebias– Softwareisnotavailable

22

ReferencesRegressionCalibration• PrenticeRL.Covariatemeasurementerrorsandparameterestimationina

failuretimeregressionmodel.Biometrika.1982Aug1;69(2):331-42.• ZhengC,BeresfordSA,VanHornL,TinkerLF,ThomsonCA,Neuhouser ML,Di

C,MansonJE,Mossavar-Rahmani Y,SeguinR,ManiniT,LaCroix AZ,PrenticeRL.Simultaneousassociationoftotalenergyconsumptionandactivity-relatedenergyexpenditurewithrisksofcardiovasculardisease,cancer,anddiabetesamongpostmenopausalwomen.AmJEpidemiol.2014Sep1;180(5):526-35.

Simex• CookJandStefanski LA.Asimulationextrapolationmethodforparametric

measurementerrormodels.JournaloftheAmericanStatisticalAssociation1995;89:1314-1328.

• Küchenhoff,H,Mwalili SM,Lesaffre E.Ageneral method for dealing withmisclassification inregression:the misclassification SIMEX.Biometrics.2006;62(1):85-96.

23

References(2)IowaStateUniversityMethod(ISU)• Nusser,S.M.,Carriquiry,A.L.,Dodd,K.W.,andFuller,W.A.1996.ASemiparametricApproach

toEstimatingUsualIntakeDistributions.JournaloftheAmericanStatisticalAssociation,91:1440-1449.

MultipleSourceMethod(MSM)• Harttig U,Haubrock J,Knüppel S,BoeingH.2011TheMSMprogram:web-basedstatistics

packageforestimatingusualdietaryintakeusingtheMultipleSourceMethod.Eur JClin Nutr.65S1:S87-9

• Haubrock J,Nöthlings U,Volatier JL,Dekkers A,Ocké M,Harttig U,Illner AK,KnüppelS,Andersen LF,BoeingH;EuropeanFoodConsumptionValidationConsortium.EstimatingusualfoodintakedistributionsbyusingthemultiplesourcemethodintheEPIC-PotsdamCalibrationStudy.JNutr,141,914-20

NCIMethod• Tooze JA,Midthune D,DoddKW,FreedmanLS,Krebs-SmithSM,Subar AF,GuentherPM,

CarrollRJ,Kipnis V.Anewstatisticalmethodforestimatingtheusualintakeofepisodicallyconsumedfoodswithapplicationtotheirdistribution<image001.gif>.JAmDietAssoc 2006Oct;106(10):1575-87.

• Tooze JA,Kipnis V,Buckman DW,CarrollRJ,FreedmanLS,GuentherPM,Krebs-SmithSM,Subar AF,DoddKW.Amixed-effectsmodelapproachforestimatingthedistributionofusualintakeofnutrients:theNCImethod<image001.gif>.StatMed2010Nov30;29(27):2857-68.

24

25

DietaryIntakePopulationStudies:SurveyMethodology

• Daterange:Jan2012–May2015

• Term“Measurementerror”nottypicallyreferredtoindietaryintakesurveys– Understoodasvariancearoundusualintake– ConductedSearchAonly

26

PhysicalActivityCohortStudies:SurveyMethodology

• Daterange:Jan2012– Sep2015

• SearchA:Verybroadsearchterms:N=8760fromsearch;randomlyselectedN=610;N=51fromabstractreview

• SEARCHB:Added"measurementerror"ORmisreport*ORmisclassif*ORbiasORattenuat*ORcalibrat*– N=610fromsearch;N=86fromabstractreview

27

AirPollutionCohortStudies:SurveyMethodology

• Daterange:Jan2012– Dec2014

• SearchAbroadsearchwithin„WebofScience“:

– SearchBAdditionalkeywords: "measurementerror”,"measurementuncertainty”,misclassif*,attenuat*

– A:4595hits,B:386hits

• Afterabstractreview:A:431hits,B:32hits

• Randomselection:SearchA:50/SearchB:25

28