View
216
Download
0
Category
Preview:
Citation preview
A Case Approach to Rating Events and A Case Approach to Rating Events and Difficulties in the National Comorbidity Survey 2 Difficulties in the National Comorbidity Survey 2
(aka “Down and Dirty with the Data”)(aka “Down and Dirty with the Data”)
Elaine WethingtonElaine WethingtonCornell UniversityCornell University
&&
Joyce SeridoJoyce SeridoUniversity of ArizonaUniversity of Arizona
May 20, 2005May 20, 2005
AcknowledgmentsAcknowledgments
Ronald C. Kessler (Harvard)Ronald C. Kessler (Harvard) George W. Brown (London)George W. Brown (London) William Eaton (Johns Hopkins)William Eaton (Johns Hopkins) StudentsStudents
Catherine J. TaylorCatherine J. Taylor Lauren Beckles, Karina Chapman,Sarah Lauren Beckles, Karina Chapman,Sarah
Howe, Ninfa Leal, Dhurgha Reddy, Jessica Howe, Ninfa Leal, Dhurgha Reddy, Jessica Richards, Richards,
Aims of the PresentationAims of the Presentation
Introduce a case-review approach to coding and Introduce a case-review approach to coding and rating rating conventional survey measures of life conventional survey measures of life events and difficultiesevents and difficulties Will apply method to prediction of onsets of disorderWill apply method to prediction of onsets of disorder
Why we did it:Why we did it: Useful to the life course approach Useful to the life course approach Make the most of conventional survey methods and a Make the most of conventional survey methods and a
pre-existing datasetpre-existing dataset Reduce cost of producing detailed data on stressorsReduce cost of producing detailed data on stressors
SampleSample
Re-interview in 2000-2003 of respondents Re-interview in 2000-2003 of respondents from National Co-Morbidity Survey (NCS I from National Co-Morbidity Survey (NCS I – 1990-1992)– 1990-1992) N=5006N=5006
85% retention rate from wave 185% retention rate from wave 1
Interview questions at the National Interview questions at the National Comorbidity Survey web site:Comorbidity Survey web site:
http://www.hcp.med.harvard.edu/ncs/http://www.hcp.med.harvard.edu/ncs/
Measures in the National Measures in the National Comorbidity Survey 2Comorbidity Survey 2
Approximately 200 questions about life events Approximately 200 questions about life events and difficulties -- based on NCS-1 (Kessler and difficulties -- based on NCS-1 (Kessler et al., 1994); Detroit Area Survey 1985 et al., 1994); Detroit Area Survey 1985 (Kessler et al., 1984); Structured(Kessler et al., 1984); Structured Life Event Life Event Interview (Wethington et al., 1995)Interview (Wethington et al., 1995)
Onsets of depression, anxiety disorders, IED, Onsets of depression, anxiety disorders, IED, PTSD, substance abuse (lifetime and 12 month); PTSD, substance abuse (lifetime and 12 month); self-reported disability associated with disorders self-reported disability associated with disorders
Social support, personality, mood, childhood Social support, personality, mood, childhood conditions, demographicsconditions, demographics
MethodsMethods
Constructed a one-year case history of Constructed a one-year case history of life events and difficulties (follow-up year life events and difficulties (follow-up year only)only)
SAS programming stepSAS programming step: : generated event and generated event and difficulty records from close-ended questions and difficulty records from close-ended questions and dates dates
Case review stepsCase review steps::1.1. Coded open-ended responses using narrative Coded open-ended responses using narrative
text and limited demographic characteristics text and limited demographic characteristics 2.2. Scanned entire case record and eliminated Scanned entire case record and eliminated
duplicate mentions of events/difficultiesduplicate mentions of events/difficulties3.3. Rated events and difficulties on key dimensionsRated events and difficulties on key dimensions
Dimensions Rated for First AnalysesDimensions Rated for First Analyses
Event vs. difficulty Event vs. difficulty Severity, defined as long-term threat (estimated Severity, defined as long-term threat (estimated
threat 10-14 days after the event occurrence)threat 10-14 days after the event occurrence) Measure presented today conflates estimated level of Measure presented today conflates estimated level of
severity and certainty of the ratingseverity and certainty of the rating NOT “contextual threat” ratingsNOT “contextual threat” ratings
ContentContent Focus (who the event happened to: subject, Focus (who the event happened to: subject,
other, joint) and relationshipother, joint) and relationship Loss (Lazarus; Brown & Harris)Loss (Lazarus; Brown & Harris) Danger (Brown & Harris; Dohrenwend)Danger (Brown & Harris; Dohrenwend)
ProcessProcess
Strategy – put together information from 3 pieces of Strategy – put together information from 3 pieces of paperpaper SAS program step to generate lists for each respondentSAS program step to generate lists for each respondent
Coding and rating done by 5 studentsCoding and rating done by 5 students
Reduced multiple event/difficulty records to one, when Reduced multiple event/difficulty records to one, when appropriateappropriate
Each case checked by investigatorsEach case checked by investigators
Data checked and re-checked extensivelyData checked and re-checked extensively
Total process: about 9 monthsTotal process: about 9 months
Unlike some case review and stressor rating Unlike some case review and stressor rating methods… methods… Social context (e.g. availability of support from others) Social context (e.g. availability of support from others)
NOT used in rating severityNOT used in rating severity ““Objective” details onlyObjective” details only (However, humiliation and entrapment ratings were not (However, humiliation and entrapment ratings were not
possible – dependent on knowing social context)possible – dependent on knowing social context)
ALL information preserved for future useALL information preserved for future use Other coding methods possible, e.g.Other coding methods possible, e.g.
Short-term threat Short-term threat More detailed information about focus and contentMore detailed information about focus and content
Comparison of SAS Generated and Comparison of SAS Generated and Case-Reviewed Events and DifficultiesCase-Reviewed Events and DifficultiesSource of Question SAS Reviewed
9/11 1066 300Traumas (12 mo.) 1203 483Health Screening 0 28R Illness 1044 1030Employment 2569 2322Finances 1751 1188Spouse/Partner Rel. 2050 1988Children 1428 1118Social Networks 5099 4976Other Life Events 1837 1316
Total 18,047 14,749
Key Characteristics of Reviewed and Key Characteristics of Reviewed and Rated Events and DifficultiesRated Events and Difficulties
Events 10,057 68.2%Difficulties 4,692 31.8%
Events SAS coding sufficient 8,688 86.2% Intervention necessary 1,389 13.8%
Difficulties SAS coding sufficient 3,473 74.0% Intervention necessary 1,219 26.0%
Focus of Rated Events/Difficulties and Focus of Rated Events/Difficulties and Relationship to SubjectRelationship to Subject
Relationship
Spouse ChildrenOther
/Partner
Self 5769 39.1%
Joint 3042 20.6% 65.6% 15.0% 19.4%
Other 5938 40.3% 9.4% 12.0% 78.6%
Number of Events/Difficulties Reported: Number Number of Events/Difficulties Reported: Number of Cases Reporting 0 to 12 or moreof Cases Reporting 0 to 12 or more
SAS After ReviewCount % Count %
0 574 11.5 652 13.01 825 16.5 970 18.52 776 15.5 924 15.63 708 14.1 782 11.94 600 12.0 595 7.35 444 8.9 365 5.66 324 6.5 280 3.67 239 4.8 181 2.08 159 3.2 102 2.39 105 2.1 57 1.110 73 1.5 37 .711 46 .9 33 .712 or more 133 2.6 38 .5
Severity Ratings, by MethodSeverity Ratings, by Method SAS ª After Review
Events Difficulties Events DifficultiesSeverity Count % Count % Count % Count %
Severe 450 4.2 445 11.4 573 5.7 507 10.8
Probably Severe 2682 25.0 2406 61.5 2126 21.1 2509 53.4
Possibly Severe 7592 70.8 1062 27.1 7179 71.4 1617 34.5
Not severe 0 0.0 0 0.0 179 1.8 59 1.3
ª 3409 entries generated from open-ended questions and not classifiable by SAS as either events or difficulties are excluded
““Severe” and “Probably Severe” Reports: Severe” and “Probably Severe” Reports: Percent by Sex (Case Review: Weighted)Percent by Sex (Case Review: Weighted)
Events Difficulties Severe Probably Severe Probably
Male (48.6%)* 43.2 48.0 37.9 46.8Female (51.4%) 56.8 52.0 62.1 53.2
*Proportion in sample
““Severe” and “Probably Severe” Reports: Percent Severe” and “Probably Severe” Reports: Percent by Age Groups (Case Review: Weighted)by Age Groups (Case Review: Weighted)
Events Difficulties Severe Probably Severe Probably
25 – 34 (22.6%)* 19.8 25.9 22.0 21.035 – 44 (28.9%)* 31.8 33.3 36.8 29.845 – 54 (29.2%)* 29.2 28.7 26.0 30.1Over 54 (19.3%)* 19.2 12.1 15.2 19.1
*Proportion in sample
““Severe” and “Probably Severe” Reports: Percent by Severe” and “Probably Severe” Reports: Percent by Level of Education (Case Review: Weighted)Level of Education (Case Review: Weighted)
Events Difficulties Severe Probably Severe
Probably
Less than HS (12.7%)* 22.7 13.3 14.9 17.6HS degree (30.5%)* 26.5 29.7 32.4 31.3
Some college (28.0%)* 30.8 30.4 27.2 29.0
College degree (28.8%)* 20.0 26.6 25.5 22.1
*Proportion in sample
Reliability and ValidityReliability and Validity
Inter-rater reliabilityInter-rater reliability Fall-off over 12 months Fall-off over 12 months Predictive validity (relationship to onsets)Predictive validity (relationship to onsets)
Inter-rater ReliabilityInter-rater Reliability
KappaKappa AlphaAlpha
Event vs. DifficultyEvent vs. Difficulty .95.95 .93.93
LossLoss (yes/no)(yes/no) .95.95 .97.97
Danger (yes/no)Danger (yes/no) .95.95 .97.97
SeveritySeverity .89.89 .90.90
FocusFocus .89.89 .89.89
Classification codeClassification code -------- .82.82
Comparison of Falloff in Reported Frequency of Severe and Probably Severe Events
0
100
200
300
400
500
600
700
800
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0
Months Prior to Interview Month
Nu
mb
er o
f Sev
ere
and
Pro
bab
ly S
ever
e E
ven
ts
SAS Generated Known Events
SAS Generated All Possible Events
Rated Events
Preliminary Analyses of Predictive Preliminary Analyses of Predictive ValidityValidity
Case review method cleanly distinguishes Case review method cleanly distinguishes events from difficultiesevents from difficulties
Both SAS generated method and Case Both SAS generated method and Case review method show:review method show: Severe “occurrences” in month of onset are Severe “occurrences” in month of onset are
related to onset of depressionrelated to onset of depression Preliminary findings indicate that severe Preliminary findings indicate that severe
events are related to onset of depression events are related to onset of depression within 30-60 days (after that effect decays)within 30-60 days (after that effect decays)
LimitationsLimitations
Men reported less detail in open-ended Men reported less detail in open-ended questions (affects rating)questions (affects rating)
Stigmatized behavior under-reported (e.g. jail Stigmatized behavior under-reported (e.g. jail time had to be inferred)time had to be inferred)
More complicated contextual rating schemes More complicated contextual rating schemes using trained interviewers are much better at:using trained interviewers are much better at: Dating onsets and offsets of difficultiesDating onsets and offsets of difficulties
Matching related events and difficulties to each otherMatching related events and difficulties to each other
FindingsFindings
Falloff appears to be reduced – Falloff appears to be reduced – perhaps perhaps artifactually?artifactually?
Reduces cost of using case-review methodsReduces cost of using case-review methods
• Trained coders but conventionally trained interviewersTrained coders but conventionally trained interviewers
• Takes less time to code more interviewsTakes less time to code more interviews
Method can be used in very large datasets Method can be used in very large datasets • 5006 cases rated and entered in 2 months5006 cases rated and entered in 2 months
• 4 months additional checking4 months additional checking
• Previous study using more complicated methods took 9 Previous study using more complicated methods took 9 months to interview, code, and rate 100 interviewsmonths to interview, code, and rate 100 interviews
SAS generation techniques could be SAS generation techniques could be applied to many pre-existing datasetsapplied to many pre-existing datasets But you have to live with ambiguity…But you have to live with ambiguity…
Preserves more information about events Preserves more information about events and difficulties than other case rating and difficulties than other case rating methodsmethods• Test hypotheses about different rating Test hypotheses about different rating
schemes?schemes?
Recommended