Friday Harbor Psychometrics 2012
Scientific SummaryUC Davis / SENAS (Spanish and English Neuropsychological
Assessment Scales)
Friday Harbor Psychometrics Workshop 2011
Acknowledgements
• Funded in part by Grant R13AG030995-01A1 from the National Institute on Aging
• The views expressed in written conference materials or publications and by speakers and moderators do not necessarily reflect the official policies of the Department of Health and Human Services; nor does mention by trade names, commercial practices, or organizations imply endorsement by the U.S. Government.
Overview
• SENAS Overview• Demographic variables and longitudinal change
• Ethnicity, clinical diagnosis, and cognitive change
• Age, education, and relationship to brain• Decomposing demographic and brain effects• Intracranial volume and cognition• Factorial invariance• Final thoughts
Friday Harbor Psychometrics 2012
SENAS Overview
Friday Harbor Psychometrics 2012
Friday Harbor Psychometrics 2012
Purpose of SENAS Project (circa 1992)
• Create matched English and Spanish language neuropsychological tests for ages 60+ New scales based upon neuropsychological model of cognitive functioning
Scales psychometrically matched•Within English and Spanish language versions
•Between English and Spanish language versions
Distribution of item difficulty appropriate to elderly population
Friday Harbor Psychometrics 2012
Approach
• Concept of item and test bias central to project
• Development and validation based on modern psychometric methods Item response theory (IRT) Latent variable modeling
• Empirically based Data based approach to making decisions about item selection and scale construction
Empirical evaluation of reliability and validity
Friday Harbor Psychometrics 2012
Guiding PrinciplesInitial Scale Development
• Scales targeted to assess neuropsychologically relevant cognitive domains
• Verbal and non-verbal measures• Non-timed• New scales, not translations of existing scales
• Examiner administered• Item generation
•New items•Broad range of difficulty
Friday Harbor Psychometrics 2012
SENAS Scales and Abilities Measured
Friday Harbor Psychometrics 2012
Model for Item Selection
Friday Harbor Psychometrics 2012
Test Information for 3MS and SENAS Object Naming
Mungas et al., 2004
Scientific Applications
Friday Harbor Psychometrics 2012
Summary Conclusions / Challenge for Conference
• Demographic variables have robust effects on baseline test scores Especially ethnicity and education
• Demographic variables have minimal effects on longitudinal change Brain and disease variables account for change
Friday Harbor Psychometrics 2012
Friday Harbor Psychometrics 2012
African AmericanN ≈ 425
Hispanic EnglishN ≈ 450
Hispanic SpanishN ≈ 900
CaucasianN ≈ 700
English
Hispanic
Community
African AmericanN ≈ 140
HispanicN ≈ 130
CaucasianN ≈ 235
Longitudinal
Clinic
N ≈350
N ≈ 175
Cognition
Clinical
MRI
LifeExperience
Morphometric
Measures
Ethnicity, education, and cognitive change
Friday Harbor Psychometrics 2012
Purpose of this study
• Understand how demographic variables relate to cognitive change
• Demographic variables of interest Race/Ethnicity Education Language of test administration
Friday Harbor Psychometrics 2012
Independent Variables
• Model 1 Verbal Memory Form Previous Evaluation Spanish Administration
Prev Eval by Spanish Interaction
• Model 3 Age & Education Gender
• Model 2 Race/Ethnicity
• Model 4 Recruitment Source Clinical Diagnosis APOE Genotype
Friday Harbor Psychometrics 2012
Language, Ethnicity, Education Effects
Episodic Memory
Friday Harbor Psychometrics 2012
Baseline Change
Effect Estimate
SE Estimate
SE
Intercept .62 .08
Time -.04 .02
Previous Eval
-.06 .04
Language (Spanish)
-.30 .13 -.04 .04
Span x PrevEval
.33 .12
Hispanic -.29 .11 .04 .03
African American
-.06 .08 .01 .02
Education .03 .01 -.00 .00
Language, Ethnicity, Education Effects
Executive Function
Friday Harbor Psychometrics 2012
Baseline Change
Effect Estimate
SE Estimate
SE
Intercept .32 .07
Time -.03 .02
Previous Eval
.01 .03
Language (Spanish)
-.08 .11 -.01 .03
Span x PrevEval
.08 .07
Hispanic -.32 .09 .02 .03
African American
-.30 .07 .04 .02
Education .05 .01 -.00 .00
Executive Function Trajectories by Education
Friday Harbor Psychometrics 2012
Average Ethnic Group Performance by Model –
Episodic Memory
Friday Harbor Psychometrics 2012
Ethnicity, clinical diagnosis, and cognitive change
Friday Harbor Psychometrics 2012
Episodic Memory by Diagnosis Change
Friday Harbor Psychometrics 2012
Mungas et al., 2010
Episodic Memory by Diagnosis Change
African American
Friday Harbor Psychometrics 2012
Episodic Memory by Diagnosis Change
Hispanic
Friday Harbor Psychometrics 2012
Episodic Memory by Diagnosis Change
Caucasian
Friday Harbor Psychometrics 2012
Age, education, and relationship to brain
Friday Harbor Psychometrics 2012
Friday Harbor Psychometrics 2012
MRI Effect Sizes and Age and Education Adjustment -
Episodic Memory
Mungas et al., 2009
Friday Harbor Psychometrics 2012
Executive Function, MRI, Age & Education
African Americans
Executive Function
Age BM Education-.55 .00
-.03 .32.36
.39-.26
-.24
Mungas et al., 2009
Friday Harbor Psychometrics 2012
Executive Function, MRI, Age & EducationHispanics
Executive Function
Age BM Education-.60 .00
-.17 .46.30
.44-.40
.01
Mungas et al., 2009
Friday Harbor Psychometrics 2012
Executive Function, MRI, Age & EducationCaucasians
Executive Function
Age BM Education-.53 .00
-.10 .29.29
.28-.36
-.07
Mungas et al., 2009
Friday Harbor Psychometrics 2012
Age and Education Influences on
MRI - Cognition Relationships• It is possible to separate disease effects from extraneous influences
• MRI effects on cognition can obscured by demographic effects on test performance, especially when
•Demographic relationship with test score is larger than relationship with disease
•Substantial heterogeneity of demographic variable in population of interest
Decomposing demographic and brain effects
Friday Harbor Psychometrics 2012
Reed et al., 2010
Friday Harbor Psychometrics 2012
Friday Harbor Psychometrics 2012
Data
• 288 subjects. 158 normal, 92 MCI, 38 demented
• 96 African Americans, 74 Hispanics (32 tested in English, 42 tested in Spanish), and 118 Caucasians.
• Education M = 12.7 yrs (range 0-25)
• Age M = 74.7 yrs (range 60-93)
• Mean evaluations = 3.5; 74% had 3 or more evaluations. N of evaluations truncated at 5.
Reed et al., 2010
Friday Harbor Psychometrics 2012
• Mem-D ~ 20% of episodic memory variance
• Mem-B ~ 20% of episodic memory variance
• Mem-R ~ 50% of episodic memory variance
Reed et al., 2010
Friday Harbor Psychometrics 2012
Relationships of memory components with global cognitive function (CDR sum of
boxes)
DV Memory Component
Standardized Coefficient
p
CDR Sum Mem-D -0.09 ns
Mem-B -0.43 0.001
Mem-R -0.44 0.001
Reed et al., 2010
Friday Harbor Psychometrics 2012
Relationships of memory components with clinical progression (conversion to MCI
or dementia)
Memory Component
Relative Risk Ratio (confidence interval)
Mem-D 1.58 (0.92 - 2.71)
Mem-B 0.19 (0.11 - 0.33)
Mem-R 0.27 (0.18 - 0.40)
Reed et al., 2010
Friday Harbor Psychometrics 2012
Relationships of memory components with longitudinal change in cognition
(executive function)
Memory Component
Random Effect
Parameter Standard Error
p
Mem-D baseline 0.364 0.043 0.001
Mem-B baseline 0.180 0.031 0.001
Mem-R baseline 0.329 0.034 0.001
Mem-D change -0.011 0.010 ns
Mem-B change 0.050 0.010 0.001
Mem-R change 0.047 0.011 0.001
Reed et al., 2010
Friday Harbor Psychometrics 2012
Effects of Mem-D?
• Substantial variance in Episodic Memory was uniquely related to demographic variables (ethnicity and education especially) Greater amount for other cognitive domains
• Mem-D was minimally related to clinical outcomes Especially longitudinal decline and conversion
• Challenge in clinical neuropsychological assessment is to separate demographic from brain influences on test scores
Reed et al., 2010
Intracranial volume and cognition
Friday Harbor Psychometrics 2012
Farias et al., 2012
Model of joint effects of intracranial volume and brain
structure
Friday Harbor Psychometrics 2012
Farias et al., 2012
Semantic Memory by ICV
Friday Harbor Psychometrics 2012
Farias et al., 2012
Cognition and ICV
• ICV related to Semantic Memory and Executive Function
• ICV not related to Episodic Memory
• Early development might have impact on cortical development
Friday Harbor Psychometrics 2012
Farias et al., 2012
Factorial invariance
Friday Harbor Psychometrics 2012
Friday Harbor Psychometrics 2012
Sample
• Community dwelling recruited using cognitive screening protocol and clinic referrals Whites - n=678 Blacks - n=352 Hispanics, English Speaking n=434 Hispanics, Spanish Speaking n=877
• Broad range of cognitive function Normal to demented
Mungas et al., 2011
Friday Harbor Psychometrics 2012
Word List Learning 2
Spatial Config Learning
Word List Learning 1
Verbal Comprehension
Verbal Expression
Verbal Abstraction
Picture Association
Object Naming
Category Fluency
Phonemic Fluency
Working Memory
Verbal Attention
Visual Attention
NonVerbal Reasoning
Pattern Recognition
Spatial Localization
Best Factor Structure
Fluency
Spatial
Verbal
Memory
Attention
Mungas et al., 2011
Friday Harbor Psychometrics 2012
Invariance of Dimensional Structure Across Ethnic and
Language Groups• Same number of dimensions• Invariant Factor Loadings
Observed test scores have same relationship to latent dimensions
• Some differences in Intercepts for tests Spanish speaker has to have greater latent attention ability to achieve a given Verbal Attention score
Mungas et al., 2011
Friday Harbor Psychometrics 2012
Verbal Attention by Attention/Working Memory
Mungas et al., 2011
Final Thoughts
Friday Harbor Psychometrics 2012
Longitudinal Trajectories
Friday Harbor Psychometrics 2012
Longitudinal Trajectories
Friday Harbor Psychometrics 2012
Conclusions
• Cross-sectional results can be misleading Confound lifelong ability and disease effects
• Longitudinal assessment important to separate these effects Ideally beginning in mid adulthood In absence of ideal – evidence that demographic effects on cognitive decline are small and distal
• Don’t assume a low score is impairment
Friday Harbor Psychometrics 2012
References
• Farias, S. T., D. Mungas, et al. (2012). "Maximal brain size remains an important predictor of cognition in old age, independent of current brain pathology." Neurobiol Aging 33(8): 1758-1768.
• Mungas, D., B. R. Reed, et al. (2004). "Spanish and English Neuropsychological Assessment Scales (SENAS): Further development and psychometric characteristics." Psychological Assessment 16(4): 347-359.
• Mungas, D., B. R. Reed, et al. (2005). "Spanish and English Neuropsychological Assessment Scales: relationship to demographics, language, cognition, and independent function." Neuropsychology 19(4): 466-475.
Friday Harbor Psychometrics 2012
References
• Mungas, D., B. R. Reed, et al. (2009). "Age and education effects on relationships of cognitive test scores with brain structure in demographically diverse older persons." Psychology and Aging 24(1): 116-128.
• Mungas, D., L. Beckett, et al. (2010). "Heterogeneity of cognitive trajectories in diverse older persons." Psychol Aging 25(3): 606-619.
• Mungas, D., K. F. Widaman, et al. (2011). "Measurement invariance of neuropsychological tests in diverse older persons." Neuropsychology 25(2): 260-269.
• Reed, B. R., D. Mungas, et al. (2010). "Measuring cognitive reserve based on the decomposition of episodic memory variance." Brain 133: 2196-2209.
Friday Harbor Psychometrics 2012
Friday Harbor Psychometrics 2012
Ethnic Differences and DIF
• SENAS Picture Association and Verbal Learning
• English Administration - N = 1113 396 Hispanics, 480 Whites, 237 Blacks
• Spanish Administration - N = 801 • Ability adjusted for all-source DIF Education, Ethnicity, Age
• MIMIC approach
Friday Harbor Psychometrics 2012
Picture AssociationEffects of DIF Adjustment
Friday Harbor Psychometrics 2012
Word List LearningEffects of DIF Adjustment
Friday Harbor Psychometrics 2012
Effect of DIF Adjustment on Individual Scores (95% confidence interval)
PictureAssociation
Word ListLearning
White ±.29 s.d. ±.10 s.d.
Black ±.41 s.d. ±.02 s.d.
Hispanic - English
±.23 s.d. ±.00 s.d.
Hispanic - Spanish
±.36 s.d. ±.06 s.d.
Friday Harbor Psychometrics 2012
SENAS and DIF
• DIF accounted for a very small amount of ethnic differences in Object Naming, Picture Association, Word List Learning
• DIF effects differ by domain Greater for measures of semantic memory
Smaller for episodic memory• DIF effects can potentially bias interpretation of individuals scores Especially in borderzone areas
Cross-sectional ethnic differences can be explained
by confounding variables
Friday Harbor Psychometrics 2012
Friday Harbor Psychometrics 2012
Semantic Memory Mean Ethnic Group Differences