Embed Size (px)
Citation preview
Change in quality of life of people with stroke over time: True change or
response shift?
Sara Ahmed1, Nancy E. Mayo2,3, Marc Corbiere4, Sharon Wood-Dauphinee5,6,7, James Hanley8,9 &Robin Cohen10,11,121Department of Epidemiology and Biostatistics, Faculty of Medicine, McGill University; 2Division of ClinicalEpidemiology, Royal Victoria Hospital, Montreal, Quebec (E-mail: [email protected]); 3Faculty ofMedicine, School of Physical and Occupational Therapy, McGill University; 4Michael Smith Foundation forHealth Research, University of British Columbia; 5School of Physical and Occupational Therapy; 6Faculty ofMedicine, Department of Epidemiology and Biostatistics, McGill University; 7Division of ClinicalEpidemiology, Royal Victoria Hospital, Montreal; 8Division of Clinical Epidemiology, Royal VictoriaHospital; 9Department of Epidemiology and Biostatistics, McGill University; 10National Cancer Institute ofCanada and Canadian Institutes of Health Research; 11Departments of Oncology and Medicine, McGillUniversity; 12McGill University Health Center Medical Scientist, Canada
Accepted in revised form 23 September 2004
Abstract
In many studies, assessments of change in self-report measures such as health-related quality of life mustaccount for potential response shift, including reconceptualization and changes in internal standards ofmeasurement. Objective: The objective of our study was to compare healthy controls and individuals withstroke on the extent to which changes in internal standards and reconceptualization of health relatedquality of life (HRQL) occurs over the first 6 months post-stroke. Methods: Confirmatory factor analysiswas used to assess invariance of the SF-36 measurement model over time among 238 individuals with strokeand 392 controls, separately. This procedure assessed changes over time in the factor loadings, variances,and covariances of responses, and compared the extent of change between individuals with stroke and thosein the control group. In addition a multisample comparison was made between individuals with stroke andmembers of the control group at the first evaluation in order to assess invariance of the SF-36 measurementmodel between the groups. The controls were considered to be a ‘proxy’ for the stroke cohort prior to thestroke. Results: We found no evidence of reconceptualization and changes in internal standards over timewhen the groups were assessed separately. There was a significant difference in the factor covariances(reconceptualization) between the two groups at the time of the first evaluation. However, measurementerror was also significant for this comparison. Conclusion: This study indicates that the improvement inHRQL over time is real rather than a result of reconceptualization or a recalibration. If response shift doesoccur with stroke it is likely to be mediated by the event itself and not the recovery process.
Key words: Change, Confirmatory factor analysis, Health-related quality of life outcomes, Response shift,Stroke
Introduction
Many investigators have moved away from relyingonly on clinical measures and have started toincorporate measures that target a self-evaluation
of the individuals’ own health status. Measures ofhealth related quality of life (HRQL) have beendeveloped to serve this purpose, and extensiveefforts have gone into their development. [1–5]Many authors, however, have agreed that the
Quality of Life Research (2005) 14: 611–627 � Springer 2005
assessment of HRQL is challenging, burdened bymethodological issues related to the subjectivenature of this construct. [6–8]. Despite efforts todevelop reliable, valid measures that are based onsound conceptual thinking, assessments of HRQLover time or after an intervention may still besubject to what has been referred to as responseshift. Response shift is defined as ‘a change in themeaning of one’s self-evaluation of a target con-struct as a result of: (1) a change in the respon-dent’s internal standards of measurement (i.e. scalerecalibration) (2) a change in the respondent’svalues (i.e. the importance of component domainsconstituting the target construct) or (3) a redefi-nition of the target construct (i.e. reconceptual-ization).’[8]. The concepts that define responseshift stem from a typology of change developed byorganizational psychologists. Golembiewskiet al. [9] distinguished three types of change thatmay occur in any type of evaluation that requiresself-reports:
‘Alpha change involves a variation in the level ofsome existential state, given a constantly cali-brated measuring instrument related to a constantconceptual domain (true change).
Beta change involves a variation in the level ofsome existential state, complicated by the fact thatsome intervals of the measurement continuumassociated with a constant conceptual domainhave been recalibrated (rubber yard-stick).
Gamma change involves a redefinition or recon-ceptualization of some domain, a major change inthe perspective or frame of reference within whichphenomena are perceived and classified, in what istaken to be relevant in some slice of reality.’
The term response shift encompasses beta (scalerecalibration) and gamma (reconceptualization ofa construct and change in values) change.Response shift is likely to occur under circum-stances when individuals experience a change inhealth status either because of illness or a treat-ment. We expect that individuals with stroke mayexperience a response shift as a means of copingand adapting to changes in health. Stroke is adisease that occurs with a sudden onset. Themajority of individuals will experience some levelof disability followed by a period of recovery,which, on average, is most rapid in the first5 weeks post-stroke and begins to plateau at about13 weeks [10–12]. These changes in health status
may impact stroke survivors’ views of the HRQLconstruct differently over time. For example, aperson who shortly after stroke is unable to walk50 m independently and judged their health asgood, who at 3 months post-stroke is able to walkin the community independently may now lookback and judge their initial health as poor (scalerecalibration). The meaning of HRQL may alsochange, where initially it meant good physicalhealth, and now encompasses relationships withfamily and friends (reconceptualization). Simi-larly, if he/she remains limited in areas of physicalfunctioning, then a strong social network andmental health may be more important for thisperson than physical health (change in values).Experiencing a response shift is seen as a naturalresponse to illness and is the result of mechanismsused to accept or adapt to changes in physical,emotional, and social health status [8, 13]. Theproblem arises in that a simple comparison ofHRQL mean scores over time or between groupsdoes not account for this response shift. Only afteridentifying and estimating the level of potentialresponse shift can observed average change bemeaningfully interpreted.
Difficulties with assessing change in HRQL arereflected in studies that have shown results that goagainst what we would intuitively expect, and raisequestions about the validity of the comparison ofHRQL measurements. Such studies include thosethat have found that individuals with lower levelsof physical functioning report higher levels ofperceived HRQL. King et al. [4] compared overallHRQL of stroke survivors to that of normativecontrols and found that HRQL scores were com-parable. A comparison of studies evaluating theoutcome of stroke have found that persons withmajor stroke have reported levels of HRQL higherthan individuals with moderate stroke [14]. Similarfindings have been reported in other patient pop-ulations. For example, patients with cancer havebeen found to report better levels of HRQL thanthe general population [15, 16]. People with spinalcord injuries surprisingly have been found toreport only slightly worse levels of HRQL ascompared to controls [17]. Response shift has beenapplied as a possible explanation for these para-doxes.
Several methods of evaluating the variouscomponents of response shift have been proposed.
612
A method, common to the educational andpsychology fields, uses factor analytic techniquesto assess data for dissimilarities in factor structureover time as an indication of response shift. Factoranalytic approaches have been used to assesschanges in self-reports of improvement for a spe-cific ability or characteristic such as self-esteem[18–21]. A group of researchers have operational-ized beta and gamma changes as changes in vari-ances and covariances of true scores underlyingthe observed responses of a group of individuals[20, 22, 23]. This can be examined through con-firmatory factor analysis (CFA) [20, 23]. With thistechnique, concepts of interest are interpreted as aset of factors in a measurement model. Gammachange is examined in terms of changes in thenumber of common factors (e.g. a one factorsolution at time 1 requires a two factor solution attime 2) or a change in the covariances amongfactors, because this reflects changes in the con-struct and the relationship between the constructs[20, 23]. Beta change is examined in terms of achange in the strength of the relationship betweenobserved variables and a common factor (i.e. thefactor loadings change which reflect the scalingunits of the factor) or a change in the variances ofthe constructs [20, 23]. The advantage of using aCFA technique is that it allows for the statisticalcomparison of separate components of the mea-surement model over time, and provides statisticaltests of fit and detailed information on departuresfrom the hypothesized structure [18, 24, 25].
Although distinction is made between the threetypes of response shift, it may be that they do notoccur independently. Researchers who haveexamined alpha, beta, and gamma change havesuggested a hierarchy between the types of change.Gamma change (reconceptualization) needs to beruled out before beta change (change in internalstandards) is evaluated, and beta change needs tobe accounted for before alpha (objective change)change can be inferred.
In the past several years steps have been takento employ methods to assess response shift [26]related to changes in health status and have beenapplied to groups with conditions such as coloncancer [27], breast cancer [28], and end stage renaldisease [29]. To date, there have been few reportsof the use of factor analysis to assess changes inHRQL.
The first objective of this study was to compareindividuals with stroke and healthy controls on theextent to which response shift, changes in internalstandards and reconceptualization of HRQL,occurs over the first 6 months post-stroke. Wehypothesized that individuals with stroke wouldexperience a greater degree of response shift ascompared to controls, reflected through greaterchanges in factor structure and loadings of ameasurement model based on the eight scales ofthe SF-36.
A secondary objective was to identify similari-ties and differences between controls and persons1 month post-stroke on indices of response shift.For this objective the control group was treated asa ‘proxy’ for individuals pre-stroke.
Methods
Subjects and data collection procedure
To investigate response shift, data from theMontreal Stroke Cohort study were used [30]. Thiswas a 2-year longitudinal prospective study,undertaken from 1996 to 1998 to determine thelong-term outcome of stroke in comparison to theimpact of natural aging [30]. This cohort consistedof community dwelling individuals with strokeadmitted to 1 of 10 acute care university hospitalsin the Montreal area, presenting with a diagnosisof stroke caused by cerebral ischemia or intrace-rebral hemorrhage. All persons meeting these cri-teria were asked to participate in the study exceptfor those presenting with another severe or chronicillness such as metastatic disease, Parkinson’s dis-ease, or if they were living in a residential andlong-term care facility at the time of the stroke.Ethical approval for this study was obtained fromall participating hospitals.
A total of 1321 individuals with stroke wereinitially eligible for the study. After excludingthose with early death (before being invited toparticipate in the study), and those discharged tolong-term care, 75% consented and 612 wereinterviewed at least once.
Participants were interviewed over the telephoneshortly after the stroke and then every 6 months fora period of 2 years. In parallel, a population-basedrepresentative sample of community dwelling
613
individuals without stroke matched for age, sex,and postal code served as a control group. Thisgroup was sampled through the use of randomlygenerated computerized telephone listings. Theonly exclusion criterion was a previous history ofstroke. Fifty percent of all eligible controls agreedto participate, leaving a total of 468 control sub-jects.
For the purpose of this study we were interestedin assessing the occurrence of response shift duringthe first 6 months post-stroke, when on averagemost of the recovery is expected to take place. Inaddition, each subject was required to have com-pleted at least two evaluations. Therefore, weselected individuals with stroke who had a firstinterview by 3 months post-stroke and who had atleast two evaluations. The number of individualswho met these two criteria was 238. Among thecontrols, 392 individuals had at least two evalua-tions.
The SF-36
In our study we used the SF-36 to evaluate HRQLpost-stroke. The SF-36 is a generic measure ofperceived health status that incorporates behav-ioral functioning, subjective well-being and per-ceptions of health by assessing eight healthconcepts: Physical Functioning (PF) (10 items),Role-Physical (RP), limitations in role activitiesdue to physical health problems (4 items), BodilyPain (BP) (2 items), General Health (GH)(5 items), Vitality (VT) (energy & fatigue) (4items), Social Functioning (SF) (2 items), MentalHealth (MH) (5 items), Role Emotional (RE),limitations in usual role activities due to emotionalproblems (3 items) [31].
All items are measured on Likert-type scales andresponse choices are varied and range from two tosix levels. The transformed scores on all scales rangefrom 0 to 100 with higher scores indicating betterhealth. [31]. The SF-36 can be self-administered oradministered by interview in person or by tele-phone. Reliability, both test–retest and internalconsistency, has been extensively demonstrated,[32–36] as have content, criterion and constructvalidity [31, 37, 38] and responsiveness to clinicalchanges. [39, 40] There is also supporting evidencefor the validity of the SF-36 among individuals withstroke [41].
The scales of the SF-36 have been combined toform mental (MCS) and physical componentsummaries (PCS) [37, 42]. Principal ComponentsAnalysis was used to provide support forhypothesized physical and mental components.Shown to have the strongest loadings on themental component were the Mental Health andRole-Emotional scales, and those that had thestrongest loadings on the physical component werethe Physical Functioning, Bodily Pain, and Role-Physical scales. The Vitality, General Health, andSocial Functioning scales were found to relate withboth the physical and mental components andcontribute to both summary scores.
Clinical measures
Clinical measures were administered to describethe physical impact of the stroke, and to comparethe extent of physical disability in the stroke andcontrol groups. Severity of stroke was assessedusing the Canadian Neurological Stroke Assess-ment (CNS). The CNS measures two aspects ofneurologic status, mentation and motor function.There is support for its reliability and validity [43].The Mini-Mental State Examination (MMSE) [44]is a cognitive test and gives a brief assessment ofthe person’s orientation to time and place, recallability, short-term memory and arithmetic ability.The Barthel Index [45] assesses the level of inde-pendence in performing basic activities of dailyliving (ADL). It has been found to be reliable [46–51], and valid [50–53] and measures 10 basicADLs, which cover independence in self-care,continence of bowel and bladder, and mobility. Itis scored from 0 to 100, with 100 indicating fullindependence. The OARS Instrumental activitiesof daily living (IADL) measure assesses higherlevel tasks than basic ADLs which are necessaryfor continued community living. The OARS IADLhas seven items, covering use of the telephone,traveling, shopping, meal preparation, housework,taking medications and handling finances [54]. TheReintegration to Normal Living Index (RNL) [55,56] is an 11-item scale that assesses handicapcovering areas such as participation in recreationaland social activities, movement within the com-munity, and how comfortable the individual is inhis or her role in the family and with other rela-tionships. It was developed simultaneously in
614
Canadian French and English, and can becompleted by either a patient or a significant other,and has been used over the telephone.
Analysis
A structural equation modeling framework wasused to carry out a confirmatory factor analysisusing Equations (EQS) software, version 5.7 [57].The use of structural equation modeling has beenreviewed by several authors [24, 58–61]. Structuralequation techniques allow for the testing ofmodels based on patterns of covariances observedbetween all dependent and independent variablesof interest. This is a technique that requires a largesample size, and a recommended ratio of numberof subjects to measured variables is 10:1 [24]. It isa confirmatory rather than an exploratory analysisbecause it statistically compares a pre-specifiedcovariance matrix with that obtained from thedata. Therefore, for a confirmatory factor analysisprior hypotheses must be specified about howcovariances between items or scales should havebeen caused by underlying factors. In this study,an a priori hypothesized model based on the eightSF-36 scales and a mental and physical healthconstruct was tested [37]. If the model did not fitthe data well, modifications were made. TheVitality, General Health, and Social Functioningscales of the SF-36 were not permitted to load onboth the physical and mental factors, and theerror terms of the Role Physical and Role Emo-tional scales were specified to correlate with eachother. The latter modification is appropriate, asthe errors of the scales on the SF-36 are highlyassociated [62].
The hypothesized relationships are depictedgraphically using the EQS software. The pathsconnecting the latent and measured variables arecollectively referred to as the measurement model.The latent factors in the model were a physicalhealth and a mental health construct. The eightscales of the SF-36 are the measured variablesexplained by the latent variables. The analysisdescribed below was performed at time 1 and time2 for the stroke and control groups separately.
Models were assessed by analyzing covariancesusing the maximum likelihood method of estima-tion, using the un-transformed scores of the SF-36.Assumptions for a structural equation modeling
analysis such as problems with multivariatenormality and diagnostics (e.g. multicollinearity,outliers) were evaluated. Robust1 [63] statisticswere used because data were expected to benon-normal, especially the scales with fewerresponse categories.
There are many indices of model fit for confir-matory factor analysis with no consensus as towhich are the best. Therefore, we used several fitindices as each reflects somewhat different aspectsof model fit. These included the v2 statistic which,in large samples and under the assumption ofmultivariate normality, is interpreted as a test ofsignificance of the difference of fit between themodel and a just-identified version of it. TheSatorra-Bentler scaled v2 statistic is a correctedtest statistic used for models under non-normaldistributional conditions. [24, 64]. A small andnon-significant v2 statistic reflects good fit, how-ever, in large samples it is unlikely to achievenon-significant values of v2. A v2 /df ratio less than3 is a suggested criteria for acceptable fit [24]. TheBentler–Bonett Comparative Fit Index [24], whichindicates the proportion of improvement of themodel relative to a null model, and the RMSEA[65], which accounts for the degrees of freedom inthe model, were also used to evaluate model fit.Values of 0.9 or greater and values less than 0.1reflect acceptable fit for the CFI and RMSEA,respectively.
Cross-sectional assessments of measurement modelsat each time
The first step in the analysis was to model the datacross-sectionally at time 1 and at time 2 for thestroke and the control group separately. Withineach group, informal comparisons were madebetween time 1 and time 2 on: the correlationmatrix; factor structure; magnitude of the factorloadings; the square of the coefficients, which is thevariance of the scales explained by the respectivelatent construct (R2); and the error variances.
The extent to which these parameters variedfrom time 1 to time 2 was compared between thestroke and control group. This was a first step to
1 The robust method of estimation provides goodness-of-fit
statistics that are corrected for the degree of skewness and
kurtosis in the data.
615
assessing whether there was evidence of a gammachange (indicated by changes in the factor struc-ture) or a beta change (indicated by changes in thefactor loadings).
Tests of longitudinal invariance to assess gammaand beta change
Further analyses involved statistical tests of lon-gitudinal invariance (i.e. no change in parts of themeasurement model over time). With this model-ing approach, changes in the latent factors orrelationships between latent factors are detectedby examining changes in variance–covariancematrices from time 1 to time 2. This is similar tothe approach suggested by Schmitt et al. [20] andis performed by imposing equality constraints oncorresponding parameters across time (e.g. equalfactor covariances, equal factor variances, equalfactor loadings, and equal error variances).
There is a hierarchy for testing alpha, beta, andgamma change [20]. The first step is to assesschange in the entire variance–covariance matrixbetween time 1 and time 2. If a significant differ-ence is found then the next steps of the analysis areto evaluate if these differences are due to (1)gamma change, (2) beta change, or (3) change inuniqueness (error variances).
Gamma change (or reconceptualization) wasassessed by testing configural invariance, which isinvariance of the factor structure (the a prioripattern of factor loadings imposed on the scales)and factor covariance. Changes in these twoparts of the measurement model represent agamma change (reconceptualization) becausethey reflect the pattern by which the scales relateto the construct (factor structure) and the rela-tionship between the constructs (covariances). Ifconfigural invariance does not hold, then theobserved scores represent different constructsassessed from one time to the next and it wouldnot make sense to test further differences overtime. If there is configural invariance then thesubsequent step would be to assess if there isbeta change (scale recalibration). This is evalu-ated by testing the equivalence of the factorvariances and the factor loadings (metric invari-ance) over time as they represent the scale metricor units of measurement. Finally a test ofinvariant uniquenesses is evaluated to examine if
changes may be a result of change in measure-ment error over time.
For this study, the above steps were followed forthe longitudinal invariance testing. The first stepinvolved evaluating invariance of the entirevariance–covariance matrix between the time 1 andtime 2 evaluations. This was followed by longitu-dinal invariance testing of five nested models. Thefirst model (model 1) served as the baseline modelfor testing, across the two time periods, the confi-gural invariance (without equality constraints) ofthe factor structure. This model is an adaptation ofthe general longitudinal covariance structuralmodel for the same group evaluated over time [66].We allowed for covariances between the errors ofthe scales, as it improved the fit of the longitudinalmodel. Factor covariances and not relationshipsthat were unidirectional over time were specified inthemeasurement model since the interest was not totest structural relationships between the constructsover time (66).
The subsequent models applied the followingconstraints over time: the factor covariances toassess gamma change (reconceptualisation) (model2); the factor variances (model 3), and the factorloadings (model 4) to evaluate beta change (scalerecalibration). A final model was tested to assessthe invariance of factor uniquenesses (i.e. theequality of measurement error associated witheach scale) (model 5). Invariance was evaluated bya scaled v2 difference test [67] between each sub-sequent pair of models. However, as the v2 dif-ference test is influenced by sample size, we alsoexamined differences in the Robust ComparativeFit Index, and the RMSEA [20, 24].
Multigroup invariance
In addition to the above longitudinal invarianceanalysis, we assessed invariance of the SF-36measurement model between individuals withstroke and the control group at time 1 i.e. a multi-group comparison. The rationale behind con-ducting this analysis was that the matched controlgroup served as a pre-stroke ‘proxy’ for the strokecohort. The same model comparisons betweenthe two groups were made as those described forthe longitudinal invariance above. However, as theEQS software does not allow for robust estimationfor a multi-group comparison, an AGLS estimation
616
method was used. AGLS is an asymptomaticallydistribution-free (ADF) estimation method, whichadjusts its results for the degree of kurtosis in thedata [57]. It provides slightly different fit indicesincluding the Yuan Bentler v2, the CorrectedComparative Fit Index, and the Adjusted FitIndex. These indices are interpreted in the sameway as those described above.
Results
Subjects
The characteristics of individuals in the MontrealStroke Cohort study have been presented exten-sively in a previous article [30]. As expected, per-sons unable to be interviewed because of death ordischarge to long-term care were quite differentfrom those who completed at least one interview.They were older, and had experienced a moresevere stroke as evidenced by a lower score on theCanadian Neurological Stroke Assessment (CNS)[43]. They also stayed in the hospital longer thanpersons eligible to be interviewed. In contrast,eligible subjects who refused to participate did notdiffer to any great extent from those subjects whoparticipated.
To assess change in HRQL we restricted anal-yses to subjects who had at least two interviews.Table 1 compares this group to those that hadonly one interview. Subjects with at least twointerviews tended to be younger and in betterphysical health.
Table 2 compares the characteristics ofindividuals with stroke and the control cohort.The stroke cohort was significantly older andperformed worse on all measures of physical andcognitive ability. Their time 1 evaluation was onaverage 1 (SD ± 1.1) month post-stroke and thetime 2 evaluation was 7.5 (SD ± 2.6) monthspost-stroke. In the control group there was onaverage a 7 (SD ± 1.7)-month difference betweenboth evaluations.
Cross-sectional assessments of measurement modelsat each time
Table 3 shows the correlation matrix of the scalesof the SF-36 for both groups at time 1 and time 2.Very few correlations changed by more than 0.1over time for the stroke or control group.
Figure 1 describes the pattern of relationshipsand shows the magnitude of the standardized fac-tor loadings between the eight SF-36 scales and themental and physical factors among the strokecohort. All factor loadings were significant andwere greater than 0.5 with very little difference inthe magnitude of factor loadings over time (sug-gesting no beta change). The covariance betweenthe physical and mental factors was high at bothtimes. Similar results were found for the controlgroup (Figure 2), and there appeared to be just asmuch change, if not more change (e.g. MH and VTscales) in factor loadings between time 1 and time 2as compared to the stroke cohort. The error vari-ances and the amount of variability explained bythe factors were also similar over time (Figure 3).
Tests of longitudinal invariance to assess gammaand beta change
The overall assessment of change in the entire var-iance–covariance matrix between time 1 and time 2was significant for the stroke cohort but not for thecontrol group. Further testing revealed that in bothgroups (Table 4a and b) all parts of the measure-ment model, for the evaluation of gamma and beta
Table 1. Characteristics of individuals with stroke for those
that had one and two evaluations post-stroke
Characteristic Only one
n = 87
Two
evaluations
n = 238
Age mean ± SD 71 ± 12.2 67 ± 12.1
Sex Male n(%) 41 (53) 145 (61)
Female n(%) 46 (47) 93 (39)
Basic ADLa (Barthel: 0–100)
Mean ± SD
81 ± 24.9 92 ± 15.8
Cognition (Mini mental: 0–22)
mean ± SD
17 ± 6 19 ± 4
Instrumental ADL (OARS: 0–14)
mean ± SD
10 ± 3.6 12 ± 2.8
Community Re-integration
(RNLb: 22 to 0) mean ± SD
7 ± 5.4 5 ± 4.4
Length of stay in acute care (days) 18 ± 17.3 12 ± 8.8
SF-36 PCS mean ± SD 39 ± 12 42 ± 11
MCS mean ± SD 46 ± 10 45 ± 12
aADL, activities of daily living.bReintegration to normal living, a lower score reflects better
re-integration.
617
change, were invariant over time based on the v2
difference test and all fit indices. This confirmed ourprevious findings when we compared the SF-36measurement model cross-sectionally in eachgroup. The only significant change based on the v2
difference test, was for the measurement errors(uniquenesses) in the stroke cohort which were notinvariant between the two evaluations. This wasmainly attributed to change in the measurementerror of the Social Functioning, General Health,and Physical Function scale.
Multigroup invariance
The assessments of invariance between the con-trol group (which served as a ‘proxy’ for thepre-stroke distribution of SF-36 scores) andindividuals with stroke at the time 1 evaluation(Table 4c) showed that there was a significantdifference between the multi-group model withoutconstraints (Model 1) and the model that hadconstraints on the factor covariances (suggests agamma change) (Model 2) according to the v2
Table 2. Characteristics of individuals with stroke and controls
Characteristic Stroke n = 238 Controls n = 392
Time 1 Time 2 Time 1 Time 2
Age mean ± SD 67 ± 13 ) 62 ± 12 )Sex Male n(%) 145 (61%) 131 (33%)
Female n(%) 93 (39%) ) 261 (67%) )Basic ADL (Barthel: 0–100) mean ± SD 92 ± 15.8 96 ± 9 99 ± 4.3 100 ± 2
Cognition (Mini mental: 0–22) mean ± SD 19 ± 4 19 ± 3 21 ± 2 21 ± 2
Instrumental ADL (OARS: 0–14) mean ± SD 12 ± 2.8 12 ± 2.5 14 ± 0.7 14 ± 0.8
Community Re-integration
(RNLb:22 to 0) mean ± SD
5 ± 4.4 3 ± 3.9 0.71 ± 1.9 0.71 ± 2.0
aADL, activities of daily living.bReintegration to Normal Living, a lower score reflects better re-integration.
Table 3. Correlationsa,b at time 1 and time 2 for individuals with stroke and the control group
Controls
PF RP BP MH RE SF VT GH
Stroke
PF – 0.37
0.28
0.41
0.38
0.29
0.19
0.33
0.26
0.41
0.39
0.37
0.37
0.44
0.38
RP 0.42
0.38
– 0.46
0.39
0.32
0.20
0.81
0.88
0.49
0.49
0.38
0.33
0.41
0.26
BP 0.30
0.37
0.35
0.35
– 0.29
0.22
0.39
0.35
0.52
0.46
0.39
0.40
0.37
0.31
MH 0.19
0.15
0.25
0.16
0.23
0.13
– 0.37
0.27
0.53
0.47
0.53
0.53
0.40
0.42
RE 0.24
0.32
0.72
0.84
0.25
0.30
0.34
0.30
– 0.54
0.50
0.37
0.33
0.33
0.20
SF 0.46
0.43
0.50
0.48
0.41
0.34
0.31
0.44
0.43
0.50
– 0.50
0.53
0.44
0.35
VT 0.37
0.41
0.49
0.36
0.37
0.27
0.50
0.46
0.32
0.35
0.40
0.42
– 0.51
0.44
GH 0.42
0.51
0.33
0.28
0.26
0.28
0.39
0.38
0.27
0.27
0.43
0.44
0.53
0.60
–
a Time1 correlations are in bold.b SF-36 scales: Physical Functioning (PF); Role Physical (RP); Bodily Pain (BP); Social Functioning (SF); Mental Health (MH); Role
Emotional (RE); Vitality (VT); General Health (GH).
618
difference test. However, because there was asignificant change in measurement error (unique-nesses) between both evaluations, we cannotcompletely attribute the change in covariance togamma change.
Alpha change
Because the confirmatory factor analyses re-vealed no significant change in factor structure(i.e. gamma or beta change) over time amongindividuals with stroke or the control group, ameaningful comparison can be made betweenobserved mean scores over time (alpha change).From Table 5 we see that, at both times, indi-viduals with stroke were below Canadian normsfor the SF-36 [68], and mean scores of thecontrol group, except on the Bodily Pain scale
which we would expect as pain is often not af-fected by stroke. The greatest amount of changewas on the role scales, on which they had per-formed the lowest on at time 1. The leastamount of change occurred on the MentalHealth (MH), Vitality (VT), and General Health(GH) scales. The control group was aboveCanadian norms at both times and changed verylittle by the second evaluation for all scales.
Discussion
In our study we assessed whether individuals withstroke compared to a non-stroke control groupexperienced a response shift. Based on the longi-tudinal tests of invariance over time, the results didnot provide evidence to support our initial
.84 R2=.290.54MH
.84 R2=.290.54
.87 R2=.250.50.78c R2=.38 0.62d
REPF
.85 R2=.27.73 R2=.46 0.520.68
0.66.73 R2=.47 0.90 .75 R2=.44Physical Mental0.68 SF
RP0.87 0.68
.73 R2=.46.83 R2=.31 0.56
.68 R2=.540.74VT
BP0.52
0.50
GH
0.72
0.67
0.73
.74 R2=.45
.87 R2=.25
.68 R2=.53
Fit index e T1 T2Satorra 2/df 73/18=4 72/18=4RMSEA(confidenceinterval))
.12 (.093, 0.15) .13 (.10,.15)
Robust CFI .92 .92
.85 R2=.27 .69 R2=.52
bPhysical Functioning (PF); Role physical (RP); Bodily Pain (BP); Social Funtioning (SF); Mental Health (MH); Role Emotional (RE); Vitality
(VT); General Health (GH)c : Error variance, R2= variance of the scales explained by the respective latent construct (R2)d: Factor Loadings.
e:RMSEA, root mean square error. Accepttable fit < 0.1; CFI, comparative fit index. Acceptable fit > 0.9; SRMR, standardized root mean squared
a: Time 1 is in bold.
residual. Acceptable fit < 0.1.
Figure 1. SF-36 measurement model for individuals with stroke at time 1a and time 2. b
619
hypotheses, that individuals with stroke wouldexperience a greater degree of response shift(gamma and/ or beta change) as compared to thecontrol group.
Our results from the longitudinal analysis sug-gest that either our stroke cohort did not experi-ence a response shift during the first 6 monthspost-stroke, or that confirmatory factor analysis islimited in its ability to assess reconceptualizationand changes in internal standards. In this study,persons with stroke who were included in the finalanalysis were those that had at least two evalua-tions within the first 6 months post-stroke, bywhich time 80% of recovery is expected to takeplace [69]. We found that, based on the disabilitymeasures, the impact of the stroke was greater forthose who had one evaluation, as compared tothose that continued to be followed and had atleast two evaluations. It is possible that in our final
sample too few individuals experienced a signifi-cant change in health status for there to have beena shift in the conceptualization and internal stan-dards of measurement of HRQL. Approximately44% of individuals changed less than five pointson the physical component scale of the SF-36.Despite this, our sample did have clinicallymeaningful loss as compared to the control groupin their ability to perform basic and moreadvanced activities of daily living and with rein-tegration into the community, thus, the expecta-tion was that they would experience a responseshift.
We incorporated the idea of a pre-stroke ‘proxy’to gain some insight about the potential of aresponse shift almost immediately after the stroke.The results from this comparison provided someevidence based on the v2 difference test for agamma type change (i.e. a reconceptualisation)
Physical Mental
0.80c R2=.36
0.84R 2=.30
PF
0.76 R2=.43RP
BP
0.60d
0.65
0.67
0.55
0.55
0.61
0.91
0.92
0.79 R2=.37
0.76 R2=.42
0.64 R2=.60
0.86 R2=.27
0.79 R2=.37RE
SF
VT
GH
0.61
0.68
0.74
0.64
0.56
0.79 R2=.38
0.74 R2=.45
0.68 R2=.54
0.77 R2=.40
0.83 R2=.31
Fit indexe T1 T2Satorra 2/df 58/18= 3 78/18= 4 RMSEA(confidenceinterval)
.098 (.08, .12) 0.11 (.089, .13)
Robust CFI .94 .92
0.73 R2=.46
0.65 R2=.580.83 R2=.30
0.65
0.76
0.77
0.52
MH0.61
bPhysical Functioning (PF); Role physical (RP); Bodily Pain (BP); Social Funtioning (SF); Mental Health (MH); Role Emotional (RE); Vitality
(VT); General Health (GH).c : Error variance, R2= variance of the scales explained by the respective latent construct (R2).d: Factor Loadings.
e:RMSEA, root mean square error. Accepttable fit < 0.1; CFI, comparative fit index. Acceptable fit > 0.9.
a: Time 1 is in bold.
Figure 2. SF-36 measurement model for the control group at time 1a and time 2.b
620
reflected through change in the factor covariancesbetween the control group and individuals withstroke at time 1. Therefore, it may be that responseshift is mediated by the event itself and not therecovery process. However, these results need to beconsidered with caution as the other fit indices werevery similar to each other. This type of comparisonneeds to be examined further in future studies.
There are other factors, such as personality andsociodemographic variables [8] that can impactwhether or what type of response shift will occur,and a change in health status does not determinethis in isolation. The influence of these factorsneeds to be evaluated in addition to whether or notthere are differences in the magnitude of responseshift among individuals who experience varyinglevels of severity of stroke and recovery.
In the organizational psychology field confir-matory factor analytic procedures have been found
to be powerful tools to examine the reconstitutionof meaning [9, 22, 23]. Several studies in this areahave provided supporting evidence of gamma andbeta change as reflected by changes in factorstructure for interventions expected to elicit thesetypes of change. However, very few of these studiesused a control group or tried to link factor struc-ture changes to changes in other outcomes [20, 23].In addition, despite the documented strengths ofusing factor analytic techniques, there has beendiscussion about the ability of this methodology toassess changes in conceptualization and internalstandards. The statistical approaches are based onthe administration of the same measure over time.Administering the same items or scales over timemay not adequately capture changes in conceptu-alization [26]. Even if they are able to detect rec-onceptualization they would not reveal the natureof this change [19]. Furthermore, the factor analytic
Physical Mental
0.80c R2=.36PF
RP
BP
0.60d
0.65
0.67
0.62
0.68
0.52
0.91
0.90
0.84 R2=.29
0.76 R2=.42
0.64 R2=.60
0.87 R2=.25
0.79 R2=.370.77
0.54
RE
SF
VT
GHP
0.50
0.65
0.66
0.68
0.74
0.64
0.67
0.78 R2=.38
0.76 R2=.43
0.85 R2=.270.68 R2=.54
0.77 R2=.40
0.74 R2=.45
Fit indexe Stroke ControlsSatorra 2/df 73/18= 4 58/18= 3 RMSEA(confidenceinterval)
12 (.093, 0.15) .098 (.08, .12)
Robust CFI .92 .94
0.73 R2=.46
0.75 R2=.440.73 R2=.47
0.74 R2=.45
MH0.61
a:Values for control cohort are in bold
bPhysical Functioning (PF); Role physical (RP); Bodily Pain (BP); Social Funtioning (SF); Mental Health (MH); Role Emotional (RE); Vitality
(VT); General Health (GH).c : Error variance, R2= variance of the scales explained by the respective latent construct (R2).d: Factor Loadings.
e:RMSEA, root mean square error. Acceptable fit < 0.1; CFI, comparative fit index. Acceptable fit > 0.9.
Figure 3. SF-36 measurement model for the control groupa and individuals with stroke at time 1.b
621
Table 4b. Results of the confirmatory factor analyses (CFA) testing factorial invariance of the sf36 between time 1 and time 2 among
the control group.
Models tested v2 Satorra v2 df v2diff*(Ddf) SRMR RMSEA Robust
CFI
NNFI
Model 1
no constraints
305 234 88 – 0.06 0.08 (0.07, 0.09) 0.92 0.91
Model 2
Covariances
constrained (Gamma
change)
313 238 89 3.6 (1) 0.06 0.08 (0.07, 0.09) 0.92 0.91
Model 3
Factor covariances
and variances
constrained (Beta
change)
315 238 91 1.2 (2) 0.07 0.08 (0.07, 0.09) 0.92 0.91
Model 4
Factor covariances,
variances, and loadings
constrained
(Beta change)
318 244 97 3 (6) 0.06 0.08 (0.07, 0.09) 0.92 0.92
Model 5
Factor covariances,
variances, loadings,
and uniquenesses
constrained
341 258 105 14 (8) 0.07 0.08 (0.07, 0.09) 0.92 0.92
* v2 differences were calculated using the formula for scaled difference chi-square test statistic. No model comparisons were significant
at the alpha = 0.05 level.
Table 4a. Results of the confirmatory factor analyses (CFA) testing factorial invariance of the SF-36 between time 1 and time 2 among
individuals with stroke
Models Tested v2 Satorra
v2df v2diff
*
(Ddf)SRMR RMSEA Robust
CFI
NNFI
Model 1
no constraints
232 213 88 – 0.06 0.08 (0.07, 0.1) 0.93 0.89
Model 2
Covariances
constrained (Gamma
change)
234 215 89 2.0 (1) 0.07 0.08 (0.07, 0.09) 0.92 0.89
Model 3
Factor covariances and
variances constrained
(Beta change)
237 219 91 3.7 (2) 0.07 0.08 (0.07, 0.10) 0.93 0.89
Model 4
Factor covariances,
variances, and loadings
constrained (Beta
change)
241 224 97 7.7 (6) 0.07 0.08 (0.07, 0.09) 0.93 0.90
Model 5
Factor covariances,
variances, loadings,
and uniquenesses
constrained
272 258 105 35 (8)* 0.06 0.08 (0.07, 0.09) 0.91 0.89
* v2 differences were calculated using the formula for scaled difference chi-square test statistic. No model comparisons were significant
at the a = 0.05 level.
622
techniques use a group measure drawing upon allitems and the entire sample to assess each factor[21], and invariance over time at the populationlevel may occur for reasons other than responseshift. Other methods that involve a more individ-ualized approach may prove to be more sensitive to
detect a response shift, either alone or in combi-nation with some of the factor analytic techniques.
A small number of studies in non-health relatedfields have compared various techniques designedfor the assessment of beta and gamma change, andthe results have been mixed. Armenakis,
Table 5. Mean scores (SD) on the SF-36 among individuals with stroke and the control group at time 1 and time 2
SF-36
ScaleaCanadian
normsdStrokeb Controlc
Time 1 Time 2 Time 1 Time 2
PF 76 (22) 60 (32) 70 (27) 86 (20) 87 (20)
RP 76 (37) 31 (41) 65 (42) 84 (34) 87 (31)
BP 74 (24) 72 (32) 76 (31) 75 (28) 76 (29)
MH 79 (15) 68 (23) 72 (22) 78 (19) 79 (17)
RE 83 (33) 46 (47) 72 (43) 85 (34) 88 (31)
SF 87 (20) 67 (29) 82 (24) 85 (21) 88 (19)
VT 68 (18) 48 (24) 53 (24) 69 (21) 67 (20)
GH 74 (18) 66 (23) 71 (21) 81 (18) 82 (18)
PCS 47 (10) 42 (11) 46 (10) 51 (8) 51 (8)
MCS 54 (8) 45 (12) 49 (11) 52 (9) 53 (8)
a Physical Functioning (PF); Role Physical (RP); Bodily Pain (BP); Social Functioning (SF); Mental Health (MH); Role Emotional
(RE); Vitality (VT); General Health (GH); Physical Component Scale (PCS); Mental Component Scale (MCS).b Paired comparison t-test was used for mean differences for all scales between time 1 and time 2 p < 0.01 for all scales except for BP
p > 0.1.c Paired comparison t-test was used for mean differences for all scales between time 1 and time 2 p > 0.05 except for SF p < 0.05.d These Canadian Norms are mean age and sex-standardized scores for the eight scales and two component scales of the SF-36 among
the 65–74 age range.
Table 4c. Results of the confirmatory factor analyses cfa testing factorial invariance of the Sf-36 between the stroke and control group
at time 1
Models Tested Yuan bentler
v2df v2diff
a (Ddf) SRMR RMSEA Corrected
CFI
Adjusted
fit index
Model 1
no constraints
93 36 ) 0.09 0.06 (0.05, 0.07) 0.93 0.94
Model 2
Covariances constrained
(Gamma change)
102 37 0 9 (1)b 0.09 0.07 (0.05, 0.08) 0.90 0.93
Model 3
Factor covariances, and
variances constrained
(Beta change)
102 39 0 (2) 0.10 0.06 (0.05, 0.08) 0.90 0.93
Model 4
Factor covariances,
variances, and loadings
constrained (Beta change)
110 45 8 (6) 0.11 0.06 (0.05, 0.07) 0.93 0.90
Model 5
Factor covariances,
variances, loadings, and
uniquenesses constrained
181 53 71 (8)b 0.26 0.09 (0.08, 0.10) 0.79 0.88
a v2 differences were calculated using the formula for scaled difference chi-square test statistic.b p<0.005.
623
Randolph, and Bedian (1982) compared two dif-ferent factor analytic techniques, the transforma-tion [9] and coefficients of congruence methods [70]and found that they led to different conclusions interms of the similarity of factors [71]. Conversely,Schmitt et al. [72] found that the results of thecovariance analysis were consistent with those ofthe coefficients of congruence method for assessinggamma change and that the analysis of covariancemethod and a design method (the then test) pro-posed by Terborg et al. [73] resulted in the sameconclusions regarding beta change [72]. Anotherstudy that contrasted the Golembiewski et al. [9]method of comparing factor structures over timeto this same design method found a 50% agree-ment between the two techniques, where the designmethod reported gamma change where it was notpredicted and the factor structure method did notreport gamma change for items for which it wasexpected. These studies helped to clarify the dif-ferences and similarities and have provided someevidence of convergent validity between themethods. Given the conflicting results, morestudies such as these that compare various meth-ods, especially in the HRQL field, would providevaluable information on the validity and the sen-sitivity of the various approaches.
Other authors have proposed growth curveanalysis for inferring response shift [26, 74, 75]This technique allows for the evaluation of indi-vidual changes over time through latent variablelongitudinal modeling. The advantage of thistechnique is that it allows changes at several timepoints to be modeled for each individual and in-terindividual differences in change can be exam-ined (e.g. stroke compared to controls), or for thesame individual where differences in the trajectoryof change of objective and HRQL outcomes canbe evaluated for evidence of response shift.
To allow for an acceptable fit we made somemodifications to the initial hypothesized measure-ment model of the SF-36. We were not able toallow the Vitality, General Health, and SocialFunctioning scales to have complex factor load-ings: i.e. load on both the physical and mentalfactors as suggested by earlier work performed onthe SF-36 [76]. A possible reason is that there werehigh correlations between some of our scales andthat an attempt to model complex factor loadingsresulted in linear dependence between the factor
loadings. There has been extensive testing of thevalidity and reliability of the SF-36, includingsupporting evidence from principal componentsanalysis that the scales of the SF-36 measure pri-marily mental and physical components [76].However, there is little evidence of the use oftechniques such as confirmatory factor analysisthat have tested the hypothesized structure of theSF-36 and that take into account the measurementdifficulties inherent in estimating the measurementmodel. This includes possible correlations betweenthe error variances of the scales [62]. In our datawe found that the error variance of the Role-Physical and Role-Emotional scales were corre-lated, and had to allow for this correlation toachieve acceptable fit of our model. A study thatassessed the factor structure of the SF-36 amongindividuals with peripheral arterial occlusive dis-ease also found that if they allowed for correlatederror structures and the interdependence of thescales in the analysis, it improved the fit of the SF-36 measurement model [62]. An advantage ofconfirmatory factor analysis is that it allows forsuch correlated error structures.
In addition, we allowed for an intercorrelationbetween the physical and mental factors. In theoriginal testing of the SF-36, Ware [37] used anorthogonal rotation which assumes no correlationbetween the factors, and provided a good justifi-cation for this choice [37]. However, Simon et al.,[77] and Nortvedt et al. [78] suggest considerationof oblique factor rotation instead of orthogonal sothat the factor analysis will reflect the interdepen-dence of the eight scales. The study mentionedabove, that assessed the factor structure of the SF-36 in patients with peripheral arterial occlusivedisease, found that allowing for a correlation be-tween the mental and physical factors improvedthe fit of the measurement model [62]. A scoringmethod for the SF-36 based on an oblique rotationbetween the physical and mental components isnow available (79).
For the stroke cohort the measurement errorsfor the Social Functioning, General Health, andPhysical Functioning scales were significantly dif-ferent between the first and second evaluationsbased on the v2 difference test. The other fit indiceswere very similar. It is not unexpected that themeasurement error may be different over time for agroup that is experiencing changes in health status.
624
Variation in error may also partially be attributedto variability in the evaluation times within thesample. An advantage of structural equationmodeling is that it allows us to look at the rela-tionship between measurement errors whereas theobserved score can only provide an imperfect esti-mate of the underlying construct. Change in mea-surement error needs to be considered whileinterpreting the observed mean scores in Table 5.
The measurement models that we found in ourstudy sample for persons with stroke and those inthe control group support some of the suggestionsmade by other investigators with regards to thefactor structure of the SF-36. Our results alsocontribute information concerning the stability ofthis factor structure over time.
Limitations
The data source used for the present study wasfrom a longitudinal study, with the primaryobjective of evaluating the long-term outcome ofstroke compared to a healthy control group. Theobjective of the analyses presented here was toappraise whether the observed change can beattributed to objective change or response shift.The analyses was restricted to those individualswho remained in the study long enough to have atleast two evaluations. The question arising iswhether the results found here, i.e. objectivechange rather than response shift, for the targetpopulation specified in the main study (communitydwelling stroke survivors) are invalidated some-what by failure to evaluate all eligible persons. Weknow those who did not have a second evaluationhad lower levels of functional ability. To under-stand the impact of low function on response shiftwould necessitate a response shift appraisal withinlow and high functioning subgroups but not en-ough low functioning individuals were availablefor this comparison. Previous work [80] has notconfirmed hypotheses that individuals with moresevere stroke or greater improvements in physicalhealth experience greater levels of response shift.To have a clearer understanding of the impact ofnot including the 87 individuals that had only oneevaluation we used an FIML approach to replacemissing SF-36 data for these individuals. When wetested invariance over time with missing datareplaced the results regarding the occurrence of
response shift were similar to those withoutincluding the 87 subjects.
Another limitation of this study is that we usedonly a statistical technique to assess response shift.This is in part because of the availability of anexisting data set, but also because design methodsand individualized approaches place greater de-mands on respondents.
Summary and conclusion
The data presently available on persons withstroke indicates that the improvement in HRQLover time is real rather than a result of reconcep-tualization or a recalibration. If response shiftdoes occur with stroke it is likely to be mediated bythe event itself and not the recovery process. Thisconclusion needs to be confirmed in other samplesof stroke patients and by employing other tech-niques. In any study or clinical situation wherechange in health status is evaluated, it is importantto interpret the health rating in light of the possi-bility that response shift occurred.
References
1. Williams LS, Weinberger M, Harris LE, Clark DO, Biller J.
Development of a stroke-specific quality of life scale. Stroke
1999; 30(7): 1362–1369.
2. Duncan PW, Samsa GP, Weinberger M, et al. Health status
of individuals with mild stroke. Stroke 1997; 28(4): 740–745.
3. de Haan R, Aaronson N, Limburg M, Hewer RL, van
Crevel H. Measuring quality of life in stroke. Stroke 1993;
24(2): 320–327.
4. King RB. Quality of life after stroke. Stroke 1996; 27(9):
1467–1472.
5. Duncan PW, Wallace D, Lai SM, Johnson D, Embretson S,
Laster LJ. The Stroke Impact Scale Version 2.0. Evaluation
of reliability, validity, and sensitivity to change. Stroke
1999; 30(10): 2131–2140.
6. Allison PJ, Locker D, Feine JS. Quality of life: A dynamic
construct. Soc Sci Med 1997; 45(2): 221–230.
7. Cronbach LJ, Furby L. How should we measure ‘change’ –
Or should we? Psychol Bull 1970; 74: 68–80.
8. Sprangers MA, Schwartz CE. Integrating response shift
into health-related quality of life research: A theoretical
model. Soc Sci Med 1999; 48(11): 1507–1515.
9. Golembiewski RT, Yeager S. Measuring change and per-
sistence in human affairs: Types of change generated by OD
designs. J Appl Behav Sci 1975; 12(2): 133–157.
10. Mayo NE. Epidemiology and recovery. Phys Med Reh:
State Art Rev 1993; 7(1): 1–25.
11. Duncan PW, Goldstein LB, Horner RD, Landsman PB,
Samsa GP, Matchar DB. Similar motor recovery of upper
625
and lower extremities after stroke. Stroke 1994; 25(6):
1181–1188.
12. Wade D, Langton HR. Functional abilities after stroke:
The first three months. J Neurol Neurosur Psych 1987; 50:
177–182.
13. Schwartz CE, Sprangers MAG. Adaptation to Changing
Health: Response Shift in Quality-of-life research. 1st ed.
Washington, DC: American Psychological Association,
2000.
14. engs TO, Yu M, Luistro E. Health-related quality of life
after stroke – A comprehensive review. Stroke 2001; 32(4):
964–971.
15. Breetvelt IS, Van Dam FS. Underreporting by cancer pa-
tients: The case of response-shift. Soc Sci Med 1991; 32(9):
981–987.
16. Statistics Canada. How healthy are Canadians? Ottawa:
Government of Canada, 1999: 9–119.
17. Decker SD, Schulz R. Correlates of life satisfaction and
depression in middle-aged and elderly spinal cord-injured
persons. Am J Occup Ther 1985; 39(11): 740–745.
18. Millsap RE, Sandra B. Alpha, beta, and gamma change in
evaluation research: A structural equation approach. J
Appl Psychol 1988; 73(3): 574–584.
19. Norman P. The interpretation of change in verbal reports:
Implications for health psychology. Psychol Health 1996;
11(2): 301–314.
20. Schmitt N. The use of analysis of covariance structures to
assess beta and gamma change. Source. Mult Behav Res
1982; 17(3): 343–358.
21. Randolph WA. How can OD consultants and researchers
assess gamma change? A comparison of two analytical
procedures. J Manage 1989; 15(4): 633–648.
22. Millsap RE, Sandra B. Alpha, beta, and gamma change in
evaluation research: A structural equation approach.
J Appl Psychol 1988; 73(3).
23. Schaubroeck J. Confirmatory factor analytic procedures for
assessing change during organizational entry. J Appl
Psychol 1989; 74(6): 892–900.
24. Kline RB. Principles and Practice of Structural Equation
Modeling. New York: The Guilford Press, 1998.
25. Lindell MK. Equivocality of factor incongruence as an
indicator of type of change in OD Interventions. Acad
Manage Rev 1980; 5(1): 105–107.
26. Schwartz CE, Sprangers MA. Methodological approaches
for assessing response shift in longitudinal health-related
quality-of-life research. Soc Sci Med 1999; 48(11):
1531–1548.
27. Bernhard J, LowyA,MaibachR,HurnyC.Response shift in
the perception of health for utility evaluation: An explorative
investigation. Eur J Cancer 2001; 37(14): 1729–1735.
28. Sprangers MA, Van Dam FS, Broersen J, et al. Revealing
response shift in longitudinal research on fatigue – the use
of the thentest approach. Acta Oncol 1999; 38(6): 709–718.
29. Postulart D, Adang EM. Response shift and adaptation in
chronically ill patients. Med Decis Making 2000; 20(2):
186–193.
30. Mayo NE, Wood-Dauphinee S, Cote R, Durcan L, Carlton
J. Activity, participation, and quality of life six months
post-stroke. Arch Phys Med Rehab 2001; 83: 1035–1042.
31. Ware Jr JE, Snow KK, Kosinski M, Gandek B. Health
Survey: Manual & Interpretation Guide. Boston, MA: The
Health Institute, New England Medical Center, 1993.
32. McHorney CA, Ware Jr JE, Lu JF, Sherbourne CD. The
MOS 36-item Short-Form Health Survey (SF-36): III. Tests
of data quality, scaling assumptions, and reliability across
diverse patient groups. Med Care 1994; 32(1): 40–66.
33. McHorney CA, Ware Jr JE, Rogers W, Raczek AE, Lu JF.
The validity and relative precision of MOS short- and long-
form health status scales and Dartmouth COOP charts.
Results from the Medical Outcomes Study. Med Care 1992;
30(Suppl 5): MS253–MS265.
34. Brazier JE, Harper R, Jones N, et al. Validating the SF-36
health survey questionnaire: New outcome measure for
primary care. Br Med J 1992; 305: 160–164.
35. Modan B, Wagener DK. Some epidemiological aspects of
stroke: Mortality/morbidity trends, age, sex, race, socio-
economic status. Stroke 1992; 23: 1230–1236.
36. Dorman P, Slattery J, Farrell B, Dennis M, Sandercock P.
Qualitative comparison of the reliability of health status
assessments with the EuroQol and SF-36 questionnaires
after stroke. United Kingdom Collaborators in the Inter-
national Stroke Trial. Stroke 1998; 29(1): 63–68.
37. Ware Jr JE, Kosinski M, Keller S–D. SF-36 Physical &
Mental Scales: A User’s Manual. Boston, Massachusetts:
The Health Institute, New England Medical Center, 1994.
38. Dorman PJ, Dennis M, Sandercock P. How do scores on
the EuroQol relate to scores on the SF-36 after stroke?
Stroke 1999; 30(10): 2146–2151.
39. van der Putten JJ, Hobart JC, Freeman JA, Thompson AJ.
Measuring change in disability after inpatient rehabilita-
tion: Comparison of the responsiveness of the Barthel index
and the Functional Independence Measure. J Neurol
Neurosurg Psychiatr 1999; 66(4): 480–484.
40. Harwood RH, Ebrahim S. A comparison of the respon-
siveness of the Nottingham extended activities of daily
living scale, London handicap scale and the SF-36. Disabil
Rehabil 2000; 22: 786–793.
41. Anderson C, Laubscher S, Burns R. Validation of the Short
Form 36 (SF-36) health survey questionnaire among stroke
patients. Stroke 1996; 27(10): 1812–1816.
42. McHorney CA, Ware JE, Raczek AE. The MOS 36-Item
Short-Form Health Survey (SF-36): II. Psychometric and
Clincal Tests of Validity in Measuring physical and mental
health constructs. Med Care 1993; 31: 247–263.
43. Cote R, Battista RN, Wolfson C, Boucher J, Adam J,
Hachinski V. The Canadian neurological scale: Validation
and reliability assessment. Neurology 1989; 39: 638–643.
44. FolsteinMF, Folstein SE,McHugh PR. ‘Mini-mental state’.
A practical method for grading the cognitive state of patients
for the clinician. J Psychiatr Res 1975; 12(3): 189–198.
45. Granger CV, Hamilton BB. Measurement of stroke reha-
bilitation outcome in the 1980s. Stroke 1990; 21(Suppl 9):
II46–II47.
46. Mahoney FI, Barthel DW. Functional evaluation: The
Barthel lndex. Md St Med J 1965; 14: 61–65.
47. Loewen SC, Anderson BA. Reliability of the Modified
Motor Assessment Scale and the Barthel Index [see com-
ments]. Phys Ther 1988; 68(7): 1077–1081.
626
48. ShinarD,GrossCR,Mohr JP, et al. Interobserver variability
in the assessment of neurologic history and examination in
the Stroke Data Bank. Arch Neurol 1985; 42: 557–565.
49. Roy CW, Togneri J, Hay E, Pentland B. An inter-rater
reliability study of the Barthel Index. Int J Rehabil Res
1988; 11(1): 67–70.
50. Granger CV, Dewis LS, Peters NC, Sherwood CC, Barrett
JE. Stroke rehabilitation: Analysis of repeated Barthel
index measures. Arch Phys MedRehabil 1979; 60: 14–17.
51. Gresham GE, Phillips TF, Labi ML. ADL status in stroke:
relative merits of three standard indexes. Arch Phys Med
Rehabil 1980; 61(8): 355–358.
52. Wade DT, Langton Hewer R. Functional abilities after
stroke: Measurement, natural history and prognosis.
J Neurol Neurosurg Psychiatr 1987; 50(2): 177–182.
53. Wellwood I, Dennis MS, Warlow CP. A comparison of the
Barthel Index and the OPCS disability instrument used to
measure outcome after acute stroke. Age Ageing 1995; 24:
54–57.
54. McDowell I, Newell C. Measuring Health: A Guide to
Rating Scales and Questionnaires. 2nd ed. New York:
Oxford University Press, 1996.
55. Wood-Dauphinee SL, Opzoomer MA, Williams JI, Marc-
hand B, Spitzer WO. Assessment of global function: The
Reintegration to Normal Living Index. Arch Phys Med
Rehabil 1988; 69(8): 583–590.
56. Wood-Dauphinee S, Williams JI. Reintegration to normal
living as a proxy to quality of life. J Chron Dis 1987; 40(6):
491–502.
57. Bentler PM. EQS Structural Equation Program Manual.
Encino, 1995.
58. Hoyle RH. Structural Equation Modeling: Concepts,
Issues, and Application. Thousand Oaks: Sage Publica-
tions, c1995, 1995.
59. Kaplan D. Structural Equation Modeling: Foundations
and extensions. Thousand Oaks, CA: Sage Publications,
c2000, 1955.
60. Hayduk LA. Structural Equation Modeling with LISREL:
Essentials and Advances. Baltimore: Johns Hopkins Uni-
versity Press, c1987, 1987.
61. Bollen KA. Structural Equations with Latent Variables.
New York: Wiley and Sons, 1989.
62. Siddiqui O, Ali MW. Linear structural equation model in
analyzing quality-of-life-data from clinical trials. J Biop-
harm Stat 1999; 9(4): 661–681.
63. Satorra A, Bentler PM. Corrections to test statistics and
standard errors in covariance structure analysis. In: von Eye,
Alexander, Clogg, Clifford C (eds), Latent Variables Anal-
ysis: Applications for Developmental Research. Thousand
Oaks, CA, US: Sage Publications, Inc. 1994: 399–419.
64. Curran PS, West SG, Finch JF. The Robustness of test
Statistics to nonnormality and specification error in con-
firmatory factor analysis. Psychol Meth 1996; 1: 16–29.
65. Nevitt J, Hancock GR. Improving the root mean square
error of approximation for nonnormal conditions in
structural equation modeling. 2000; 68(3): 251–268.
66. oreskog KG, Sorbom D. Advances in Factor Analysis and
Structural Equation Models. Cambridge, MA: ABT Books,
1979.
67. Jarnlo GB, Ceder L, Thorngren KG. Early rehabilitation at
home of elderly patients with hip fractures and consump-
tion of resources in primary care. Scand J Prim Health Care
1984; 2: 105–112.
68. Hopman WM, Towheed T, Anastassiades T, et al. Cana-
dian normative data for the SF-36 health survey. Can Med
Assoc J 2000; 163(3): 265–271.
69. Jorgensen HS, Nakayama H, Raaschou HO, et al. Out-
come and time course of recovery in stroke. Part II: Time
course of recovery. The Copenhagen Stroke Study. Arch
Phys Med Rehab 1995; 76(5): 406–412.
70. Armenakis AA, Wilmoth JN. An algorithm for assessing
factor structure congruence. Educ Psychol Measure 1977;
37(1): 213–214.
71. Randolph WA, Elloy DF. A comparison of two methods
for evaluating the similarity of factor analytic solutions.
J Manage 1989; 15: 633–648.
72. Schmitt N, Pulakos E, Lieblein A. Comparison of three
techniques to assess group-level beta and gamma change.
J Appl Psychol 1984; 8: 249–260.
73. Terborg JR, Howard GS, Maxwell SE. Evaluating plan-
ned organizational change: A method for assessing alpha,
beta, and gamma change. Acad Manage Rev 1980; 5(1):
109–121.
74. Muthen BO, Curran PJ. General longitudinal modeling of
individual differences in experimental designs: A latent
variable framework for analysis and power estimation.
Psychol Meth 1997; 2(4): 371–402.
75. Willett JB, Sayer AG. Using covariance structure-analysis
to detect correlates and predictors of individual change over
time. Psychol Bull 1994; 116(2): 363–381.
76. McHorney CA, Ware Jr, JE Raczek AE. The MOS 36-Item
Short-Form Health Survey (SF-36): II. Psychometric and
clinical tests of validity in measuring physical and mental
health constructs. Med Care 1993; 31(3): 247–263.
77. Simon GE, Revicki DA, Grothaus L, Vonkorff M. SF-36
summary scores: are physical and mental health truly dis-
tinct? Med Care 1998; 36(4): 567–572.
78. Nortvedt MW, Riise T, Myhr K, Nyland HI. Performance
of the SF-36, SF-12, and RAND-36 summary scales in a
multiple sclerosis population. Med Care 2000; 38(28 ref):
1022–1028.
79. University of Kansas Pepper Center for Aging. Stroke
Toolbox. 2002. www2.kumc.edu/coa/Stroke_Toolbox/
stroke-tool.htm.
80. Ahmed S, Mayo NE, Wood-Dauphinee S, Hanley J.
Response Shift In The Assessment Of Health Related
Quality of Life (HRQL) Post-Stroke. Qual Life Res 2001;
10(3): 202–204.
Address for correspondence: Sara Ahmed, Royal Victoria Hos-
pital, Division of Clinical Epidemiology, 687 Pine Avenue
West, Ross 4.29, Montreal, Quebec, Canada, H3A 1A1
Phone: +1-514-934-1934 ext. 31564; Fax: +1-514-843-1493.
E-mail: [email protected]
627
