1
Figure 1. Mean levels and 95% CI for a) Overall Composite scores, b) Working Memory Domain scores and c) Speed of Processing Domain. (Blue Squares) = Screening levels (T1), English norms (Green Circles) = Baseline levels (T2), English norms (Yellow Squares) = Screening levels (T2), language spec. norms (Red Circles) = Baseline levels (T2), language specific norms BACKGROUND The MATRICS Consensus Cognitive Battery (MCCB) is the primary cognitive test battery recommended for clinical trials in schizophrenia populations. The MCCB has been successfully translated to multiple foreign languages and used in many international clinical trials, which has generated an interest to test whether language and regional differences have an influence on the psychometric characteristics of the MCCB. The primary aim of the present study was to examine regional differences and the influence of language specific norming on test-retest stability and practice effects of the MCCB. We present results using both English speaking norms and language/region specific norms. METHODS We examined a pooled cohort of 2846 schizophrenia patients from 10 clinical trials. All participants completed the MCCB twice before starting treatment: once at a screening (T1) and once as part of the baseline assessment (T2). There were 1910 participants from English speaking countries and 936 participants from non-English speaking countries (171 patients from South America, 302 patients from Asia and 463 patients from Europe). The Overall composite MCCB score and the 2 cognitive domains, ‘Working memory’ (WM) and ‘Speed of Processing’ (SoP) were studied. RESULTS Mean levels at screening and baseline. Comparison between English norms and language/region specific norms for the Overall Composite, the SoP domain and the WM domain means are presented in Figure 1. Mean levels differed significantly for all 3 measures when comparing English norms and language specific norms within the South American, Asian and European regions (p’s <.001). There were also significant differences in mean levels between the 4 regions at both time points, when English and language specific norms were applied. However, the practice effects were low and similar within all 4 regions (see Figure 1 and Table 1). Examining effects of language and region on test-retest reliability of the MATRICS Consensus Cognitive Battery (MCCB) Georgiades, A 1, 2 ; Davis, VG 1 ; Atkins, AS 1 ; Stroescu, I 1 ; Keefe, RSE 1, 2 1 NeuroCog Trials, Inc. (Durham, NC), 2 Duke University Medical Center, Department of Psychiatry (Durham, NC) English Speaking South American Asian European CONCLUSION Practice effects for the Overall Composite score, the WM and SoP domains were low and similar in English speaking and non- English speaking patient populations (see Figure 1 and Table 1). Test-retest stability of the Overall Composite score was high in all regions and only marginally affected by norming method (see Figure 2 and Table 1). Disclosure: Anastasia Georgiades is currently and has in the past 3 years received funding from the National Heart, Lung and Blood Institute and Duke University School of Medicine. She is also currently a part time employee of NeuroCog Trials. V.G. Davis is an employee of NeuroCog Trials. A.S. Atkins and I. Stroescu are currently or have in the past 3 years received funding from the National Institute of Mental Health and are full time employees of NeuroCog Trials. English Speaking South American Asian European English Speaking South American Asian European South America English Speaking Europe Asia Figure 2. Histograms I-IV presenting the distribution of the Overall Composite scores at screening (green) and baseline (yellow) within (I) the English speaking region, (II) the South American region, (III) Europe and (IV) Asia. Scatterplots depicting associations between screening and baseline values of Overall Composite scores within the same regions, by country. (I) (II) (III) (IV) Table 1. Practice effects, Cohen’s d (effect size), ICC’s and Pearson correlations by region English Norms Language Specific Norms English Norms Language Specific Norms Region Mean Diff (SD) (screening-baseline) Cohen’s d Mean Diff (SD) (screening-baseline) Cohen’s d ICC [95% CI] Pearson’s corr. ICC [95% CI] Pearson’s corr. English Speaking Overall Comp. (N=1815) 1.9 (5.5) # .15 - - .89 [.88-.90] .90 - - (98% North America) WM (N=1909) 1.4 (7.1) # .12 - - .80 [.79-.82] .81 - - SoP (N=1910) 2.2 (7.0) # .18 - - .82 [.80-.83] .83 - - South America Overall Comp. (N=131) 3.0 (5.7) # .27 2.0 (4.0) # .25 .86 [.82-.90] .87 .86 [.82-.90] .87 WM (N=168) 1.6 (7.4)** .13 1.3 (5.4)** .14 .82 [.76-.86] .82 .83 [.78-.87] .82 SoP (N=171) 2.9 (7.1) # .20 2.0 (4.8) # .20 .86 [.82-.90] .88 .87 [.83-.90] .89 Asia Overall Comp. (N=270) 2.8 (5.5) # .22 2.7 (5.3) # .22 .88 [.85-.90] .90 .89 [.86-.91] .91 WM (N=299) 1.2 (8.2)* .09 1.3 (8.2)** .10 .77 [.72-.82] .77 .77 [.72-81] .77 SoP (N=302) 3.6 (6.1) # .27 3.4 (6.9) # .24 .86 [.83-.89] .90 .85 [.82-.88] .88 Europe Overall Comp. (N=413) 2.2 (5.0) # .17 2.4 (5.2) # .18 .91 [.90-.93] .93 .91 [.89-.92] .92 WM (N=463) 1.7 (6.5) # .13 1.7 (6.7) # .13 .86 [.84-.88] .87 .86 [.84-.88] .87 SoP (N=463) 2.9 (7.1) # .22 2.9 (7.3) # .21 .83 [.80-.86] .85 .84 [.81-.86] .86 Overall Composite score at screening and baseline Overall Composite score at screening and baseline Overall Composite score at screening and baseline Overall Composite score at screening and baseline R.S.E. Keefe is currently or has in the past 3 years received investigator-initiated research funding support from the Department of Veteran’s Affair, Feinstein Institute for Medical Research, National Institute of Mental Health, Psychogenics, Research Foundation for Mental Hygiene, Inc., and the Singapore National Medical Research Council. He currently or in the past 3 years has received honoraria, served as a consultant, or advisory board member for Abbvie, Akebia, Asubio, Avanir, AviNeuro/ChemRar, BiolineRx, Biogen Idec, BiolineRx, Biomarin, Boehringer-Ingelheim, EnVivo/FORUM, GW Pharmaceuticals, Janssen, Johnson & Johnson, Lundbeck, Merck, Minerva Neurosciences, Inc., Mitsubishi, Neuralstem, Neuronix, Novartis, NY State Office of Mental Health, Otsuka, Pfizer, Reviva, Roche, Sanofi/Aventis, Shire, Sunovion, Takeda, Targacept, and the University of Texas South West Medical Center. Dr. Keefe receives royalties from the BACS testing battery, the MATRICS Battery (BACS Symbol Coding) and the Virtual Reality Functional Capacity Assessment Tool (VRFCAT). He is also a shareholder in NeuroCog Trials and Sengenix. Means and 95% CI Means and 95% CI Means and 95% CI Paired t-test * P <.05, ** P <.01, # P <.001 Practice effects: The effect size of the practice effects between screening and baseline, measured as Cohen’s d, was low in all regions and only marginally affected by how the scores were normed (see Table 1). Test-retest: The Overall Composite score, WM domain and the SoP domain mean scores showed highly significant associations between the 2 time points T1 (screening) and T2 (baseline). Pearson’s correlations and ICC’s within the English speaking and the non-English speaking regions were similar (see Table 1 and Figure 2). a) Overall Composite score b) Working Memory Domain c) Speed of Processing Domain

Examining effects of language and region on test-retest ......A.S. Atkins and I. Stroescu are currently or have in the past 3 years received funding from the National Institute of

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Examining effects of language and region on test-retest ......A.S. Atkins and I. Stroescu are currently or have in the past 3 years received funding from the National Institute of

Figure 1. Mean levels and 95% CI for a) Overall Composite scores, b) Working Memory Domain scores and c) Speed of Processing Domain. (Blue Squares) = Screening levels (T1), English norms (Green Circles) = Baseline levels (T2), English norms (Yellow Squares) = Screening levels (T2), language spec. norms (Red Circles) = Baseline levels (T2), language specific norms

BACKGROUNDThe MATRICS Consensus Cognitive Battery (MCCB) is theprimary cognitive test battery recommended for clinicaltrials in schizophrenia populations. The MCCB has beensuccessfully translated to multiple foreign languages andused in many international clinical trials, which hasgenerated an interest to test whether language andregional differences have an influence on thepsychometric characteristics of the MCCB. The primaryaim of the present study was to examine regionaldifferences and the influence of language specificnorming on test-retest stability and practice effects ofthe MCCB. We present results using both Englishspeaking norms and language/region specific norms.

METHODSWe examined a pooled cohort of 2846 schizophreniapatients from 10 clinical trials. All participantscompleted the MCCB twice before starting treatment:once at a screening (T1) and once as part of the baselineassessment (T2). There were 1910 participants fromEnglish speaking countries and 936 participants fromnon-English speaking countries (171 patients from SouthAmerica, 302 patients from Asia and 463 patients fromEurope). The Overall composite MCCB score and the 2cognitive domains, ‘Working memory’ (WM) and ‘Speedof Processing’ (SoP) were studied.

RESULTSMean levels at screening and baseline. Comparisonbetween English norms and language/region specificnorms for the Overall Composite, the SoP domain andthe WM domain means are presented in Figure 1. Meanlevels differed significantly for all 3 measures whencomparing English norms and language specific normswithin the South American, Asian and European regions(p’s <.001). There were also significant differences inmean levels between the 4 regions at both time points,when English and language specific norms were applied.However, the practice effects were low and similarwithin all 4 regions (see Figure 1 and Table 1).

Examining effects of language and region on test-retest reliability of the MATRICS Consensus Cognitive Battery (MCCB)

Georgiades, A1, 2; Davis, VG1; Atkins, AS1; Stroescu, I1; Keefe, RSE1, 2

1 NeuroCog Trials, Inc. (Durham, NC), 2 Duke University Medical Center, Department of Psychiatry (Durham, NC)

EnglishSpeaking

South American

Asian European

CONCLUSIONPractice effects for the Overall Compositescore, the WM and SoP domains were lowand similar in English speaking and non-English speaking patient populations (seeFigure 1 and Table 1). Test-retest stabilityof the Overall Composite score was high inall regions and only marginally affected bynorming method (see Figure 2 and Table1).

Disclosure: Anastasia Georgiades is currently and has in the past 3

years received funding from the National Heart, Lung and Blood Instituteand Duke University School of Medicine. She is also currently a part timeemployee of NeuroCog Trials. V.G. Davis is an employee of NeuroCogTrials. A.S. Atkins and I. Stroescu are currently or have in the past 3 yearsreceived funding from the National Institute of Mental Health and are fulltime employees of NeuroCog Trials.

EnglishSpeaking

South American

Asian European

EnglishSpeaking

South American

Asian European

South America

English Speaking Europe

Asia

Figure 2. Histograms I-IV presenting the distribution of the Overall Composite scores at screening (green) and baseline (yellow) within (I) the English speaking region, (II) the South American region, (III) Europe and (IV) Asia. Scatterplots depicting associations between screening and baseline values of Overall Composite scores within the same regions, by country.

(I)

(II)

(III)

(IV)

Table 1. Practice effects, Cohen’s d (effect size), ICC’s and Pearson correlations by region

English

Norms

Language Specific

NormsEnglish Norms

LanguageSpecific Norms

Region Mean Diff (SD) (screening-baseline)

Cohen’s

d

Mean Diff (SD) (screening-baseline)

Cohen’s

dICC [95% CI]

Pearson’s

corr.ICC [95% CI]

Pearson’s

corr.

English Speaking Overall Comp. (N=1815) 1.9 (5.5)# .15 - - .89 [.88-.90] .90 - -

(98% North America) WM (N=1909) 1.4 (7.1) # .12 - - .80 [.79-.82] .81 - -

SoP (N=1910) 2.2 (7.0) # .18 - - .82 [.80-.83] .83 - -

South America Overall Comp. (N=131) 3.0 (5.7) # .27 2.0 (4.0) # .25 .86 [.82-.90] .87 .86 [.82-.90] .87

WM (N=168) 1.6 (7.4)** .13 1.3 (5.4)** .14 .82 [.76-.86] .82 .83 [.78-.87] .82

SoP (N=171) 2.9 (7.1) # .20 2.0 (4.8) # .20 .86 [.82-.90] .88 .87 [.83-.90] .89

Asia Overall Comp. (N=270) 2.8 (5.5) # .22 2.7 (5.3) # .22 .88 [.85-.90] .90 .89 [.86-.91] .91

WM (N=299) 1.2 (8.2)* .09 1.3 (8.2)** .10 .77 [.72-.82] .77 .77 [.72-81] .77

SoP (N=302) 3.6 (6.1) # .27 3.4 (6.9) # .24 .86 [.83-.89] .90 .85 [.82-.88] .88

Europe Overall Comp. (N=413) 2.2 (5.0) # .17 2.4 (5.2) # .18 .91 [.90-.93] .93 .91 [.89-.92] .92

WM (N=463) 1.7 (6.5) # .13 1.7 (6.7) # .13 .86 [.84-.88] .87 .86 [.84-.88] .87

SoP (N=463) 2.9 (7.1) # .22 2.9 (7.3) # .21 .83 [.80-.86] .85 .84 [.81-.86] .86

Overall Composite score at screening and baseline

Overall Composite score at screening and baseline

Overall Composite score at screening and baseline

Overall Composite score at screening and baseline

R.S.E. Keefe is currently or has in the past 3 years received investigator-initiated research funding support from the Department of Veteran’s Affair, Feinstein Institute for Medical Research, National Institute of Mental Health,Psychogenics, Research Foundation for Mental Hygiene, Inc., and the Singapore National Medical Research Council. He currently or in the past 3 years has received honoraria, served as a consultant, or advisory board member forAbbvie, Akebia, Asubio, Avanir, AviNeuro/ChemRar, BiolineRx, Biogen Idec, BiolineRx, Biomarin, Boehringer-Ingelheim, EnVivo/FORUM, GW Pharmaceuticals, Janssen, Johnson & Johnson, Lundbeck, Merck, MinervaNeurosciences, Inc., Mitsubishi, Neuralstem, Neuronix, Novartis, NY State Office of Mental Health, Otsuka, Pfizer, Reviva, Roche, Sanofi/Aventis, Shire, Sunovion, Takeda, Targacept, and the University of Texas South West MedicalCenter. Dr. Keefe receives royalties from the BACS testing battery, the MATRICS Battery (BACS Symbol Coding) and the Virtual Reality Functional Capacity Assessment Tool (VRFCAT). He is also a shareholder in NeuroCog Trials andSengenix.

Me

ans

and

95

% C

IM

ean

s an

d 9

5%

CI

Me

ans

and

95

% C

I

Paired t-test * P <.05, ** P <.01, # P <.001

Practice effects: The effect size of the practice effectsbetween screening and baseline, measured as Cohen’sd, was low in all regions and only marginally affected byhow the scores were normed (see Table 1).Test-retest: The Overall Composite score, WM domainand the SoP domain mean scores showed highlysignificant associations between the 2 time points T1(screening) and T2 (baseline). Pearson’s correlations andICC’s within the English speaking and the non-Englishspeaking regions were similar (see Table 1 and Figure 2).

a) Overall Composite score

b) Working Memory Domain

c) Speed of Processing Domain