Human Movement Science 30 (2011) 143–146

Contents lists available at ScienceDirect

Human Movement Science

journal homepage: www.elsevier .com/locate/humov

Response to Cauraugh, J. H., et al. Bilateral movement training and stroke motor recovery progress:A structured review and meta-analysis. Human Movement Science (2009), doi: 10.1016/j.humov.2009.09.004

Dear EditorWe read the structured review and meta-analysis by Cauraugh, Lodha, Naik, and Summers (2009)

with interest. We agree with the authors that whether ‘‘bilateral movement training protocolsimprove motor capabilities in the upper extremities of stroke survivors’’ is an enduring strokerehabilitation question warranting further systematic review and appropriate meta-analyses.However we do not agree with Cauraugh et al.’s conclusion that the evidence ‘‘clearly show(s) thatpost stroke rehabilitation involving bilateral arm training leads to improved motor capabilities andprogress toward recovery’’. Our own interpretation of the existing evidence-base leads us to concludethat there is currently insufficient high quality evidence to recommend bilateral training as a clinicalintervention. We are concerned that the way Cauraugh et al.’s meta-analysis has been carried out mayhave overestimated the effects of this training and resulted in misleading conclusions. We set out thereasons for these concerns below.

We have recently completed a Cochrane review and meta-analysis investigating the effectivenessof simultaneous bilateral training for improving arm function after stroke (Coupar et al., 2010). In con-trast to Cauraugh, we concluded that there is insufficient good quality evidence to make recommen-dations about the effect of simultaneous bilateral training compared to placebo, no intervention orusual care. Our review included 18 randomized controlled trials (549 participants). Data from 14 ofthe studies (421 participants) were available to be combined within a series of meta-analyses,grouped according to type of control group and outcomes. Evidence was identified which suggeststhat bilateral training may be no more (or less) effective than usual care or other upper limb interven-tions for performance in activities of daily living, functional movement of the upper limb or motorimpairment outcomes.

The differences between our conclusions and those of Cauraugh et al. are clearly the result of con-trasting methodological criteria between these reviews. We can identify three key methodological dif-ferences between these two reviews, which relate broadly to decisions concerned with:

(1) The methodological quality of included studies.(2) The combination of outcomes.(3) Comparison groups included within meta-analyses.

1. Methodological quality of included studies

Our review, like most other Cochrane reviews, focused only on randomized trials, becauserandomization is the only way to prevent systematic differences between baseline characteristics ofparticipants in different intervention groups in terms of both known and unknown confounders

0167-9457/$ - see front matter � 2010 Elsevier B.V. All rights reserved.doi:10.1016/j.humov.2010.10.003

144 Correspondence / Human Movement Science 30 (2011) 143–146

(Higgins and Green, 2009). In contrast, Cauraugh et al. cite Rosenthal and DiMatteo (2001), an author-ity on meta-analysis, in support of their decision to combine a very varied group of studies, combiningrandomized controlled trials with uncontrolled studies. It is our understanding however that, whileRosenthal and DiMatteo clearly argue in support of the merits of combining ‘apples and oranges’,in doing so he emphasizes the need for addressing and appropriately weighting studies accordingto methodological quality. Rosenthal and DiMatteo also specifically highlight the need for sensitivityanalyses to explore the impact of methodological issues. While Cauraugh et al. have reported someaspects of the methodological quality of the studies included in their meta-analysis, they have not ap-plied weighting to studies or carried out sensitivity analyses to explore the impact of the methodolog-ical variability on the overall standardized mean difference. We believe sensitivity analysis is essentialhere to explore the impact of combining data from non-randomized non-controlled studies with datafrom randomized controlled trials and to enable readers to assess the potential biases within the pre-sented data. Another approach to interpretation of findings of studies with diverse methodologies isthe examination of moderator variables. Rosenthal and DiMatteo suggest that reviewers should ex-plore these to allow for further testing of details of theory and a better understanding of the natureof the included studies. Cauraugh et al. have used moderator variable analysis to explore the typeof bilateral arm training technique, and also intended to explore impairment level and functional lim-itations. We would argue that used in isolation from other methodological biases, the analysis of thesevariables is insufficient to account for potential biases of combining studies irrespective of design andoutcomes used. There are also a number of other potential methodological biases which Cauraughet al. do not address; these include selection, detection and reporting bias (Higgins and Green,2009), which inevitably have the potential to influence findings.

2. The combination of outcomes

Cauraugh et al. combined, within a single analysis, outcomes which clearly measure very differentdomains. Combined outcomes included assessments of performance in activities of daily living, func-tional movement of the hand and arm, and motor impairment of the upper limb (including motorimpairment scales, temporal, spatial and strength outcomes). In contrast we felt it was not clinicallymeaningful to combine outcomes which measured different domains as defined within the ICFframework for measuring health and disability (World Health Organisation, 2001). We identifiedthe domain of motor impairment, aligned with the ICF category of body functioning, and threedomains, performance in activities of daily living, functional movement, performance in extendedactivities of daily living and motor impairment, aligned with the ICF category of activity and partic-ipation, and carried out separate analyses for each of these pre-defined outcome classifications. Thisapproach led to an analysis that could be interpreted clinically in relation to meaningful outcomes.We appreciate the considerable difficulty which exists in deciding which outcomes should andshould not be combined within meta-analyses of complex rehabilitation trials, and we acknowledgethat there is no accepted best practice to guide reviewers in this decision. However, given that othermeta-analyses (e.g., Barclay-Goddard, Stevenson, Poluha, Moffatt, & Taback, 2004; Mehrholz, Platz,Kugler, & Pohl, 2008) have demonstrated significant differences in measures of impairment butnot measures of functional ability, we again believe that sensitivity analyses to explore the effectof combining such heterogeneous outcomes are essential. The need for sensitivity analyses whenthere is considerable variation between studies is again highlighted by Rosenthal and DiMatteo(2001).

3. Comparison groups within meta-analyses

Cauraugh et al. decided to estimate the cumulative effect of bilateral arm movement trainingregardless of the comparison groups. We believe that this is potentially problematic. It is very difficultto interpret in a single meta-analysis studies comparing bilateral training to a range of controltreatments; unilateral training, control interventions such as walking (e.g., Higgins et al., 2006),electrical stimulation (e.g., Hesse et al., 2005), other therapeutic interventions and no control groupor intervention (e.g., Richards, Senesac, Davis, Woodbury, & Nadeau, 2008; Stinear & Byblow, 2004).

Correspondence / Human Movement Science 30 (2011) 143–146 145

We argue that combining studies with such diverse control interventions into one meta-analysis pro-vides limited clinically useful insight into the relative effectiveness of bilateral training. In contrast toCauraugh et al., we compared the effect of bilateral arm movement training with three pre-statedcomparison groups: (1) placebo or no intervention, (2) usual care, (3) other specific upper limb inter-ventions or programs. Furthermore, where studies included another intervention as an adjunct tobilateral training, which was also delivered to the control group, we included these studies in theappropriate comparison groups, regardless of the adjunct intervention, but completed a sensitivityanalysis to explore the effect of including studies where the simultaneous bilateral training was com-bined with another intervention. Rosenthal and DiMatteo (2001) emphasize the importance of clearlyformulating research questions in order to produce informative meta-analyses. Unfortunately, thequestion of whether bilateral movement training is more effective than anything else, including notreatment and any other active treatments, is not sufficiently focused to enable useful clinical conclu-sions to be made.

Clearly Cauraugh et al. have, like ourselves, carried out considerable work in identifying and syn-thesizing the evidence in relation to the effectiveness of bilateral training for patients with stroke.However, the clearly opposing conclusions which have been drawn from the contrasting analyses re-ported within the two reviews is a matter for concern, and is likely to confuse rather than clarify themessage given to clinicians making treatment choices for individual patients.

We concur that there is no single correct way to perform a meta-analysis (Hall & Rosenthal, 1995),and we are aware of the complexities associated with systematic reviews of multifaceted rehabilita-tion interventions. However we suggest that the methodological decisions made by Cauraugh et al., inparticular the decision to combine heterogeneous study designs, outcomes and comparison groupshas led them to draw conclusions which are potentially clinically misleading.

Yours faithfully,A. Pollock, J. Morris, F. van Wijck, F. Coupar, P. Langhorne.

References

Barclay-Goddard, R. E., Stevenson, T. J., Poluha, W., Moffatt, M., & Taback, S. P. (2004). Force platform feedback for standingbalance training after stroke. Cochrane Database of Systematic Reviews 2004, Issue 4. Art. No.: CD004129. doi: 10.1002/14651858.CD004129.pub2.

Cauraugh, J. H., Lodha, N., Naik, S. K., & Summers, J. J. (2009). Bilateral movement training and stroke motor recoveryprogress: A structured review and meta-analysis. Human Movement Science, 29, 853–870.

Coupar, F., Pollock, A., van Wijck, F., Morris, J., & Langhorne, P. (2010) Simultaneous bilateral training for improving armfunction after stroke. Cochrane Database of Systematic Reviews 2010, Issue 4. Art. No.: CD006432. doi: 10.1002/14651858.CD006432.pub2.

Hall, J. A., & Rosenthal, R. (1995). Interpreting and evaluating meta-analysis. Evaluation and the Health Professions, 18,393–407.

Hesse, S., Werner, C., Pohl, M., Rueckriem, S., Mehrholz, J., & Lingnau, M. L. (2005). Computerized arm training improves themotor control of the severely affected arm after stroke: A single-blinded randomized trial in two centers. Stroke, 36,1960–1966.

Higgins, J., Salbach, N. M., Wood-Dauphinee, S., Richards, C. L., Cote, R., & Mayo, N. E. (2006). The effect of a task-oriented intervention on arm function in people with stroke: A randomized controlled trial. Clinical Rehabilitation, 20,296–310.

Higgins, J. P. T., & Green, S. (Eds.). 2009. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2 [updatedSeptember 2009]. The Cochrane Collaboration. Available from <www.cochrane-handbook.org>.

Mehrholz, J., Platz, T., Kugler, J., & Pohl, M. (2008) Electromechanical and robot-assisted arm training for improving armfunction and activities of daily living after stroke. Cochrane Database of Systematic Reviews 2008, Issue 4. Art. No.:CD006876. doi: 10.1002/14651858.CD006876.pub2.

Richards, L. G., Senesac, C. R., Davis, S. B., Woodbury, M. L., & Nadeau, S. E. (2008). Bilateral arm training with rhythmicauditory cueing in chronic stroke: Not always efficacious. Neurorehabilitation and Neural Repair, 22, 180–184.

Rosenthal, R., & DiMatteo, M. R. (2001). Meta-analysis: Recent developments in quantitative methods for literature reviews.Annual Review of Psychology, 52, 59–82.

Stinear, J. W., & Byblow, W. D. (2004). Rhythmic bilateral movement training modulates corticomotor excitability andenhances upper limb motricity poststroke: A pilot study. Journal of Clinical Neurophysiology, 21, 124–131.

World Health Organisation (2001). ICF: International classification of functioning, disability and health. Geneva: World HealthOrganisation.

146 Correspondence / Human Movement Science 30 (2011) 143–146

Alex PollockGlasgow Caledonian University, Scotland, UK

Nursing Midwifery and Allied Health Professions (NMAHP) Research Unit,Glasgow Caledonian University,

Buchanan House,Cowcaddens Road,

Glasgow G4 0BA, Scotland, UKTel.: +44 0141 331 8100

E-mail address: alex.pollock@gcu.ac.uk

Jacqui MorrisUniversity of Dundee,

Scotland, UK

Frederike van WijckGlasgow Caledonian University,

Scotland, UK

Fiona CouparPeter Langhorne

University of Glasgow,Scotland, UK