2
Physical Therapy in Sport 8 (2007) 57–58 Editorial Statistical significance and clinical relevance—Does it matter, and what is the difference? When was the last time you read an article that was reporting on quantitative experimental research where the results reported ‘no significant difference’? Have you ever wondered why there tend to be fewer of these articles in the published literature? I have three main hunches as to why this might be. Firstly, I think researchers think it is not worth publishing if they have not found statistically significant findings. Having seen researchers disappointed by their findings, I would always encourage budding researchers to start off with the null hypothesis, i.e. there is no difference and then you can ‘win’ either way. Obviously this approach is much better science, though it is sometimes difficult to persuade those who are relatively new to research that this is the case. In this instance, the line of enquiry is usually based on personal interest and experience, where the investigators are keen to show a difference. This brings me on to the second hunch. When the primary analysis has been completed on a data set and the null hypothesis confirmed, the natural consequence is then to examine the data differently, this may include looking at sub-groups, in order to try and identify trends and/or statistical significance elsewhere. Looking through abstracts in a literature search, these papers appear to be showing significant differences, however, these findings do not always relate entirely to the original research question. There are many examples of these, and I would encourage readers to look out for them, since the conclusions in such studies often overestimate the scale of the findings. Editors’ and reviewers’ are the final potential barriers to publication of statistically non-significant results. There is a perception in some circles that these studies do not contribute as much to the literature as those reporting statistically significant results, and I would disagree with this. Indeed you will spot one article of original research in this issue that presents non- significant results. Particular care needs to be taken in the presentation of the discussion so it is evident how the findings contribute to the overall knowledge, or inter- pretation of the knowledge, of that particular area. Having stated all this, I cannot leave this arena without putting the subject of statistical significance into context. Testing an experimental hypothesis requires statistical analysis however, I think we can get too bogged down with p values, and sometimes miss the bigger picture. A statistically significant difference does not mean it is clinically important, and vice versa, something that is clinically important may not be statistically significant. This is rudimentary, but vitally important. You do not have to be bamboozled by fancy statistical analysis—have a look at the descriptive statistics, do they mean anything clinically? If the clinical relevance is not highlighted in the paper, and this may not be clear, step back and ask yourself the question. Onto this issue, you might have noticed that there have been some changes on the Editorial and Interna- tional Advisory Board. We would like to extend our thanks and gratitude to Jack Taunton, Ken Wright, Elizabeth Gass, Yves Larequi and Liz Mendl for all their hard work and making Physical Therapy in Sport the journal that it is today. Additionally, we would like to welcome Glen Bergeron onto the International Advisory Board, and we look forward to his future contributions to assist further development of the Journal. There is a lower limb theme to this issues, original research and review paper. Lancaster, Nyland and Roberts present a study investigating the validity of the motion palpation test (MPT) for pre-operatively grading patellofemoral joint articular cartilage damage. They report this to be a useful tool when the crepitation sub-classification was judged to be severe. Debates on the benefits of pre-season musculoskeletal screening invariably raise the question of what is ‘normal’? Corkery and colleagues present the results of lower extremity muscle length conducted on 72 students aged 18–22. The clinical assessment included tests for gastrocnemius, hamstrings, rectus femoris and iliopsoas. The results presented may inform clinicians of what normative values to expect for these tests in this particular age group. ARTICLE IN PRESS www.elsevier.com/locate/yptsp 1466-853X/$ - see front matter r 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.ptsp.2007.02.001

Statistical significance and clinical relevance—Does it matter, and what is the difference?

Embed Size (px)

Citation preview

Page 1: Statistical significance and clinical relevance—Does it matter, and what is the difference?

ARTICLE IN PRESS

1466-853X/$ - s

doi:10.1016/j.pt

Physical Therapy in Sport 8 (2007) 57–58

www.elsevier.com/locate/yptsp

Editorial

Statistical significance and clinical relevance—Does it matter, andwhat is the difference?

When was the last time you read an article that wasreporting on quantitative experimental research wherethe results reported ‘no significant difference’? Have youever wondered why there tend to be fewer of thesearticles in the published literature? I have three mainhunches as to why this might be. Firstly, I thinkresearchers think it is not worth publishing if they havenot found statistically significant findings. Having seenresearchers disappointed by their findings, I wouldalways encourage budding researchers to start off withthe null hypothesis, i.e. there is no difference and thenyou can ‘win’ either way. Obviously this approach ismuch better science, though it is sometimes difficult topersuade those who are relatively new to research thatthis is the case. In this instance, the line of enquiry isusually based on personal interest and experience, wherethe investigators are keen to show a difference.

This brings me on to the second hunch. When theprimary analysis has been completed on a data set andthe null hypothesis confirmed, the natural consequenceis then to examine the data differently, this may includelooking at sub-groups, in order to try and identify trendsand/or statistical significance elsewhere. Lookingthrough abstracts in a literature search, these papersappear to be showing significant differences, however,these findings do not always relate entirely to theoriginal research question. There are many examples ofthese, and I would encourage readers to look out forthem, since the conclusions in such studies oftenoverestimate the scale of the findings.

Editors’ and reviewers’ are the final potential barriersto publication of statistically non-significant results.There is a perception in some circles that these studiesdo not contribute as much to the literature as thosereporting statistically significant results, and I woulddisagree with this. Indeed you will spot one article oforiginal research in this issue that presents non-significant results. Particular care needs to be taken inthe presentation of the discussion so it is evident how thefindings contribute to the overall knowledge, or inter-pretation of the knowledge, of that particular area.

ee front matter r 2007 Elsevier Ltd. All rights reserved.

sp.2007.02.001

Having stated all this, I cannot leave this arena withoutputting the subject of statistical significance into context.Testing an experimental hypothesis requires statisticalanalysis however, I think we can get too bogged downwith p values, and sometimes miss the bigger picture. Astatistically significant difference does not mean it isclinically important, and vice versa, something that isclinically important may not be statistically significant.This is rudimentary, but vitally important. You do nothave to be bamboozled by fancy statistical analysis—havea look at the descriptive statistics, do they mean anythingclinically? If the clinical relevance is not highlighted in thepaper, and this may not be clear, step back and askyourself the question.

Onto this issue, you might have noticed that therehave been some changes on the Editorial and Interna-tional Advisory Board. We would like to extend ourthanks and gratitude to Jack Taunton, Ken Wright,Elizabeth Gass, Yves Larequi and Liz Mendl for alltheir hard work and making Physical Therapy in Sportthe journal that it is today. Additionally, we would liketo welcome Glen Bergeron onto the InternationalAdvisory Board, and we look forward to his futurecontributions to assist further development of theJournal.

There is a lower limb theme to this issues, originalresearch and review paper. Lancaster, Nyland andRoberts present a study investigating the validity ofthe motion palpation test (MPT) for pre-operativelygrading patellofemoral joint articular cartilage damage.They report this to be a useful tool when the crepitationsub-classification was judged to be severe. Debates onthe benefits of pre-season musculoskeletal screeninginvariably raise the question of what is ‘normal’?Corkery and colleagues present the results of lowerextremity muscle length conducted on 72 students aged18–22. The clinical assessment included tests forgastrocnemius, hamstrings, rectus femoris and iliopsoas.The results presented may inform clinicians of whatnormative values to expect for these tests in thisparticular age group.

Page 2: Statistical significance and clinical relevance—Does it matter, and what is the difference?

ARTICLE IN PRESSEditorial / Physical Therapy in Sport 8 (2007) 57–5858

Altered biomechanics and fatigue are two possibleintrinsic risk factors associated with injury in running.Cheung and Ng investigate the effect of motion controlshoes and neutral shoes in over-pronating runners,before and after fatigue. Their findings indicated thatthe motion control shoes could control excessive rear-foot motion in these runners regardless of the state offatigue. The next article also investigates fatigue, but inthis instance its role on proprioception at the ankle joint.South and George investigate the effect of peronealmuscle fatigue on joint position sense (JPS). No effecton JPS was reported with fatigue and the authorssuggest that proprioception is fatigue resistant in theperoneal muscles or alternatively that other structures inthe ankle such as the capsule and ligaments play asignificant proprioceptive role. In the first of two review

papers, Pull and Ranson present a comprehensivereview on eccentric muscle training and the implica-tions for injury prevention and rehabilitation. Theinfluence of the peroneal reflex on functional instabilityin the ankle remains controversial. In the secondreview, Delahunt presents the literature to date in thisarea.

I hope you enjoy reading this issue of PhysicalTherapy in Sport.

Zoe HudsonEditor