Embed Size (px)
Citation preview
Schizophrenia Research 153 (2014) 237–239
Contents lists available at ScienceDirect
Schizophrenia Research
j ourna l homepage: www.e lsev ie r .com/ locate /schres
Letter to the Editor
Response to the letters of Dr Amos andDr Preti and colleagues
Dear Editors,
The letters of Amos et al. and Preti et al. provide us with the oppor-tunity to rebut some of their criticisms and to further clarify the designand outcome of our meta-analysis of randomized trials in the preven-tion of first psychosis.
In his criticism of our meta-analysis, Amos relies on the arguments ofthe pre meta-analysis era: this can be characterized as “vote counting”.Vote counting concludes that if most individual studies do not find statis-tically significant results, then there is no effect. However such a conclu-sion may not be correct when the primary studies are underpoweredand could not therefore detect effects as statistically significant even ifthe effect sizes can be seen as clinically relevant (Borenstein et al., 2009).
To illustrate the key problem with vote counting we discuss a meta-analysis from hypertension research. The effect of streptokinase on theprevention of death following heart attack was studied from 1959 to1988 (Lau et al., 1992). Only 6 of 33 trials showed statistical significance.Meta-analysis was not available at the time and the use of streptokinasein preventingdeathwas criticized because of the 27negative studies. Nar-rative reviews used vote counting as a criterion, as does Amos. Lau et al.conducted cumulative meta-analyses in 1992 and found a risk reductionof 21% with a cumulative p-value of 0.0000008. The individual studiesall had statistical power of about 20%, while the meta-analysis exceeded99.9%. Had meta-analysis been applied earlier in this research field, thenstatistical significance could have been demonstrated in 1973 after only8 trials! The contribution of the subsequent 25 trials, which enrolled anadditional 35,542 patients, had little or no impact on the odds ratio estab-lishing efficacy, but simply narrowed the 95 percent confidence interval.
Underpowered trials might become an issue when rare events arestudied, such as incidence of first psychosis and death after myocardialinfarction. For such outcomes it often becomes problematic to recruitenough patients in trials, rendering the studies underpowered and con-sequently their outcomes statistically insignificant. Using vote-countingtechniques in such a research field may subsequently give the false im-pression that the whole endeavour must be cast aside due to an appar-ent failure to demonstrate statistically significant effects. A related erroris to further conclude that statistically insignificant effects are clinicallyunimportant. In fact, for establishing clinical relevance the p-valuemight be an inappropriate and even misleading metric, because it is acomplex function of sample size, effect size and its variance. From theperspective of clinical relevance, the effect size is key. In the context ofprevention and early intervention trials, we have an interest in risk re-duction and corresponding effect sizes, as captured by statistical vari-ables such as Risk Ratio (RR) or Odds Ratio (OR). The advantage ofmeta-analysis is that the clinically relevant metric, such as the RR, canbe pooled across a series of smaller and larger studies, such that thepooled effect size can be assessedwithmore power and greater accuracy.
The Button paper referred to by Amos states that publication biasand low quality of the primary studies may inflate true effect-sizes in
0920-9964/$ – see front matter © 2014 Elsevier B.V. All rights reserved.http://dx.doi.org/10.1016/j.schres.2014.01.004
meta-analysis (Button et al., 2013). This is correct and therefore weevaluated and reported the impact of publication bias and study qualityon our meta-analytic outcomes. We made two observations. First, pub-lication bias was small. Second, the quality of the primary studies washardly correlated with the effect sizes and could therefore not have bi-assed our appreciation of the pooled effect size in a substantial way(van der Gaag et al., 2013).
Whether one should offer preventive interventions in routine careor not, will partly depend on the cost-utility analysis (CUA). The CUAof the study by van der Gaag et al. (2012) is now under review(VanDerGaag 2012e).
If Dr Amos can agree with the scientific community that p-values ofindividual trials are less important in demonstrating risk reduction inrare disorders and that risk reduction is the key focus in preventingrare diseases, then he should also agree that meta-analysis is an appro-priate method to pool the available evidence. Actually, the pooled dataclearly demonstrate that interventions for preventing or delaying firstonsets in psychosis are more effective than the treatment of highblood pressure to prevent or delay stroke.
The comments and remarks by Preti and colleagues are on statistical,clinical and conceptual matters. We will address these one by one.
Statistical comments
One primary study had three arms leading to double counting of 28control subjects in a total of 1150 subjects. In subsequent (sensitivity)analyses no double counting occurred. Preti and colleagues have per-formed another sensitivity analysis without double counting andshow that the risk ratios in the low quality studies do not differ (RR inboth 0.282) and in the high quality studies the risk ratios differ 0.002(RR is 0.526 vs. 0.528), thus not affecting our conclusions materially.Preti et al.'s additional sensitivity analysis underscores the robustnessof our findings.
To distinguish low quality studies from high quality studies, theCTAM uses an arbitrary cut-off at 65. Preti and colleagues used a conve-nience cut-off at 80. This shows a difference in risk reduction of 72% inlow and high quality studies versus 47% in very high quality studies.The decrease in effect-sizes in methodologically more rigorous studieshas been foundmore often.We are pleased to see that the risk reductionis still substantial and statistically significant (all p-values b 0.003) instudieswithoutflaws, showing the robustness of thefindings oncemore.
Clinical comments
In a prevention trial it would be a problem to include peoplewho arepsychotic, but have not been recognized as such. This problem is inher-ent to all prevention studies relying on psychiatric and other diseaseclassifications. To illustrate, in Nelson and Yung's study, GPs under-diagnosed psychosis in about 11% of all referrals (Nelson and Yung,2007). All UHR studies have been aware of these risks and have putgreat effort in preventing the inclusion of dissimulating psychotic pa-tients as having a sub threshold or UHR mental state. In general UHR
238 Letter to the Editor
researchers have been successful at ensuring that strict inclusion criteriaare met, and only few patients were detected as already psychotic atbaseline during crosschecks. For instance, the study by van der Gaagand colleagues did intention-to-treat and sensitivity analysis removingthe misclassifications and actually found better results after removingthe exclusion violations (van der Gaag et al., 2012). So the awarenessof including already psychotic patients with a UHR status is important,but faulty inclusions have probably decreased the effect-sizes ofinterventions, as they are present in treatment and control conditionsequally and push the risk ratio toward 1, thus strengthening the null-hypothesis of no effect.
Preti and colleagues alsomention research in 12-year olds (Hameedet al., 2013; Sullivan et al., 2013) and state that these young peoplepresent with other problems and only have mild psychotic-like symp-toms. We are aware that the developmental tracks to psychosis arevery diverse and non-specific. The presented UHR studies in the meta-analysis have been using quite strict inclusion criteria including geneticrisk, attenuated symptoms and brief psychosis shorter than a weekwith spontaneous remission, combined with help-seeking for an axis 1or axis 2 disorder and a recent decline in social functioning. So this is a se-lection of people who have developed a need for care and are at immi-nent risk for developing psychosis in about 22% of the cases within12 months (Fusar-Poli et al., 2013).We are aware that young adolescentscan also be on the track to psychosis but their risk for developing psycho-sis while having hallucinations is only 0.6% in the next year (Rubio et al.,2012). Interventions in this grouphave not been developed and exploreduntil this moment and could not be evaluated in this meta-analysis.
Conceptual comments
Preti and colleagues warn that antipsychotic medications might notbe protective in the long run, because negative symptoms might stillevolve. As far as we know there is no research in effectively preventingnegative symptoms. The treatment options of established negativesymptoms are still characterised by poor results and in need of furtherresearch. Our meta-analysis showed that antipsychotic medication hasno advantage over CBT in postponing or preventing a first episode ofpsychosis and we support the idea of CBT as a first line treatment. Inthose people who do not respond to CBT several other treatmentoptions could be chosen, such as omega-3 fatty acids or low dose anti-psychotic medication, but these strategies require more evidence.
Negative symptoms have been predictive of transition into psycho-sis in people who have subclinical psychotic symptoms and also play arole in the functional outcome of UHR patients (Valmaggia et al.,2013). The field must acknowledge this fact and try to detect patientswith predominantly negative symptoms especially in those patientswith a major decrease in social functioning. Conceptually, negativesymptoms are therefore best seen as both a potential risk factor andan outcome target.
Preti and colleagues mistakenly state that the psychosocial treat-ment protocols are generally focussed on psychoeducation for symp-toms and stigma. This is a misperception of the treatment in UHRpatients. Education regarding traditional diagnostic categories andsymptoms is not a prominent focus in cognitive–behavioural interven-tions. Normalisation through education on perceptual aberrations, ex-periences of salience, and cognitive biases takes away fear from oddexperiences and helps to develop alternative explanations. Once alter-native explanations are considered real change through experientiallearning can take place. Conviction, preoccupation and fear only changeafter successful behavioural experiments, wherein a non-delusional ap-praisal is confirmed. So “insight-based coping” is not the issue, but realchange of ideas and the breakdown of safety and avoidance behaviours.Continued selective attention for threat and avoidance behaviours pro-pels pre-psychotic ideations and contributes to the collected “evidencein favour” of potentially delusional explanations, until psychotic thresh-old is reached. Better coping is helpful but insufficient, and a more
ambitious target is disconfirmation of psychotic explanations of odd ex-periences and the removal of their behavioural consequences.
Preti and colleagues suggest transcending the categorical outcomessuch as transition/non-transition, and here we fully agree, as we havenoted before (Yung et al., 2010). Non-transitions do improve on sub-clinical psychotic symptoms, but some persist without transitioning tofrankpsychosis. Also anxiety disorder at one-year follow-up is still prev-alent in 21% to 38% of the cases and depression in 16% to 29% of thosenot transitioning (Addington et al., 2011; van der Gaag et al., 2012).Lin and colleagues state that “The at-risk criteriamay therefore beusefulfor detecting young people who are at heightened risk for chronic non-psychotic psychopathology, which could be targeted in clinical care”(Lin et al., 2012).
This alsomeans that those who do not transition to psychosis are byno means all false-positive cases, but a majority of them are at risk fordeveloping persisting or recurrent (non-psychotic) mental illnesses.The majority of all UHR patients are at risk of substantial functional im-pairment and failure to fulfil their vocational potential. Many UHRpatients still attend school or have a paid job, but are at risk of vocationalfailure. Outreach to schools and workplaces might reduce the risks ofdropping out.ManyUHRpatients have troubled relationships, with par-ents, friends and spouses and some psychoeducational outreach mightwell improve their social functioning. Many UHR patients use alcoholand drugs excessively and more focussed interventions might takeaway this additional risk factor. A proportion of UHR patients have atrauma history and also present with (sub)clinical posttraumatic stressdisorder (Thompson et al., in press). Trauma-focused therapies mightreduce trauma symptoms, but also ameliorate subclinical psychoticsymptoms. Notwithstanding the multiple outcome targets which exist,demonstrating that interventions aimed at preventing the transitioninto first episode psychosis as demonstrated in the meta-analyses byPreti and ours, are extremely important. Sustained psychotic illnessessuch as schizophrenia are major source of serious health burden. Onceschizophrenia develops the recovery rates with traditional care areonly 13.5% (Jääskeläinen et al., 2013), though early psychosis care maywell be able to greatly improve on this. The health gainmaywell be sub-stantial even when delaying and ameliorating a transition and muchmore so when a transition to psychosis can be fully prevented. Otherinterventions can be added to reduce concurrent morbidity and to im-prove functioning and well-being.
ContributorsAll authors have contributed to the final version and have approved the final
manuscript.
Conflict of interestThe authors disclose no conflicting interests and this meta-analysiswas accomplished
without funding.
AcknowledgementsNo acknowledgements for this response to two letters.
References
Addington, J., Cornblatt, B.A., Cadenhead, K.S., Cannon, T.D., McGlashan, T.H., Perkins, D.O.,et al., 2011. At clinical high risk for psychosis: outcome for nonconverters. Am.J. Psychiatry 168 (8), 800–805.
Borenstein, M., Hedges, L.V., Higgins, J.P.T., Rothstein, H.R., 2009. Introduction to Meta-analysis. Wiley and Sons Ltd., Chichester, West Sussex, United Kingdom.
Button, K.S., Ioannidis, J.P., Mokrysz, C., Nosek, B.A., Flint, J., Robinson, E.S., et al., 2013.Power failure: why small sample size undermines the reliability of neuroscience.Nat. Rev. Neurosci. 14 (5), 365–376.
Fusar-Poli, P., Borgwardt, S., Bechdolf, A., Addington, J., Riecher-Rössler, A., Schultze-Lutter, F., et al., 2013. The psychosis high-risk state: a comprehensive state-of-the-art review. JAMA Psychiatry 70 (1), 107–120.
Hameed, M.A., Lewis, A.J., Sullivan, S., Zammit, S., 2013. Child literacy and psychotic expe-riences in early adolescence: findings from the ALSPAC study. Schizophr. Res. 145(1–3), 88–94.
Jääskeläinen, E., Juola, P., Hirvonen, N., McGrath, J.J., Saha, S., Isohanni, M., et al., 2013. Asystematic review and meta-analysis of recovery in schizophrenia. Schizophr. Bull.39 (6), 1296–1306.
239Letter to the Editor
Lau, J., Antman, E.M., Jimenez-Silva, J., Kupelnick, B., Mosteller, F., Chalmers, T.C., 1992. Cu-mulative meta-analysis of therapeutic trials for myocardial infarction. N. Engl. J. Med.327 (4), 248–254.
Lin, A., Nelson, B., Yung, A.R., 2012. ‘At-risk’ for psychosis research: where are we head-ing? Epidemiol. Psychiatr. Sci. 21, 329–334.
Nelson, B., Yung, A.R., 2007. When things are not as they seem: detecting first-episodepsychosis upon referral to ultra high risk (“Prodromal”) Clinics. Early. Interv. Psychi-atry. 1 (2), 208–211.
Rubio, J.M., Sanjuán, J., Flórez-Salamanca, L., Cuesta, M.J., 2012. Examining the course ofhallucinatory experiences in children and adolescents: a systematic review.Schizophr. Res. 138, 248–254.
Sullivan, S., Rai, D., Golding, J., Zammit, S., Steer, C., 2013. The association between autismspectrum disorder and psychotic experiences in the Avon longitudinal study of par-ents and children (ALSPAC) birth cohort. J. Am. Acad. Child Adolesc. Psychiatry 52(8), 806–814.e2.
Thompson, A.D., Nelson, B., Yuen, H.P., Lin, A., Amminger, G.P., McGorry, P.D., et al., 2013.Sexual trauma increases the risk of developing psychosis in an ultra high-risk “pro-dromal” population. Schizophr. Bull. (in press).
Valmaggia, L.R., Stahl, D., Yung, A.R., Nelson, B., Fusar-Poli, P., McGorry, P.D., et al., 2013.Negative psychotic symptoms and impaired role functioning predict transition out-comes in the at-risk mental state: a latent class cluster analysis study. Psychol.Med. 43, 2311–2325.
Van der Gaag, M., Nieman, H., Rietdijk, J., Dragt, S., Ising, K., Klaassen, R.M.C., et al., 2012.Cognitive behavioral therapy for subjects at ultrahigh risk for developing psychosis: arandomized controlled clinical trial. Schizophr. Bull. 38 (6), 1180–1188.
van der Gaag, M., Smit, F., Bechdolf, A., French, P., Linszen, D.H., Yung, A.R., et al., 2013.Preventing a first episode of psychosis: meta-analysis of randomized controlled pre-vention trials of 12 month and longer-term follow-ups. Schizophr. Res. 149, 56–62.
Yung, A.R., Nelson, B., Thompson, A., Wood, S.J., 2010. The psychosis threshold in ultrahigh risk (prodromal) research: is it valid? Schizophr. Res. 120, 1–6.
Mark van der GaagVU University and EMGO Institute of Health and Care Research,
Amsterdam, The Netherlands
Parnassia Psychiatric Institute, The Hague, The NetherlandsCorresponding author at: Vu University, Department of Clinical
Psychology, Van der Boechorststraat 1, 1081 BT Amsterdam, TheNetherlands. Tel.: +31 6 45780463; fax: +31 20 5988758.
E-mail address: [email protected].
Filip SmitVU University and EMGO Institute of Health and Care Research,
Amsterdam, The NetherlandsTrimbos Institute, Utrecht, The Netherlands
Paul FrenchUniversity of Manchester, Manchester, United Kingdom
Alison R. YungUniversity of Manchester, Manchester, United Kingdom
University of Melbourne, Australia
Patrick McGorryUniversity of Melbourne, Australia
Pim CuijpersVU University and EMGO Institute of Health and Care Research,
Amsterdam, The Netherlands
31 December 2013
