at SciVerse ScienceDirect

Journal of Psychiatric Research 46 (2012) 233e238

Contents lists available

Journal of Psychiatric Research

journal homepage: www.elsevier .com/locate/psychires

Review

Can we combine symptom scales for collaborative research projects?

John P. Lyne a,b,*, Anthony Kinsella a, Brian O’Donoghue a,b

aDETECT Services, Avila House, Block 5 Blackrock Business Park, Co. Dublin, IrelandbCollege of Life Sciences, University College Dublin, Belfield, Dublin 4, Ireland

a r t i c l e i n f o

Article history:Received 21 July 2011Received in revised form4 September 2011Accepted 10 October 2011

Keywords:ScalesSymptomatologyCollaborative researchRemission criteriaPsychosis

* Corresponding author. DETECT Services, Avila Honess Park, Co. Dublin, Ireland. Tel.: þ353 01 2791700;

E-mail address: johnlyne@mail.com (J.P. Lyne).

0022-3956/$ e see front matter � 2011 Elsevier Ltd.doi:10.1016/j.jpsychires.2011.10.002

a b s t r a c t

Collaborative research projects have the potential to answer important research questions, which mayotherwise require huge resources, funding, and time to complete. There are several scales for measuringpsychotic symptoms in schizophrenia and other psychotic disorders, with the Scale for Assessment ofPositive Symptoms (SAPS), Scale for Assessment of Negative Symptoms (SANS), Positive and NegativeSymptom Scale (PANSS), and the Brief Psychiatric Rating Scale (BPRS) being among the most commonlyused. High quality research efforts have used these three scales in different projects, and in order tomerge study efforts, some means of combining data from these scales may be necessary. We reviewedcorrelations in published studies for these three scales, finding them to be highly correlated, however oncomparison of the three scales there were considerable clinical differences between them. The paperdiscusses potential methods for combining the scales in collaborative research, including use of therecently developed standardised remission criteria for schizophrenia.

� 2011 Elsevier Ltd. All rights reserved.

1. Introduction

There has been much research into symptoms of psychosis overthe last few decades, with new findings leading to better treatmentand management of patients with mental illness (Tandon et al.,2010). Large sample studies take years to complete, and requirehuge financial input with heavy reliance on research funding.Collaboration between research projects is a more efficient way toobtain a large study sample, without the need for new datacollection. This has the potential to increase statistical power foranswering research hypotheses for which sample size is an issue,for example when investigating long-term outcomes of psychosisor where large sample sizes are necessary for genetic studies.

If different scales have been used in research projects, somemethod for combining scales is necessary to allow collaboration toproceed. Three of the most commonly used scales for psychoticsymptoms include: (i) Scale for Assessment of Positive Symptoms(SAPS) and the Scale for Assessment of Negative Symptoms (SANS),(ii) Brief Psychiatric Rating Scale (BPRS), and (iii) Positive andNegative Syndrome Scale (PANSS) (Andreasen, 1984; Andreasen,1989; Kay et al., 1987; Overall & Gorham, 1962). The aim of thispaper is to investigate the challenges of combining these three

use, Block 5 Blackrock Busi-fax: þ353 01 2791799.

All rights reserved.

scales for collaborative research purposes, and in the discussionwewill explore possible methods for achieving this.

2. The rating scales

2.1. SAPS

The SAPS is a widely used tool for measurement of positivesymptoms. The scale has been shown to have good interrater andtest-retest reliability (Moscarelli et al., 1987; Norman et al., 1996;Andreasen & Flaum, 1991), and is divided into 31 item ratings andfour global factor ratings. Each item is scored on a six point scale(0e5). TheSAPSglobal total is thesumof the fourglobal ratings,whilethe SAPS composite total is the sum of the thirty one item ratings.

2.2. SANS

The SANS is recognised as a valuable tool in assessment of neg-ative symptoms (Kirkpatrick et al., 2006), with sound psychometricproperties, such as interrater reliability, internal consistency, andpredictive validity (Andreasen, 1990a; Andreasen, 1990b;Andreasen, 1982). The scale is divided into 19 item ratings, and 5global factor ratings. Like the SAPS, each item is scored on a sixpoint scale (0e5). The SANS global total is the sum of the five globalratings, while the SANS composite total is the sum of the nineteenitem ratings. Andreasen has reported that summary scores based

J.P. Lyne et al. / Journal of Psychiatric Research 46 (2012) 233e238234

on global ratings is probably a more sensitive index than thecomposite score for both the SAPS and SANS (Andreasen, 1982).

2.3. BPRS

Overall andGorham introduced the Brief Psychiatric Rating Scale(BPRS) in 1962, and it is still a commonly used measure of symp-tomatology today (Overall & Gorham,1962). The initial 16 item scalewas subsequently expanded to 18 items (Overall & Klett,1972), withmore recent versions expanded further to 24 items (Lukoff et al.,1986). BPRS items are rated on a seven point scale (1e7), with theBPRS total score calculated by summing all items. The BPRS has beenshown to be a sensitive and specific measure, with good interraterreliability (Ventura et al., 1993; Roncone et al., 1999).

The BPRS can be divided further into subscales, including theBPRS-PS, and the BPRS-WR. The BPRS-PS is used to measure posi-tive psychotic symptoms, comprising the items hallucinatorybehaviour, unusual thought content, suspiciousness, and concep-tual disorganisation (Ventura et al., 2000; Harris et al., 2005). TheBPRS-WR comprises a withdrawal-retardation factor, including theblunted affect, emotional withdrawal and motor retardation items,which is used as a measure of negative symptomatology (Welhamet al., 1999; Eckert et al., 1996). The sensitivity of the BPRS-WR tonegative symptom change has been questioned by Eckert et al. whoshowed that an additional measure of negative symptoms, such asthe SANS, may increase the ability to detect changes in negativesymptoms with substantially fewer subjects than the BPRS-WRalone (Eckert et al., 1996).

2.4. PANSS

The PANSS was constructed by combining items from the BPRS,and the Psychopathology Rating Scale (Singh & Kay, 1975). The

Table 1Comparison of symptoms in the SAPS, PANSS-Pos, and the BPRS-PS.

SAPS

Hallucinations Auditory hallucinationsVoices commentingVoices conversingSomatic or tactile hallucinationsOlfactory hallucinationsVisual hallucinations

Delusions Persecutory delusionsDelusions of jealousyDelusions of sin or guiltGrandiose delusionsReligious delusionsSomatic delusionsIdeas and delusions of referenceDelusions of being controlledDelusions of mind readingThought broadcastingThought insertionThought withdrawal

Bizarre Behaviour Clothing and appearanceSocial and sexual behaviourAggressive and agitated behaviourRepetitive or stereotyped behaviour

Positive Formal Thought Disorder Derailment (loose associations)TangentialityIncoherence (word salad)IllogicalityCircumstantialityPressure of speechDistractible speechClanging

Other Inappropriate affect

a BPRS-PS items.b Also contained in the PANSS general subscale.

PANSS consists of 30 items rated on a seven point scale (1e7).Seven of the items contribute to an overall positive syndromescore, the PANSS Positive Scale (PANSS-PS), while another sevenitems contribute to a negative syndrome score, the PANSS Nega-tive Scale (PANSS-NS); 16 items contribute to a measure of generalpsychopathology. The PANSS has been recommended as appro-priate for use in positive and negative symptom evaluation (Alphs,2006), and has been validated by van der Gaag et al. (2006a;2006b).

3. Comparison of items present in the three scales

3.1. Positive symptom item comparisons

Table 1 illustrates the positive symptom items for the SAPS,PANSS-PS, and BPRS. For ease of comparison the subscale headingsof the SAPS have been used to classify items for each of the scales.There are items which are similar across the three scales for thedelusions, hallucinations, and positive formal thought disorderfactors. The bizarre behaviour subscale of the SAPS has no clearlycomparable items with the PANSS-PS, however Andreasen et al.have positioned the mannerisms/posturing item, present in thegeneral PANSS scale, alongside the SAPS bizarre behaviour subscalein their remission criteria paper (mannerisms/posturing item hasbeen excluded from the PANSS-PS section of Table 1 as it is part ofthe general PANSS scale) (Andreasen et al., 2005). The BPRS alsocontains the mannerisms/posturing item, and the extended versionBPRS includes a bizarre behaviour item. Some items contained inthe PANSS-PS, including grandiosity, hostility, and excitement, arealso present in the BPRS, however these items are not included inthe BPRS-PS. Likewise, a BPRS-PS item, unusual thought content, ispresent as a general subscale item on the PANSS.

PANSS-PS BPRS

Hallucinatory behaviour Hallucinatory behavioura

DelusionsSuspiciousness/PersecutionGrandiosity

Unusual thought contenta,b

Suspiciousnessa

Grandiosity

Bizarre behaviourMannerisms and posturinga

Conceptual disorganisation Conceptual Disorganisationa

ExcitementHostility

ExcitementHostility

J.P. Lyne et al. / Journal of Psychiatric Research 46 (2012) 233e238 235

3.2. Negative symptom item comparisons

Table 2 illustrates the negative symptom items for the SANS,PANSS-NS, and BPRS. Again for ease of comparison the subscaleheadings of the SANS have been used to classify items for each ofthe scales. There are fewer items on the PANSS-NS and BPRS-WR,however the content of some of these items overlap with morethan one SANS item. Some of the general PANSS items, alsooverlap with other negative symptom items on the SANS andBPRS. For example, the general PANSS item, disturbance of voli-tion could be viewed as overlapping with the grooming andhygiene item and the recreational interests and activities item ofthe SANS. The motor retardation item is considered part of theBPRS-WR subscale, but is included in the general subscale in thePANSS. The motor retardation item is not directly comparablewith any single item of the SANS, but could be considered tooverlap with a number of the SANS items such as decreasedspontaneous movements, paucity of expressive gestures,increased latency of response, and physical anergia. The generalPANSS item poor attention may overlap with social inattentive-ness in the SANS, however there have been doubts expressed as towhether attention items should be considered under the negativesymptom construct (Toomey et al., 1998; Palacios-Araus et al.,1995; Miller et al., 1993).

3.3. Comparison of other items on the PANSS and BPRS

The general scale of the PANSS and other items of the BPRS arepresented in Table 3. Some of these items may represent othersymptoms/factors in schizophrenia, which are not adequatelyrepresented on the positive or negative scales. As the PANSS wasderived from the BPRS many of these items overlap, however itshould be noted that the item definitions were changed when thePANSS was created, and Bell et al. showed that several of theseitems, which are identically named on both scales, were modifiedsufficiently to make them not necessarily interchangeable (Bellet al., 1992). When comparison between shared items wasmeasured with weighted kappas, the kappa coefficient was in theexcellent range for only three items (blunted affect, hallucinations,and grandiosity), none of which are contained in the general scaleof the PANSS.

Table 2Comparison of symptoms in the SANS, PANSS-Neg, and BPRS-WR.

SANS

Affective Flattening or Blunting Unchanging facial expressionDecreased spontaneous movementsPaucity of expressive gesturesPoor eye contactAffective non-responsivityLack of vocal inflections

Alogia Poverty of speechPoverty of content of speechBlockingIncreased latency of response

Avolition-Apathy Grooming and hygieneImpersistence at work or schoolPhysical anergia

Anhedonia-Asociality Recreational interests and activitiesSexual interest and activityAbility to feel intimacy and closenessRelationships with friends and peers

Attention/Other Social InattentivenessInattentiveness during mentalstate testing

a BPRS-WR items.

4. Review of studies comparing the intercorrelation of thethree scales

4.1. Correlation coefficients

Correlation coefficients (r) are a way of summarising the inter-relationship between instruments by calculating the correlationbetween two scale total scores. Redundancy analysis is also used forstudying the relationship between scales, by maximising theexplained variance (or redundancy) between them (Welham et al.,1999; Czobor et al., 1991). We performed an electronic search for allpublished studies including at least two of the terms ‘SAPS’, ‘SANS’,‘PANSS’, and ‘BPRS’ in the title for the MEDLINE, Embase, CINAHL,and PsycINFO databases. A similar search was performed for theunabbreviated names of these scales. All studies which reporteda correlation figure between two or more of these scales is reportedhere (Table 4). The references of each article selected were alsoreviewed in search of further studies.

4.2. SAPS/SANS and PANSS

Norman et al. examined the inter-correlations between totalscores on SAPS/SANS and PANSS correlating: (i) Average score ofthe same rater for both scales, and (ii) Average score using differentraters for each instrument (Norman et al., 1996). Using the ratingsfor each tool by the same rater, high and significant correlationbetween the SAPS global scores and PANSS positive scale (0.91,N¼ 85) were found, while correlations across different raters werealso highly correlated (0.81). For negative symptoms, significantcorrelation was found between SANS global scores and PANSSnegative scale when average scores by the same rater were used(0.88), and only moderate correlation was found when measuredby different raters (0.58). The interrater reliabilities for itemsassessing positive symptoms tended to be higher than that fornegative symptoms, and the greater number of items in the SAPS/SANS were noted to provide more detailed assessment of specificsymptoms where necessary.

Peralta et al. also found high correlation for the SAPS and PANSSpositive scale (0.70), as well as for SANS and PANSS negative scale(0.80; N¼ 100), however the negative symptom scale correlationwas higher than that for the positive symptom scales (Peralta et al.,

PANSS-NS BPRS

Blunted affect Blunted affecta

Lack of spontaneity and flow ofconverstation

Passive/apathetic social withdrawal Self-neglect

Emotional withdrawalDifficulty in abstract thinkingStereotyped thinkingPoor rapport

Emotional withdrawala

Motor retardationa

Table 3Comparison of the PANSS general scale items and other BPRS items.

PANSS e general scale items BPRS e other items

Somatic concern Somatic concernAnxiety AnxietyGuilt feelings GuiltTension TensionDepression DepressionUncooperativeness UncooperativenessDisorientation DisorientationPoor attention DistractibilityMannerisms and posturinga SuicidalityUnusual thought contentb Elevated moodMotor retardationc Motor hyperactivityLack of judgement and insightDisturbance of volitionPoor impulse controlPreoccupationActive social avoidance

a Also contained in BPRS scale, presented in Table 1.b Also contained in BPRS-PS, presented in Table 1.c Also contained in BPRS-WR, presented in Table 2.

J.P. Lyne et al. / Journal of Psychiatric Research 46 (2012) 233e238236

1995). Fenton and McGlashan also found high correlation betweenseveral measures of negative symptoms, including PANSS negativescale and the SANS (0.82; N¼ 187) (Fenton &McGlashan, 1992), butcorrelation between just the SANS and PANSS negative scale wasnot reported.

4.3. SAPS/SANS and BPRS

Thiemann et al. found the BPRS-WR to correlate well with theSANS (0.7; N¼ 35) (Thiemann et al., 1987), and suggested that dueto considerable intra-set redundancy in the SANS, negative symp-toms may be more efficiently assessed by shorter scales. Czoboret al also found high intercorrelation between the SANS and BPRSanergia subscale (contains the items for the BPRS-WR, as well as thedisorientation item), however using redundancy analysis, foundthat SANS individual items contained information independent ofthat given by the BPRS anergia subscale (Czobor et al., 1991). Guret al. also found high summary score correlations between the BPRSand SAPS/SANS (0.87 for positive scales, and 0.95 for negativescales; N¼ 47) (Gur et al., 1991).

4.4. PANSS and BPRS

Bell et al. showed that PANSS and BPRS syndrome scale scoreswere highly correlated (0.82 for negative scales, 0.92 for positivescales, and 0.84 for scale totals;N¼ 154) (Bell et al., 1992), while thegeneral scales were only moderately correlated (0.61). Ten of thetwelve items of the PANSS not measured on the BPRS, had low zeroorder correlations with BPRS items, indicating that they provide

Table 4Range of correlations coefficients (r) between scales in published studies.

Scales compared Range of correlationcoefficients, r

SAPS/PANSS-PS 0.70e0.91SAPS/BPRS-PS 0.87PANSS-PS/BPRS-PS 0.92SANS/PANSS-NS 0.80e0.88a

SANS/BPRS-WR 0.70e0.95PANSS-NS/BPRS-WR 0.82e0.85PANSS/BPRS scale totals 0.84PANSS/BPRS general scales 0.61

a Included correlation for same raters by Norman et al., but excluded correlationbetween different raters (r¼ 0.58) (Norman et al., 1996).

more detailed assessment by measuring symptoms not covered inthe BPRS.

4.5. SANS, PANSS-NS, and BPRS-WR

Welham et al. found the negative symptom subscale of thePANSS, the negative symptom subscale of the BPRS, and the SANSwere all highly correlated (SANS and PANSS-NS had correlation of0.86, SANS and BPRS-WR had correlation of 0.72, PANSS-NS andBPRS-WR had correlation of 0.85; N¼ 47) and highly redundant(Welham et al., 1999). However, individual items and subscalescores were not found to be redundant, with the SANS containingadditional information to the BPRS and, to a lesser extent, thePANSS-NS. All three scales appeared to measure a single general‘affective’ component (core negative dimension) of the negativesyndrome, while the PANSS and SANS also measured additionalcomponents, identifying further cognitive, anergic and socialdimensions.

5. Discussion

If you were to combine the items or factors of the SAPS/SANS,PANSS, and BPRS in a research study, ideally items would be clin-ically similar, and correlate well with each other. Our review foundpositive and negative subscales of these tools to be highly corre-lated, but there were substantial clinical differences betweenindividual items on the three scales, and the items and subscaleswere not redundant, with additional information measured forexample in the SANS than the BPRS-WR. These differences suggestthat combining these scales for collaborative research is notstraightforward, and in the discussion we will examine potentialapproaches to this challenge.

5.1. Comparing scale scores with outcomes

One method which has been utilised for combining scales, iscomparing scale scores with an external clinical outcome, forexample as measured by the Clinical Global Impression (CGI).Leucht et al. corresponded percentage and absolute scaleimprovement, on the BPRS and PANSS, with clinical globalimpression outcomes (Leucht et al., 2005b; Leucht et al., 2005a;Leucht et al., 2006), a method which was subsequently used toevaluate the clinical relevance of antipsychotic medication forstudies incorporating the BPRS and PANSS (Lepping et al., 2011).This provides a useful method for collaborative research, howevera limitation includes the difficulty equating outcomes withpercentage change in scores (Mortimer, 2007).

5.2. Recoding global scores as categories of severity

Given the high correlation of global scores between scales,a potential approach for combining scales would be to createa bridging system by recoding global scores for each scale. Forexample in the SANS which has a range of scores of 0e25, a score of0 could recode as absent (bridged scale score of 0), a score of 1e8recode as mild (bridged scale score of 1), a score of 9e17 recode asmoderate (bridged scale score of 2), and a score of 18e25 recode assevere (bridged scale score of 3). For the PANSS-NS a score of 7could recode as absent (0), a score of 8e21 recode as mild (1),a score of 22e35 recode as moderate (2), and a score of 36e49recode as severe (3). Similar scoring systems would need to becreated for the 3 scales, and this approach would require validation,possibly through using established databases, to ensure that thenew bridged scores correlate well with scores on each of the scales.

Table 5Remission criteria items for schizophrenia.

Dimension of Psychopathology DSM IV criterion SAPS/SANS item PANSS item BPRS item

Reality Distortion/Psychoticism Delusions Delusions DelusionsUnusual thought content

GrandiositySuspiciousnessUnusual though content

Hallucinations Hallucinations Hallucinatory behaviour Hallucinatory behaviourDisorganisation Disorganised speech Positive formal thought disorder Conceptual disorganisation Conceptual disorganisation

Grossly disorganised orcatatonic behaviour

Bizarre behaviour Mannerisms/Posturing Mannerisms/Posturing

Negative symptoms/Psychomotor poverty

Negative symptoms Affective flattening Blunted affect Blunted affect

Avolition-apathyAnhedonia-asociality

Passive/Apathetic Socialwithdrawal

No clearly related symptom

Alogia Lack of spontaneity andflow of converstation

No clearly related symptom

J.P. Lyne et al. / Journal of Psychiatric Research 46 (2012) 233e238 237

5.3. Remission criteria

Perhaps themost promising approach to combining scales is useof absolute threshold remission criteria to create a binary outcomevariable of remission/not in remission across the three scales. TheRemission in Schizophrenia Working Group included the SAPS/SANS, PANSS and BPRS in their remission criteria in schizophrenia,where parallel cross-scale remission criteria items for the threescales were proposed (Andreasen et al., 2005). For the SAPS/SANSa score of two or less on eight selected items for a period of at leastsix months is deemed as remission, while for the PANSS a score ofthree or less on eight selected items for a period of at least sixmonths is defined as remission. Seven items were selected from theBPRS, and a score of three or less on each of these items for sixmonths is necessary for remission (Table 5). It should be noted thatthese remission criteria were developed for remission in schizo-phrenia, and the scores should be for a minimum period of sixmonths, which is information not usually included whenmeasuring the SAP/SANS, PANSS, and BPRS (Leucht et al., 2007). Fordatabases where monthly measurements are not available, adap-tation of the criteria for how frequently patients require assessmentwould be necessary.

Somemedication trial studies have already shown correlation ofthe remission criteria with established measures of symptomseverity, functioning, and quality of life (Sethuraman et al., 2005;Lasser et al., 2005; Kissling et al., 2005; Nasrallah & Lasser, 2006;Gharabawi et al., 2005; Docherty et al., 2007; Dunayevich et al.,2006; van et al., 2006). Using remission criteria in the same studywith different scales would require validation by comparingremission status criteria across the three scales in the same patientpopulation. Ideally such a study would also include validation ofthe scales with other important outcome measures, such as func-tioning and quality of life. This simple approach may offer the bestoption for combining research with different scales to a singleresearch project, and to our knowledge no previous studies haveused this method for combining the SAPS/SANS, PANSS and BPRSfor collaboration purposes.

6. Conclusion

Combining the SAPS/SANS, PANSS, and BPRS scales couldprovide large patient samples in collaborative research to answerimportant clinical questions. There is high correlation among bothpositive and negative scale totals across the SAPS/SANS, PANSS, andBPRS, however items and subscales may not be interchangeable.Given the high correlation across scale totals, it may be possible tocombine the three scales for collaborative research projects, forexample by comparing scale scores with outcomes or creating

a bridging system across these scales. Further studies to validate thepotential methods discussed are needed and, based on currentevidence, use of recognised remission criteria appears to offer themost promise for collaborative studies with different symptomscales.

Conflict of interestThe authors declare they have no conflict of interest.

Role of Funding Source

None.

Acknowledgement

The authors wish to acknowledge the help of Ms. Daria Brennanfor her contribution to thewriting of this article. The authors wouldalso like to acknowledge the assistance of Dr. Mary Clarke andDr. Sharon Foley for their contribution to ideas in this article.

References

Alphs L. An industry perspective on the NIMH consensus statement on negativesymptoms. Schizophr Bull 2006;32:225e30.

Andreasen NC. Negative symptoms in schizophrenia. Definition and reliability.Arch Gen Psychiatry 1982;39:784e8.

Andreasen NC. The Scale for the Assessment of Negative Symptoms (SANS). IowaCity, Iowa: University of Iowa; 1984.

Andreasen NC. The Scale for the Assessment of Negative Symptoms (SANS):conceptual and theoretical foundations. Br J Psychiatry Suppl; 1989:49e58.

Andreasen NC. Methods for assessing positive and negative symptoms. Mod ProblPharmacopsychiatry 1990a;24:73e88.

Andreasen NC. Positive and negative symptoms: historical and conceptual aspects.Mod Probl Pharmacopsychiatry 1990b;24:1e42.

Andreasen NC, Carpenter Jr WT, Kane JM, Lasser RA, Marder SR, Weinberger DR.Remission in schizophrenia: proposed criteria and rationale for consensus.Am J Psychiatry 2005;162:441e9.

Andreasen NC, Flaum M. Schizophrenia: the characteristic symptoms. SchizophrBull 1991;17:27e49.

Bell M, Milstein R, Beam-Goulet J, Lysaker P, Cicchetti D. The positive and negativesyndrome scale and the brief psychiatric rating scale. Reliability, comparability,and predictive validity. J Nerv Ment Dis 1992;180:723e8.

Czobor P, Bitter I, Volavka J. Relationship between the brief psychiatric rating scaleand the scale for the assessment of negative symptoms: a study of theircorrelation and redundancy. Psychiatry Res 1991;36:129e39.

Docherty JP, Bossie CA, Lachaux B, Bouhours P, Zhu Y, Lasser R, Gharabawi GM.Patient-based and clinician-based support for the remission criteria in schizo-phrenia. Int Clin Psychopharmacol 2007;22:51e5.

Dunayevich E, Sethuraman G, Enerson M, Taylor CC, Lin D. Characteristics of twoalternative schizophrenia remission definitions: relationship to clinical andquality of life outcomes. Schizophr Res 2006;86:300e8.

Eckert SL, Diamond PM, Miller AL, Velligan DI, Funderburg LG, True JE.A comparison of instrument sensitivity to negative symptom change. PsychiatryRes 1996;63:67e75.

Fenton WS, McGlashan TH. Testing systems for assessment of negative symptoms inschizophrenia. Arch Gen Psychiatry 1992;49:179e84.

J.P. Lyne et al. / Journal of Psychiatric Research 46 (2012) 233e238238

van der Gaag M, Cuijpers A, Hoffman T, Remijsen M, Hijman R, de HL, van MB, vanHarten PN, Valmaggia L, de HM, Wiersma D. The five-factor model of thePositive and Negative Syndrome Scale I: confirmatory factor analysis fails toconfirm 25 published five-factor solutions. Schizophr Res 2006a;85:273e9.

van der Gaag M, Hoffman T, Remijsen M, Hijman R, de HL, van MB, van Harten PN,Valmaggia L, de HM, Cuijpers A, Wiersma D. The five-factor model of thePositive and Negative Syndrome Scale II: a ten-fold cross-validation of a revisedmodel. Schizophr Res 2006b;85:280e7.

Gharabawi G, Rodriguez S, Turkoz I. Exploring the evidence for a relationshipbetween symptomatic remission and functioning in patients with schizo-phrenia. Neuropsychopharmacology 2005;30:s1e270.

Gur RE, Mozley PD, Resnick SM, Levick S, Erwin R, Saykin AJ, Gur RC. Relationsamong clinical scales in schizophrenia. Am J Psychiatry 1991;148:472e8.

Harris MG, Henry LP, Harrigan SM, Purcell R, Schwartz OS, Farrelly SE, Prosser AL,Jackson HJ, McGorry PD. The relationship between duration of untreatedpsychosis and outcome: an eight-year prospective study. Schizophr Res 2005;79:85e93.

Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) forschizophrenia. Schizophr Bull 1987;13:261e76.

Kirkpatrick B, Fenton WS, Carpenter Jr WT, Marder SR. The NIMH-MATRICSconsensus statement on negative symptoms. Schizophr Bull 2006;32:214e9.

Kissling W, Heres S, Lloyd K, Sacchetti E, Bouhours P, Medori R, Llorca PM. Directtransition to long-acting risperidoneeanalysis of long-term efficacy.J Psychopharmacol 2005;19:15e21.

Lasser RA, Bossie CA, Gharabawi GM, Kane JM. Remission in schizophrenia: resultsfrom a 1-year study of long-acting risperidone injection. Schizophr Res 2005;77:215e27.

Lepping P, Sambhi RS, Whittington R, Lane S, Poole R. Clinical relevance of findingsin trials of antipsychotics: systematic review. Br J Psychiatry 2011;198:341e5.

Leucht S, Beitinger R, Kissling W. On the concept of remission in schizophrenia.Psychopharmacology (Berl) 2007;194:453e61.

Leucht S, Kane JM, Etschel E, Kissling W, Hamann J, Engel RR. Linking the PANSS,BPRS, and CGI: clinical implications. Neuropsychopharmacology 2006;31:2318e25.

Leucht S, Kane JM, Kissling W, Hamann J, Etschel E, Engel R. Clinical implications ofBrief Psychiatric Rating Scale scores. Br J Psychiatry 2005a;187:366e71.

Leucht S, Kane JM, Kissling W, Hamann J, Etschel E, Engel RR. What does the PANSSmean? Schizophr Res 2005b;79:231e8.

Lukoff D, Nuecheterlein KH, Ventura J. Manual for the expanded brief psychiatricrating scale. Schizophr Bull 1986;12:594e602.

Miller DD, Arndt S, Andreasen NC. Alogia, attentional impairment, and inappro-priate affect: their status in the dimensions of schizophrenia. Compr Psychiatry1993;34:221e6.

Mortimer AM. Symptom rating scales and outcome in schizophrenia. Br J PsychiatrySuppl 2007;50:s7e14.

Moscarelli M, Maffei C, Cesana BM, Boato P, Farma T, Grilli A, Lingiardi V,Cazzullo CL. An international perspective on assessment of negative and posi-tive symptoms in schizophrenia. Am J Psychiatry 1987;144:1595e8.

Nasrallah HA, Lasser R. Improving patient outcomes in schizophrenia: achievingremission. J Psychopharmacol 2006;20:57e61.

Norman RM, Malla AK, Cortese L, Diaz F. A study of the interrelationship betweenand comparative interrater reliability of the SAPS, SANS and PANSS. SchizophrRes 1996;19:73e85.

Overall JE, Gorham DR. Brief psychiatric rating scale. Psychol Rep 1962;10:799e812.Overall JE, Klett CJ. Applied multivariate analysis. New York: McGraw Hill; 1972.Palacios-Araus L, Herran A, Sandoya M, Gonzalez de la HE, Vazquez-Barquero JL, ez-

Manrique JF. Analysis of positive and negative symptoms in schizophrenia.A study from a population of long-term outpatients. Acta Psychiatr Scand 1995;92:178e82.

Peralta V, Cuesta MJ, de LJ. Positive and negative symptoms/syndromes in schizo-phrenia: reliability and validity of different diagnostic systems. Psychol Med1995;25:43e50.

Roncone R, Ventura J, Impallomeni M, Falloon IR, Morosini PL, Chiaravalle E,Casacchia M. Reliability of an Italian standardized and expanded Brief Psychi-atric Rating Scale (BPRS 4.0) in raters with high vs. low clinical experience. ActaPsychiatr Scand 1999;100:229e36.

Sethuraman G, Taylor CC, Enerson M, Dunayevich E. A retrospective comparison ofcumulative time spent in remission during treatment with olanzapine or ris-peridone among patients with schizophrenia. Schizophr Res 2005;79:337e40.

Singh MM, Kay SR. A comparative study of haloperidol and chlorpromazine interms of clinical effects and therapeutic reversal with benztropine in schizo-phrenia. Theoretical implications for potency differences among neuroleptics.Psychopharmacologia 1975;43:103e13.

Tandon R, Nasrallah HA, Keshavan MS. Schizophrenia, “just the facts” 5. Treatmentand prevention. Past, present, and future. Schizophr Res 2010;122:1e23.

Thiemann S, Csernansky JG, Berger PA. Rating scales in research: the case ofnegative symptoms. Psychiatry Res 1987;20:47e55.

Toomey R, Faraone SV, Simpson JC, Tsuang MT. Negative, positive, and disorganizedsymptom dimensions in schizophrenia, major depression, and bipolar disorder.J Nerv Ment Dis 1998;186:470e6.

van OJ, Drukker M, Campo J, Meijer J, Bak M, Delespaul P. Validation of remissioncriteria for schizophrenia. Am J Psychiatry 2006;163:2000e2.

Ventura J, Green M, Shaner A, Liberman RP. Training and quality assurance on theBPRS: ‘The Drift Busters’. International Journal of Methods in PsychiatricResearch 1993;3:221e44.

Ventura J, Nuechterlein KH, Subotnik KL, Gutkind D, Gilbert EA. Symptom dimensionsin recent-onset schizophrenia and mania: a principal components analysis of the24-item Brief Psychiatric Rating Scale. Psychiatry Res 2000;97:129e35.

Welham J, Stedman T, Clair A. Choosing negative symptom instruments: issues ofrepresentation and redundancy. Psychiatry Res 1999;87:47e56.