E L S E V I E R Schizophrenia Research 19 (1996) 213-219

SCHIZOPHRENIA RESEARCH

Validity of self-administered symptom scales in clients with schizophrenia and schizoaffective disorders

Edna K. Hamera a,,, Joanne Kraenzle Schneider b, Miriam Potocky c, Mary Ann Casebeer d

a School o f Nursing, University of Kansas Medical Center, Kansas City, Kansas, USA b School o f Medicine, Washington University, St. Louis, MissourL USA

¢ Social Work Department, Florida International University, North MiamL Florida, USA a Independent Practitioner, Coffeyville, Kansas, USA

Received 15 August 1995; revision 6 October 1995; accepted 21 October 1995

Abstract

Few studies have used self-administered symptom scales as outcome measures with individuals who have schizophrenia. However, with the increase in community-based treatment for the serious and persistently mentally ill and the emphasis on client empowerment, their ability to monitor and report symptoms needs to be assessed. Two forms of self-administered items, symptom distress statements and symptom intensity statements, were formed from 10 BPRS symptoms and administered to 29 individuals with schizophrenia. Both forms of self-administered items were highly correlated with BPRS items, supporting concurrent validity. Self-administered responses for positive symptoms of schizophrenia and nonpsychotic symptoms may be more valid than self-administered responses for negative or deficit symptoms.

Keywords: Self-administered symptom scale; Schizophrenia; Validity

1. Introduction

Subjective symptom monitoring has become more feasible with advances in neuroleptic drugs and more desirable with community-based treat- ment. However, self-administered scales rarely have been developed and used to measure thera- peutic response in individuals with schizophrenia. In a recent survey of 79 drug trials published from 1986 to 1989, Collins et al. (1991) reported that almost all of the trials included measures of symp-

* Corresponding author.

0920-9964/96/$15.00 © 1996 Elsevier Science B.V. All rights reserved SSDI 0920-9964 (95) 00100-X

tomatology, but only five assessed symptoms by using client self-administered scales. Although self-administered scales are often viewed with scepticism, others (Strupp and Hadley, 1977) sug- gest that clients perception of symptoms are impor- tant in their own right. The present study is predicated upon this view.

There are numerous reasons clients with schizo- phrenia may not accurately report their symptom experience. Denial, shame, or trust about how the person collecting the information will use it are some of the major factors contributing to inaccu- rate symptom reporting. Inaccuracy may arise from cultural and environmental demands. Prior

214 Edna K. Hamera et al./Schizophrenia Research 19 (1996) 213-219

to the community mental health movement, and more recent mental health reform, individuals with schizophrenia often concealed or under-reported symptoms in order to be released from the hospital. In the present mental health system, the opposite may be true. Clients sometimes report exaggerating symptoms to convince mental health professionals that they need hospitalization. The fact that symp- tom measures may be contingent on the goals that individuals with schizophrenia want to achieve attest to the need to use diverse methods of assess- ing symptomatology.

Another source of inaccuracy is information processing deficits that may impair ability to report symptom experience. Deficits can include problems with attention, concentration, memory, abstrac- tion, and concept formation. However, these problems are not unique to individuals with schizo- phrenia and are not as prominent during residual phases of the illness.

Few investigators have developed new measures or evaluated the accuracy of standardized self- administered symptom measures with individuals who have schizophrenia. The Symptom Check- list 90 (SCL-90) and Minnesota Multiphasic Personality Inventory (MMPI) are the most common standardized self-administered scales given to individuals with schizophrenia.

Using the self-administered MMPI and the Brief Psychiatric Rating Scale (BPRS), Faustman et al. (1989) found modest (0.26 to 0.50) but significant correlations between inpatients' and clinicians' rat- ings of inpatients on positive BPRS symptom ratings. Turner et al. (1983) compared agreement among clients', case managers', and independent evaluators' ratings of clients' symptoms using the SCL-90. Case managers' ratings of clients' symp- toms correlated more highly with clients' self- ratings (0.87) than independent evaluators' ratings of clients (0.67) but both showed good agreement. Blanchard et al. (1992) reported that self-ratings of negative mood using the SCL-90 and clinicians' ratings using the Raskin Depression Scale and BPRS did not correlate initially but did at a six month follow-up. The investigators speculated that the initial disparity between self and interviewer ratings might be due to higher thought disorder

ratings in clients initially as compared to six months later.

Several investigators have developed and eval- uated the concurrent validity of self-administered scales specifically designed for individuals with schizophrenia. Hogan et al. (1983) developed a self-report scale to measure attitudes and subjective experiences of neuroleptic treatment. In a subse- quent study (Hogan and Awad, 1992), the scale was given to 55 individuals with schizophrenia along with a semi-structured interview (Van Putten and May, 1978) designed to elicit subjective experi- ences of medication. Responses from the self- administered scale and codings from the semi- structured interview were correlated 0.76 at 24 h and 0.74 at 48 h after medication was initiated. In a larger sample (N=96) of community mental health center outpatients with a diagnosis of schizophrenia, Michaels and Mumford (1989) found that nurses' ratings of akathisia were moder- ately correlated (0.67) with items measuring akath- isia on a 40-item self-report scale of feelings associated with taking antipsychotic medications. Agreement between nurses' ratings and self-ratings of akinesia were lower (0.29). The results of these studies indicate that both observation and self- report are important in detecting side effects.

The accuracy of client self-report was measured indirectly in a study of early warning symptoms (Birchwood et al., 1989). The Early Signs Scale was completed by 19 outpatients with schizophre- nia and their significant other every two weeks for nine months. The correlation between clients' and significant others' ratings was 0.81 at the onset of the study but had dropped to 0.31 at the end of nine months due in part to the attrition of nine clients who relapsed.

In a study (Ernst and Vingiano, 1989) of a broader spectrum of clinical symptoms, investiga- tors constructed a 29-item graphic and 29-item verbal self-rating scale derived from the BPRS and Research Diagnostic Criteria. The two scales were administered in counterbalanced order to 73 psy- chiatric inpatients, 24 of whom had schizophrenia or schizoaffective disorders. All but two of the graphic items were significantly correlated with corresponding items from the verbal scale. The graphic scales also appeared to be sensitive to

Edna K Hamera et al./Schizophrenia Research 19 (1996) 213-219 215

changes over time and ratings corresponded to expected differences in diagnostic groups. Unfortunately, the investigators did not compare the graphic scale with a measure that has known validity and reliability. However, with further development the graphic scale could be useful for clients who are illiterate or with clients who have significant information processing deficits.

The studies reviewed have some limitations for clinical and research purposes. Standardized self- administered questionnaires show relatively good agreement with clinicians ratings and add another dimension to outcome measures. However, the standardized symptom questionnaires are too long for frequent assessment. Questionnaires specifically designed for individuals with schizophrenia often focus on a subset of the symptoms experienced by individuals with schizophrenia. A short self-admin- istered questionnaire measuring a broader spectrum of clinical symptoms would be useful clinically and as an outcome measure.

The purpose of the present study was to develop a short self-administered symptom questionnaire for clients with schizophrenia. Two kinds of self- administered items were evaluated, ones that focused on symptom intensity and ones that focused on symptom distress. In addition, the two kinds of self-administered items were compared to interviewer ratings on the Brief Psychiatric Rating Scale (BPRS). The specific research questions were: (1) What is the relationship between client self-report of symptoms using self-administered questionnaires and interviewer ratings of the client on the same symptoms using the BPRS? and (2) How does the wording of self-administered items affect the relationship between client self-report of symptoms and interviewer ratings of client symp- toms using the BPRS?

2. Method

2.1. Subjects

The subjects (N=29) were recruited from a community support program of a county mental health center. The county has a per capita income of $23 346 and a 6% minority population. The

center serves 453 clients; approximately 50% have a diagnosis of schizophrenia or schizoaffective disorder. To protect confidentiality, the 76 clients with clinical diagnoses of schizophrenia or schizo- affective disorders were initially invited to partici- pate in the study by their case managers. Thirty- seven (48.7%) agreed to participate. Clinical diag- noses of schizophrenia or schizoaffective disorders were confirmed using the Structured Clinical Interview for the DSM-III-R (SCID) (Spitzer et al., 1990) in 29 of the potential subjects who formed the sample for the study.

Of the 29 subjects participating in the study, 12 (41.4%) were females and 17 (58.6%) males. Twenty-four (82.8%) had SCID diagnoses of schizophrenia and five (17.2%) had SCID diagno- ses of schizoaffective disorder. Mean age was 38 years and ages ranged from 22 to 65 years. Twenty- six (89.7%) subjects were White and two (6.9%) were Black. Subjects reported a mean duration of emotional or psychiatric problems to be 16 years (SD = 10.6). The majority (53.6%) reported five or more previous hospitalizations while 46.4% reported four or less hospitalizations. All subjects reported having at least a high school education. Two (6.9%) reported post high school education other than college, 13 (44.8%) reported having some college education, four (13.8%) had a bache- lor's degree, and one (3.4%) reported having edu- cation beyond a bachelor's degree. Eleven (37.9%) reported being employed part time, three (10.3%) were employed full time. The remaining subjects (51.7%) were unemployed. Mean monthly income was $628 with a range from $265 to $2200.

2.2. Measures and procedures

Ten symptoms that are frequently experienced by community-based individuals with schizophre- nia were selected from the expanded version of the BPRS (Lukoff et al., 1986). Four were positive symptoms, i.e. conceptual disorganization, suspi- ciousness, hallucinations, and unusual thought content; two were negative symptoms, i.e. emo- tional withdrawal and motor retardation. The remaining four symptoms were commonly experi- enced by individuals with schizophrenia, i.e. depression, anxiety, guilt, and hostility. Two client

216 Edna K. Hamera et al./Schizophrenia Research 19 (1996) 213-219

self-administered symptom questionnaires were developed from the 10 BPRS symptoms. For each symptom, at least one and sometimes two items reflecting symptom intensity, and one and some- times two items reflecting symptom distress were generated for self-administered questionnaires. The 14-item symptom intensity questionnaire focused on the severity of the symptoms. Subjects were asked to respond on a five-point scale, from 'none' to an 'extreme amount'. The 12-item symptom distress questionnaire focused on how disturbing subjects found the symptom. Subjects responded on a five-point scale, from 'not distressing at all' or 'did not have symptom' to 'extremely distressing'.

The content validity of the items was evaluated by having two masters-prepared nurses indepen- dently match the self-administered items with their corresponding BPRS symptoms. Both nurses matched the self-administered items to BPRS symptoms with 100% accuracy. Sample items from the BPRS and the self-administered questionnaires measuring the symptom suspiciousness are shown in Appendix A.

After obtaining informed consent the two self- administered questionnaires along with the expanded BPRS (Lukoff et al., 1986) were admin- istered in an alternating order. Subjects were asked to respond to all three questionnaires based on their experiences in the last 24 h. In addition, demographic and treatment data were collected. Subjects were paid $10.00 for their participation.

Inter-rater reliability among investigators administering the BPRS was performed by calcu- lating an intraclass correlation (Lahey et al., 1983; Barko, 1991). One investigator conducted the interview while another observed with 11 (37.9%) of the subjects. Each completed the BPRS indepen- dently. The first author was always one of the raters. The intraclass correlation was 0.78 (p<0.01).

3. Results

The 10 BPRS items that corresponded to the 12 symptom distress items and 14 symptom intensity items of the self-administered questionnaires were

analyzed. Internal consistency of each question- naire was estimated using Cronbach's alpha. All three questionnaires produced adequate alpha coefficients, 0.81 for the BPRS, 0.78 for the symp- tom intensity, and 0.77 for the symptom distress. Mean, standard deviation, and range of scores for each questionnaire are listed in Table 1. Mean scores are low in relation to the possible ranges of all three questionnaires, showing that subjects did not have severe symptomatology. This is not sur- prising since the time frame was brief (the previous day) and the subjects were in residual stages of illness and living in the community.

To answer the first research question, the corre- lations between the total scores of the three ques- tionnaires were calculated using Spearman Rank Order correlation coefficients. Scores among the three questionnaires were significantly correlated. The correlation of the BPRS with the symptom distress questionnaire was 0.81 (p<0.01) and between the BPRS and the symptom intensity questionnaire 0.66 (p < 0.01 ). Finally, the correla- tion between symptom distress and intensity was 0.77 (p<0.01).

The second research question was answered by correlating the BPRS symptoms with their corre- sponding symptom distress and symptom intensity items. Correlations and significance levels are reported in Table 2. Neither the distress nor inten- sity items were significantly correlated with the corresponding BPRS items for the two negative symptoms, motor retardation and emotional with- drawal. Three of the four intensity items measuring positive symptoms, i.e. suspiciousness, hallucina- tions, and unusual thought content, were signifi- cantly correlated with the BPRS while only two of the positive distress items were significantly correlated. Three of the nonpsychotic distress items

Table 1 Score means and dispersion for BPRS, symptom intensity, and symptom distress questionnaires

Questionnaire Mean SD Minimum Maximum Possible range

BPRS 19.69 8.31 10.0 37.0 10-70 Distress q. 21.31 8.32 12.0 47.0 12-60 Intensity q. 22.03 7.28 14.0 39.0 14-70

Edna K. Hamera et al./Schizophrenia Research 19 (1996) 213-219 217

Table 2 Correlations of BPRS items and corresponding items on the symptom distress and symptom intensity questionnaires ( N = 29)

BPRS items Distress items Intensity items

Anxiety 0.43 a Depression 0.59 b Guilt 0.14 Hostility 0.41 a Suspiciousness 0.27 Hallucinations 0.73 b Conceptual 0.28

disorganization Unusual thought 0.47 b

content Emotional withdrawal 0.11 Motor retardation - 0.02

- 0 . 0 5

0.52 b

0.04 0.21 0.21 0.24 0.37 a 0.79 b

- 0 . 1 8

0 . 1 4

0.34

0.52 b

0.67 b 0.67 b

- 0 . 2 4

- 0.003

Note: Some BPRS items have two symptom distress items (i.e., conceptual disorganization and unusual thought content) and some have two symptom intensity items (i.e., anxiety, depres- sion, hostility and unusual thought content). ap<0.05. bp<O.Ol.

were significantly related to corresponding BPRS items while only one, hostility, was significantly correlated with the BPRS when presented in the intensity format.

4. Discussion

The results of the study suggest that a self- administered questionnaire is a valid means of assessing positive symptoms of schizophrenia and general nonpsychotic symptoms in community- based clients with schizophrenia who are in a residual state. Self-report of negative or deficit symptoms appears less valid. Negative symptoms by their nature may be difficult to self-rate. They are more behavioral than experiential and on the BPRS interviewers use their observation to rate clients rather than base their ratings on client responses to interview questions.

There was a difference in the two self-adminis- tered questionnaires evaluated. For nonpsychotic symptoms more of the symptom distress items were significantly correlated with corresponding BPRS items than for the symptom intensity items.

However, these item-to-item correlations should be viewed with caution. The lack of variability on any of the three questionnaires for these items may have caused the low correlations. This was particu- larly true for emotional withdrawal, motor retard- ation, guilt and conceptual disorganization. The low variability on the items is likely due to the short time frame for ratings, i.e. one day, and the fact that the population was community-based and probably in a residual phase of illness.

There are some limitations with the standardized scale, the BPRS, used to establish validity for the self-administered questionnaires. Like other inter- viewer rating scales, the BPRS can have low inter- rater reliability. The version of the BPRS (Lukoff et al., 1986) chosen for this study had a semi- structured interview and anchor points to provide consistency in administration and rating. This ver- sion of the BPRS also was chosen because it is applicable to an outpatient population. In future research predictive validity for the self-adminis- tered questionnaires needs to be assessed by com- paring ratings with functional outcomes and treatment indices.

The clients for this methodological study pro- vided an ideal sample for initially evaluating the ability of clients to self-report their symptoms. The sample consisted of community-based clients in the residual phases of illness who live in a relatively affluent county. Future research could utilize a cross-sectional design with a variety of samples from diverse settings (i.e., persons in varying phases of the illness) or a longitudinal design.

The time frame for recalling symptoms, i.e. the previous day, provided evidence of the concurrent validity of self-administered symptom question- naires for a recent but brief time period. Future research should assess clients ability to accurately recall their symptoms over longer time periods. Significant cognitive deficits in some individuals with schizophrenia may limit the length of the time frame they are able to validly respond to self- administered questionnaires. Thus brief self-rating scales like the ones tested in this study that can be administered more frequently may be most useful.

A limitation of this study was the mediocre response rate of clients who were invited to partici- pate. Although the study was described to each case manager and specific suggestions given for

218 Edna K. Hamera et al./Schizophrenia Research 19 (1996) 213-219

how to approach potential subjects, we had little control over how subjects were actually recruited and no way to compare responders with nonre- sponders. Those who chose not to participate may have more paranoia than participants in this study. Thus the validity of symptom self-administered questionnaires by nonresponders is unknown and should be addressed in future studies.

This study has demonstrated that clients with schizophrenia can validly monitor and report their own symptoms. Such self-monitoring will become more important as more clients are enabled to maintain independent community living for longer periods of time. Symptom self-monitoring over time can be useful for clinical decision making with regard to both medication and case manage- ment, as it provides a measure of treatment response. Finally, self-monitoring is consistent with the goals of promoting and enhancing clients' self-care. It provides clients greater involvement in the therapeutic process.

A c k n o w l e d g m e n t

This research was supported by Grant RO3 MH46650 from NIMH, ADAMHA.

A p p e n d i x A

A. 1.1. B P R S item Suspiciousness: Expressed or apparent belief

that other persons have acted maliciously or with discriminatory intent. Include persecution by supernatural or other nonhuman agencies (e.g., the devil).

1 Not present 2-3 Mild Seems on guard.

Unresponsive to 'personal' questions. Describes inci- dents where other persons have harmed or wanted to harm him/her that sound plausible. Patient feels as if others are laughing at or crit- icizing him/her in public.

4 5 Moderate Says other persons are talk- ing about him/her mali- ciously or says others intend to harm him/her. Beyond likelihood of plausibility but not delusional.

6-7 Severe Delusional. Speaks of Mafia plots, the FBI, or others poi- soning food.

NA Not assessed.

Do you ever feel uncomfortable as if people are watching you?Is anyone trying to harm or interfere with you in any way?Are you concerned about anybody's intentions toward you?Have you felt that any people are out to get you?

A.1.2. Symptom intensity self-administered item How much did you think that others were

watching you or paying particular attention to you yesterday?

- None - A little - Some - A lot

An extreme amount

A. 1.3. Symptom distress self-administered item How distressing was it for you to feel paranoid

yesterday? - Not distressing at all - A little distressing

Somewhat distressing - A lot distressing - Extremely distressing - Did not have symptom.

R e f e r e n c e s

Barko, J.J. (1991) Measurement and reliability: Statistical thinking considerations. Schizophr. Bull. 17, 483-489.

Birchwood, M., Smith, J., MacMillan, F., Hogg, B., Prasad, R., Harvey, C. and Bering, S. (1989) Predicting relapse in schizophrenia: the development and implementation of an early signs monitoring system using patients and families as observers, a preliminary investigation. Psychol. Med. 19, 649 656.

Blanchard, J.J., Mueser, K.T. and Bellack, A.S. (1992) Self- and Interview-rated negative mood states in schizophrenia:

Edna K. Hamera et al./Schizophrenia Research 19 (1996) 213 219 219

their convergence and prediction of thought disturbance. J. Psychopath. Behav. Assess. 14, 277-291.

Collins, E.J., Hogan, T.P. and Desai, H. (1991) Measurement of therapeutic response in schizophrenia. Schizophr. Res. 5, 249-253.

Ernst, M. and Vingiano, W. (1989) Development of a graphic psychiatric self-rating scale. Comp. Psychiatry 30, 189-194.

Faustman, W.O., Moses, J.A., Csernansky, J.G. and White, P.A. (1989) Correlations between the MMPI and the Brief Psychiatric Rating Scale in schizophrenic and schizoaffective patients. Psychiatry Res. 28, 135-143.

Hogan, T.P. and Awad, A.G. (1992) Subjective response to neuroleptics and outcome in schizophrenia: a re-examination comparing two measures. Psychol. Med. 22, 347-352.

Hogan, T.P., Awad, A.G. and Eastwood, R. (1983) A self- report scale predictive of drug compliance in schizophrenics: reliability and discriminative validity. Psychol. Med. 13, 177-183.

Lahey, M.A., Downey, R.G. and Saal, F.E. (1983). Intraclass correlations: there's more there than meets the eye. Psychol. Bull. 93, 586-595.

Lukoff, D., Neuchterlein, K.H. and Ventura, J. (1986) Symptom monitoring in the rehabilitation of schizophrenic patients. Schizophr. Bull. 12, 578-602.

Michaels, R.A. and Mumford, K. (1989) Identifying akinesia and akathisia: the relationship between patient's self-report and nurses's assessment. Arch. Psychiatr. Nurs. 3, 97-101.

Spitzer, R.L., Williams, J.B.W., Gibbon, M. and First, M.B. (1990) Structured Clinical Interview for DSM-III-R. Patient edition. American Psychiatric Press, Washington, DC.

Strupp, H.H. and Hadley, S.W. (1977) A tripartite model of mental health and therapeutic outcomes: with special refer- ence to negative effects of psychotherapy. Am. Psychol. 32, 187-196.

Turner, R.M., McGovern, M. and Sandrock, D. (1983) A multiple perspective analysis of schizophrenics' symptoms and community functioning. Am. J. Community Psychol. 11,593-607.

Van Putten, T. and May, P.R.A. (1978) Subjective response as a predictor of outcome in pharmacotherapy. Arch. Gen. Psychiatry 38, 477-480.