A Comparison of Clinical and Structured Interview ...cmhsr.wustl.edu/Resources/Documents/A comparison of clinical and...A Comparison of Clinical and Structured Interview ... may account

Community Mental Health Journal, Vol. 33, No. 6, December 1997

A Comparison of Clinical andStructured Interview Diagnoses in a

Homeless Mental Health Clinic

Carol S. North, M.D.David E. Pollio, Ph.D.

Sanna J. Thompson, M.S.W.Daniel A. Ricci, CM., L.C.S.W.

Elizabeth M. Smith, Ph.D.Edward L. Spitznagel, Ph.D.

ABSTRACT: Objective. This study compared psychiatric diagnoses ascertained byindependent clinicians with structured research interviews of homeless psychiatric pa-tients assessed in a mental health clinic and in the community. Problems of both over-diagnosis and underdiagnosis in structured research interviews compared to clinicianassessment were predicted. Method. Over a period of a year, 97 patients referred to amental health clinic for homeless people were assessed with the Diagnostic InterviewSchedule (DIS) administered by a clinical social worker who then completed a fullclinical psychiatric social work assessment. These same patients received a thoroughand systematic clinical psychiatric evaluation by a psychiatrist or psychologist, bothexperienced with this population. These clinicians gathered data from multiple sources,often with extended observation over time. The DIS and clinician diagnoses were madeblind to one another and then compared; the clinician was often made aware of some ofthe symptoms that the social worker had elicited, but not whether the elicited materialwas from the DIS or from the clinical assessment. Diagnoses of 33 clinic patientspreviously assessed by trained nonclinician DIS interviews in an epidemiologic study ofthe homeless population in the community were also compared to clinician diagnoses,

All the authors are affiliated with Washington University, St. Louis, Missouri, except for Dan-iel A. Ricci, who is affiliated with the Grace Hill Neighborhood Health Center, St. Louis, Mis-souri. Dr. Smith died March 7, 1997.

Address correspondence to Carol S. North, M.D., Washington University, School of Medicine,Department of Psychiatry, 4940 Children's Place, St. Louis, MO 63110.

This research was supported by National Institute on Alcohol Abuse and Alcoholism Grant#AA007549 to Dr. Smith.

531 © 1997 Human Sciences Press, Inc.

and no information from these patients' survey DIS interviews was made available tothe clinicians. Results. Compared to clinician assessment, structured interviews under-diagnosed antisocial personality disorder (ASPD) and overdiagnosed major depression.Alcohol use disorder and schizophrenia showed only small discrepancies by assessmentmethod. Drug use disorder revealed no bias according to method of ascertainment, butshowed very discrepant kappa levels comparing DIS to clinician assessment in the twodifferent comparison contexts. Conclusions. If structured research methods assessingthe homeless population actually overestimate depression, underestimate ASPD, andmisclassify drug abuse, then policies stemming from structured interview researchrecommendations may call for levels and types of services not optimally suited to thereality of this population's needs. Because mental illness and substance abuse arethought to be critical factors in the generation and perpetuation of homelessness, theissue of accurate diagnosis is tantamount to understanding and providing workablesolutions to the problem of homelessness. Further research is needed to untangle poten-tial confounders of the homeless situation to psychiatric diagnosis.

INTRODUCTION

The last decade of research into homelessness has benefitted from in-creased attention to systematic study of psychiatric disorders in thispopulation. Random, representative sampling methods and systematicassessment of psychiatric disorders of the homeless have greatly im-proved scientific confidence in the findings. For example, rates ofschizophrenia in this population, previously estimated at 50% to 75%(Torrey, 1986; Lipton, Sabatini, & Katz, 1983) by less sophisticatedmethods, have subsequently been honed to a more realistic 5% to 15%(Bassuk, Rubin, & Lauriat, 1986; Koegel, Burnam, & Farr, 1988;Breakey, et al. 1989; Susser, Struening, & Conover, 1989; Herrman,McGorry, Bennett, van Riel, & Singh, 1989).

Although methodologic improvements have advanced the field sig-nificantly, current emphasis on use of structured instruments has pre-ceded investigation into the validity of such instruments in this specialpopulation. Much of the existing knowledge about mental illness in thehomeless population has derived from studies based on structured in-terviews (Koegel, Burnam, & Farr, 1988; Koegel & Burnam, 1988;Smith, North, & Spitznagel, 1992; Smith, North, & Spitznagel, 1993)such as the Diagnostic Interview Schedule (DIS) (Robins, Helzer,Croughan, Williams, & Spitzer, 1981). Questions have been raisedabout potential lack of sensitivity of the DIS to psychiatric disorderssuch as schizophrenia and depression in homeless populations (Fischer,Shapiro, Breakey, Anthony, & Kramer, 1986; Breakey, et al. 1989;Fischer, 1989).

Drake and colleagues emphasize that valid diagnostic assessment iscritical for study of the homeless population. Both Drake's group

532 Community Mental Health Journal

(Drake, Osher, & Wallach, 1991) and Fischer (1989) have pointed outthat standardized instruments need to be normalized for homelesspopulations. Drake and colleagues (Drake, Osher, & Wallach, 1991)stressed the need to aggregate observations over time and situation,collect information from collateral sources, and modify standard instru-ments for assessment of behavioral dimensions that are relevant tothis population to maximize validity of diagnosis. Susser and Struen-ing (1990) identified a need to validate structured interview assess-ment through longitudinal observation and consideration of informa-tion from multiple sources who are familiar with homeless subjects.

Fischer (1989) observed that prevalence estimates from standard-ized scales overdiagnose psychiatric disorders relative to psychiatricexamination in homeless populations. Although concern has been ex-pressed that antisocial personality disorder (ASPD) may be overdiag-nosed in homeless populations due to confounders of the homeless sit-uation (Koegel & Burnam, 1992), North and colleagues (1993) havedocumented the validity of the diagnosis of ASPD ascertained by DISinterview in this population.

Potentially confounding effects of stressors inherent in homelessnessmay generate difficulties in distinguishing stress-related behaviorfrom psychopathology. For example, distinguishing state-dependentdistress (e.g., odd appearance, deteriorated hygiene, dysphoria and de-moralization) from enduring psychopathology (e.g., psychosis, majordepression) may be very difficult (Susser, Conover, & Struening, 1989).North (North, 1995) reported that homelessness-related situationalfactors such as miserable weather on the day of interview increase thelikelihood of a diagnosis of major depression. Therefore, use of struc-tured interviews and assessment scales devised for other populationsmay account for some portion of overdiagnosis of psychiatric illness inhomeless settings.

Structured instruments may also have problems with underdiag-nosis. Fischer and colleagues (1992) have observed substantial utiliza-tion of psychiatric services among homeless people with no history ofmajor mental illness or substance use disorder. North and coworkers(1994) found that more than half of homeless people reporting a pastsuicide attempt described no history of major depression. Koegel andBurnam (1992) documented an 8.5 times increase in risk for suicideattempts among nondepressed homeless people over a domiciled popu-lation. While these findings might be considered evidence of under-diagnosis of major psychiatric disorders in homeless populations, alter-natively, suicidality and utilization of psychiatric services could also

Carol S. North, M.D., et al. 533

be considered indicators of the magnitude of psychological distress andgeneral misery in this population that does not fit traditional defini-tions of psychiatric disorder.

Further study is needed to compare the performance of structuredinterviews such as the DIS with clinician ratings in the homeless pop-ulation. Therefore, the purpose of the present study was to compareclinician diagnoses with diagnoses made by a standardized instrumentto determine differences in rates of diagnoses as well as disagreementon individual cases. Based on evidence in the literature, it was hypoth-esized that certain disorders might be overdiagnosed by standardizedinstruments relative to clinical assessment, while others might be un-derdiagnosed, and both forces might be operative in some diagnoses.Specifically, it has been predicted (North, 1995) that major depressionin homeless subjects will be overdiagnosed by structured instruments.The current study also predicted that ASPD and substance use disor-ders will be underdiagnosed by a single cross-sectional research in-terview due to subjects' desire to create positive impressions. Thediagnosis of schizophrenia, because of misperception of adaptation tohomelessness as psychosis, as well as denial of symptoms on a cross-sectional structured interview assessment, is expected to show ele-ments of both overdiagnosis and underdiagnosis relative to clinicianassessment, manifested by agreement in rates but disagreement onwhich individuals constitute the cases.

METHODS

Two homeless samples, in which structured interviews were compared with clinicianassessments, comprise the data base for this study. In the first data set, both assess-ments occurred in close temporal proximity (2 weeks) of one another in a mentalhealth clinic. In the second, assessments in the mental health clinic were comparedwith structured interview diagnoses made in a separate homeless population study.

Sample 1: Comparison of DIS and Clinician Assessment in a MentalHealth Clinic

In 1988, the Health Care for the Homeless Program at the Grace Hill NeighborhoodHealth Center in St. Louis set up a mental health clinic to serve homeless individuals.The mental health team for the clinic, composed of three of the authors of this paper(C.S.N., E.M.S., D.A.R), evaluated patients in the clinic and on visits to shelters. Thesepatients were referred from a variety of medical and social service sources in the city,and referrals were not connected with the survey study to be described later in thispaper. During the first year of this program's operation, patients were initially screenedand assessed by the team social worker (D.A.R.) who administered the DIS as part ofthe routine evaluation. Depending on whether the services of a psychiatrist (C.S.N.) or a


psychologist (E.M.S.) were more immediately appropriate, the patients were then re-ferred respectively for treatment.

Clinician diagnoses were made according to DSM-III-R criteria by the psychiatristor the psychologist, who were blind to diagnoses from the DIS research interview. Theclinicians, unlike the social worker, had considerable experience in psychiatric assess-ment and treatment of homeless individuals. They also had the luxury of assessmentover time (usually with multiple clinic visits, sometimes over a period of years fromthe initiation of the study) and the advantage of multiple sources of information. Clini-cal data were gathered from as many sources as possible, including medical recordsfrom other treatment centers and information from shelter and social service person-nel who often had extensive contact with the individuals. In the first year of theclinic's existence, 97 consecutive patients received a DIS interview and a clinician in-terview in the clinic.

Sample 2: Comparison of Clinician Assessment with DIS Data from aCommunity Homeless Population Study

In 1989-1990, 600 men and 300 women were randomly sampled for an epidemiologicstudy of the homeless population from all overnight and daytime shelters located inthe city of St. Louis that serve the homeless, as well as locations on the street or otherpublic areas where the homeless are known to congregate. Sampling was conductedproportionally to the numbers of persons in the various programs. Lists of guests ofshelters and day centers were randomized, and sets of random numbers were gener-ated by computer to select subjects proportionately to center size from daily logs ofattendance. This random sampling procedure provided this study with a sample thatis believed to be truly representative of homeless men and women in St. Louis whoutilize shelters and day centers.

The majority (70%) of the 600 male subjects and all of the 300 women were sampledfrom shelters. From overnight shelters, 195 men and 251 women were interviewed;150 men and 29 women were interviewed from day centers; 76 men were sampledfrom specialized rehabilitation programs; and 20 women were sampled from 24-houremergency shelters. The remaining male subjects (N = 179) were recruited system-atically from streets, parks, and other public areas. Women were not available in largeenough numbers in these settings to permit sampling there. The DIS was adminis-tered in the field survey sample by trained nonclinician interviewers. The completionrate for the men was 91% and for the women was 96%. More details of the samplingmethods utilized in this study are provided elsewhere (Smith, North, & Spitznagel,1992; Smith, North, & Spitznagel, 1993; Smith, North, & Spitznagel, 1991).

Comparison of names from the list of study subjects to the roster of clinic patientsfound that 33 subjects from the Grace Hill mental health clinic had participated in thehomeless population study. Clinical diagnoses in this group of 33 made by the psychi-atrist and the psychologist were blind to all data from the DIS interviews in the popu-lation study.

Data Analysis

Diagnoses included in the data analyses were schizophrenia, major depression, alcoholand drug use disorders, and ASPD, which were selected for their prevalence and sig-nificance in homeless populations in the published literature.

Clinician diagnoses were compared with DIS diagnoses separately for the 97 clinicpatients who received the DIS in the clinic and the 33 clinic patients who had receivedthe DIS in the community survey study. Reliability estimates on diagnoses were made


by kappa comparisons. (Strictly speaking, kappa is a measure of test-retest reliability,meaning that the two measurement methods should be equivalent; nonetheless, kappahas been used successfully in a variety of contexts in reporting agreement betweentrained nonclinician and clinician raters.) Where prevalence rates of diagnoses werebelow 20%, the kappa statistic is negatively affected and considered by some to be anunacceptable measure. When this occurred, following Spitznagel and Helzer (1985), aY statistic (coefficient of colligation) was also calculated (Yule, 1912). Y scores approxi-mate what the kappa value would have been had the prevalence been within optimalrange. Kappa values above .75 are considered excellent, .4 to .75 fair to good, and lessthan .4 poor (Fleiss, 1981).

To examine partiality of DIS diagnosis relative to the clinician as standard, Mc-Nemar's chi square was calculated for each diagnosis. In the absence of a gold stan-dard for psychiatric diagnostic validity in the homeless population, clinician diagnosesyielded more opportunities for the inclusion of data from multiple sources and overtime, and were therefore used as the comparison standard for this study. If the propor-tion of differences between the DIS and clinician showed directional tendencies (i.e.,either the clinician or the nonclinician DIS tended to make a diagnosis more oftenthan the other), then the McNemar test would find a differential diagnostic preva-lence. For example, if the DIS was consistently positive and clinician consistently neg-ative, negative clinician partiality would be described. McNemar's chi square does notregister disagreement if it is evenly distributed in positive and negative directions.Statistical significance for the McNemar's x2 was set at the .05 level.

RESULTS

Demographic characteristics of the two samples were quite similar.Because the survey sample was prospectively designed to be two-thirds male, the gender rates differ from the clinical sample, whichwas two-thirds (68%) female. Both samples were predominantly young(men about 35 and women about 30 years of age), African-American(75% of survey and 69% of clinic subjects), and unmarried (>90% ofboth samples), and averaged an eleventh grade level of education.

Table 1 presents comparison of DIS diagnoses with clinician diag-noses both made in the clinic. Schizophrenia and substance use disor-ders had acceptable (Y and kappa .5 to .6 range) reliability, ASPD onlyfair (kappa = .4), and major depression poor (kappa < .4) reliability.ASPD and depression were the only two diagnoses showing significantdifferences in diagnostic prevalence by diagnostic method. The clini-cian diagnosed many more cases of ASPD (McNemar's x2 = 12.96, df= 1, p < .001), and the DIS diagnosed more cases of major depression(McNemar's x2 = 17.45, df = 1, p < .001).

Diagnoses by the clinician versus nonclinician DIS survey diagnoseson the 33 population study subjects who were seen in the clinic aresummarized in Table 2. In these comparisons, alcohol use disorder andschizophrenia had acceptable (kappa and Y >.6) reliability and the


other three diagnoses had poor (kappa < .4) reliability. The one diag-nosis with differential diagnostic prevalence was ASPD, which againwas much more often diagnosed by the clinician (McNemar's x2 =

5.82, df = 1, p < .05). The nonclinician-administered DIS identified amarginal but nonsignificant excess of major depression. While the re-liability of drug diagnoses was very poor in this comparison context,the two diagnostic methods identified similar rates of diagnoses, al-though identifying different subjects for diagnosis, and without direc-tional partiality.

DISCUSSION

This study compared psychiatric diagnoses of homeless clinic patientsmade by clinicians who have considerable experience in working withhomeless populations with diagnoses ascertained by the DIS (by a cli-nician DIS rater inexperienced with the homeless in one comparison,and by trained nonclinician DIS administrators in another). Comparedto experienced clinician assessment of homeless psychiatric patients,structured interviews underdiagnosed ASPD and overdiagnosed majordepression. Alcohol use disorder and schizophrenia showed only smalldiscrepancies by assessment method. Drug use disorder showed no di-rectional partiality according to method of assessment, but reliabilityof the diagnosis comparing DIS to clinician assessment yielded verydiscrepant kappa levels in the two different comparison contexts.

The hypothesis that ASPD would be underdiagnosed by the DIS rel-ative to the clinician was upheld, ais demonstrated by the low kappasand significant directional partiality detected across both comparisons.It is therefore likely that the experienced clinician procedure of obtain-ing information from multiple sources and by observation over timeyields significant gains in data not obtained in a single face-to-faceinterview that depends entirely upon the subject's candidness.

Similarly, the hypothesis that major depression would be overdiag-nosed by the DIS was strongly supported by the findings of low kappasfound in both comparisons and significant directional partiality foundin the first comparison. This finding agrees with previous speculationthat the diagnosis of major depression in this population may be con-founded by the miserable conditions of homelessness that the struc-tured nature of the DIS on cross section cannot sort out in the waythat a clinician experienced with this population can. Clinical experi-ence of this team has been that homeless people presenting for treat-



merit of depressive complaints seem to present with a context-orientedset of symptoms that temporally parallels the course of homelessness.To determine whether this syndrome clinically simulating major de-pression in cross section represents a valid case of major depression asit presents in other populations, external validators such as consis-tency on follow-up especially after stable housing is achieved, familyhistory, and response to treatment for depression should be sought infuture studies.

The hypothesis that substance use disorders are systematically un-derdiagnosed by the DIS was not supported by the data. Alcohol usedisorder showed the highest kappa of all five disorders examined, andno directional partiality of method of diagnosis was ascertained. Druguse disorder also showed no directional partiality, and in the first com-parison the kappa was relatively good; however, in the comparison ofnonclinician DIS diagnoses made in the field with clinician diagnoses,the kappa was the lowest of all reliability statistics across either set ofcomparisons. This lack of directional partiality by method for this di-agnosis could indicate that subjects were equally likely to conceal theirdrug history from the DIS interviewer in the field (potentially due tofear of being expelled from programs if this history is learned) as theywere to conceal it from the clinician (due to desire to make a positiveimpression). In both comparisons, the clinician and the DIS assess-ments diagnosed similar overall rates of drug use disorder, but therates of disagreement indicate that the two methods of assessmentoften identified different cases.

Part of the discrepancy in drug diagnoses may be due to the lengthof time between interviews, sometimes up to four years apart in thefield-DIS assessed sample (while the clinic-DIS assessed sample re-ceived both interviews within a week or two of one another). This issupported by the 41% rate of disagreement in the field-DIS assessedsample, compared to only 16% disagreement in the clinic-DIS assessedsample. The combined rate of cases diagnosed by the clinician and/orDIS in the field-DIS assessed sample was 56%, much higher than theclinic-DIS assessed rate of 39%. This suggests that in different set-tings and in separate time periods subjects' stories change, and theyield may be optimized by combining information from as disparate ofsources and time periods as possible.

The final hypothesis, that schizophrenia would show no significantbias, was upheld, but the expectation of unacceptably low reliabilitywas not borne out in either set of comparisons. Therefore the DIS as-sessment of schizophrenia in this population may be functionally


equivalent to clinician diagnosis. Situational factors of homelessnessdo not appear to confound structured diagnostic assessment of schizo-phrenia any more than they confuse the seasoned clinician.

Strengths of this study included the standardized definition of home-lessness and random sampling of field subjects from shelters, day cen-ters, and street settings, as well as the systematic interviewing withthe DIS. The expertise of the clinicians with this population and theopportunity to collect information over time and from many sourcesargue in favor of their role as the standard of comparison in this study.Probably the greatest limitation to this study is that because it as-sessed psychiatric disorders only among clinic attenders, the findingsmay not generalize to assessment of other homeless populations. Thestudy was also limited in its assessment of only five psychiatric diag-noses, due to limited prevalence of diagnoses, and in assessment oflifetime as opposed to current diagnosis. Finally, the sample was rela-tively small and inter-rater reliability of clinicians was not ascer-tained. Therefore, attention to these limitations is needed in futureresearch.

The findings of this study suggest the need for more research indiagnostic assessment of homeless populations to further clarify theeffects of potential confounders of the homeless situation to psychiatricdiagnosis, especially for the diagnoses of major depression and ASPD,and possibly drug use disorder. Because it cannot be determinedwhether the diagnostic discrepancies found in this study were due todifferences in evaluator expertise with the homeless population or todifferences in assessment method, future studies might consider con-trolling for level of clinician expertise with homeless populations.Studies to collect repeated diagnoses over time across changes inthe individuals' homelessness status might help clarify the impact ofhomelessness on validity of diagnosis. A potentially fruitful next stepin diagnostic methodology might be to study the effects of the findingsof incorporation of methodological elements of multiple sources ofdata, longitudinal observation of diagnostic information and its inter-play with homelessness, and degree of assessors' clinical experiencewith this population.

Drake and colleagues (Drake, Alterman, & Rosenberg, 1993) havepointed out that diagnostic dilemmas created by unrecognized pres-ence of substance abuse include misdiagnosis of other disorders, inap-propriate treatment with medications, neglect of appropriate interven-tions for substance abuse, and inappropriate treatment planning andreferral. These researchers concluded that when substance abuse is


not addressed, mental health treatment for dually diagnosed individ-uals is "markedly ineffective." In the homeless population, where men-tal illness and substance abuse are thought to be critical factors in thegeneration and perpetuation of homelessness, the issue of proper diag-nosis is tantamount to understanding and providing workable solu-tions to the problem of homelessness.

If nonclinician structured assessment of the homeless populationactually overestimates depression, underestimates ASPD, and mis-classifies drug abuse, then policies stemming from these research rec-ommendations may call for levels and types of services not optimallysuited to the reality of this population's needs. Overestimation of ma-jor mental illness in the homeless population may lead to policies over-emphasizing and isolating mental illness in service development whileignoring the very stressors that may be causing the overestimation bystandardized instruments. Underestimation of Axis II disorders andsubstance abuse in this population will interfere with therapy of otherdisorders and delay recognition of need for substance abuse treatment,probably the greatest psychiatric treatment need of this population.The findings of this study suggest the need for interpretation of find-ings by clinicians experienced with this population in recommendingprogram designs. Further work is needed by clinicians who are bothexperienced in homelessness and in methods of academic research onpsychiatric diagnosis.

REFERENCES

Bassuk, E.L., Rubin, L., & Lauriat, A.S. (1986). Characteristics of sheltered homeless families.American Journal of Public Health, 76(9), 1097-1101.

Breakey, W.R., Fischer, P.J., Kramer, M., Nestadt, G., Romanoski, A.J., Ross, A., Royall, R.M., &Stine, O.C. (1989). Health and mental health problems of homeless men and women in Bal-timore. The Journal of the American Medical Association, 262(10), 1352-1357.

Drake, R.E., Alterman, A.I., & Rosenberg, S.R. (1993). Detection of substance use disorders inseverely mentally ill patients. Community Mental Health Journal, 29, 175-192.

Drake, R.E., Osher, EC., & Wallach, M.A. (1991). Homelessness and dual diagnosis. AmericanPsychologist, 46(11), 1149-1158.

Fischer, P.J. (1989). Estimating the prevalence of alcohol, drug and mental health problems inthe contemporary homeless population: A review of the literature. Contemporary Drug Prob-lems, 16(3), 333-390.

Fischer, P.J., Drake, R.E., & Breakey, W.R. (1992). Mental health problems among homelesspersons: A review of epidemiological research from 1980 to 1990. In H.R. Lamb, L.L.Bachrach, & F.I. Kass (Eds.), Treating the Homeless Mentally III: A Report of the Task Forceon the Homeless Mentally III. Washington, DC: American Psychiatric Association, (pp. 75-93).

Fischer, P.J., Shapiro, S., Breakey, W.R., Anthony, J.C., & Kramer, M. (1986). Mental health andsocial characteristics of the homeless: A survey of mission users. American Journal of PublicHealth, 76(5), 519-524.


Fleiss, J. (1981), Statistics for rates of proportions. New York: John Wiley & Sons.Herrman, H., McGorry, P., Bennett, P., van Riel, R., & Singh, B. (1989). Prevalence of severe

mental disorders in disaffiliated and homeless people in inner Melbourne. American Journalof Psychiatry, 146, 1179-1184.

Koegel, P., & Burnam, A. (1988). Alcoholism among homeless adults in the inner city of LosAngeles. Archives of General Psychiatry, 45, 1011-1018.

Koegel, P., Burnam, A., & Farr, R.K. (1988). The prevalence of specific psychiatric disordersamong homeless individuals in the inner city of Los Angeles. Archives of General Psychiatry,45, 1085-1092.

Koegel, P., & Burnam, M.A. (1992). Problems in the assessment of mental illness among thehomeless: An empirical approach. In M.J. Robertson & M. Greenblatt (Eds.), Homelessness:A National Perspective. New York: Plenum, (pp. 77-99).

Lipton, F.R., Sabatini, A., & Katz, S.E. (1983). Down and out in the city: The homeless mentallyill. Hospital and Community Psychiatry, 34, 817-821.

North, C.S. (1995). Homelessness: Psychiatric and Cultural Dimensions. American PsychiatricAssociation presentation in Miami, Florida.

North, C.S., Smith, E.M., & Spitznagel, EX. (1993). Is antisocial personality a valid diagnosisamong the homeless? American Journal of Psychiatry, 150, 578-583.

North, C.S., Smith, E.M., & Spitznagel, EX. (1994). Violence and the homeless: an epidemiologicstudy of victimization and aggression. Journal of Traumatic Stress, 7(1), 95—110.

Robins, L.N., Helzer, J.E., Croughan, J., Williams, J.B.W., & Spitzer, RX. (1981). NIMH Diagnos-tic Interview Schedule: Version III (May 1981). National Institute of Mental Health.

Smith, E.M., North, C.S., & Spitznagel, EX. (1991). Are hard-to-interview street dwellers neededin assessing psychiatric disorders in homeless men? International J of Methods in Psychi-atric Res. 1, 69-78.

Smith, E.M., North, C.S., & Spitznagel, EX. (1992). A systematic study of mental illness, sub-stance abuse, and treatment in 600 homeless men. Annals of Clinical Psychiatry, 4(2), 111-120.

Smith, E.M., North, C.S., & Spitznagel, EX. (1993). Alcohol, drugs, and psychiatric comorbidityamong homeless women: An epidemiologic study. Journal of Clinical Psychiatry,34, 82-87.

Spitznagel, EX., & Helzer, J.E. (1985). A proposed solution to the base rate problem in the kappastatistic. Archives of General Psychiatry, 42, 725-728.

Susser, E., Conover, S., & Struening, EX. (1989). Problems of epidemiologic method in assessingthe type and extent of mental illness among homeless adults. Hospital and Community Psy-chiatry, 40, 261-265.

Susser, E., Struening, EX., & Conover, S. (1989). Psychiatric problems in homeless men. Ar-chives of General Psychiatry, 46, 845-850.

Susser, E.S., & Struening, EX. (1990). Diagnosis and screening for psychotic disorders in a studyof the homeless. Schizophrenia Bulletin, 16, 133-145.

Torrey, E.F. (1986). Forced medication is part of the cure. The New Physician, December, 34-37.Yule, G.U. (1912). On the methods of measuring association between two attributes. Journal of

the Royal Statistical Society, 75, 581-642.
