Administration and Policy in Mental Health Vol. 20, No. 6, July 1993
COLLABORATIVE RESEARCH IN MENTAL HEALTH
Bentson H. McFarland, M.D., Ph.D., Frederick A. DiBlasio, Ph.D., and John R. Belcher, Ph.D.
ABSTRACT: This paper describes advantages and disadvantages of the National Institute of Mental Health's program for Public Academic Liaison (PAL). Useful administrative lessons gleaned from longstanding connections between state mental health agencies and universities in Ohio, Virginia, and Oregon are reviewed. Important issues such as budgeting and staffing for both agency and university administrators to consider when establishing collaboration are outlined.
At the end of the last decade, the National Institute of Mental Health proclaimed a new initiative entitled "Public Academic Liaison" (Bevilacqua, 1991; Glick, 1989). The purpose of the Public Academic Liaison (PAL) program is to enhance the relationship between academic researchers and the public mental health system. Since PAL is emphasized in National Institute of Mental Health (NIMH) requests for applications (RFAs), mental health ad- ministrators will benefit from learning more about this program. It will also be of interest to review results from three states which have had longstanding public academic linkages designed to facilitate research.
While there is growing literature on public academic collaboration with
Bentson McFarland is Director of the Western Mental Health Research Center at Oregon Health Sciences University and Investigator at the Kaiser Permanente Center for Health Research in Portland, Oregon. Frederick DiBlasio is Associate Professor and John Belcher is Assistant Professor, both at the School of Social Work and Community Planning at the University of Maryland, Baltimore. Address correspondence to Bentson McFarland, Dept. of Psychiatry, Mail Code OP-02, Oregon Health Sciences University, Portland, OR 97201.
This paper was presented in part at the National Conference on Mental Health Statistics in San Diego on June 2, 1989, and at the 141st Annual Meeting of the American Psychiatric Association in New York on May 15, 1990. Support from the National Institute of Mental Health grant number P50 MH43458 to the Western Mental Health Research Center is gratefully acknowledged. Dr. McFarland is Burroughs- Wellcome Scholar in Pharmacoepidemiology.
421 9 1993 Human Sciences Press, Inc.
422 Administration and Policy in Mental Health
regard to training (Boust, 1991; Cutler, Bloom, & Shore, 1981; Faulkner et al., 1987; Keefer, Kraus, Parker, Elliot, & Patton, 1991; Talbott & Ra- binowitz, 1986; Weintraub, Nyman, &Harbin, 1991; Yank et al., 1991), little has been written about collaborative research endeavors (Pardes, 1990). Of course, there are many examples of successful collaborative research enter- prises (Bloom, 1989; Peterson, 1991). However, the recent emphasis on public-academic liaison from the NIMH has resulted in a need for mental health administrators to better appreciate the subtleties of these collaborative enterprises.
University and mental health administrators recognize the value of research for developing and evaluating mental health programs. Despite this shared recognition, a gap exists in the structure of universities and mental health agencies in the process of doing collaborative research. This process is hindered by many conceptual and pragmatic obstacles that seldom are addressed and resolved. However some scholars have noted that as states' budgets shrink, state funding for research will also decrease (Jerell & Larsen, 1985), and new challenges will generate the need for unique ways of promoting agency and academic liaison relationships.
From the universities' perspective, graduate schools are increasingly depen- dent upon direct funding from student enrollments. Moreover, decreased financial support for graduate professional education has reduced enrollments. The National Institute of Mental Health and the National Institutes of Health once provided major financial support for graduate professional education; such funds are now largely non-existent (McPheeters, 1983). Reduced enroll- ments pressure universities to reduce faculty lines, which leave remaining faculty with increased responsibility.
As funds decrease and responsibilities increase for both the state and univer- sity systems the relationship between the two becomes more distant. McPhee- ters (1985) suggests that the two systems have been distant for more than 30 years partially because of the perceived differences in goals. Public agencies are usually concerned with direct service delivery and universities are interested in education and research.
This paper summarizes some of the practical and ideological concerns in designing a collaborative research system and suggests initial approaches to resolve issues that block collaboration. Many of the issues presented here were evident in Colorado's (Shern & Glover, 1984) and Virginia's (DiBlasio, 1985) experiences in setting up a process for collaboration. Experiences from Ohio and Oregon are also pertinent.
TYPES OF COLLABORATIVE RESEARCH
Some definitions will be helpful. In this context public refers to state mental health agencies, state mental hospitals, and community mental health pro-
Bentson H. McFarland, Frederick A. DiBlasio, and John R. Belcher 423
grams. Academic generally refers to universities which may be public or private. Also included in the academic sector are research institutions which may be public or private, non-profit, or for profit. It should be noted that some agencies in fact encompass both roles, i.e., public and academic. For example, a university may be chiefly interested in teaching and research, but could also operate a community mental health program. Private providers such as a general hospital might operate a public mental health program under contract to a state agency.
Paying for Research
One means of categorizing collaborative public-academic research is by the way in which it is financed (Table 1). Internally financed studies will typically be relatively small projects-perhaps involving record review or other types of retrospective data analysis. Usually the work will in fact be done by students, residents, or fellows seeking research training in the public sector while being supervised by academic professionals. Program evaluation is another impor- tant area for collaborative research. It is not uncommon for public sector agencies to contract with academic institutions to evaluate specific mental health projects. Finally, a collaborative pilot project may be launched in order to obtain data prior to beginning a large research program. External financing for collaborative research typically comes from the private sector such as foundations or the pharmaceutical industry or from federal agencies such as the NIMH.
There is, of course, a large literature on program evaluation (Table 2). With regard to collaborative efforts it is important to distinguish between formative program evaluation and summative evaluation (King, Morris, & Fitzgibbon, 1987). Formative evaluation typically takes place early in a project and is used to clarify the objectives of the program. It is not uncommon for a formative evaluation to lead to changes in the program. In contrast, summative evalua-
TABLE 1 Collaborative Research
1. Internally Financed a. Students, residents, fellows b. Program evaluation c. Pilot projects
2. Externally Financed a. Private
(1) Foundations (2) Pharmaceutical
b. Federal (NIMH)
424 Administration and Policy in Mental Health
TABLE 2 Program Evaluation*
1. Formative a. Clarifies objectives b. Early in program's life c. May lead to program
changes 2. Summative
a. Recapitulates objectives b. Later in program's life c. Evaluates accomplishments
*Modified fi'orn Randolph (1990)
tion is done toward the end of a project, is designed to evaluate its accomplish- ments, and hopefully recapitulates the objectives of the program (Randolph, 1990). In conducting collaborative program evaluation it is essential that both parties understand the nature of the evaluation to be performed. Specifically, it is important for the academic evaluator to determine whether the public agency is interested in formative or summative evaluation.
With regard to program evaluation, it is important to understand that the mission of direct practice agencies is the delivery of services to directly meet needs of clients suffering from mental health problems, many of whom are the chronic mentally ill. The mission of universities is to generate new knowledge and to produce graduates. Ultimately, the mental health system must provide long-term care for mentally incapacitated people, who carry negative stigma and are viewed as burdens to taxpayers. The disparity between end products creates tension between the two systems as agencies sometimes view the universities as uncaring and removed from the everyday struggles of service delivery, while universities often view agencies as bureaucratic and resistant to the scientific process.
Conflicts in missions lead to disagreement as to the purpose and direction of research. Administrators often desire to participate in research that is directly applicable to their clinical setting to implement prevention strategies or meet immediate service delivery needs. In addition, they emphasize the evaluative aspect of applied research, particularly to evaluate the effectiveness of an intervention program or strategy. Shern and Glover (1984) reported that providers of services viewed academically housed research as abstract and contrived, and for the most part not applicable in applied settings. University researchers also are skeptical as they question the scientific quality of many of the evaluative research designs. Evaluative designs are often time-limited, compromised in design and conceptualization, and therefore restrict gener- alizability (Shern & Glover, 1984).
Bentson H. McFarland, Frederick A. DiBlasio, and John R. Belcher 425
Service providers and administrators should understand the vocabulary used in describing research projects. Pharmaceutical research is especially notewor- thy since much terminology devised by the Food and Drug Administration is now employed in other areas (Table 3). Pharmaceutical studies are typically divided into pre-clinical and clinical components (Nightingale, 1981; Young, Norris, Levitt, & Nightingale, 1988). The pre-clinical work is done in the laboratory involving extensive animal tests. Upon completion of pre-clinical studies the pharmaceutical manufacturer typically applies to the Food and Drug Administration for an investigational new drug (IND) permit. Once the IND has been obtained the manufacturer will begin phases I, II, and III of the clinical research. Phase I is the safety study done in healthy male volunteers. Phase II is efficacy research usually done in an open fashion in patients. Phase III is typically a double blind randomized clinical trial attempting to determine both safety and efficacy in patients. Upon completion of phase III the manu- facturer then obtains marketing approval from the Food and Drug Administra- tion. Approval may perhaps be contingent upon conducting further studies which would be termed phase IV.
There are analogies between pharmaceutical research and studies of psycho- social treatment programs. Specifically, phase II pharmaceutical studies for efficacy are similar to demonstration projects. The idea in a demonstration project is to determine whether a theoretically appealing program can actually be provided to the population in question. A secondary objective of the demonstration project is to determine (roughly) whether the desired interven- tion appears to have some efficacy in the target population. In federally sponsored demonstration projects it is usually the case that 95 % of the funding will go for service and 5 % will be assigned to research-evaluation. Typically in a federally funded demonstration project the grantee must be a public agency.
A recent development in federally financed projects is the research- demonstration grant (Table 4). This type of grant (labeled R-18 by the
TABLE 3 Pharmaceutical Research
Phase Purpose Subjects Method
Investigational New Drug I Safety Healthy Open II Efficacy Patients Open III Safety and Efficacy Patients Blind
New Drug Application Approved IV Indications New Patients Blind IV Surveillance All Patients Cohort
426 Administration and Policy in Mental Health
TABLE 4 Federally Financed Research
Mechanism Number Phase Grantee % Service % Research
Demonstration -- II State 95 % 5 % Research-Demo R 18 III Anyone 60 % 40 % Research R01 IV Anyone 5 % 95 %
NIMH) corresponds to the phase III pharmaceutical trial. Specifically, the idea in the research-demonstration project is to compare the efficacy of a well defined treatment program against a control condition. Typically in a research-demonstration project the intervention will be "model-driven." In other words, it is presumed that the researcher will have assimilated and/or generated a large body of theory and preliminary data that will motivate the intervention's development as well as give one hope that the new program will be successful. On the average, 60% of the federal research dollars in a research-demonstration project can go for service, but at least 30-40% must be for research purposes. Randomized designs with well defined treatment and control groups are key parts of the research-demonstration project. It is also important that the intervention being tested is generalizable. Having a manual of procedures that specifies the intervention is one way to achieve this gener- alizability. The research-demonstration project should be conducted in a well- defined population so that the intervention can be easily replicated by others working in the field.
Finally, the traditional research project (R01 in the federal nomenclature) corresponds to a phase IV pharmaceutical trial. Generally in an R01 project at least 95 % of the budget is dedicated to research. One expects that the interven- tion in question will be well defined and will have been shown to be efficacious in previous work. Often an R01 project is aimed at testing the application of a well-defined intervention to a new target population.
With this terminology we now review advantages and disadvantages of collaborative research.
ADVANTAGES OF PUBLIC ACADEMIC COLLABORATION
There are considerable benefits to both sides in public academic liaison. Persons in the public sector can collaborate with academic specialists and can consult with individuals who may have particular skills or knowledge that are not generally available. Furthermore, the academic sector can deliver profes- sional education, continuing education, and specialized training to workers in the p...