Toward collaborative mental health outcomes systems

A growing recognition that customers are important health partners has created the needfor collaborative systems of accountability in behavioral health services.

Toward Collaborative Mental Health Outcomes Systems Jean Campbell

In the face of health care reform and the proliferation of managed behavioral health care systems, the organization and delivery of mental health services has confronted unprecedented challenges to the way quality is managed and the effectiveness of treatments and services determined. A growing recognition that customers are important health care partners has created the need for collaborative systems of accountability However, neither measures of outcomes nor the communication technologies that facilitate information management are value free. Beliefs and attitudes play an important part (Campbell and John- son, 1995; Campbell, 1994). Incongruities between the perceptual and expe- riential framework of those who research, manage, and deliver services and those who receive mental health services have raised questions concerning professional assessments and research protocols. In many cases, consumers, their families, and mental health professionals do not agree on the identification and relative importance of clinical and service outcomes (Ridgway, 1988). There appear to be wide differences between consumers and professionals on the relative importance of treatment goals (Dimsdale, Klerman, and Shershow, 1979), identification of problems (Mitchell and others, 1983), barriers to service (Lynch and Kruzich, 19861, and needs and preferences for housing and sup- ports (Minsky, Riesser, and Duffy, 1995; Ridgway and Carling, 1987). The experiences of people with mental illness stand as a robust critique of bio- medical psychiatry and the mental health services delivery system (Chamber- lin, 1978; Susko, 1991; Zinman, Harp, and Budd, 1987).

Historically, changes in and expected improvements to health services have been based on the common assumption that the expert knows best about

NEW DIRECTIONS FOR MENTAL HEALTH SERVICES, no. 71, Fall 1996 0 Jossey-Bass Publishers 69

70 USING CLIENT OUTCOMES INFORMATION TO IMPROVE TREATMENT

the operating principles of the health service delivery system. In general, medical professionals assume that the principles of good health care ovemde individual treatment preferences. But Eddy and Billings (1988) have cautioned that “this seduction by authority” inevitably influences future behavior to mimic current practice and has led to the creation of current practice standards without knowledge of “how people would compare the benefit and harm” (p. 15).

From the perspective of health professionals, a diagnosis indicates pathol- ogy, but to the individual, a diagnosis can become an assault on identity and sense of self. Professionals monitor symptom control, but people judge their services by the impact they have on quality of life. Some researchers fear that acceptance of a broad definition of health that encompasses mental and social well-being, and not just the absence of disease, may compromise the ability to measure concepts of health care effectiveness (Gould, 1993). On the other hand, consumers point out that with the focus on chronicity and disability in mental health research, the capacity of people with mental illness to heal and recover is relatively unexplored (ConsumerlSunivor Mental Health Research and Policy Work Group, 1992). Therefore, illness represents not a shared real- ity but two quite different realities, the meaning of one being significantly different from the other (Toombs, 1987).

Some critics within the medical profession have also argued that clinical decisions should be based on outcomes that are important to patients and their families (Eddy, 1989). Wennberg (1988) observes that “in many clinical situ- ations, the physician’s treatment and outcomes preferences unduly influence the clinical decision process, in part because so little attention is now given to the crucial task of helping patients discover and choose the care they really want” (p. 99).

In response to the growing emphasis on outcomes management and the debate on appropriate outcomes indicators, this chapter argues that mental health care delivery systems must engage in a shift away from how mental health services are currently assessed toward a multistakeholder model based on collaboration. That divergent views can be shared and reconciled through reciprocal theory and practice implies major conceptual leaps in how one thinks about the production of mental health outcomes information and how one uses this information to manage services and facilitate individual recipi- ent choice.

From Patients to Customers The growing tide of health consumerism is one of the most compelling forces in the future of mental health information management and quality assurance systems. It holds that persons who seek health services are consumers just as are persons who seek other types of services. The consumer listens to the thoughts of the provider but ultimately makes his or her own decisions (Reeder, 1972). Consumerism implies that “values derived from principles of good medical care must be interpreted and operationalized through reference

TOWARD COLLABORATIVE MENTAL HEALTH OUTCOMES SYSTEMS 71

to the patient’s personal health care values and desires” (Beisecker and Beisecker, 1993, p. 53).

Because of pervasive cultural stereotypes of people with illness as violent and incompetent, psychiatric consumers must overcome tremendous personal and political barriers to gain control over and responsibility for their own lives in the medical marketplace and to be seen truly as customers. Often services that philosophically support consumer empowerment withhold power from recipients to make and enforce decisions. For example, although self-determi- nation is a basic tenet of psychiatric rehabilitation and social work practice (Anthony, Cohen, and Cohen, 1983; Meenaghan and Mascari, 1971), consumer perspectives are frequently not considered in the development of programs or individual service plans by social workers or psychiatric rehabilitators (Kopolow, 1979). Significantly, the results of a survey of consumers, families, and providers indicated that many mental health professionals presume that people with mental illness do not know what is in their own best interests (Campbell and Schraiber, 1989). Yet the consumer responses in the study indicated that the failure of mental health professionals to provide treatment information, show respect, or listen and validate consumer insights had a detrimental effect on their well-being (Campbell, 1992).

People with mental illness and their families have fought with limited but significant success to redress perceived injustices in the delivery of mental health services by organizing advocacy groups. Mental health consumers have built a national self-help movement, with a growing number now participat- ing in research and evaluation (Campbell, Ralph, and Glover, 1993; Scott, 1993; Sherman and Kaufmann, 1995) and taking leadership roles in policy and the administration of mental health services. Some professionals and policymakers have responded to consumer demands by redesigning professional roles and creating opportunities for people who receive services to provide input and perspective (Blanch, 1993).

Clearly, consumer perceptions about treatment efficacy and their satisfaction and dissatisfaction with the type of treatment received, as well as the way services are delivered, determine which treatments are sought or complied with and which outcomes are valued (Mirin and Namerow, 1991). With mental health consumers emerging as customers, “the mental health care industry is beginning to grapple with the need to be customer driven” (Chowanec, Neun- aber, and Krajl, 1994, p. 47).

Prioritizing Outcomes Among System Stakeholders

Mental health programs operate within a political system in which data are often exercised in struggles for influence. Since outcome data have an impact on various constituencies who have both power and a stake in the service delivery system, the choice of measures and indicators is usually contested. Some stakeholders are primarily interested in whether the system relieves human suffering, others are concerned about costs, still others care about


issues of access and equitable resource allocation, and most stakeholders have multiple concerns about the service system (MHSIP Task Force, 1993).

Ideally, an outcomes management system considers the range of stakeholders, what they value about the operation of the mental health service system, and what they want to know about it. It is as responsive as possible to these concerns, but obviously no one set of outcome indicators covers all sit- uations or satisfies all perspectives. Effective selection of outcomes depends on the careful application of methods for prioritizing the multiple perspectives of key stakeholders. While both quality improvement and accountability are linked to information management, it cannot be assumed that if data elements are refined and collected through statistically valid methods, system functions will be accepted or optimized.

The national focus on efforts to identify outcome domains, the growth of the consumer movement, and the public policy support of consumer empowerment and choice has translated into consumers’ taking a role in outcomes assessment. Seeing the need for consumer input and guidance, national consumer leaders began the systematic articulation of consumer values and outcomes in the 1990s. Support from the Center for Mental Health Services (CMHS) through the Mental Health Statistics Improvement Program (MHSIP) enabled the newly formed Consumer/Survivor Mental Health Research and Policy Work Group to conduct a series of focus groups with key mental health consumer informants who had expertise in advocacy, peer support, or research. According to most focus group participants, traditional mental health systems pathologize problems in living, hold low expectations of consumer achieve- ment, are paternalistic, offer a limited range of options, and are too quick to define anger as symptomatic. Some key concerns included the threat of invol- untary treatment, subtle forms of coercion, lack of respect toward consumers by mental health professionals and providers, and the debilitating side effects of medications. It was observed that researchers fail to ask questions that cap- ture adverse or negative effects of treatment and care or outcomes such as recovery, personhood, well-being, and liberty (Consumer/Survivor Mental Health Research and Policy Work Group, 1992).

Following this initial effort, a consumer research project to identify desired outcomes for state mental health service recipients for development of the MHSIP Consumer-Oriented Mental Health Report Card and to draft a consumer component for the National Association for State Mental Health Pro- gram Directors (NASMHPD) State Mental Health Profiling System led to two pilots using concept mapping, a structured group process with multivariate statistical analysis (Dumont and Campbell, 1994; Trochim, Dumont, and Campbell, 1993). The content domains from the combined pilot results included legal issues, consumer impact on service delivery and system development, oppression and racism, healing and recovery, coercion and control, personhood, damaging effects of treatment, alternatives to traditional services, citizenship, quality of life, employment, and validity of research (Dumont and Campbell, 1994).


The outcomes that different stakeholders prioritize will vary based on services and administrative organization, the current policy context, information system capacity, the stakeholders themselves, and demographical and geographical variables that are unique to a particular service system. A study by the Colorado Division of Mental Health and the Western Interstate Com- mission for Higher Education (WICHE) has attempted to learn to what extent consumers, families, clinicians, and administrators’ ratings of the relative importance of different outcomes agree or disagree (McGuirk, Zahniser, Bartsch, and Engleby, 1994). From a content analysis of outcomes for mental health services in the professional literature, the researchers identified ten general outcome domains: community tenure, consumer involvement, consumer satisfaction, family involvement, family relief, improved social func- tion, personhood, safety, skilled coping, and symptom reduction. The method of paired comparisons was used in obtaining stakeholders’ ratings of the importance of the different outcome dimensions. Although there was significant overlap in the stakeholders’ ratings, there were significant areas of dif- ference as well. Consumers rated family relief significantly more important than did providers and administrators, consumers and family members rated family involvement significantly higher than did administrators, consumers and family members rated safety significantly less important than did both providers and administrators, and community tenure was rated relatively low in importance by all stakeholder groups. Consumer satisfaction was also rated relatively low in importance. There was also diversity among consumer groups (for example, hospitalized consumers in comparison to those living in the community). The only outcome that is frequently used in a wide range of research and evaluation efforts that was highly rated was symptom reduction.

Developing Collaborative Models

Promoting multistakeholder participation is particularly salient for health care reform efforts. Such an approach has great power precisely because it includes a diversity of system stakeholders within an open architecture of knowledge production. Integration of personal experiences of illness, service, and treatment with professional knowledge recognizes the importance of each person’s needs, expectations, and experience but retains an understanding of organiza- tional behavior and its management. In mental health services research, mean- ingful inclusion of consumers has enriched the process of “good science” by illuminating important discrepancies between perceptions of consumers and professionals. In November 1994, the National Alliance for the Mentally Ill, Johns Hopkins University, and the National Institute of Mental Health con- vened the first multistakeholder outcomes conference. Bringing together consumers, families, providers, employers, purchasers, and private and public sectors, this conference signaled a shift from dependency on professionals to collaborative accountability. As a result, new questions, methods, and ways of


looking at data have begun to emerge in the margins of traditional discourse and in the practice of mental health care.

On the other hand, the challenge to multistakeholder participation is to synthesize the divergent assumptions that underlie every outcome measure. How does a multistakeholder process determine the expected results of services? How are service effects to be measured? When should effects be measured? Who should be measured, and how are data analyzed? Without developing ways for constructive dialogue to occur and shared decision making to take place, the focus on mental health outcomes reinforces a “kind of turf war over controlling human beings in a landscape that includes an entire array of service options and widely divergent goals and definitions of mental health and quality of life” (Scott, 1993, p. 5).

The Final Report of the Task Force on the Design of Performance Indica- tors Derived from the MHSIP Content (MHSIP, 1993) suggests that the ideal environment for the development of a system of performance indicators is one in which “intents of all stakeholders are articulated and shared; there is a culture of respect for and constructive use of data; changes are accomplished through participatory development; resistance is reduced through disclosure of fears and implementation of safeguards that address those fears” (p. 41). As an alternative to polarizing debate, consumers, professionals, and policymakers have begun to use participatory methods for discussion and exchange of perspective. The collaborative models discussed below point to a multistakeholder process that can inform the development of outcomes management systems.

Continuous Quality Improvement. The Total Quality Management approach adapted from industry empowers an entire health delivery system to improve the quality of goods and services continuously in order to satisfy its customers. High-performance organizations develop strategic plans to find out what customers want by involving them through customer surveys, focus groups, consumer councils, complaint tracking systems, and test marketing (Osborne and Gaebler, 1992). Feedback from customers provides the data necessary for establishing a reiterative process called Continuous Quality Improvement.

Participatory Action Research. The participatory character of participatory action research (PAR) involves subjects in becoming co-researchers (Carr and Kemmis, 1983; Fals-Borda, 1984; Gaventa, 1988) through a repet- itive and cyclical process of diagnosis, analysis, action, and evaluation with constant feedback loops (Chein, Cook, and Harding, 1948; Elden, 1981; Tichy and Friedman, 1983). PAR stresses respect for and reliance on the needs and expertise of the people under study from start to finish (Chesler, 1991).

Constituency-Oriented Research and Dissemination. The National Institute on Disability and Rehabilitation (NIDRR) in 1992 undertook Con- stituency-Oriented Research and Dissemination (CORD), an initiative to draft a policy statement for the agency that would ensure that constituencies would actively participate in research needs identification, priority setting, requests for proposal, the application preparation process, peer review, making awards,


conducting projects, disseminating and using results, and conducting evalua- tions. CORD promotes the opportunity for each individual to share and use his or her skills, background, and experiences SO that common objectives can be defined and achieved in partnership with people with disabilities. In the C O W model, beneficiary and constituency participation in the NIDRR research and dissemination process is the cord that ties together the producers, dissemina- tors, and users of research outcomes to achieve a common goal: improving the lives of individuals with disabilities (Fenton, Batavia, and Roody, 1993).

Pioneer Dialogues. Pioneer Dialogues is an innovative process devel- oped by mental health consumers in Florida to overcome barriers due to an imbalance of power that inhibit collaboration between individuals from dis- parate backgrounds and experiences. Two Pioneer Dialogues were organized by the Consumer/Survivor Mental Health Research and Policy Work Group and NASMHPD to bring consumers and state mental health administrators together to discuss system reform and its implications for research and evaluation. These meetings contributed to changes in state mental health policies, raised the consciousness of participants regarding the traditional imbalance of power between consumers and staff, and began to establish an appreciation for the value of collaboration (Loder and Glover, 1992). Similarly, in New York State psychiatrists and consumers also conducted several dialogues to exchange perspectives, develop a shared vision of recovery, and consider ways in which treatment relationships could be more collaborative (Blanch, Fisher, Tucker, and Chassman, 1993). Psychiatrists were surprised by the degree to which people recounted negative experiences within the mental health system. Despite differences, both groups were willing to listen and respond thoughtfully to each other.

Conclusion

If mental health research and evaluation can enhance individual choice, power, and knowledge, the services delivery system has a great opportunity to take a quantum leap forward to where individual and system decisions can be made in dramatically more effective and humane ways. Collaborative efforts to define outcome indicators are an important first step. Yet without incorporating mechanisms for data distribution and feedback in outcomes management systems, there is little opportunity to integrate quality improvement goals of system stakeholders. Since the structure or formal design elements of information systems shape their content, character, and influence on service delivery, it is important that they are collaborative as well. Although the preceding discussion has concentrated on collaborative models to define outcome indicators, complex issues of access, connectivity, privacy, and how the structure of the outcomes management system shapes its content, character and influence on the service delivery system must also be addressed within a participatory framework. The culture of the mental health care system, the roles of its participants, and the engineering of information systems are slowly changing


toward that end. To the extent that mental health consumers and professionals embrace the concept that shared decision making is useful, cost-effective, and a consumer’s right, there will be a broad adoption of technologies to facilitate this process. Collaborative knowledge holds the promise for fluid and continuing reform if system stakeholders work together as equals and, most important, consumer participation is valued and supported at all levels.

Now is a rare moment, a clearing horizon of historic opportunity, for individuals and communities to consider together values and goals and proceed thoughtfully into all areas where health quality and consumer choice are ensured and policy decisions are made. The introduction of multistakeholder partnerships offers much promise to the development of mental health services that are responsive to the needs of all citizens.

References Anthony, W., Cohen, M., and Cohen, B. “Philosophy, Treatment Process, and Principles of

the Psychiatric Rehabilitation Approach.” In L. Bachrach (ed.), Deinstitutionalipation. New Directions for Mental Health Services, no. 17. San Francisco: Jossey-Bass, 1983.

Beisecker, A,, and Beisecker, T. “Using Metaphors to Characterize Doctor-Patient Relation- ships: Paternalism Versus Consumerism.” Health Communication, 1993.5 (l), 41-58.

Blanch, A. “The Community Support Program and Psychology: Trends and Opportunities.” In P. Wohlford, H. F. Myers, and J. E. Callan (eds.), Serving the Seriously Mentally Ill: Pub- lic-Academic Linkages in Services, Research and Training. Washington, D.C.: American Psy- chological Association, 1993.

Blanch, A,, Fisher, D., Tucker, W., and Chassman, J. “Consumer-Practitioners and Psychi- atrists Share Insights About Recovery and Coping.” Disability Studies Quarterly, 1993, 13 (21. 17-20.

Campbell, J. “The Well-Being Project: Mental Health Clients Speak for Themselves.’’ In Third Annual Conference Proceedings on State Mental Health Agency Research. Alexandria, Va.: National Association for State Mental Health Program Directors Research Institute, 1992.

Campbell, J. ‘Values and the Generation of Outcome Measurements.” In Outcome Issues in a Managed Care Environment. Report of the Ninth Annual WICHE Decision Support Con- ference, San Francisco, 1994.

Campbell, J., and Johnson, J. “Struggling to Reach Common Ground.” Behavioral Health- care Tomorrow, 1995,4 (3),40,45-46.

Campbell, J., Ralph, R., and Glover, R. “From Lab Rat to Researcher: The History, Models, and Policy Implications of Consumer/Survivor Involvement in Research.” In Fourth Annual Conference Proceedings on State Mental Health Agency Services Research and Program Evaluation. Alexandria, Va.: National Association for State Mental Health Program Direc- tors Research Institute, 1993.

Campbell, J., and Schraiber, R. In Pursuit of Wellness: The Well Being Project. Sacramento: California Department of Mental Health, 1989.

Carr, W., and Kemmis, S. Becoming Critical: Knowledge Through Action Research. Victoria, Australia: Deakin University Press, 1983.

Chamberlin, J. On Our Own: Patient-Controlled Alternatives to the Mental Health System. New York: Hawthorne Books, 1978.

Chein, I., Cook, S., and Harding, J. “The Field of Action Research.” American Psychologist,

Chesler, M. “Participatory Action Research with Self-Help Groups: An Alternative Paradigm for Inquiry and Action.” American Journal of Community Psychology, 1991, 19 (51,

1948,3,43-50.

757-768.


Chowanec, G., Neunaber, D., and Krajl, M. “Customer Driven Mental Healthcare and the Role of the Mental Healthcare Consultant.” Consulting Psychology journal, 1994, 46 (4), 47-54.

Consumer/Survivor Mental Health Research and Policy Work Group. Focus Group Meeting on Client Outcomes Report 1, 2. Fort Lauderdale, Fla.: Consumer/Survivor Mental Health Research and Policy Work Group, 1992.

Dimsdale, J., Klerman, G., and Shershow, J. “Conflict in Treatment Goals Between Patients and Staff.” Social Psychiatry, 1979, 14, 1-4.

Dumont, J., and Campbell, J. A Preliminary Reportfor the Mental Health Reform Report Card Task Force of the MHSIP Ad Hoc Advisory Group. Rockville, Md.: Center for Mental Health Services, 1994.

Eddy, D. “Clinical Decisionmaking: From Theory to Practice. Connecting Value and Costs. Whom Do We Ask, and What Do We Ask?”Journal of the American Medical Association,

Eddy, D., and Billings, J. “The Quality of Medical Evidence: Implications for Quality of

Elden, M. “Sharing the Research Work: Participative Research and Its Role Demands.” In P.

1989,262, 914-919.

Care.” Health Affairs, 1988, 7 (11, 19-32.

Reason and J. Rowan (eds.), Human Inquiry. New York: Wiley, 1981. Fals-Borda, 0. “Participatory Action Research.” Development:. Seeds of Change, 1984, 2,

18-20. Fenton, J. , Batavia, A., and Roody, D. Proposed Policy Statementfor NZDRR on Constituency-

Oriented Research and Dissemination (CORD). Washington, D.C.: National Institute for Disability Rehabilitation Research, 1993.

Gaventa, J. “Participatory Research in North America.” Convergence, 1988,21 (2/3), 41-40. Gould, B. “Improving Managed Behavioral Healthcare Through Outcomes Management.”

Behavioral Healthcare Tomorrow, 1993,Z (2), 36-40. Kopolow, L. “Consumer Demands in Mental Health Care.” InternationalJournal of Law and

Psychiatry, 1979,2, 46-53. Loder, A., and Glover, R. “New Frontiers: Pioneer Dialogue Between Consumers/Survivors

and Commissioners.” MHSIP Updates, Nov. 1992, pp, 13-14. Lynch, M., and Kmzich, J. “Needs Assessment of the Chronically Mentally 111: Practitioner

and Client Perspectives.” Administration in Mental Health, 1986,4, 237-248. McGuirk, F., Zahniser, J., Bartsch, D., and Engleby, C. “Capturing Outcome Values: Prior-

ities of Key Stakeholders.” In Outcome Issues in a Managed Care Environment. Report of the Ninth Annual WICHE Decision Support Conference, San Francisco, 1994.

Meenaghan, T., and Mascari, M. “Consumer Choice, Consumer Control in Service Deliv- ery.” Social Work, 1971, 16, 50-61.

MHSIP Task Force on the Design of Performance Indicators Derived from the MHSIP Con- tent. Performance Indicatorsfor Mental Health Services: Values, Accountability, Evaluation, and Decision Support. Rockville, Md.: Center for Mental Health Services, 1993.

Minsky, S., Riesser, G., and Duffy, M. “The Eye of the Beholder: Housing Preferences of Inpatients and Their Treatment Teams.” Psychiatric Services, 1995,46 (21, 173-176.

Mirin, S., and Namerow, J. “Why Study Treatment Outcome?” Hospital and Community Psy- chiatry, 1991,42 (lo), 1007-1013.

Mitchell, J., Pyle, R., and Hatsukami, D. “A Comparative Analysis of Psychiatric Problems Listed by Patients and Physicians.” Hospital and Community Psychiatry, 1983, 34 (91,

Osbome, D., and Gaebler, T. Reinventing Government: How the Entrepreneurial Spirit Is Trans- forming the Public Sector. Reading, Mass.: Addison-h’esley, 1992.

Reeder, L. “The Patient-Client as a Consumer: Some Observations on the Changing Profes- sional-Client Relationship.”Journal of Health and Social Behawor, 1972, 13,406-412.

Ridgway, P. The Voice of Consumers in Mental Health Systems: A Call for Change. South Burlington, Vt.: Center for Community Change Through Housing and Support, 1988.

848-849.


Ridgway, P., and Carling, P. A User’s Guide to Needs Assessment in Community Residential Rehabilitation. Boston: Center for Psychiatric Rehabilitation, 1987.

Scott, A. “ConsumerdSurvivors Reform the System, Bringing a Human Face to Research.” Resources, 1993, 5, (11, 3-6.

Sherman, P., and Kaufmann, C. A Compilation ofLiterature on What Consumers Wantfrom Mental Health Services. Cambridge, Mass.: Evaluation Center at Human Services Research Institute, 1995.

Susko, M. (ed.). Cry of the Invisible: Writingsfrom the Homeless and Survivors of Psychiatric Hospitals. Baltimore: Conservatory Press, 199 1.

Tichy, N., and Friedman, S. “Institutional Dynamics of Action Research.” In R. Kilman and C: Thomas (eds.), Producing Useful Information for Organizations. New York Praeger, 1983.

Toombs, S. “The Meaning of Illness: A Phenomenological Approach to the Physician-Patient Relationship.”Joumal of Medicine and Philosophy, 1987, 12, 219-240.

Trochim. W., Dumont, J., and Campbell, J. A Reportfar the State Mental Health Agency Pro- filing System: Mupping Mental Health Outcomesfrom the Perspective of Consumers/Survivors. Alexandria, Va.: National Association of State Mental Health Program Directors Research Institute, 1993.

Wennberg, J. E. “Improving the Medical Decision-Making Process.” Health Affairs, 1988, 7,

Zinman, S., Harp, H., and Budd, S. Reaching Across: Mental Health Clients Helping Each Other. 99-105.

Everside: California Network of Mental Health Clients. 1987.

JEAN CAMPBELL, Ph.D., is research assistant professor in psychiatry at the Missouri Institute of Mental Health, School of Medicine, University of Missouri-Columbia, St. Louis, Missouri.

Documents

Toward collaborative mental health outcomes systems