INTERMEDIATE CARE FACILITIES FOR PERSONS … · PERSONS WITH MENTAL RETARDATION COMPENDIUM OF STATE QUALITY ... Methods And Measures Used In Outcome-Focused Quality ... retardation/developmental

INTERMEDIATE CARE FACILITIES FOR

PERSONS WITH MENTAL RETARDATION

COMPENDIUM OF STATE QUALITY

ASSURANCE SYSTEMS

Prepared by:

Valerie J. Bradley, Co-Project Director; HSRI

Tecla Jaskulski, Jaskulski and Associates

Celia Feinstein, Co-Project DirectorTemple University Institute on Disabilities

Lynda Brown, HSRI

Julie Silver, HSRI

Barbara Raab, HSRI

October 31, 1996

This project was supported by the Health Care Financing Administration, contract number 500-

94-0074. The views expressed herein do not necessarily represent the views or policies of this

agency

.

i

PREFACE

The Compendium of State Outcome-Focused Quality Assurance Systems was developed in the spring of 1995 as part of a project funded by the Health Care Financing Administration (HCFA) under Contract No. 500-94-0074. The Human Services Research Institute was awarded this contract, in collaboration with the Temple University Institute on Disabilities and Jaskulski & Associates, to evaluate the Quality Assurance System for Intermediate Care Facilities for Persons with Mental Retardation (ICFs/MR). Elizabeth Couchoud served as the HCFA Project Officer. The project was comprised of three major tasks:

Task A: Evaluation of the ICFs/MR Program Quality Assurance System

Project staff examined the degree to which individual outcomes such as self-determination, dignity, community integration, individualism, personal growth and health and safety (the outcomes of quality) are present in operative, innovative quality assurance systems. An initial canvass of states was undertaken in order to identify 6-8 exemplary systems for in-depth analysis.

Task B: Identification and Evaluation of Quality Indicators

Project staff examined existing automated databases in order to identify specific data which provide reliable information about the outcomes of quality beneficiaries experience. Because existing databases of quality indicators were generally found to be insufficient in scope to a HCFA quality indicator data base, project staff developed a primary set of quality indicators suitable for incorporation into such a data base and designed a plan for the development and testing of the quality indicators.

Task C: Establish a Baseline of Current Psychoactive Drug Utilization in ICFs/MR

Project staff determined the extent to which psychoactive drugs are being utilized in a representative sample of large and small ICFs/MR.

The final report on these tasks was submitted to HCFA in June 1996, including recommendations for HCFA to consider in making changes in the ICF/MR quality assurance system as well as extensive analysis of the outcome-focused QA systems in the eight state selected for in-depth analysis (Colorado, Massachusetts, Missouri, New York, Oklahoma, Oregon, Utah and Wyoming).

This compendium was initially prepared as an interim report on Task A. It built on previous collections of data on State QA systems, in particular the work done by the Temple University Institute on Disabilities for the City of Philadelphia, provided by Celia Feinstein. Charlie Lakin, University of Minnesota Institute on Community Integration and a member of the Technical Working Group, provided information collected through that agency's Reinventing Quality publication series. Updates of these materials and other information on State QA systems previously gathered by HSRI and by Jaskulski & Associates provide an interesting snapshot of practices and issues as of early 1995.

ii

TABLE OF CONTENTS

PREFACE i

I. OVERVIEW 1

State Of The States Analysis 1

What We Looked At 2

Methods Used In Reviewing State Quality Assurance Systems 3

Summary Of Key Findings 4

Issues 5

Implications For The ICF/MR Quality Assurance Project And For ICF/MR Quality Assurance Reform 8

II. THE MOVE TO OUTCOME-FOCUSED QUALITY ASSURANCE 9

The Degree Of Quality Assurance Reform 9

Quality Assurance And System Reform Parallels 13

Selection Of Quality Outcomes 14

Characteristics Of Outcome-Focused Quality Assurance Systems 17

Interface With Licensure And Certification 17

Interface With Private Accreditation Bodies 17

How Outcome-Focused Quality Assurance Systems Are Being Developed 20

Scope Of Facilities/Residential Services And Populations Covered 22

Methods And Measures Used In Outcome-Focused Quality Assurance/Quality Enhancement Systems 25

Special Features Of Outcome-Focused Quality Assurance /Quality Enhancement Systems 29

Future Plans And Emerging Trends 30

III. THE RELATIONSHIP BETWEEN HEALTH, SAFETY, INDIVIDUAL RIGHTS AND

OTHER QUALITY OUTCOMES 36

IV. USING OUTCOME-FOCUSED QUALITY ASSURANCE IN ICF/MR FACILITIES

OR WITH SIMILAR POPULATIONS 38

V. DATA COLLECTION AND ANALYSIS ON THE PRIMARY OUTCOMES 41

Quality Indicators 44

VI. CONCLUSION 46

ii

TABLE OF EXHIBITS

EXHIBIT 1: CATEGORY OF QUALITY ASSURANCE REFORM 11

EXHIBIT 2: SELECTION OF PRIMARY OUTCOMES 15

EXHIBIT 3: IMPLEMENTATION OF PRIMARY OUTCOMES 19

EXHIBIT 4: USE OF RELIABILITY AND/OR VALIDITY ANALYSES 21

EXHIBIT 5: SCOPE OF RESIDENTIAL SERVICES AND POPULATIONS COVERED 23

EXHIBIT 6: METHODS USED IN OUTCOME-FOCUSED SYSTEMS 27

EXHIBIT 7: PLANNED QA REFORM ACTIVITIES AS OF EARLY 1995 32

EXHIBIT 8: STATUS OF PRIMARY OUTCOME DATA 42

1

I. OVERVIEW

The Human Services Research Institute (HSRI) conducted extensive research on state quality assurance systems for residential services to people with mental retardation/developmental disabilities, from October 1994 - April 1996. This work was carried out in collaboration with the Institute on Disabilities at Temple University and with Jaskulski and Associates. The work was undertaken for the Health Care Financing Administration (HCFA), to support their evaluation of the Intermediate Care Facilities for Persons with Mental Retardation (ICFs/MR) Quality Assurance System and plans for ICF/MR Quality Assurance reform.

STATE OF THE STATES ANALYSIS

Task A of the project called for research on the "state of the States" in quality assurance (QA) systems, in particular their focus on selected quality outcomes (primary outcomes). Based on this analysis, project staff proposed eight states for in-depth analysis of outcome-based quality assurance systems, including on-site validation. This analysis is covered in a separate report. As suggested by the project's Technical Working Group and approved by HCFA, the primary outcomes of interest are as follows:

Self-determination Individualism

Dignity Personal Growth

Health and Safety Community Integration/Inclusion

Consumer Satisfaction Relationships/Social Connections Throughout the report, the phrase “primary outcomes” is used to refer to the outcomes identified by the Technical Working Group and outlined by HCFA in the request for proposal for this project. The Technical Working Group included representatives of major stakeholder organizations such as the Arc, The Accreditation Council, American Association on Mental Retardation, American Network of Community Options, American Health Care Association, Association of Public Developmental Disabilities Administrators, Commission on Accreditation of Rehabilitation Facilities, National Association of Developmental Disabilities Services Directors, National Association of Developmental Disabilities Councils as well as the Office of the Assistant Secretary for Planning and Evaluation of the Department of Health and Human Services, the HCFA Office of Research and Demonstrations, the Medicaid Bureau, ICF/MR program staff, and the project team. This report is a compendium of the State quality assurance systems as of spring 1995. The review was undertaken at a time when the field of mental retardation/developmental disabilities was experiencing a major shift in its approach to services, and a corresponding shift in the approach to quality assurance. Although there remains considerable variation among the States, virtually every State has significantly expanded the availability of

2

community residential services, has greatly reduced its use of institutional services, and has developed at least some individualized supported living arrangements. In mission statements, legislation and service standards, more and more States are defining desired outcomes for people with mental retardation/developmental disabilities in terms of power, choice, community membership, and consumer satisfaction, in addition to the traditional benefits of individually-planned services, quality health care, and safe and healthy environments. Trends in quality assurance reform reflect the evolution of services:

When there were few community-based services and services emphasized improving conditions in institutions, and quality assurance methods emphasized environmental factors (sanitation, meals, numbers of individuals per toilet) and staffing ratios.

When the emphasis shifted to individual development and potential movement into the community, quality assurance approaches that focused on the process of individualized program planning (interdisciplinary assessment and team planning, monitoring of compliance with the individual plan) and access to specialized professional services based on individual need were added to environmental and staffing standards..

As services shifted further to community membership and participation, empowerment, and "quality of life," quality assurance systems began to encompass techniques that looked more closely for outcomes associated with well-being and consumer satisfaction, as well as focusing on outcomes of individualized planning and planning processes.

As services increasingly are seen as supports to help individuals reach their personal (and personally-chosen) goals, with service recipients defined as customers to be satisfied rather than recipients of professionally-determined services, quality assurance system designers look not only at outcomes defined by the service system, but at outcomes defined by and with each individual, and measured through the lens of each person's attainment of desired outcomes.

This "state of the States" in quality assurance as of early 1995 reflects vestiges of all four of these approaches, but with a clear momentum toward outcome-focused quality assurance mechanisms that assess quality of life outcomes as well as essential health, safety and individual rights.

WHAT WE LOOKED AT

The focus of this "snapshot" was on the extent to which the primary outcomes identified by the Technical Working Group and approved by HCFA as the focus of inquiry are being used in State quality assurance systems for residential services to people with mental retardation and related disabilities. Although particularly targeted to quality assurance in

3

relation to standards for compliance, our review of quality assurance systems included quality enhancement (QE) activities as well; as used in this report, the terms "Quality Assurance" and "Quality Assurance system" are intended to include quality enhancement as well as compliance and other traditional quality assurance concepts. Information was collected on the inclusion of primary outcomes and measures in State quality assurance systems; the development of outcome-focused quality assurance systems; States’ perspectives on the applicability of primary outcomes and outcome-based quality assurance systems to institutions, ICFs/MR and individuals with severe disabilities; the States’ identification of quality indicators; and the States’ views on the effectiveness of outcome-focused quality assurance. In addition to collecting information for this compendium, we also considered States for in-depth review (case studies and on-site validation) later in the project.

METHODS USED IN REVIEWING STATE QUALITY ASSURANCE SYSTEMS

A three stage process was used to collect information about State quality assurance systems. These stages were:

Review of existing data: Including report and data base of outcome-focused quality assurance systems compiled by Temple University Institute on Disabilities for the City of Philadelphia in 1994, Reinventing Quality volumes I and II (University of Minnesota, 1993, 1994), and materials previously collected by HSRI and Jaskulski & Associates.

Telephone discussions with one or more knowledgeable individuals in each State to obtain an update on their quality assurance system

Review of written materials provided by States as a follow-up to the telephone discussion

At least one person in each of the 50 States and the District of Columbia was contacted for this part of the project. The majority of primary informants were directors of quality assurance within the State MR/DD program agency (33 States); other were directors of State MR/DD agencies (7 States) and staff with administrative responsibilities for community residential services (7 States). Additional informants included State mental retardation/developmental disabilities service directors, State and non-state staff with responsibility for data collection and analysis, coordinators of quality assurance demonstration projects, and other State officials with service system responsibilities. Specific questions asked of these contacts were: 1) To what extent are the primary outcomes or equivalent being used?

Where are they being used? Settings/program funding (ICFs/MR, other residential settings; auspices; size) Populations being served

4

Statewide or regional How are the primary outcomes identified by the Technical Working Group being used? Application methods Relationship to licensure and certification, other aspects of QA Relationship to health and safety, protection of individual rights Relationship to incentives and adverse actions

2) What data are collected on primary outcomes? 3) What primary outcome measures are being used?

Sources Reliability and validity

4) To what extent are quality indicators being used?

What quality indicators have been defined? How are they being used?

5) How effective are outcome -focused systems? What is the rationale for such systems? Evaluation results Future plans

Efforts were made to obtain information across the study questions through use of a discussion guide, however, there was less than 100 percent consistency in what was covered. In particular it should be noted that tallies of responses indicate the information volunteered by the respondents. It should not be assumed that States which did not mention a particular quality assurance reform activity, outcome-focused quality assurance component or perspective are not engaged in reform or pursuing outcomes-focused quality assurance systems. This report has not been provided to the State respondents for verification. It is based on our best understanding of their activities and perspectives elicited through informal telephone discussion and review of any documents provided. Overall, however, there was sufficient information to obtain a current profile of State approaches to outcome-focused quality assurance in residential services for people with mental retardation/developmental disabilities as of early 1995.

SUMMARY OF KEY FINDINGS

The "state of the States" of quality assurance systems for residential services to people with mental retardation/developmental disabilities appears to include the following:

Quality assurance systems in motion and at various stages of reform, with a clear trend toward outcome-focused quality assurance systems.

5

Widespread application of the primary outcomes identified by the Technical Working Group and of outcome-focused quality assurance to individuals with severe and multiple disabilities (those with the same level of disability and characteristics as ICF/MR beneficiaries).

Exclusion of ICFs/MR from most outcome-focused quality assurance systems, primarily because of the traditional separation between ICF/MR quality assurance and non-ICF/MR quality assurance.

General use of the primary outcomes defined by HCFA and the Technical Working Group.

Decreasing focus on document review with a corresponding increasing focus on observation and interviews, in particular increased contact with service recipients.

Continuing use of process measures in outcome-focused quality assurance reviews, but with greater emphasis on processes that are seen as directly tied to specific primary outcomes.

A mixed picture as to how much "traditional" quality assurance is being retained as primary outcomes, in particular those associated with quality of life.

General trend toward reduction of prescriptive rules and standards and development of streamlined licensure and certification procedures.

Increased focus on individuals, their satisfaction, and choices, as well as an emerging approach with individuals as the organizing principle of the quality assurance systems, (and the service system) rather than provider agencies.

Consensus on quality assurance reform growing among stakeholders in most States, and recognition of the importance of involving stakeholders in developing and building consensus on outcome-focused quality assurance systems.

Emerging trends: increased decentralization of quality assurance activities, tying quality assurance reforms to managed care and using cross-disability quality indicators.

Overall, there is strong interest and support for outcome-focused quality assurance systems in residential services, for individuals at all levels of mental retardation/developmental disability, and in all settings.

ISSUES

6

Relationship to traditional health and safety concerns

State perspectives on quality assurance reform also reflect an awareness of various issues to be addressed in moving to an outcome-focused quality assurance system. Several State respondents noted, in various forms, that outcome measures and outcome-focused methods need to be supplemented with traditional (and streamlined) health and safety assurances. There was a deep sense of responsibility for protecting essential health, safety and individual rights of a vulnerable population. Although specific quality assurance strategies varied, there were only a few States involved in quality assurance reform that did not articulate the need for special attention to health and safety, and for retention of some traditional standards and monitoring techniques. This perspective was expressed by States moving to the vanguard of quality assurance methods as well as States which were hesitating to move ahead to an outcome-focused system because of their concerns about health and safety protections.

Health and Safety Risks

Some States brought up a related issue: that new service and quality assurance models based on consumer choice required checks and balances in protecting people from risks. In addition to public sector accountability, there were also comments regarding issues of provider liability and the potential political backlash against community services and support models if a harmful incident occurred and was publicized. At the same time, some States recognized that outcome-focused monitoring, especially when the quality assurance system includes citizen monitoring and an emphasis on community membership and relationship outcomes, can be effective in preventing health and safety problems, as well as in their early identification and intervention.

Outcomes versus Current Service Practices

Some States acknowledged the difficulty of shifting to measuring outcomes, especially when the desired outcomes reflect new service approaches (e.g., person-centered planning, relationship and community membership goals, individual support models). From the tools available for review, it is clear that process measures continue to be used extensively, even when desired outcomes are clearly stated as the basis.

Outcomes and People with Severe Disabilities

Another issue addressed by several States was the need for careful consideration of the primary outcomes for people with severe disabilities and limited or challenging communication. This issue is highlighted in quality assurance systems which focus on choice and consumer satisfaction, and which rely extensively on interviews to measure the primary outcomes. Although most respondents felt that this challenge can be addressed successfully, there was consensus on the importance of dealing with it in defining quality

7

assurance methodologies, in training surveyors, and in analyzing quality assurance results in relation to individual characteristics.

Relevance to Institutions

Questions also were raised about the applicability of the primary outcomes and outcome-focused quality assurance systems to institutions and to ICFs/MR, both large facilities and smaller community-based units. Among the State respondents who commented on this issue, there was overall consensus that outcome-based standards should and can apply to all settings. Some indicated, however, that outcome-focused quality assurance methods need some adaptations, such as flexibility in quality assurance guidelines on choice and community integration/inclusion, when applied in large institutional settings. These observations were based on the nature of institutions, and not on the characteristics of those receiving the services. Further, a few States noted the inherent conflict the primary outcomes create in evaluating quality in ICF/MR programs. The right to choice is necessarily impinged upon in the ICF/MR system where consumers cannot choose a less restrictive setting.

Reliability and Validity Concerns

A few States reported questions about the reliability and/or validity of outcome measures, from either the State or provider perspective. Overall, however, there was little concern in States which had not previously conducted reliability and validity tests. This may be partly due to the work most States have done prior to formal adoption of outcome-focused quality assurance systems involving stakeholders in designing and developing the system, building consensus on quality assurance reforms, and, in several States, reviewing quality assurance pilots.

Resource Constraints

Finally, resource constraints were mentioned by several States as they consider service system and quality assurance reform. Some States reported that State quality assurance personnel have been cut back, and that they are falling further behind in maintaining their survey schedules. In a few States resource constraints have been part of the impetus for streamlining the licensure standards and survey, and for decentralizing or privatizing quality assurance programs. Some States also are concerned about provider response to outcome-focused quality assurance if it becomes costly for providers to participate in quality assurance activities and to reach the desired outcomes. A related emerging issue in a few States is the pressure to make sure that every service recipient is receiving the services the providers are paid to deliver, a concern that lends itself to traditional documentation review rather than outcome-focused quality assurance. Because resource constraints are likely to continue in the near term, issues of cost-effectiveness in quality assurance approaches are likely to find voice in the discussion and consideration of systems reform.

8

IMPLICATIONS FOR THE ICF/MR QUALITY ASSURANCE PROJECT AND FOR

ICF/MR QUALITY ASSURANCE REFORM

The review of State quality assurance systems supports the presumption that relatively few States that are using strong outcome-focused systems also have data on the primary outcomes, service recipients, and additional health and safety measures. Sufficient information was available, however, to develop recommendations for six case study States, for in-depth analysis. There is a general lack of data, empirical investigation and analysis of the experiences with outcome-focused quality assurance. The subsequent case studies and verification review of State outcome-based quality assurance systems provide a major opportunity for HCFA and the field to learn more about what is working, as well as the applicability to ICFs/MR and to individuals with similar characteristics as ICF/MR beneficiaries. The case studies include application of State quality assurance instruments in ICFs/MR. The experience of States which are using the primary outcomes in their State institutions (Colorado, Massachusetts and Oklahoma) will be thoroughly examined. It is apparent, however, that many consumers similar to ICF/MR service recipients are already covered by outcome-focused quality assurance systems, even if few ICF/MR-MR certified facilities are involved. The special place given to health and safety outcomes confirms the need to focus on the ways States attempt to measure these outcomes and the related procedures used to maximize positive health and safety outcomes. There is an additional need to look at the extent to which other primary outcomes and findings on outcomes in general correspond to indicators of health, well-being, safety and protection of individual rights. This was examined in each of the case study States. As well, States are interested in looking for the processes that are associated with the primary outcomes as well as the outcomes themselves. In some cases, systems and instruments that are described as measuring outcomes appear to focus more on identifying the presence of processes rather than the presence of outcomes. This was pursued in analyzing the actual practices in the case study States, as well as the implications for ICF/MR quality assurance reforms. Finally, the issue of choice can be addressed in the case study States particularly when the issue of freedom of choice may mean an alternative to ICF/MR services and choice of a different provider. Attention was paid to the implications for quality assurance reform in person-centered planning, expression of personal choice as a primary outcome , and consumer satisfaction outcomes.

9

II. THE MOVE TO OUTCOME-FOCUSED QUALITY ASSURANCE

There is a clear trend nationally toward outcome-focused quality assurance systems in residential services for individuals with mental retardation/developmental disabilities. There are only a handful of States that are not at least actively considering how to use outcome measures as part of their quality assurance systems. Although the specific impetus varies, there is a general sense that traditional quality assurance, in particular comprehensive licensure and certification surveys, focuses too heavily on environment and process and not enough on outcomes for the individual (consumer) or on "quality of life" issues. In some States, quality assurance reform is associated with litigation and the movement of large numbers of individuals from State institutions to the community (e.g., Connecticut, Oklahoma, Washington, Wyoming). In other States, quality assurance reform that includes the primary outcomes may be tied to an interest in regulatory reform and an emphasis on making State quality assurance more cost-efficient (e.g., Ohio). Across the States there appears to be a relationship between the evolution of the State's mental retardation/developmental disabilities service system and the degree of quality assurance reform toward an outcome-focused system.

THE DEGREE OF QUALITY ASSURANCE REFORM

Three categories were developed to characterize the degree to which quality assurance reform approaches an outcome-focused system, as well as two "transition" categories. The need for the "transition" categories became apparent in the discussions with the States, reflecting both the strong interest in quality assurance reform and the current fluidity of quality assurance nationally. Several States also reported that they were projecting changes for later this year. As noted above, almost every State is actively moving toward an outcome-focused quality assurance system. The categories and number of States in each category as of spring 1995 are as follows:

Category I: Significant movement toward outcome-focused quality assurance:

(10 States)

This category includes States that may not have completely eliminated traditional quality assurance, comprehensive licensure and certification (L&C) surveys, etc., but include definitive outcome focus and systems include the primary outcomes of interest. These States may not have included all settings (e.g. their ICFs/MR and state institutions), but are using outcome-focused quality assurance in settings serving individuals with the same characteristics (e.g., severe and profound mental retardation, complex disabilities).

10

Transition II to I: States actively in transition to an outcome-focused system:

(12 States) This category includes States that have made the decision to reform their quality assurance systems, and the reform is underway. For example, some States in this category are conducting a demonstration to determine how to implement such a system.

Category II: Some reform aimed at outcome-focused quality assurance including

the primary outcomes of interest: (9 States) States in this category may be applying the primary outcomes in some form of a demonstration project, such as a pilot in one part of the State that may become the future model, or an outcome-focused quality enhancement program such as citizen monitoring, or an outcome-focused quality assurance that applies only to one type of residential services (for example, supported living - if possible, however, one that includes at least some individuals with same characteristics as ICF/MR residents.) These States continue to rely on traditional licensure and certification process.

Transition III to II: Beginning the process of developing some aspects of an

outcome-focused quality assurance system: (5 States) States in this category are actively planning a pilot project, may have a task force that has recently issued its first report, new legislation that will require them to move ahead with reform, but are not committed, as yet, to total quality assurance reform..

Category III: Little or no movement toward outcome-focused quality assurance

reforms: (12 States) These States may have a task force beginning to address the issues, a pilot that fizzled, some token activities focused on quality of life measures, but are still relying primarily on traditional quality assurance.

Other: (3 States) In this category are States with deemed status provisions (CARF and/or Accreditation Council) [2 States] and one that is moving from Category I to Category II. Exhibit 1 on the following pages illustrates the category of quality assurance reform for each State.

11

EXHIBIT 1:

CATEGORY OF QUALITY ASSURANCE REFORM

State I II>>>I II III>>>II III Other

AL X

AK X

AZ X

AR X

CA X

CO X

CT X

DE X

FL X

GA X

HI X

ID X

IL X

IN X

IA X

KS X

KY X

LA X

ME X

MD X

MA X

MI X

MN X

MS X

MO X

12

EXHIBIT 1 (CONTINUED)

State I II>>>I II III>>>II III Other

MT X

NE X

NV X

NH X

NJ X

NM X

NY X

NC X

ND X

OH X

OK X

OR X

PA X

RI X

SC X

SD X

TN X

TX X

UT X

VT X

VA X

WA X

WV X

WI X

WY X

DC X

TOTAL 10 12 9 5 12 3

13

QUALITY ASSURANCE AND SYSTEM REFORM PARALLELS

There appears to be a parallel between moves toward an outcome-focused quality assurance system and the extent of service system reforms based on a consumer-centered support model, and a de-emphasis on institutions and facility-based services. This can be illustrated as follows:

Service Orientation Quality Assurance Systems Traditional Services Traditional Quality Assurance Partial Shift to Supports Model Traditional Quality Assurance and some Quality Enhancement for priority outcomes Few/no ICFs in Community Limited Licensure and Certification Primary Outcomes review important Few/no large State ICFs Minimal Licensure and Certification for health and safety Primary Outcomes review central Support model/ $ direct to individual Little/no Licensure and Certification All Quality Assurance based on individual This trend is particularly noticeable in States that have reoriented their mental retardation/developmental disabilities service system to a support model and have embraced values of consumer choice, quality of life and well-being as the core values of their systems. Colorado, for example, is attempting move from facility-based services (employment and residential services) to individually-tailored supports and consumer choice. The effectiveness of the system is measured in relation to outcomes for individuals defined in the mission statement of the Colorado Division for Developmental Disabilities, as follows:

The mission of the Colorado Division for Developmental Disabilities is to join with others to offer the necessary supports with which all people with developmental disabilities have their rightful chance to:

14

Be included in Colorado community life.

Make increasingly responsible choices.

Exert greater control over their life circumstances.

Establish and maintain relationships and a sense of belonging.

Develop and exercise their competencies and talents.

Experience personal security and self-respect. Consumer satisfaction also is assessed in Colorado's ongoing evaluation of its progress towards the reform mission. The Alaska system has moved almost entirely to a support and consumer-based system. The director of the State developmental disabilities agency feels strongly that the reforms in quality assurance are driven by the reforms in the service system. Because resources go to individuals and families, the State feels that the appropriate quality assurance measures are their satisfaction with the services and supports they are receiving.

SELECTION OF QUALITY OUTCOMES

States that have moved toward an outcome-focused quality assurance system and have defined important outcomes as part of that effort have selected outcomes that are remarkably consistent with those selected by HCFA and the Technical Working Group (the primary outcomes). Based on a review of quality assurance instruments and materials from 30 States with defined important outcomes (including some States that have changed but have not completely re-oriented their quality assurance systems), the outcomes

appearing most frequently in State quality assurance efforts are: Individualization (30

States) and Integration/Inclusion (29 States). The primary outcome found least

frequently is Consumer Satisfaction (19 States). Based on additional information, however, it appears that consumer satisfaction is a part of most other States' quality assurance systems, even if not specified as one of their primary outcomes. The number of States for all eight primary outcomes are illustrated in Exhibit 2.

15

EXHIBIT 2:

SELECTION OF PRIMARY OUTCOMES

Primary Outcome

Number of States

How Primary Outcome Typically Defined

Individualization

30 Services based on individual choices, preferences, needs - primary emphasis on choice (22 of the 30)

Integration/Inclusion 29 Community presence, participation, membership; social inclusion

Relationships/Social Connections

26 Relationships with friends and family

Health and Safety 25 Health and safety; well-being; personal security

Personal Growth

25 Various - personal growth, competence, attainment of individual goals, skill acquisition

Self-determination 25 Various - control, choice-making, decision-making, power, independence

Dignity 24 People treated with respect and dignity; age-appropriate services (some)

Consumer Satisfaction 19 Consumer satisfaction

In looking at the consistency between the study's primary outcomes and those being used in State quality assurance systems, the greatest challenge is clarifying Individualization and Self-Determination outcomes. Individualization was identified when outcomes and indicators referred to services and supports that were tailored to the needs, choices and/or preferences of each individual. Self-determination was identified when outcomes and indicators reflected the individual's exercise of choice, as well as increasing independence and control over decisions. In most of the 30 States, the central concept for

Individualization and Self-Direction was choice. Definitions and criteria for Individualization, for example, included:

Personal choice

Chooses daily routine; makes choices

People choose where and with whom they live

16

Opportunities to make choices in pursuit of a personal future

Services are built around what individual wants and needs

Person-centered planning facilitates empowerment of individuals to attain personal goals

Supports are flexible; ask for and listen to desires and needs

Making choices that are meaningful and express individual identity

Personal goals

Services build on individual strengths and are anchored in clear understanding of each person's needs and preferences

Services/lifestyles reflect cultural preferences

Many States combine empowerment and choice in a single value statement or outcome, with measures and criteria that intertwine Individualization and Self-determination concepts as a result. When personal choice outcomes without additional reference to control, empowerment, decision-making, etc., are included under Self-determination, then the total number of States examining a Self-determination outcome increases to 29. Although there is strong overall consistency among the States and between their emphasized outcomes and the primary outcomes focused on in the study, there are also some important outcomes under examination in the States that were not on the list of eight primary outcomes. Altogether, 20 States included one or more of these desired outcomes in their quality assurance efforts:

Productivity, employment (7 States)

Economic security, sufficient resources to realize goals (7 States)

Exercise rights and responsibilities (3 States)

Home comfortable, nice appearance of home (3 States)

Consumers have increasing choices among providers (2 States)

Opportunities to increase status; participation in socially valued roles (2 States)

Access to advocacy (2 States)

Training in natural context; receiving appropriate generic services and supports (2 States)

17

Many of these outcomes could be "folded in" in some way to one or more of the eight primary outcomes. For example, "access to advocacy" could be an outcome indicator for Health and Safety. However, outcomes of productivity/employment and economic security/adequate personal resources are examples of important outcomes that reflect categorically different values and goals than the eight primary outcomes. Likewise, individual choice among providers, along with similar values of freedom of choice and competition among providers, seems to go beyond the concepts in Individualization and Self-determination. Although defined as a primary outcome by only two States, the goal of consumer choice among providers, and the related criticism of current quality assurance as not promoting freedom of choice, was cited by additional States. Although not included in formal quality assurance measures, Utah publishes "consumer reports" on residential and other service providers, to help individuals and family members make choices. This is consistent with increasing emphasis on empowerment throughout the disability field, including the vision of people being able to choose the programs they participate in rather than being limited to making choices within the program/facility.

CHARACTERISTICS OF OUTCOME-FOCUSED QUALITY ASSURANCE SYSTEMS

Nineteen States have experience with an outcome-focused quality assurance system or are using outcomes as a critical component of their overall quality assurance program, i.e., for quality enhancement (QE). As illustrated in Exhibit 3 this focus on outcomes includes:

Four States using outcome-based reviews as their primary quality assurance mechanism,

Nine States using outcome-based reviews in conjunction with licensure and certification (L&C) surveys or equivalent,

Two States using outcome-based reviews primarily for quality enhancement(although typically as a requirement of provider participation),

Three States with a quality assurance demonstration project based on outcomes, and

One State using outcome-based reviews for program and policy evaluation.

INTERFACE WITH LICENSURE AND CERTIFICATION

As reflected in Exhibit 3, only a few States have replaced all or most traditional compliance monitoring and licensure/certification surveys with an outcome-focused review. Even among some of the "Category I" States (see Exhibit 1), there is still a licensure and certification process - typically one that has been "streamlined" to focus on critical health, safety and rights issues.

INTERFACE WITH PRIVATE ACCREDITATION BODIES

18

A few States that have adopted an outcome-focused system work with the Accreditation Council (AC) or the Council on the Accreditation of Rehabilitation Facilities (CARF), or give "deemed status" to provider agencies that are accredited by either Accreditation Council or CARF. North Dakota and South Dakota are in effect shifting to a more outcome-focused quality assurance system through deemed status of Accreditation Council accreditation, where provider agencies are increasingly surveyed under the 1992 Accreditation Council standards. Wyoming, which requires CARF accreditation, is working with that body to pilot new, more outcome-focused, standards and methodology. Montana, which gives deemed status to both Accreditation Council and CARF accreditations, has participated in the piloting of the proposed CARF standards and has had several residential service provider agencies undergo Accreditation Council reviews with the 1992 Accreditation Council standards. In Oklahoma, the State bases its licensure standards on the Accreditation Council standards (1990 version), and plans to adopt the 1992 outcome-focused standards within the next year. Agencies in Oklahoma may also be accredited by the AC. Those scoring 85 percent or higher are deemed to be in compliance with State licensure standards, and are granted a two-year license. Additionally, providers must participate in the longitudinal outcome assessment required by the Hissom suit and consumer/citizen monitoring through OK-AIM.

19

EXHIBIT 3:

IMPLEMENTATION OF PRIMARY OUTCOMES

State How Outcome-Focused System Implemented

Other Considerations/ Related Components

Year Adopted

AK QA + minimal licensure and certification (L&C)

Assessment based on each individual's desired services

1991

CO NA - used as policy/program evaluation of MR/DD system

Streamlined L&C system is used for quality assurance along with case management

1986

FL

Pilot in one State district adapted from Accreditation Council outcome standards/methods

"Desk audit" of agency's compliance with L&C, other inspections

1994

ME

QA + very minimal L&C Major focus on consumer satisfaction, in conjunction with case management

1994

MA QA (QUEST, adapted from AC, replaced all quality assurance /QE surveys)

1994

MN QA (demonstration project in 5 counties)

1995

MO

QA/QE (compliance/QA with core standards. on health & safety, rights;quality enhancement of rest of primary outcomes, quality enhancement plan required and monitored)

Also give deemed status to agencies accredited by either Accreditation Council or CARF

1995 (piloted 1993-1994)

NH QA Mixed

NY

QA/QE (COMPASS demonstration project with 24 agencies - to evaluate as option to State L&C of non-Medicaid residential services)

Would be option only to agencies with good records - others would continue with regular State L&C

1993

OH

QA - streamlined L&C that focuses on some outcomes in observation and interview process

Comprehensive L&C survey performed if surveyor has concerns

1993

OK QA/QE Require AC, CARF and/or State L&C in addition to other outcome-focused reviews

Mixed

20


State How Outcome-Focused System Implemented

Other Considerations/ Related Components

Year Adopted

OR QA/QE - Provider Self-reported outcome data (ROS, replacing VOIS/ORLIS) + State "Probe" survey focused especially on health & safety

Encourage participation in quality enhancement based on Continuous Quality Improvement (CQI); also have citizen monitoring

ROS: 1995 VOIS/ ORLIS: 1986

RI QE + streamlined L&C and separate Human Rights Monitoring system

In process

TN QA 1992

UT QA + streamlined L&C 1991

VT QA/QE + streamlined L&C Multiple "audits" including outcome assessment

unknown

WI QA + streamlined L&C unknown

WA QA + health & safety L&C Based on primary outcomes, but focuses on process in actual survey

1990

WY QA (piloting new CARF outcome-focused standards.) + existing L&C

Also using Residential Lifestyle Inventory (adapted from Oregon's ROS on valued outcomes), consumer satisfaction

1993 (RLI) 1994 (CARF outcome survey)

HOW OUTCOME-FOCUSED QUALITY ASSURANCE SYSTEMS ARE BEING

DEVELOPED

Some States volunteered information on how their outcome-focused quality assurance system was developed. Commonly, a task force of stakeholders was instrumental in defining outcomes, measures and procedures in States that had developed outcome-focused quality assurance systems. These task forces typically included providers, consumers, family members and other advocates, State officials, and local officials in States with decentralized service systems. Another common approach was to pilot, prior to adoption, new proposed standards, instruments, and procedures. Reliability and validity testing and empirical data analysis are much less common for outcome-focused instruments. States which reported that such testing and analysis are performed are listed in Exhibit 4.

21

EXHIBIT 4:

USE OF RELIABILITY AND/OR VALIDITY ANALYSES

State Reliability, Validity or Other Empirical Review

CT Reliability and validity check of quality assurance instruments (limited focus on outcome)

FL Adapted measures from Accreditation Council 1992 which have undergone reliability analysis

MA Reliability analysis done as well as field test of QUEST measures done in State institutions, plus statistical methodological consultation on sampling procedure

MO Reliability analysis in process on newly adopted outcome measures

NM Reliability and validity analysis of earlier version, reliability of current version tested

NY Factor analysis of COMPASS indicators plus comprehensive evaluation of effectiveness in relation to compliance with standards

OK Reliability and validity tested

OR Some reliability and validity testing reported

SD Using Accreditation Council 992 (deemed status - also Montana and North Dakota)

UT Validity and item analysis performed

VT Ongoing reliability checks

Only two States cited weakness in reliability and validity as an issue. The apparent lack of conflict over reliability and validity may reflect States' ability to build consensus in developing outcome-focused quality assurance systems and measures. Several States have used teams of stakeholders, including providers, advocates and consumers, to define desired outcomes and to develop and approve outcome measures and procedures. Surveyor training also was mentioned by some States as an important component in moving to an outcome-focused quality assurance system, which may further enhance reliability even if not formally tested and verified.

22

SCOPE OF FACILITIES/RESIDENTIAL SERVICES AND POPULATIONS COVERED

Most States that are using outcome-focused quality assurance/QE in a systematic fashion exclude ICFs/MR from the process. The most common reason given for this exclusion is that these facilities are certified by a different State agency (i.e., the Health Department). As illustrated in Exhibit 5, however, some States do include ICFs/MR in their quality assurance activities, notably:

Ohio uses the same licensure and certification outcome-focused survey on all residential service agencies, including State institutions certified as ICF/MR facilities;

Massachusetts uses its QUEST tool as a quality enhancement effort in its large State ICFs/MR;

Oregon requires participation in the Residential Outcomes Survey (ROS) for community ICFs/MR; and

Colorado uses the COPAR outcome assessment on a sample of service recipients that includes some ICF/MR residents (including State facility residents).

In addition, Minnesota is piloting a demonstration outcomes focused quality assurance system (adapted form the 1992 Accreditation Council standards) in ICFs/MR with the intention of revamping its licensure and certification processes.

States with outcome-focused quality assurance systems apply measures in agencies serving significant numbers of individuals with the same characteristics as ICF/MR residents (severe or profound levels of mental retardation and multiple disabilities). With the exception of the COMPASS demonstration in New York State, the outcome-focused systems are applied to Home and Community Based (HCB) waiver service providers as well as to non-Medicaid units serving similar individuals. In the New York demonstration, 31 percent of individuals on whom outcomes were measured have diagnoses of mental retardation at the severe or profound level, despite the focus of COMPASS on agencies licensed by the State Office of Mental Retardation and Developmental Disabilities, rather than Medicaid providers. In Washington, 500 of the 900 former State institution residents whose outcomes are now being monitored in community residences are individuals with severe or profound mental retardation.

Overall, there was a strong consensus among these States that outcome-focused quality assurance systems, including in particular the focus on consumer choices, quality of life and consumer satisfaction, are appropriate for all people, including those with severe disabilities.

23

EXHIBIT 5:

SCOPE OF RESIDENTIAL SERVICES AND POPULATIONS COVERED

State Scope of Residential Services Covered

Coverage of Individuals with Same Characteristics as ICF/MR Residents

Additional Comments

AK All but ICFs/MR (one small State facility and five small private ICFs/MR)

Yes

CO

Includes State institutions (ICFs/MR) but not community ICFs/MR except indirectly; adults only

Yes

Because COPAR survey includes sample of community day program participants, some community ICF/MR residents included as well as State institution residents

FL Pilot demo in one State HRS District; includes HCB waiver but no ICFs/MR

Similar functioning level, however, ICFs/MR serve those who are medically fragile

ME All community residences

Yes

MA

All community residences, including many HCB waiver (State has no community ICFs/MR); using in State institutions for quality enhancement and piloted QUEST in State institution

Yes

MN Demonstration project with 5 ICF/MR providers in 5 counties

Yes, although community ICFs/MR may serve somewhat less severe/complex disabilities than State institutions

Participating agencies will apply the new quality assurance to 150 ICF/MR residents + an additional 300 in HCB or Supported Independent Living services

24


MO Primary focus on HCB waiver services

Yes

NH All community residences (NH has no State institution)

Yes Mixed

NY

COMPASS demonstration project with 24 OMRDD-licensed community residential service agencies/1,000 individuals, including both State- and privately-operated. A few ICF/MR and HCB waiver beneficiaries included indirectly.

Yes - 24 percent with severe MR, 7 percent with profound MR; also 14 percent with physical disabilities and/or "wheelchair dependent"

State plans to apply for demonstration of COMPASS in community ICFs/MR and to extend to HCB waiver services in the near future

OH All, including State institutions

Yes

OK All community residential services; longitudinal outcome data also being collected in State institutions

Yes

OR All except State institutions, however, they use for QE

Yes OR-AIM (citizen monitoring focused on outcomes, quality of life) required for people moving from State institution to community

RI All, including State institutions

Yes

TN All except ICFs/MR Yes

UT All except ICFs/MR Yes

VT All community residences Yes

25


WI HCB waiver services Yes Targeted especially to individuals who have moved from State institution to community residences

WA All except ICFs/MR Yes - 500 of the 900 people who have moved from the State institution and whose outcomes are monitored are individuals with either severe or profound mental retardation.

WY All except the State institution (no other ICFs/MR)

Yes

METHODS AND MEASURES USED IN OUTCOME-FOCUSED QUALITY

ASSURANCE/QUALITY ENHANCEMENT SYSTEMS

All but two of the States with outcome-focused systems have developed a specific instrument and protocol for determining whether the primary outcomes and related processes are present. The exceptions were Maine, where providers are required to focus on consumer satisfaction and to conduct self-assessments, but no specific instrument is required; and New Hampshire, where area regional agencies determine the instruments and measures used to oversee providers. There is also some variety in the Minnesota demonstration, since each of the five participating counties has made a few additions to the basic instrument adapted from the Accreditation Council 1992 standards.

As illustrated in Exhibit 6, most States use a combination of instruments and procedures, and often include a provider self-assessment component. Several States have separate consumer satisfaction and/or third-party monitoring instruments, many of which are not reflected on the exhibit. Case management monitoring also is not cited, however, this is typically part of the States’ quality assurance systems. The majority of States use State surveyors (employees of the State mental retardation/developmental disabilities agency) to conduct the reviews. Other methods include:

26

A team led by a State surveyor that includes a consumer (person with a disability), family member or provider (3 States).

An independent contractor conducting the outcome surveys (2 States).

Provider agencies collecting outcome data (2 States).

Nearly all States emphasize interviews and observation as the primary tools in determining the presence of outcomes and related processes, with document review limited to verification of essential procedures and essential health and safety compliance. Several States volunteered the observation that outcome-focused approaches require surveyor training and re-training, in order to maintain the focus on outcomes actually observed and validated by the service recipients.

27

EXHIBIT 6:

METHODS USED IN OUTCOME-FOCUSED SYSTEMS

State Methods of Outcome- Focused quality assurance /QE

Personnel Additional Information

AK Interview, observation, and limited document review

Independent survey teams (consumer, family member, local business person, provider)

51 percent of review time must be spent talking with the service recipient

CO Personal survey Independent contractor

COPAR surveys are for policy and program evaluation, not quality assurance

FL Interview, observation, and limited document review (based on Accreditation Council methodology); desk audit of L&C surveys

State surveyors

ME No standard instrument; each agency assesses consumer satisfaction

Self-assessment

Coupled with ongoing monitoring by case managers

MA Observation, discussion and document review with QUEST instrument

State surveyors

Review outcomes for 35 percent sample of service recipients

MN Interview, observation, and limited document review (adapted from AC methodology)

County monitors as well as State surveyors

Some differences among counties in specific instruments, but overall the same primary outcomes

MO Interview, observation, and limited document review

Team of State surveyor and peer, consumer or family member

Minimum 10 percent sample of service recipients per agency

NH Different instruments and procedures in each service area

Area agency staff May include citizen monitors in some areas

28


State

Methods of Outcome- Focused quality assurance /QE


OH Observation, interview, and limited document review in unannounced surveys

State surveyors Observation of problems can trigger traditional comprehensive survey

OK Interview, observation, and document review; review of CARF or Accreditation Council accreditation and on-site validation, as appropriate; separate OK/AIM and Quality of Care reviews

State surveyors CARF or Accreditation Council surveyors (at provider agency option) Consumers, family members and other interested citizens (OK/AIM) OSU surveyors (Quality of Care)

Oklahoma has recruited 350 monitors for OK/AIM: 35% consumers, 50% family members and 15% other citizens.

OR Self-reported valued outcome data (ROS); document review, observation and interviews for L&C survey and for AIM reviews; self-assessment for CQI program (voluntary)

Providers report ROS data, reviewed by State surveyors and local case managers; State staff do L&C survey; consumers and family members do AIM reviews; CQI done by providers

State surveyors do 1/2 day "probe survey," focused on health and safety, using self-assessment data as the base. Univ. of Oregon participates in data collection and analysis. Paradigm, Inc. supports CQI process.

RI Primarily interview and observation, with limited document review, coupled with streamlined L&C surveys (unannounced)

Volunteers (Regional Monitoring Boards, adapted from RI's CSLA program) for major outcome survey State surveyors for L&C

Also have Independent Human Rights Committees (all providers must establish) which do ongoing monitoring of all human rights issues

TN Interview, observation and document review

State surveyors

29


State

Methods of Outcome- Focused quality assurance /QE


VT Observation, interview and limited document review; also self-assessment and separate health and safety review

Team of State surveyor and consumer for outcome survey Self-assessment State surveyor for health and safety review

Outcome survey of 20 percent sample of service recipients Self-assessment and plan for improvement are required

WI Primarily interviews and observation

Either State or county surveyor, depending on size of facility/agency

WA Interviews, observation and document review, including health and safety standards

Independent contractor

Sample of consumers drawn for outcome review, with 100% sample required for individuals who have moved from the State institution to the community

WY CARF survey; additional L&C survey; self-reported data on valued outcomes (RLI); additional consumer satisfaction survey

CARF surveyors State surveyors Providers report RLI data, collect satisfaction data

RLI instrument adapted from OR's ROS instrument predecessor (VOIS/ORLIS)

SPECIAL FEATURES OF OUTCOME-FOCUSED QUALITY ASSURANCE /QUALITY

ENHANCEMENT SYSTEMS

A few States reported features of their outcome-focused quality assurance systems for residential services that they felt strengthened the systems' effectiveness. The feature mentioned most often in this context is a tiered rating system on outcomes tied to incentives and sanctions. The usual "reward" is a longer period of certification.

The certification system in Massachusetts is based on QUEST (Quality Enhancement Survey Tool) which has essentially replaced all previous licensure and monitoring surveys. QUEST reviews outcomes in six "quality of life" areas: Rights and Dignity, Individual Control, Community Membership, Relationships, Personal Growth and

30

Accomplishments, and Personal Well-Being. A seventh measure focuses on outcome measures for the organization. Numerical ratings, based on individual surveys of a 35 percent sample of service recipients, are used to assess the impact of the provider's services on quality of life and the agency's overall level of quality. Depending on the score, agencies receive certification at one of four levels: (1) Full two year certification; (2) One year certification; (3) Certification with conditions; or (4) Non-certification.

In Oklahoma, agencies that have chosen to obtain Accreditation Council accreditation are given deemed status as licensed agencies for a two year period if they achieve an 85 percent score or better on the Accreditation Council survey, and a one-year license if their score is between 65 and 85 percent. Agencies scoring below 65 percent must develop a plan of correction and obtain technical assistance in their areas of weakness.

Another feature noted by States is the ability to trigger a full "traditional" comprehensive survey if outcomes are not found or if basic concerns arise regarding health, safety or individual rights. In Ohio, for example, State surveyors move from the streamlined version to a full survey if there is concern about compliance. Although the survey process has been streamlined, the comprehensive standards are still in effect. The State reports that more agencies have been de-certified under the new approach than previously, in part because surveyors are more able to focus on outcomes and essential protections.

New York's COMPASS system of self-assessment coupled with collaborative State review and quality enhancement, is envisioned as quality assurance only for those agencies which have demonstrated significant compliance with existing licensure standards. Agencies with poor records are not permitted to participate in COMPASS until they improve their record, and agencies coming under COMPASS that are found lacking in outcome reviews and in quality enhancement activities can be removed from the COMPASS program. In addition, agencies scoring at the 25 - 75 percent level on outcomes must undergo further review of their self-survey, their management plans and/or their consumer input process. Those scoring below 25 percent must undergo a full traditional licensure survey. As in Ohio, a full set of standards and related surveys remains in place, to be used as needed. Some States also highlighted the use of "Quality Enhancement plans" as part of the quality assurance /quality enhancement process. In Missouri, for example, the program adopted July 1, 1995 requires agencies to develop an Enforcement Plan to correct any deficiencies in relation to core outcomes (health, safety and basic rights) and an Enhancement Plan to address issues in community membership and self-determination, including relationships and control and choice outcomes.

FUTURE PLANS AND EMERGING TRENDS

As indicated above, virtually every State is at least exploring outcome-focused quality assurance/ quality enhancement, and at least 20 States have taken action to incorporate

31

one or more outcome-focused components, and a few States are moving almost completely in that direction. Information on future plans was obtained from 50 States: 22 States in Category I or moving from Category II to Category I; 18 States in Category II or moving from Category III to Category II; and 7 States in Category III. (Three States are categorized as "other" - see Exhibit 1.) This information is summarized in Exhibit 7. It should be noted, however, that these are activities planned or getting underway as selectively reported by the States. It is very likely that other States are planning various in activities quality assurance/quality enhancement, though the respondent(s) did not mention it. Further, Exhibit 7 does not reflect information on States which are already engaging in the activities shown. For example, few "Category I" States mentioned making quality assurance more outcome focused in their future plans because they already have an outcome-focused quality assurance system.

32

EXHIBIT 7:

PLANNED QA REFORM ACTIVITIES AS OF EARLY 1995

33


34


35

Emerging trends reflected in future plans, with particular attention to States that have already made significant moves toward outcome-focused quality assurance /quality enhancement, include the following:

A focus on measuring outcomes rather than process.

Increasing use of self-assessment and of self-reported data on outcomes and other indicators.

Require or strongly support continuous quality improvement (CQI) activities among providers.

Expanding use of consumer surveys and citizen monitoring, and incorporating these activities within quality assurance systems more formally.

Decentralization of quality assurance/quality enhancement activities, with decreased focus on State surveys and increased emphasis on local or regional monitoring, case management, and quality enhancement activities.

Shifting role of the State, placing more emphasis on quality enhancement and collaboration with providers, and less on comprehensive compliance monitoring.

Rapidly emerging interest in managed care, including quality assurance reforms that will be relevant to the managed care system.

Plans to extend outcome-focused quality assurance /QE to State institutions and to ICFs/MR in the community.

States that are moving more slowly toward an outcome-focused system identified several factors that have delayed reform processes, including: resource constraints, lack of consensus on primary outcomes and on outcome-based approaches, and provider resistance. Regardless of the pace and focus of reforms, however, most States identified some issues in relation to health, safety and governmental responsibilities in quality assurance as part of the reform debate, as summarized in the following section.

36

III. THE RELATIONSHIP BETWEEN HEALTH, SAFETY, INDIVIDUAL

RIGHTS AND OTHER QUALITY OUTCOMES

There was strong consensus across the States, regardless of the current level of outcome-based quality assurance/quality enhancement, that promoting health and safety outcomes is an essential component of their responsibilities, requiring the continuation of some traditional compliance-oriented quality assurance activities. For example, several States are continuing to review medical records regarding medications and physician orders, even if document review had been otherwise de-emphasized. Several respondents also indicated that they expected the State to continue to monitor some essential health, safety and individual rights standards on a compliance basis, while other primary outcomes are monitored on more of a quality enhancement basis. Overall, there was some indication that the health and safety outcome is a different category, even when listed along with the other outcomes that are more associated with "quality of life." For example: Massachusetts’ QUEST survey looks at the areas of rights and dignity and of personal well-being and includes the following outcomes and measures: 1.3 People exercise their rights. 1. My rights are protected (legal, civil, human). 2. There are safeguards when there are limitations of my rights. 3. I have privacy. 4. When there is a need, I have support to exercise and safeguard my rights. 5. My need for a fiduciary is reviewed regularly. 6.1 People are safe. 1. My home is safe. 2. I am safe when I am at home. 3. The place that I work in is safe. 4. I am free from abuse, neglect and mistreatment. 6.2 People enjoy optimal health. 1. I have health care services that are responsive and meet my needs. 2. I maintain good health. Supplementary "QUEST Guides" relate these outcomes and measures to State regulatory standards and include worksheets and procedures to be used in the review (such as cross-referencing the Medication Administration Chart with physician's orders and a physical check of safety features in the residence). This difference in approach to health and safety also is exemplified in the Missouri outcome-based standards and methodology for recipients of HCB waiver services. As described above, the Missouri review looks at "core issues" in health, safety and essential

37

individual rights on a compliance basis, and at community membership and self-determination on a quality enhancement basis. A related issue identified by some respondents was the challenge of balancing risk-taking in consumer-focused services with the State's responsibility to protect individuals from harm. One State specifically identified this concern as the major factor in the decision to move more slowly in shifting to a consumer-centered service model and related outcome-focused quality assurance. Only two States provided information indicating that they place primary emphasis on individual choices and satisfaction, as well as a tolerance for some potential risks to health and safety. Several States reported that they are working with providers and case managers to increase the emphasis on consumer choice, the support model, and individualized service and support plans that are based on each person's goals, strengths and needs. Similar training was mentioned by some States in relation to those who conduct the quality assurance surveys. Although much of the emphasis of these efforts is reported to be on moving to a more progressive, person-centered approach, a few States also mentioned the need to help providers and case managers understand that this is not a license to abrogate responsibilities for helping people make responsible choices. The need for guidelines in this area appears to cut across States at various stages of quality assurance reform.

38

IV. USING OUTCOME-FOCUSED QUALITY ASSURANCE IN ICF/MR

FACILITIES OR WITH SIMILAR POPULATIONS

Primarily because ICFs/MR are certified separately from other types of residential services for people with mental retardation/developmental disabilities, very few States have incorporated outcome-focused quality assurance into their ICF/MR quality assurance program. As described above, however, some States report using outcome-based measures for quality enhancement in their State institutions certified as ICFs/MR. For example, Illinois and Texas use the 1992 Accreditation Council standards and methodology; Louisiana uses quality indicators related to outcomes derived from the 1992 Accreditation Council standards; and Massachusetts uses the QUEST program for quality enhancement and piloted the instrument in one of the State facilities. Oregon also reports that one of its State institutions is beginning to use the ROS program (data collection on valued activities) for quality enhancement. New York and Iowa indicate that they hope to obtain a waiver from HCFA in order to apply an outcome-focused quality assurance system in ICFs/MR, and Minnesota has a demonstration underway to apply adaptations of the 1992 Accreditation Council standards and methods in community ICFs/MR.

It is clear from discussions with State contacts that there is considerable support

for the application of outcome-focused quality assurance and the primary outcomes

in ICFs/MR, in institutional settings, and in residential services to people with severe

or profound mental retardation. Respondents from 39 States commented on the applicability of the primary outcomes and outcome-focused quality assurance systems to institutions, ICFs/MR, and to people with severe disabilities. Twenty-six of the 39 indicated support for one or more of the following approaches:

Application of the same primary outcomes and outcome-based quality assurance methodologies. (17 States)

Same primary outcomes and standards but with some adaptations and flexible guidelines when applied in institutional settings. (4 States)

Use of Accreditation Council outcome standards and methodology or equivalent, supplemented with some additional health and safety assurances. (9 States)

Regarding consumer-focused outcome measures, 18 States specifically indicated that communication issues can (and are being) addressed effectively in assessing services to people with severe disabilities and little or no verbal communication. Three States, including one which supports the application of the primary outcomes and outcome-focused quality assurance to all facilities, noted that there are still questions to be addressed on communication and expressions of choice for people with severe disabilities and communication challenges. This is an area that continues to attract attention as the

39

emphasis on consumer choice, satisfaction and consumer-centered quality assurance increases. Six States indicated their belief that the structural characteristics of institutions, rather than the characteristics of the service recipients, make it very difficult to apply the primary outcomes in a quality assurance system. Three of these respondents also support their application regardless of the difficulty; two of the three indicating that this will be helpful in spotlighting the problems of institutions in relation to the primary outcomes and hastening their closure. It is also clear from the responses that many States with outcome-focused quality assurance systems are applying them in settings that serve people with the same characteristics as those typically found in ICF/MR beneficiaries. As indicated in Exhibit 5, all but two of the 19 States with outcome-focused quality assurance systems include people with very severe and complex disabilities. Settings that serve people with similar characteristics to people residing in ICFs/MR include: services to people who have recently left State ICF/MR-certified institutions, HCB waiver service recipients, and, in three of the States, ICF/MR residents. States apply outcome-focused quality assurance for ICF/MR residents and people with comparable challenges in the following ways:

In Alaska most people are served in their own homes, in shared homes or in non-ICF/MR community residences, including children and adults with severe, multiple disabilities.

Maine serves large numbers of people with severe disabilities in community residences.

Massachusetts converted all its community ICFs/MR to HCB waiver residences.

New Hampshire closed its only State institution and is serving people with severe and multiple disabilities primarily in small individualized settings for six or fewer individuals.

Oregon and Washington target additional outcome-focused quality assurance to those who have moved from State institutions to community residences.

Wisconsin serves people with severe disabilities in its HCB waiver program (the CIP - Community Integration Program).

Wyoming is developing a network of individualized supported living arrangements and community residences for individuals moving out of the single State institution.

40

Outcome-focused systems in Colorado, Oklahoma and Ohio include the State ICF/MR-certified institutions.

41

V. DATA COLLECTION AND ANALYSIS ON THE PRIMARY OUTCOMES

The majority of State quality assurance systems do not include a database on the primary outcomes (i.e., on individualization, self-determination, dignity, personal growth, health and safety, community integration/inclusion, consumer satisfaction, and relationships/social connections) that are the focus of this review of State activity. Even among the ten States in Category I (those States which have moved to outcome-focused quality assurance systems that include these primary outcomes) only eight indicated that they maintain a database on outcomes, and two are maintained manually. State reports on primary outcome databases, organized in relation to their category of quality assurance reform, are summarized in Exhibit 8. It should be noted, however, that States may have additional databases on selected primary outcomes, particularly on health and safety, that are not reflected in this summary. Data collected by the private sector, such as the Arc in Michigan which contracts with the State to collect data on quality of life focused outcomes in community residential programs, may not have been included as well. Some States also reported plans to develop automated data systems in the near future. Overall, however, it does not appear that very many States have developed data systems on primary outcome or on compliance with specific State standards, comparable to the Federal OSCAR database on ICF/MR surveys.

42

EXHIBIT 8:

STATUS OF PRIMARY OUTCOME DATA

Category States Primary Outcome Data

I (10 States)

Alaska Colorado Maine Massachusetts Oklahoma Oregon Tennessee Utah Vermont Wisconsin

X (Manual) X X X X X X X (Manual)

II>>>I (12 States)

Arkansas California Florida Georgia Iowa Kentucky Missouri New York North Dakota Rhode Island Texas Wyoming

X (Manual) X (CSLA) X (Pilot) X (Pilot) X (AC Pilot) X (CSLA) X (CARF Pilot)

II (9 States)

Arizona Connecticut Illinois Minnesota Nebraska New Hampshire New Mexico Pennsylvania Washington

X (Health & Safety) X (Demonstration) X (Pennhurst) X (Manual/Regional)

43


Category States Database on primary outcomes of interest

III>>>II (5 States)

Alabama Kansas Nevada North Carolina South Carolina

III (12 States)

Delaware District of Col. Hawaii Idaho Indiana Louisiana Maryland Michigan Mississippi New Jersey Virginia West Virginia

X (Citizen monitoring. data) X (CSLA) X (CSLA)

Other (3 states)

Montana Ohio South Dakota

X (Health & Safety)

Analysis of primary outcome data is relatively new in most States that have moved to an outcome-focused quality assurance system. As noted previously, most of these systems have been developed over the past few years. Exceptions include the following:

Colorado has collected and analyzed COPAR data on the primary outcomes for several years. In addition to its examination of State policy and program goals, COPAR has been used to examine special issues such as the use of psychotropic medications and the extent to which outcomes vary in relation to the size and type of residential setting.

Connecticut has developed extensive monitoring systems through its database on health and safety outcomes.

Massachusetts is conducting an ongoing review of QUEST implementation, in order to promote quality, maintain State standards, and manage the quality assurance system itself.

44

Minnesota has begun collecting data on the primary outcomes in the ICF/MR outcome-focused quality assurance demonstration project which will be analyzed as part of a comprehensive evaluation study, including a comparison with non-participating agencies on compliance with ICF/MR standards.

New York is completing an extensive evaluation of the COMPASS demonstration project, including an assessment of the extent to which participating agencies maintained compliance with existing State quality standards.

Oklahoma is working with Oklahoma State University on a longitudinal analysis of primary outcomes, as part of the settlement of the Hissom litigation.

Oregon has analyzed data from the VOIS/ORLIS data collection system on desired outcomes for several years as a local (county) and State management tool. ROS data (which replaces VOIS/ORLIS as of 1995) will be analyzed by the University of Oregon as part of an ongoing quality review system.

Pennsylvania continues to collect and analyze primary outcome data on former residents of Pennhurst.

Utah is collecting and analyzing primary outcome data to track the service system's progress toward meeting new outcome-focused standards.

Most of these examples illustrate States' growing interest in using the primary outcome data for management purposes. Yet little is known about how well the service system does in producing the kinds of outcomes important to the people it serves.

QUALITY INDICATORS

A few States have developed additional quality indicators that are collected on a systematic basis and are assumed to have predictive validity in relation to the primary outcomes. These are primarily indicators of health and safety outcomes, as found in Connecticut's CAMRIS "red flag" system, and in critical indicator systems for State institutions in Alabama and Illinois. Like outcome data analysis, these are used primarily for management purposes, such as identifying patterns in specific regions, provider agencies, facilities/units, and staffing levels. Georgia indicates that cross-disability (mental health, substance abuse, and mental retardation/developmental disabilities) quality indicators, as well as indicators specific to mental retardation/developmental disabilities, will be developed over the coming year as part of system reforms that include decentralization, health reform, and cross-disability

45

accountability and quality assurance. Evaluation of the Minnesota demonstration will include identification of quality indicators, and Alabama hopes to identify quality indicators for community residences complementary to those they are already using in their State institutions. Most of the work on quality indicators to date has been limited to health and safety indicators somewhat analogous to those being considered in nursing homes and home health care. It is also plausible to consider items covered in streamlined licensure and certification reviews as quality indicators, such as the Ohio licensure survey, Massachusetts' QUEST survey, and the Tennessee outcome-focused survey. The Accreditation Council 1992 standards and survey may be considered an example of quality indicators. But unlike those based on health and safety, these quality indicators cover the full range of the primary outcomes. Detailed information on State experience with quality indicators is incorporated in a separate report.

46

VI. CONCLUSION

This review confirms the move across the United States to place more emphasis on outcomes important to consumers in QA systems covering residential services to individuals with developmental disabilities. It also reflects the continuing focus on health and safety and on individual rights, and the interest among State officials in retaining more traditional compliance-focused quality assurance measures in these areas. There is an overall consensus that outcome-focused QA based on the primary outcomes is appropriate for people with severe disabilities, with some acknowledgment that interview-based approaches require thoughtful adaptations for individuals with significant cognitive limitations and/or communication challenges. This is coupled with some mixed views on optimal applications in large institutions; most of the feasibility reservations, however, are based on institutional characteristics, such as lack of opportunities for making choices and for community inclusion, rather than the characteristics of those receiving services. The analysis summarized in this report formed the basis for in-depth review of outcome-based QA in eight States, as well as the frame of reference for looking at HCFA's current ICF/MR QA system. The intertwined issues of focusing on the primary outcomes associated with quality of life while maintaining health and safety and individual rights can be expected to surface in Federal and State and local efforts to make QA more effective. Already tight QA resources are being further challenged as the number of residential settings continues to increase and become smaller and more dispersed. Consumer-centered support models in these dispersed settings also require adaptations of traditional QA, while largely interview-based QA in outcome-focused systems requires adaptation of recently developed procedures for people with significant cognitive and communication disabilities. Most States indicated in early 1995 that they were continuing or planning to make changes toward more outcome-focused QA. Continued monitoring of the evolution in QA is strongly recommended, including Federal ICF/MR QA reforms and local innovations as well as State QA systems. There is widespread interest in making improvements in QA that continue to protect vulnerable individuals while promoting valued outcomes. Research on the effectiveness of new approaches needs to be carried out and assessed, such as the work completed by HSRI and the Temple University Institute on Disabilities on the eight States selected for in-depth analysis in conjunction with this project; the five-county demonstration in Minnesota using outcome-based QA in ICFs/MR and other community residences, being conducted by the University of Minnesota Institute on Community Integration; and the analysis being done by the Accreditation Council on the application of its 1994 Accreditation Standards. As state of the art outcome-based QA is identified and verified, it will be important that resources are available to share information among States and other stakeholders.

Documents

INTERMEDIATE CARE FACILITIES FOR PERSONS … · PERSONS WITH MENTAL RETARDATION COMPENDIUM OF STATE QUALITY ... Methods And Measures Used In Outcome-Focused Quality ... retardation/developmental