MENTAL HANDICAP RESEARCH, 1 988; 1 :2, 1 97- 2 10 0 BlMH Publications

Evaluating the quality of life of people with mental handicaps:

197

A SOCIAL VALIDATION STUDY Brian Stanley, Principal Clinical Psychologist, Lynfield Mount Hospital, Bradford BD9 6DP Ashok Roy, Senior Registrar in Mental Handicap, Chelmsley Hospital, Marston Green, Birmingham B37 7HL

ABSTRACT This paper describes an attempt to evaluate quality of life using standards set by the local community. Several parameters derived from community psychology, such as actual levels of functioning and degree of satisfaction within several life domains, are used to develop a social validation approach. This avoids certain methodological pitfalls common to this type of evaluation.

INTRODUCTION As more people with mental handicaps leave institutional settings, researchers and service providers have become concerned with the quality of the services they receive (Lakin and Bruininks, 1985; Towell, 1985; Blunden and Beyer, 1986). Quality of life (QOL) evaluations frequently focus on three aspects: the quality of the environment in which people live (King, Raynes, and Tizard, 1971); the qualityofcare they receive from the staff who look after them (Hemming, Lavender, and Pill, 1981); and their degree of integration into the neighbourhood, as measured by their use of community facilities (Butler and Bjannes, 1978).

Quality of life is difficult both to define and to quantify. Nevertheless, Landesman (1986) has argued that the development and adoption of measurable subjective and objective standards would offer better services to people with mental handicaps and their families. However, despite living in a homely environment, having easy access to community facilities, and being looked after by caring staff, people with mental handicaps may still have unmet needs which they may find difficult to express and fulfil (Tyne, 1978). So, how can a working definition of quality of life be developed?

Any correspondence should be addressed to Brian Stanley at the address given.

198 STANLEY AND ROY

The present study considered five possibilities:

measuring aspects ofservice delivery, such as the quality ofcare or environment, and equating these with quality of life;

using the researchers’ personal definitions of quality of life and developing some means of measuring them;

developing consumer feedback methods which would allow assessment of people’s quality of life;

looking at community studies of quality of life and drawing up measures based upon them;

involving members of the local community in the development of a working definition.

It was felt that solution (a) tended to confuse the definition still further and, in any case, measures of service delivery were worth taking alongside a measure of quality of life. Solution (b) was considered to provide no further insights into the problem of quality of life, since it is not derived from the context to which it is to be applied. It is rather like travellers who only ex- amine another culture in their own terms, rather than attempting to accept it on its own terms. Solution (c), then, was thought to be the ideal solution to the problem, but was it a practical solution? Recent studies have demon- strated that, with care, it is practical to involve people with mental handicaps in service evaluation (Flynn, 1986; Flynn and Saleem, 1986; Atkinson, 1988). Clearly, the assessment of subjective quality of life is vital; but our main interest lay in developing a new way of appraising objective quality of life. Eventually it was decided to employ a mixture of solutions (d) and (e).

For some time now, community psychologists have been examining conceptual issues which relate to quality of life. For example, Campbell (1981) has proposed the following twelve “domains”: marriage, family life, friendships, standard of living, work, neighbourhood, place of residence, the country, housing, education, health, and self. Each life domain can be measured in three ways: relevance, frequency, and satisfaction. These measures attempt to assess the value of that domain to the individual, the individual’s level of engagement within it, and whether that level of engagement is acceptable to the individual.

With this framework to use for defining quality oflife, it had to be decided whose quality of life should be measured. Typically, an objective quality of

EVALUATING THE QUALITY OF LIFE 199

/

life study compares a group of people who have left hospital with a group who remain; hence there is an emphasis upon “quality of care” issues. However, if the domains listed above are employed, it becomes more obvious to use memebers of the local community as the comparison group. This method of comparison is called social validation (Kazdin, 1977).

The present study set out to measure the quality of life of people with mental handicaps who were discharged from a hospital in the West Midlands to live in a nearby group home. It had three groups to compare: the local community; the hospital community; and the people living in the new group home. Each group could be measured on frequency and satis- faction within the quality of life domains by obtaining scores for “Actual QOL” and “Ideal QOL” on each item. “Actual” scores would provide frequency ratings, and “Ideal minus Actual” scores would provide satisfaction ratings.

Although Actual scores are reasonably straightforward to conceptualise, Ideal scores are more difficult, the basic problem being: What constitutes a set of Ideal QOL scores for people with mental handicaps? Are they the same, different, or do they overlap with those of the local community? The answers to these questions are not known in detail and further studies which go beyond the scope ofthis inquiry would be required to ascertain them. For the purposes ofthe present study, therefore, it was assumed that Ideal QOL

FIGURE 1. Predicted Actual Scores (a, c, e) and predicted Satisfaction Scores (by d, r) for community, group home, and hospital groups.

200 STANLEY AND ROY

Sex

Mean age

Mean intelligence quotient

Mean dependency

levels for people in the group home and the hospital were the same as for members ofthe local community. This is in accordance with the principle of normalisation, which affirms that valued and familiar methods should be used to provide each person with life conditions and opportunities which are as socially valued as those of the average citizen (Wolfensberger, 1972).

Figure 1 illustrates the predicted Actual and Satisfaction scores for all three groups studied.

Group home Hospital

5 women 5 women 2 men 2 men

30.13 32.0 r = 0.739 NS (5.83) (8.19)

52 58 t=0.844 NS (10.3) (14.03)

14 16 T = 9 N S

METHOD

Design A prematched control group design was adopted. This method involves matching two groups of people as closely as possible on factors which, if left unmatched, may themselves explain the results. The procedure, therefore, attempts to eliminate all other explanations for the results other than the factors being investigated. The matching was achieved in the following way.

Group home group Seven people who were living in hospital but were due to move to a local group home were selected from the hospital resident population by staff who considered that they would be able to benefit from the move and live together relatively harmoniously.

Hospital group Staff selected seven people who were to continue living in hospital whose age, sex, intelligence quotient, and dependency levels were not significantly different from people in the group home, as measured using the t-test and Wilcoxen Test for Matched Groups (Cohen and Holliday, 1982) (see Table 1).

Local community group In order to obtain as random a group as possible, 100 people consecutively

TABLE 1. Characteristics of the matched group home and hospital groups

EVALUATING THE QUALITY OF LIFE 201

attending a follow-up clinic in the casualty department of the local hospital were interviewed. All were minor accident victims, not regular attenders.

Measures Four measures were employed to assess the effects on people with mental handicaps of changing the living environment. The Potential Development Index (Toogood, 1974) was used to assess the level of dependency of people in both groups two months before those in the group home left hospital and again 10 months later. This Index has three parts: physical dependency, behavioural dependency, and educational potential. It is a simple checklist designed to be completed by direct care staff with special knowledge of the person being assessed. The study predicted that the dependency of people in the group home would change on moving to the community, but it did not predict the direction ofchange.

The quality of care evident in the two living environments was assessed using a revision of Gunzburg’s Steps List (May, 1985). This scale is an adaptation of Gunzburg’s original 39 Steps (Gunzburg, 1973) and measures both the quality of the physical environment (for example, the type oflighting used) and the nature of the management practices which are present (for example, socially distant staff, depersonalising and rigid routines). It was predicted that the ordinary living arrangements in the group home would register a significantly higher quality of care than those in the hospital environment.

Community participation is often used as an objective measure of quality of life (Butler and Bjannes, 1978). A modified version ofthe scale used by the NIMROD Project (Humphreys, Lowe, and Blunden, 1983) was employed to provide a fre- quency count of usage of community facilities such as shops, community centres, and public transport. It was predicted that people living in the group home would use local amenities to a significantly greater extent than those living in hospital.

The fourth measure required was an objective means of measuring quality oflife for which it was necessary to develop a specific scale. The development of this measure was a three-step process. First, the content for the measure had to be established; next, a typical range of scores (or “norms”) had to be derived from the community; and, finally, a scale had to be created for staff to rate the quality of life of the people in the group home and hospital groups.

Quality of life measure

Content Campbell’s (1 98 1) life domains, as already mentioned, represent a fairly comprehensive list of potential items for inclusion in a scale for measuring quality of life. Nevertheless, it was necessary to establish the relevance of items prior to their use (Murrel and Norris, 1983). A questionnaire was devised and administered to 25 health service workers. This questionnaire listed a number of aspects of day- to-day life, based upon Campbell’s list, which workers were asked to rate in terms of each item’s importance to them. Low importance items were removed, and the resulting list of eleven items is presented in Table 2.

Norms Having derived the list of relevant items, the second step was to obtain ratings of relevance, frequency, and satisfaction from the local community group. This was

202 STANLEY AND ROY

No. Domain

1

1 1

Having a job outside the home

Having a group of friends at work

Having the support of your family

Having support from close friends

Having friendly contact with neighbours

Going out with family or friends

Staying in during the evening

Receiving a fair wage for the work that you do

Having a work routine

Feeling a part of the local neighbourhood

Feeling a sense of control in your life

achieved by producing a preliminary questionnaire in which each of the eleven items was required to be rated, as follows:

How important is x to you? (to establish item relevance)

How often do you do x? (to measure frequency of “actual” level)

How often would you like to do x? (to measure satisfaction or “ideal” level).

One of the questions taken from the questionnaire is displayed in Table’3 as an example.

All eleven items were rated as “moderately” or “very” relevant by the local community group, and were therefore retained. Community norms were established in the following manner.

The actual scores for each item were added up for each of the 100 people interviewed. T h e 100 total scores were used to provide a median (or average) score, and a semi-interquartile range (or spread of scores). To establish Satisfaction scores for each ofthe items, the Actual score was subtracted from the Ideal rating to obtain a discrepancy measure (Gutek et al., 1983). Having obtained a total discrepancy score for each person, the median scores and semi-interquartile ranges were calculated.

A final set of norms was obtained from the community participation total scores (see above). Once again, the same pair of statistics was calculated.

Rating The questionnaire was next converted into a staff rating scale by removing the “relevance” and the “ideal” components, thus leaving only the “actual” frequency items. In order to establish that staff could reliably rate another person in the same environment, they were asked to use the scale to rate their spouse and to obtain an

EVALUATING THE QUALITY OF LIFE 203

Having a job outside the home

a How important is it for you to have a job? Not at all Slightly Moderately Very

b How many hours a week do you work? More than 30 hours Between 20-29 hours Between 10-19 hours Less than 10 hours Do not work

How many hours a week would you like to work? More than 30 hours Between 20-29 hours Between 10-19 hours Less than 10 hours Do not like to work

c

TABLE 3. Sample item from quality of life questionnaire

independent self-rating from that person. The resulting inter-rater reliability was 0.84.

Two members of the group home staff were then asked to rate the group home members they knew best. This resulted in an inter-rater reliability of 0.96. Staff also reported the frequency of community participation of the group home residents in the previous month.

For people in the hospital group, both the rating scale and the community participation questionnaire were completed by key workers. It was predicted that the quality of life of people living in the group home would be significantly higher than for those who had remained in hospital.

RESULTS

Dependency The Potential Development Index scores which were obtained for group home and hospital group members two months prior to group home members leaving hospital and 10 months later are presented in Table 4. This shows that the two groups ofpeople were exhibiting approximately the same levels of dependency both before the change (Wilcoxen T = 9 , p>.05, NS) and 12 months later (Wilcoxen T= 12.5, p> .05, NS). It is possible that, given a longer time interval, the dependency levels of people in the group home would reduce. Nevertheless, it is encouraging that their ability levels did not deteriorate after the move.

204 STANLEY AND ROY

Person Before After

Group home Hospital Group home Hospital

TABLE 4. Potential Development Index scores for people in group home hospital groups before and after discharge of group home members

70 - 60 -

50 .

and

Group home environment

0

20. Institution level

I

Ward 1 Ward 2 Ward 3 Ward 4 Group Home

FIGURE 2. Quality of care for people in the group home and in hospital compared to May (1985) norms for community residential facilities and

institutions.

EVALUATING THE QUALITY OF LIFE 205

Quality of care The score for quality ofcare for people in the group home (53.0) was almost identical to May’s (1985) norm for residential facilities in the community (mean = 52.7).

The score for the hospital wards (mean=39.0) (May’s mean for institutions = 23.9, though lower than that of the group home, still approximates towards the community level. Thus, the present study did not demonstrate that the two environments in question are significantly different (see Figure 2).

Community participation People living in the group home (median = 1 1.3) and in hospital (median = 9.6) had comparable levels of community participation ( Wilcoxen T=8.5, p> .05, NS). The level of participation of both these groups, however, was much lower than the range oflevels obtained for people in the local community (range= 33-75) (see Figure 3). r I 9 4

Range for community

50 -

Hospital group

2 3 4 5 6 7 Matched pairs

1

FIGURE 3. Community participation scores for group home and hospital group in relation to the range for the local community group.

Actual score The range for the local community was 33 to 45. The corresponding scores for the people in the group home lay within this range (median = 37.3), but those for the hospital group (median = 26.6) were significantly lower ( Wilcoxen T= 0, p< .001). These results are in line with the study prediction (see Figure 4).

206 STANLEY AND ROY

1 2 3 4 5 6 i Matched pairs

FIGURE 4. Actual QOL scores for group home and hospital group in relation to the range for the local community group.

Satisfaction scores The range of Satisfaction scores for the local community group was - 1 to + 5. Some people in the group had negative Satisfaction scores on certain items, implying that their Ideal level lay below their Actual level of functioning. This was partly due to housewives and people approaching retirement who possibly wanted a more restful life style. Several people did not show any discrepancy, indicating satisfaction with their current level of functioning.

The people living in the group home registered a Satisfaction median of 5.43 which is almost within the community range. The hospital group registered a Satisfaction discrepancy score median of 17.43, which is significantly different from that ofthe group home median (Wilcoxen T = 0, p< 0.1) and even more different from the community scores (see Figure 5).

EVALUATING THE QUALITY OF LIFE 207

0 - Group home @

range for community ‘ Interquarrile

I \ I I I

I 28

1 2 3 5 6 7

Matched pairs

FIGURE 5. Satisfaction discrepancy scores for group home and hospital group in relation to the range for the local community group.

DISCUSSION Evaluation is now increasingly emphasised in the delivery of new services. The basic problem, however, is: What constitutes an appropriate service evaluation?

Landesman (1986) has argued that both subjective and objective standards need to be developed and used, and that this would lead to better services for people with mental handicaps. Typically, the problem for service evaluation has been how to employ subjective responses without exploiting the people who are receiving the services, and at the same time ensuring some degree of response validity. Thankfully, there is now “a growing body of literature on the challenges faced by researchers who seek the viewpoint of consumers, and some ideas are now emerging about how to meet those challenges most effectively” (Atkinson, 1988). In terms of

208 STANLEY AND ROY

subjective quality of life and satisfaction, the work of Flynn and Saleem (1986) and Atkinson (1988) seems to offer ingenious possibilities for service evaluators. However, the main purpose of this study is to contribute to the debate on objective quality of life research.

The present authors’ principal argument is that typical operational definitions of quality of life suffer from bias, in that they are too personal to researchers’ own subjective views, or that they are measuring other things, such as quality of care and environments or degree of community participation. The solution in this study was complementary to the subjective research, namely, to ask members of the local community what they understand by the term “Quality of Life.”

There are two ways in which this could have been done. First, local people could have been asked what they understand by the term, how it applies to them, what is involved in its definition, and so on. This could be a useful exercise and could produce very interesting results. However, as a first attempt at using a social validation approach, it was felt that it would be useful to draw upon the considerable body of work that has been produced by community psychologists investigating quality of life.

Thus, an attempt was made to evaluate quality of life using standards set by the local community. Members of the group home and hospital groups shared the same environment prior t3 the study and had virtually identical facilities and opportunities in the hospital. Traditional measures of quality of life, such as community participation or management practices, would normally be expected to be able to discriminate between the two groups some time after discharge of one of them from hospital. However, in this study that was not the case. Using the method of social validation and measures such as actual levels of functioning and satisfaction scores, it was possible to find significant differences between the groups. This emergence of disparate findings would seem to indicate that both subjective and objective forms of data would be necessary for a comprehensive evaluation ofnew service developments. Alone, neither would appear to be sufficient as judged from the present findings.

The method of social validation used has certain distinct advantages. It reduces the effect of subjective bias on the part of researchers by employing the community’s own definition of quality of life. This allows new community care initiatives to be evaluated using that community’s own behaviour and values. Furthermore, this approach begins to incorporate the concepts and methods of community psychology into community care

EVALUATING THE QUALITY OF LIFE 209

developments. A practical difficulty with this form of evaluation is that community norms are essentially local in nature and hence may not “travel” well. It would probably be necessary for evaluators to develop community norms in their own geographical area.

Certain methodological issues are worth mentioning. Selecting a truly random sample ofthe general population was difficult and so a more readily accessible population was chosen. Interpretations based on small samples, such as those in the current study, need to be viewed with caution unless confirmed by similar observations on larger groups over a longer period of time. The bias introduced by the staff rating the people in their care could not be eliminated due to the subjective and personal nature of the items. It would be possible to overcome this by using the method of subjective evaluation rather than that of social comparison. This could then be a more direct method of eliciting the community’s evaluation of service provision. Finally, it should be borne in mind chat not all life domains could be included. Certain items were excluded from the final rating scale because it was felt that their inclusion would have needlessly compromised the group home and the hospital groups when comparing them with the community sample. Both these groups, for example, enjoyed poorer health than members of the community. Similarly, their marital status and educational background were not felt to be open to the same form of influence as work, leisure, and friendships.

The approach to quality of life research in mental handicap services used in this study could focus on important areas of people’s lives, enabling improvements beyond the quality of care, the quality of the environment, and the degree of community participation. This would clarify the factors affecting personal satisfaction and successful adaptation into the community, and thus help people with mental handicaps to establish more valued lives outside institutions.

Acknowledgements The authors are grateful to Dr. Ahmed and Dr. Bloxham for giving them access to their clinics; to Professor John Corbett, Professor Raymond Cochrane, Richard Toogood, and Dr. Brian Oliver for helpful discussions; to David Spellman and Louise Fletcher for help in the collation of data; and Lorraine Essex and Rebecca Edney for secretarial assistance. They also express thanks to the anonymous journal reviewers for their helpful comments and suggestions.

REFERENCES ATKINSON, D. (1988). Research interviews with people with mental handicaps. Menral Handicap

Research, 1, 75-90.

STANLEY AND ROY

BLUNDEN. R., BEYER. S. (eds). (1 986). Pursuing Quality. London: Independent Development Council for People with Mental Handicap.

BUTLER, E. W., BJANNES, A. T . (1978). Activities and use of time by retarded persons in community care facilities. In Sackett, G. P. (ed). ObservingBehavior(Volume 1). Baltimore: University Park Press.

CAMPBELL, A. (1981). The Sense of WellBeing in America. New York: McGraw-Hill. COHEN, L., HOLLIDAY, M. (1982). Statistics for SocialScientists. London: Harper and Row. FLYNN, M.C. (1986). Adults who are mentally handicapped as consumers: issues and guidelines for

interviewing. Journal of Mental Deficiency Research, 30, 369-377. FLYNN, M. C., SALEEM, J. K. (1986). Adults who are mentally handicapped and living with their

parents: satisfaction and perceptions regarding their lives and circumstances. Journal of Mental Deficiency Research, 30, 379-387.

GUNZBURG, H. C. (1973). “39 Steps”. Leading towards normalised living practices in living units for the mentally handicapped. British Journal of MentalSubnormality, 37,9 1-99.

GUTEK, A. G., ALLEN, K., TYLER, T. R., LAU, R. L., MAJCHRZAK, A. (1983). The importance of internal referents as determinants of satisfaction. Journal of Community Psychology, 11, 11 1 - 120.

HEMMING, H., LAVENDER, T., PILL, R. (1981). Quality of life of mentally retarded adults transferred from large institutions to small units. AmericanJournalof Mental Deficiency, 86, 157- 169.

HUMPHREYS, S., LOWE, K., BLUNDEN, R. (1983). The long term eveluation of services for mentally handicapped people in Cardiff: research methodology. Cardiff: University of Wales, Applied Research Unit.

KAZDIN, A. E. (1977). Assessing the clinical or applied importance of behaviour change through social validation. Behaviour Modification, 1,427-45 1.

KING, R. D., RAYNES, N. V., TIZARD, J. (1971). Patterns of Residential Care: sociological studies in institutions for handicapped children. London: Routledge and Kegan Paul.

LAKIN, K. C., BRUININKS, R. H. (eds). (1985). Strategies for Achieving Community Integration of Developmentally Disabled Citizens. Baltimore: Paul H. Brooks.

LANDESMAN, S. (1986). Quality of life and personal life satisfaction. Mental Retardation, 24, 141 - 143. MAY, A. E. (1985). Steps towards normalisation: a revision of “39 Steps”. British Journal of Mental

M U R R E L , ~ . A., NORRIS,F. H. (1983). Quality of life as the criterion for need assessment and community

TOOGOOD, R. J. (1974). Calculating nursing needs. Apex (now Mental Handicap), 1, 17- 18. TOWELL, D. (1985). Residential needs and services. In Craft, M., Bicknell, J., Hollins, S. (eds). Mental

TYNE, A. (1978). Looking at Life in a Hospital, Hostel or Home Unir. London: Campaign for Mentally

WOLFENSBERGER, W. (1972). The Principle of Normalisation in Human Services. Toronto: National

Subnormality, 31, 108- 113.

Psychology. Journal of Community Psychology, 11,89-97.

Handicap: a multidisciplinary approach. London: Bailliere Tindall.

Handicapped People.

Institute of Mental Retardation.