Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Quality in Health Care 1995;4:31-36
Patient satisfaction: an indicator of quality indisablement services centres
C Smith, M McCreadie, J Unsworth, H I Wickings, A Harrison
AbstractObjectives-To develop a patient satisfac-tion system for disablement servicescentres and to report on how the initialfindings have been used in audit to im-prove their quality ofcare and services.Design-Interview survey of randomlyselected users attending in three centres:Birmingham (centre X), Oxford (centreY), and Cambridge (centre Z) to establishcore topics for developing a patient satis-faction questionnaire with incorporationinto a computer patient satisfactionsystem (PATSAT) to enable collation ofresponses to the questionnaire. A pilot ofthe questionnaire was undertaken in thecentres to assess the sensitivity of thequestionnaire, which was subsequentlyused as part of clinical audit processduring June 1991 and April 1992 in centreX and the patient satisfaction system usedto monitor changes in routine practice.Patients-123 amputees in the develop-ment phase, selected by cluster sampling,and 1103 amputees in the pilot study.Main measures-Satisfaction scores forcomponents ofthe service.Results-The questionnaire included 16core topics contributing to quality of careand services, including comfort of limbs,appointments, interpersonal aspects ofcare, a system of support and counselling,and organisation. The pilot survey demon-strated high satisfaction scores for aspectsof interpersonal care, organisation, andphysical surroundings of the centres andlower satisfaction for counselling services,comfort of the limb and the number ofalterations made before the limb wasconsidered acceptable. During the auditin centre X these results prompted changesto care and services which producedsignificant improvements in satisfaction.Conclusions-The early results suggestthat the questionnaire, coupled withPATSAT software system, enable users'views to be expressed, collated, and fedback to staff; the information provided hasalready prompted change, and the systemis sufficiently sensitive to measurechanges in satisfaction with the service.(Quality in Health Care 1995;4:31-36)
Keywords: patient satisfaction, quality indicator,disablement services, audit, amputees.
IntroductionMeasures of patient satisfaction are being usedincreasingly as part of the assessment of the
quality of care and they are being incorporatedinto the process of clinical audit. Satisfactionis an important component of the total assess-ment of the value of treatment and care.Satisfaction with care is an important influencedetermining whether a person seeks medicaladvice, complies with treatment, and maintainsa relationship with the practitioner.' For usersof artificial limbs the difficulties that resultfrom dissatisfaction with services may beprofound, and any alternative accessible help isunlikely.
Several studies have examined patientsatisfaction within the artificial limb service.None has attempted to develop a system toassess satisfaction routinely or to integrateinformation about satisfaction into the processof clinical audit. This paper describes thedevelopment of a patient satisfaction systemdeveloped for disablement services centres,reports how the initial findings were used in anaudit to improve the quality of care andservices provided to patients, and shows theeffect of incorporating users' views into theclinical audit of prosthetic services.
MethodsThe study was done at three disablementservices centres: Birmingham (centre X), oneof the largest centres in the country, with about5000 current prosthetic patients; Oxford(centre Y), with about 2500 current attendees;and Cambridge (centre Z), one of the smallercentres, with about 1700 patients. The servicesprovided by the three centres vary, as do theresources available at each.
DEVELOPMENT OF QUESTIONNAIREWe developed a patient satisfaction question-naire based on issues relevant to users of theservice. Users of artificial limbs and newamputees attending clinics in the three centreswere selected randomly by cluster sampling,and the selected 123 patients were interviewedat the centre after their clinic appointment.The purpose of the study was explained by anurse at the centre, who confirmed users'willingness to take part. Semistructuredinterviews were carried out by interviewersfrom CASPE Research, and the intervieweeswere asked to identify those components ofcare and services which influenced theirsatisfaction.
In addition, the views of managers, doctors,the prosthetist, and other clinicians wereobtained to identify the issues the staffconsidered important for inclusion in thequestionnaire. These interviewees were
CASPE Research,London W2 4HUC Smith, researcherH I Wickings, directorSheffield DisablementServices Centre,Northern GeneralHospital, SheffieldS5 7ATM McCreadie, clinicalaudit coordinatorRegionalRehabilitation Centre,SBHA, BirminghamB29 6JFJ Unsworth, directorA Harrison, generalmanager rehabilitationservicesCorrespondence to:Ms C Smith,PO Box 101 13 Gouger St,Adelaide, South Australia5000
Accepted for publication1 July 1994
31
on June 13, 2020 by guest. Protected by copyright.
http://qualitysafety.bmj.com
/Q
ual Health C
are: first published as 10.1136/qshc.4.1.31 on 1 March 1995. D
ownloaded from
Smith, McCreadie, Unsworth, Wickings, Harison
selected by purposive sampling, and interviewswere carried out using a different semi-structured interview schedule.Data from the interviews were ranked to
identify the topics that interviewees identifiedas being important to their satisfaction andthese were ranked in order of frequency. Alldata were entered on to a microcomputer,using general purpose database software. Thuscore topics for the questionnaire wereidentified and questions were designed toexplore each topic in the questionnaire. Inaddition to the core topics, each centre wasable to select up to eight topics which reflectedlocal staff concerns. The questions were
designed to be short and unambiguous.Once the questionnaire had been construc-
ted convenience sampling was carried out atthe three centres, and a further 50 service userswere selected and interviewed to confirm thecontent validity of the questions, by examiningtheir interpretation of each question. Forquestions scoring less than 85% agreement on
content, the question was reworded andretested. Where there were problems in inter-preting questions a detailed content analysiswas carried out, by examining the content ofwords or descriptions within a redefinedcoded category.Each question was posed on a 4 point scale:
very satisfied, satisfied, somewhat dissatisfied,and very dissatisfied, and scores were allocatedacross a Likert scale (1 very dissatisfied, 4 verysatisfied). Satisfaction indices were thencalculated for each topic and each respondentand the total score divided by the maximumpossible score. The range of scores across theindex of satisfaction was 0-25 (very dissatis-fied) to 1-0 (very satisfied). In this way indicesof satisfaction can be produced for any topic.The appropriateness of this scale has beenassessed previously.2 3 Additional research hasshown such indices are sufficiently sensitiveto detect changes in satisfaction at local levels,such as in wards or departments.6 Space on
the questionnaire was also available foradditional comments, which were analysed bya text and retrieval package which reports ontopics relating to, for example, limb comfort.
COMPUTER PROGRAMMING
The second phase of development entailedprogramming the computers. The patientsatisfaction system being developed here(PATSAT) and the disablement services centredatabase Limbsys (the management infor-mation system) were made compatible,enabling users' satisfaction scores to be linkedto information on aspects of service deliverysuch as the doctor, prosthetist, or site ofamputation. This facility allows assessment ofsatisfaction for each core topic according totype and level of amputation, age and sex ofrespondent, and the prosthetic contractor.This level of analysis is not reported in thispaper, although it is routinely produced forusers of the PATSAT system, allowingcomparisons to be made. In addition, therespondents' verbatim comments can be linkedto satisfaction scores through search andretrieval text software.
PILOT OF PATIENT SATISFACTION SYSTEM FOR
USERS OF DISABLEMENT SERVICES CENTRES
Once the questionnaire and software had beendeveloped, the system for monitoring patientsatisfaction was piloted in three centres for sixmonths from March 1991. Every patientvisiting the centre received a personalised letterand questionnaire from reception staff.Patients were identified from the appointmentsystem, which interacted with the PATSATsoftware and identified their eligibility to takepart in the study. Those who had received aquestionnaire in the previous six months wereexcluded from the study. Patients were
encouraged to complete their questionnaire on
site, and at the end of their appointment. Forthe pilot study the responders completedquestionnaires at various stages in the processof obtaining a limb. Non-responders were notfollowed up.
FEEDBACK OF RESULTS AND MONITORING
CHANGES IN ROUTINE PRACTICE
Data were processed routinely by CASPEResearch and reports were generated monthlyand disseminated to clinical and managerialstaff in the centres.
After introduction of the patient satisfactionsystem we examined whether the system couldtrigger improvements in the quality of care.
Accordingly, in centre X (Birmingham) satis-faction scores were analysed in June 1991 andwere compared nine months later, in April1992. In June 1991, 160 amputees completedthe questionnaire and in April 1992, 114 didso. Once again users could re-enter the studyonly every six months, therefore some of therespondents had an opportunity to commenttwice.
STATISTICAL ANALYSIS
Between the two periods the percentageof satisfied respondents was compared. Owing
Core topics identified from pilot1 Surroundings in the centre2. Facilities while waiting3. Friendliness and politeness of the doctors4. Friendliness and politeness of the
prosthetist (fitter)5. Waiting time in the prosthetist's (fitting)
room6. Kindness and help given to you by staff7. Staffs willingness to talk and listen to you8. Availability of information about your limb,
fitting, aftercare9. Availability of counselling and support
services10. Skill of the prosthetist (fitter) with fitting
your limb11. Comfort of your limb12. Usefulness of your limb for daily activities13. Number of alterations to your limb before it
was acceptable14. Arrangements for booking your
appointments15. Layout of the centre16. Overall way the centre is organised
32
on June 13, 2020 by guest. Protected by copyright.
http://qualitysafety.bmj.com
/Q
ual Health C
are: first published as 10.1136/qshc.4.1.31 on 1 March 1995. D
ownloaded from
Patient satisfaction in disablement centres
Table 1 Response rate by sex and age group for three disablement services centres. Figures are perecentages (numbers)
Centre X Centre Y Centre Z(n = 1155) (n = 566) (n = 222)
Responders Non-responders Total* Responders Non-responders Total* Responders Non-responders Total*
SexMale 62 6(508) 37 4(303) 811 48-2(196) 51-8(211) 407 51.9(84) 48-1(78) 162Female 63-1(217) 36-9(127) 344 43-4(89) 56-6(90) 159 50 9(29) 49-1(28) 57
Age (years):10-19 72-4(21) 27.6(8) 29 ** ** 15 ** ** 320-29 57-9(44) 42-1(32) 76 42 6(20) 57-4(27) 47 41-7(10) 58-3(14) 2430-39 62-9(66) 37-1(39) 105 51-9(28) 48-1(26) 54 ** ** 1040-49 55 2(69) 44 8(56) 125 53-1(34) 46 9(30) 64 35-3(12) 64 7(22) 3450-59 64-6(102) 35-4(56) 158 52 4(44) 47-6(40) 84 61-5(16) 38-5(10) 2660-69 64-4(163) 35 6(90) 253 52 0(52) 48 0(48) 100 64-6(31) 35-4(17) 4870-79 66-7(174) 33-3(87) 261 48-0(61) 52 0(66) 127 61-1(33) 38-9(21) 5480-89 63-2(72) 36-8(42) 114 29-6(21) 70 4(50) 71 40 0(8) 60-0(12) 20
*Missing data in each case.**Data for <20 cases.Centre X (X2 = 7-16, df= 7; p > 0-05)Centre Y (X2 = I 1 -59, df = 7; p > 0-05)Centre Z (X2 = 15-67, df= 5; 0-01 < p < 0-05).
to the limited ability of the satisfaction scale todiscriminate between categories with smallnumbers, the scale was reduced to a 2 pointscale measuring, respectively, the percentagesof responders who were satisfied and of thosewho were dissatisfied.2 A test between sampleproportions was used to examine differencesbetween the proportions of users satisfiedbetween June 1991 and April 1992.
In all three centres the characteristics ofresponders and non-responders were exam-ined. A test between sample proportions wasused to examine differences in responsesbetween men and women and a x2 test toexamine differences in age between respondersand non-responders. Non-parametric testswere used to determine significance for datawith non-normal distributions.
ResultsDEVELOPMENT OF QUESTIONNAIREOver 40 topics were described as important tosatisfaction by limb amputees during thedevelopment phase of the questionnaire, whichincluded: comfort in wearing the limb,appointments, interpersonal aspects of care, asystem of support and counselling, and aspectsof organisation and efficiency of running of thecentres. The rank order of topics (box)determined the content of the questionnaire,the 16 topics forming the core questions. Noappreciable difference in ranking was evidentbetween the three centres. Content validity ofthe questionnaire was found to be high: onequestion, concerning the perceptions oforganisation of the centre, scored less than
Table 2 Response rate by level of amputation for two centres. Figures are percentages(numbers)
Centre X Centre Y
Site of amputation Responders Non-responders Total Responders Non-responders Total
Lowerlimb below knee 65 2(375) 34-8(200) 575 48-5(159) 51-5(169) 328Lower limb above knee 67-1(192) 32-9(94) 286 44 0(55) 56 0(70) 125Upper limb below elbow 42-9(21) 57-1(28) 49 ** ** 9Upper limb above elbow 42-9(12) 57-1(16) 28 ** ** 13Congenital amputation 40-0(12) 60-0(18) 30 ** ** 8Multiple amputation 59 0(46) 41-0(32) 78 74 3(29) 25-6(10) 39
**Data for <20 cases.Site X: x2= 20-15, df= 5; p <005.
85% agreement and was reworded to "theoverall way the centre is organised."
PILOT OF PATIENT SATISFACTION SYSTEM
In the six month pilot study a total of 1943amputees were sent questionnaires and askedto take part; 1103 responded, giving a responserate of 568%. For each centre the number ofresponders was 62-8% (725/1155, centre X)46-8% (265/566, centre Y), and 50 9%(113/222, centre Z).Tables 1 and 2 show characteristics of
responders and non-responders. No differencein the response rate between men and womenwas found. The age of responders and non-responders was similar in two centres (X andY). A x2 test of the data within each centreshowed a significant difference in age betweenresponders and non-responders only in centreZ, in which non-responders tended to beyounger than the responders.Over 70% (1380/1943) of users of the
disablement services were male (table 1), anduse of the services increased above the 50-59age group, with the greatest proportional useamong the 70-79 age group. Data on site ofamputation could be extracted only for centresX and Y (table 2). Amputations of the lowerlimb were the most common amputation incentres X and Y, and within this category therewas a greater proportion of below kneeamputations (66-7% and 72-4% respectively).In centre X a significant difference in responserate was found between upper and lower limbamputees. Too few data were available forcentre Y to enable analysis.
SATISFACTION INDICES
Figure 1 shows the satisfaction indices for alltopics for the three centres. A score of 1-0represents a response of "very satisfied" and0-25 "very dissatisfied." For all three centresuser satisfaction was high (0-80-0 90) orhigher for surroundings, layout, booking ofappointments, waiting facilities, and aspects ofinterpersonal care such as staff help and theattitudes of doctors and prosthetists, and limbfitting. Four main topics scored lower: waitingtime in the fitting room, availability of
33
on June 13, 2020 by guest. Protected by copyright.
http://qualitysafety.bmj.com
/Q
ual Health C
are: first published as 10.1136/qshc.4.1.31 on 1 March 1995. D
ownloaded from
Smith, McCreadie, Unsworth, Wickings, Hamrson
1.00
0.75
0.50
0.25
1 00
a)" 075
0
utin
0.5
(1)
0-25
C0 %zo !;i?
Questionnaire topic
= Centre X (n = 545) m Centre Y (n = 265) Centre Z (n = 113)
Fig1 Patients' satisfaction with main aspects of care in disability services cenztres, March and August 1991
counselling services, comfort of the limb, andthe number of alterations before the limb was
considered to be acceptable.
AUDIT AT BIRMINGHAM DISABLEMENT SERVICES
CENTRE
Each centre had the opportunity to assess anduse information from the patients' views of itscare and services. The results from theBirmingham centre were chosen to illustratehow the results from the routine use of thequestionnaire linked to PATSAT softwarewere used in practice. Satisfaction scores forservices provided by the Birmingham centre (Xin fig 1) indicated satisfaction with aspects ofservice delivery and many of the interpersonalaspects of care. Lower levels of satisfactioncompared with the two other centres were
visually apparent (fig 1) for waiting time in thefitting room, information, counselling, fitting
Table 3 Comparison of users' satisfaction with selected aspects of services at Birminghamafter changes in practice
Topic No ofpatients Satisfied
JUiie 1991 April 1992 Significance*
Information 70(106/152) 87(90/104) p < 0 01Counselling 67(85/127) 81(84/104) p < 001Fitting of limb 85(126/148) 91(95/104) 0 01 <p<O005Comfort of limb 57(85/149) 74(81/110) p < 0 01Usefulness of limb 80(116/146) 84(92/109) p > 0 05Alterations to limb 67(95/141) 80(82/102) 001 <p< 005Booking appointments 84(127/152) 92(100/109) p < 0 01
*Test between sample proportions.
of the limb, comfort of the limb, and thenumber of alterations before the limb wasacceptable.These results were reviewed in the
Birmingham centre throughout the auditprocess during 1991, and in response severalchanges were introduced to improve thequality of care and services (box). Mostartificial limbs are prescribed in an environ-ment which does not offer the range ofdifferent surfaces on which the limb might beused. An outdoor track where users can testout their artificial limbs in more realisticconditions, such as on grass and slopes hasbeen introduced.Through the clinical audit process the
comfort and usefulness of limbs and thenumber of alterations to limbs were highlightedas topics of major concern to users. As a resultat the initial appointment the amputee,prosthetist, and healthcare professional now
agree on a realistic outcome as a result ofproviding a prosthesis. The amputee's progressis monitored against this goal throughout the
period of aftercare. In addition, the basis ofcontracts with the manufacturers of prostheseshas been changed to reflect the need to achieveimproved quality of care and user satisfactionthrough fewer visits to the centre and a more
comfortable limb fitting. This will continue tobe monitored and discussed every month withthe contractors, according to feedback receivedon user satisfaction.
MONITORING CHANGES IN PRACTICE
To examine whether the changes introduced inthe Birmingham centre led to improvedsatisfaction and quality of care, satisfactionrecorded in June 1991 was compared with thatin April 1992 for seven selected topics. Theresponse rate in June 1991 was 64% and in
April 1992, 53%. Table 3 shows that for eachselected core topic, except for usefulness of the
Changes made by Birmingham centreto improve service* Information folder on disability* Publicity on the information department* Nursing auxiliaries reassigned and given new
responsibilities* Fitting room made available for couples* Introduction of an outdoor test track* Funding to employ a social counsellor
34
on June 13, 2020 by guest. Protected by copyright.
http://qualitysafety.bmj.com
/Q
ual Health C
are: first published as 10.1136/qshc.4.1.31 on 1 March 1995. D
ownloaded from
a)
(.1)
0-
35
100
90
80
70
60
50
40
30
20
10
0
Questionnaire topic
= June 1991 (n = 160) M April 1992 (n = 114)
Fig 2 Patients' satisfaction with main aspects of care in Birmingham (centre X) idn June 1991 and April 1992
limb, the percentage of satisfied respondents in
April 1992 had increased significantly and theimprovements were evident across the entiresatisfaction scale (fig 2)
DiscussionEarly results from the use of this questionnaireof users' satisfaction with the service providedby three disablement services centres show thatthis tool is successful in enabling users toexpress their opinion of the service standardsthey receive. The system was sufficientlysensitive to measure variations in satisfactionwith the service, it provided insights into areasof satisfaction and dissatisfaction with theprovision of artificial limbs, and it highlightedsome important differences between thecentres. Feeding back this information to thecentres has resulted in changes in servicedelivery whose effect can be measured bychanges in satisfaction indices.
The questionnaire was developed byranking topics raised by responders during theinitial interviews. The actual rankings andsatisfaction scores for each individual topicwere different, and this highlights a conceptualdifference between the two issues. To improveon future questionnaire design, incorporatinga weighting factor, such as the degree ofimportance, during the initial rankings mighthelp to identify the relative importance of thecore questions. This would facilitate a subse-quent review of satisfaction levels in audit andplanning of proposed action.The response rates were low in the pilot
study, for reasons that are not clear. Only one
set of questionnaires was sent out, and non-
responders were not followed up, owing to a
lack of resources. Among the respondentsupper limb amputees were underrepresented,which is likely to be related to their physicaldifficulty in completing the questionnaire.
Alternative methods of including the views ofthis group of users need to be considered.However, since the routine introduction ofthese questionnaires the response rates haverisen and now range from 63% to 89% in thethree centres.Once patient satisfaction data are regularly
generated, centres will be able to set localstandards including minimum standards(below which patient satisfaction should not beallowed to fall) and target standards (the levelof patient satisfaction which should beregularly achieved). Monitoring patient satis-faction consists of routinely comparing current
results against these standards. The intro-duction of a measure of one aspect of thequality of care which is available on a regularbasis is an important step forward.The facility to make comparisons lies at the
heart of any audit programme. The systemdescribed here allows such comparisons.Incorporation of PATSAT system into clinicalaudit (Committee on Clinical Audit Rehabili-tation Medicine, disablement services annualreport, April 1992) will provide information tohelp managers and health professionals to lookfor gaps in service provision and areas forimprovements. Not only does it provide a
framework for improving quality within theprovider unit it is also a mechanism whichpurchasers may require, and pay for, to ensure
the quality of services commissioned.This system has now been integrated into the
routine operation of 11 disablement servicescentres. Local and national user groups havebeen kept informed of the developments ofPATSAT. The patient satisfaction sysem has a
facility to explore topics in greater depth, andthis has been undertaken for comfort of thelimb. Future uses of these data includestandard setting for each of the topics includedin the questionnaire and monitoring the
100
90
80
70
60
50
40
Patient satisfaction in disablement centres
on June 13, 2020 by guest. Protected by copyright.
http://qualitysafety.bmj.com
/Q
ual Health C
are: first published as 10.1136/qshc.4.1.31 on 1 March 1995. D
ownloaded from
Smith, McCreadie, Unsworth, Wickings, Hamrson
subsequent achievement of increased quality ofservice. The topics that are monitored may beincreased or changed and, of course, as
standards are achieved the target standard maybe raised.
1 Larsen DE, Rootman I. Physicians' role performance andpatient satisfaction. Soc Sci Med 1976;10:29-32.
2 Green J. On the receiving end. Health Services Journal 4 Aug1988:880- 1.
3 Smith C. Validatation of a patient satisfaction system in theUnited Kingdom. Qual Assur Health Care 1992;4:171-7.
4 Kerruish A, Wickings I, Tarrant P. Information frompatients as a management tool: empowering managers toimprove the quality of care. Hospital and Health ServicesReview April 1988.
5 Wickings I, Crown J, Harvey J, Kerruish A. Proof of thepudding. Health Services journal, 31 Aug 1987: 1070-1.
6 Brent Health District. Effect of presenting nmanageoientinfonnation to clinicallv accountable teanis. Brent: BrentHealth District, 1975. (Available from HealthInformation Unit and Management Research Team,Brent Health District.)
36
on June 13, 2020 by guest. Protected by copyright.
http://qualitysafety.bmj.com
/Q
ual Health C
are: first published as 10.1136/qshc.4.1.31 on 1 March 1995. D
ownloaded from