View
217
Download
0
Tags:
Embed Size (px)
Citation preview
David Stockwell
Clinical Biochemistry Service Manager
ABMU NHS Trust
Tim von Pokorny
Pathology Quality Manager
Gwent Healthcare NHS Trust
The User’s Perspective of Quality
PoliciesUser requirement Standards
SMART objectives -Accuracy Timeliness RelevanceCost Benefit Framework
Quality Management System Overview
PoliciesUser requirement Standards
SMART objectives -Accuracy Timeliness RelevanceCost Benefit Framework
Local/Corporate Strategy
National Strategy
PoliciesUser requirement Standards
SMART objectives -Accuracy Timeliness RelevanceCost Benefit Framework
Local/Corporate Strategy
National Strategy
Strategicalignment
PoliciesUser requirement Standards
SMART objectives -Accuracy Timeliness RelevanceCost Benefit Framework
Local/Corporate Strategy
National Strategy
LegislationUser Requirement
PoliciesUser requirement Standards
SMART objectives -Accuracy Timeliness RelevanceCost Benefit Framework
Local/Corporate Strategy
National Strategy
LegislationUser Requirement
Service Delivery ModelTechnology/Workforce
PoliciesUser requirement Standards
SMART objectives -Accuracy Timeliness RelevanceCost Benefit Framework
Local/Corporate Strategy
National Strategy
LegislationUser Requirement
Service Delivery ModelTechnology/Workforce
Procedures
PoliciesUser requirement Standards
SMART objectives -Accuracy Timeliness RelevanceCost Benefit Framework
Local/Corporate Strategy
National Strategy
LegislationUser Requirement
Service Delivery ModelTechnology/Workforce
Procedures
Training&
Competency Assessment
PoliciesUser requirement Standards
SMART objectives -Accuracy Timeliness RelevanceCost Benefit Framework
Local/Corporate Strategy
National Strategy
LegislationUser Requirement
Service Delivery ModelTechnology/Workforce
Procedures
Output
Training&
Competency Assessment
PoliciesUser requirement Standards
SMART objectives -Accuracy Timeliness RelevanceCost Benefit Framework
Local/Corporate Strategy
National Strategy
LegislationUser Requirement
Service Delivery ModelTechnology/Workforce
Procedures
Output
AuditMeet SMART objectives?
Accuracy Timeliness Relevance? Meet user requirement?
Training&
Competency Assessment
Why Measure User Satisfaction
“Ongoing evaluation and improvement processes are essential to ensure that the service provided by the laboratory meets the needs and requirements of users”
CPA standards V2.00 September 2007
PoliciesUser requirement Standards
SMART objectives -Accuracy Timeliness RelevanceCost Benefit Framework
Local/Corporate Strategy
National Strategy
LegislationUser Requirement
Service Delivery ModelTechnology/Workforce
Procedures
Output
AuditMeet SMART objectives?
Accuracy Timeliness Relevance? Meet user requirement?
Training&
Competency Assessment
H
G
A
E F
B
C D
An approach to measuring User Satisfaction
User Satisfaction
…Measure of how your organisation’s total product performs in relation to a set of customer requirements.
“How to Measure Customer Satisfaction”. 2nd edition.
Capturing information on user satisfaction
• Compliments
• Complaints
• Meeting forums
• One to one conversation
• Correspondence (email, letter)
• Questionnaires
Questionnaire Structure
• Importance
• Satisfaction
• Comments
Q1. How important is the availability of access to advice / information from the Pathology Department?
Clinical Biochemistry NA 1 2 3 4 5 6 7 8 9 10Cytology (Gynae.) NA 1 2 3 4 5 6 7 8 9 10Cytology (non Gynae.) NA 1 2 3 4 5 6 7 8 9 10Haematology NA 1 2 3 4 5 6 7 8 9 10Histology NA 1 2 3 4 5 6 7 8 9 10Microbiology NA 1 2 3 4 5 6 7 8 9 10
How satisfied are you with the availability of access to advice / information from the Pathology Department?
Clinical Biochemistry NA 1 2 3 4 5 6 7 8 9 10Cytology (Gynae.) NA 1 2 3 4 5 6 7 8 9 10Cytology (non Gynae.) NA 1 2 3 4 5 6 7 8 9 10Haematology NA 1 2 3 4 5 6 7 8 9 10Histology NA 1 2 3 4 5 6 7 8 9 10Microbiology NA 1 2 3 4 5 6 7 8 9 10
Comments:
Survey Analysis
Response Analysis Cross Discipline
Q1
Mean ImportanceScore
Weightingfactor
StandardDeviation
Mean SatisfactionScore
StandardDeviation
Weightedscore
Chemistry 8.3 0.174 2.25 7.4 2.25 1.29
Gynae Cytology 7.1 0.149 2.55 5.8 2.19 0.86
Non Gynae Cytology 7.4 0.155 2.14 7 2.56 1.09
Haematology 8.4 0.176 2.03 7.4 2.57 1.3
Histology 8.1 0.17 2.1 7.4 2.28 1.26
Microbiology 8.4 0.176 2.03 7.8 2.32 1.37
Total of Mean 47.7
Mean of Mean 7.95 7.13
Graph showing combined mean importance score vs combined mean satisfaction score for each question
0
1
2
3
4
5
6
7
8
9
Mean score
1 2 3 4 5 6 7 8 9 10
Question number
Mean Importance Score
Mean Satisfaction Score
Priorities
For
Improvement
(PFI)
Attributes requiring the greatest improvement in
performance for our Primary Care Users 5 Continue to participate in the Trust’s transport review, with a
view to addressing the sample pick-up time requirements of our Primary Care users most effectively within the resources made available to Pathology.
6 The appropriate disciplines to develop and communicate to our Primary Care users a clear process for test requesting outside national guidelines.
8 Disciplines to develop a system of providing timely information to our Primary Care users on new diagnostic tests we introduce.
Quick Win PFIs
8 The Directorate should introduce an email correspondence
service to improve access to clinical advice for our Primary Care users
9 Provide data on practice test requesting patterns after
consultation of requirements with our Primary Care users.
10 Inform Primary Care of the Pathology website hosted by
HOWIS.
User
Comments
Q9. How important is it to have a clear process for test requesting outside the national guidelines (e.g. repeat test within 58 days for HbA1c?)
The problem is lack of co-ordination between GP and hospital testing – we do not know if a test has just been done in clinic.HbA1c will not be done, realise research and why, but on occasion may be required for urgent assessment – sample always returned not done.If we request outside it doesn’t get done, I then can re-request when appropriate – good to get reminded of the national guidelines.We have only had recent notification of tests being performed re guidelines. Too soon to give accurate comment on satisfaction. Agree with principle of not over testing.If GP requests chol (?cholesterol) earlier than guidelines – test is often denied.Message simply on screen does not allow for eg tests of T3 due to reduced TSH. Just rejects whole sample.When we have a high triglyceride result, we request for a fasting triglyceride level. Request is turned down; the reason we were told was it was too soon. Don’t know what they are.No opinion, we very rarely need test in this category.Need to allow more clinical freedom.Would value up-to-date info ?website.This needs clarification and clear guidelines.It stops waste of clinical time and resources. It helps clarify the indications for the request.Saves unnecessary appointments and procedures and dispels confusion.
GP
Practice Manager.
GP
Practice Nurse
Practice Nurse
GP
GP
GP
GP
GP
GP
Satisfaction IndexBiochemistry
Question Number Mean Importance Weighting Factor Mean Satisfaction Weighted Score
1 8.3 0.104 7.4 0.7696
2 7.5 0.019 6.5 0.1235
3 8.5 0.107 6.9 0.7383
4 8.1 0.102 6.8 0.6936
5 8.4 0.106 6.4 0.6784
6 9.1 0.114 5.5 0.627
7 7.2 0.09 6.9 0.621
8 5.5 0.07 6.7 0.469
9 8.5 0.107 6.5 0.6955
10 8.5 0.107 7.7 0.8239
Total 79.6 6.2398
Satisfaction index 62.40%
Mirror Survey
• Survey Employees
• Highlights Underestimation of Importance
• Facilitates Internal Feedback
WEQAS on-line survey Project
Aims - Seek to demonstrate the potential of structured assessment of user requirement and satisfaction as a reliable, valid, robust indicator of service quality and its continued improvement.
Potential benefits:
• Demonstrate the potential of user satisfaction as a reliable indicator of service quality.
• Promote and roll out good practice – do once and share.
• Reduce duplication of effort. – CPA requires regular surveys to be carried out
• Giving all services access to “good” survey question design.
• The results of structured all Wales surveys following a schedule of topics, will inform future service planning and improvement.
• Compare within and between lab performance of meeting user requirement by use of the overall “satisfaction index”.
• Contributing to developing key performance indicators that capture the value of Pathology services.
Method:
A proof of concept on-line survey using Survey Monkey has been conducted
Pilot to All Wales distributed to Quality ManagersApril 2009: On line survey for national use distributed comprising 10 questions from the Gwent Healthcare NHS Trust user survey
Pilot to local users in ABMU (Swansea/Bridgend area)April 2009: On line survey for local use distributed using the survey principles of the Gwent Healthcare NHS Trust user survey
Step 1 – Design the surveyHow does it work?
To complete the survey- please follow the attached link:
http://www.surveymonkey.com/s.aspx?sm=ZpY5MHue4oUTAPLMI6Ui4Q_3d_3d
Users receive an email with a link to the survey:
….. or post the link on your local trust web site:
Step 2 – User completes survey
Step 3 – Collect the results
Results for all user types…
or filtered by user category… (in this case Consultant medical Staff)
and if you have asked for comments click “view ….”
Export the results to calculate the satisfaction index score….
Performance assessment - the satisfaction index (parametric example)
Mean and standard deviation of the “importance” and “satisfaction” responses will be calculated for each question. A weighting factor will be calculated from the (mean importance / ∑ importance). The “mean satisfaction” value will be multiplied by a weighting factor to calculate a weighted score based on importance. Satisfaction index (%) = ∑ weighted score *10A satisfaction index is calculated for each Laboratory (and a sub-score for each discipline). Performance is assessed within and between Laboratories using the Satisfaction Index.
Satisfaction Index Calculation
Further reading:
“How to Measure Customer Satisfaction”2nd Edition, Nigel Hill, John Brierley andRob Mac Dougall. Gower Publishing Limited.ISBN 0-566-08595-X
“Handbook of customer satisfactionand loyalty measurement” 2nd Edition.Nigel Hill and Jim Alexander. Gower Publishing Limited.ISBN 0-566-08194-6
www.leadershipfactor.co.uk
http://www.nliah.com/portal
A&E – sample questions• Is there an agreement between A&E Consultant staff regarding
requesting certain tests e.g. Thyroid function?
• Does your Pathology deliver a service compatible with achieving A&E targets?
• Is there a difference between the Pathology service:– Out-of-hours and within working hours– Between-Pathology disciplines
• Form filling:– How dissatisfied are you with manual filling in of request forms?– How important is electronic test requesting?
• Add-on tests:– How important is the ability to retrospectively add on tests?– How easy do you find this?
• On-call mortician– How important is your on-call mortician service?– How satisfied are you with your on-call mortician service?
• Toxicology:– Are you satisfied with the local toxicology service that is
provided?
• POCT pregnancy:– Do you use POCT pregnancy testing?– Are you satisfied with local provision?
• POCT glucose:– How satisfied are you with the training and assessment of
competency of users of POCT glucose devices?
• POCT blood gases:– How compliant are medical staff with POCT blood gas analysis
procedures? (e.g. individual password use, disinfection etc)
• Results:– How important is the paper report?– How important is it to see on the computer whether the sample
has been received yet by the laboratory (even if not yet analysed)?
• Incidents:– How responsive is your Pathology Service to investigating
relevant incidents which arise in A&E?
• Emergency Planning:– Does your Trust have one system for assigning emergency
specimen numbers for a major incident?– Does your A&E department have a stock of in-date tubes for
sample collection for chemical incidents?
Who to contact?
• To whom should questionnaire be sent?
• Suggest:– All lead Consultants– Named Lead Nurse– Service Manager
Results
• Report for All-Wales
• Sub-report for each individual Trust
• ABMU to provide support for collation of results
In summary
Thank you
• The assessment of user requirement and satisfaction is a CPA requirement.
• It informs your policies
• It is important to have a structured approach to measuring this.
• The degree of user satisfaction is as legitimate and important a measure of quality as the more traditional assessments of accuracy/precision and timeliness.
• Electronic survey methods enable Laboratories to compare their satisfaction scores (when distributing identical surveys) and set bench marking standards.
• It is a driver for continuous service improvement