Embed Size (px)
Citation preview
Continuous Quality Improvement for CBT Groups
Jennifer Lux, M.A.Research Associate, Corrections
InstituteCenter for Criminal Justice Research
University of Cincinnati
First Annual SymposiumQuality Assurance
Continuous Quality Improvement
Objectives
• Discuss the importance of fidelity
• Identify and review key measures of CQI
• Review and practice how to provide CQI for group interventions –Using group facilitator evaluation form
Without Observation
• No chance to…– Reinforce good work– Correct mistakes or inconsistencies– Identify gaps in skills– Assure that there is fidelity to model
Fidelity to an Evidence-Based Practice
• Fidelity:Implementing the intervention as closely as possible to the way it was designed and delivered during the research stage
• Evidence-Based Practice:A practice that has been shown to work through use of scientific research
Evidence Based Practice is:
• Easier to think of as Evidence Based Decision Making
• Involves several steps and encourages the use of validated tools and treatments.
• Not just about the tools you have but also how you use them
Principles of Effective Intervention
The most effective programs are based on the principles of effective intervention:
Fidelity Principle
Making sure the program is delivered as designed and with integrity:
• Ensure staff are modeling appropriate behavior, are qualified, well trained, well supervised, etc.
• Make sure barriers are addressed but criminogenic needs are targeted
• Make sure appropriate dosage of treatment is provided
• Monitor delivery of programs & activities, etc.
• Reassess offenders in meeting target behaviors
What Happens When We Don’t Have Fidelity?
DRIFT•Accidental adaptation can pose significant problems
•Too much adaptation might decrease an intervention's effectiveness
What Happens When We Don’t Have Fidelity?
Schoenwald et al. (2003) – Therapist adherence to the model predicted post-
treatment reductions in problem behaviors of the clients
Henggeler et al. (2002)– Supervisors’ expertise in the model predicted
therapist adherence to the model
Sexton (2001)– Direct linear relationship between staff competence
and recidivism reductions
What Happens When We Don’t Have Fidelity?
Landenberger and Lipsey (2005)– Compared different cognitive-behavioral curricula to
determine which was most effective
– “Brand” of CBT didn’t matter but quality of implementation did
– Implementation defined as low dropout rate, close monitoring of quality and fidelity, and adequate training for providers
– Programs with good implementation demonstrated better outcomes than programs with poor implementation
Evidence Supporting Supervision
• Conditions– Manual Only– 14 hour Workshop– Workshop + Feedback– Workshop + Coaching– Workshop + Feedback + Coaching
All had initial skill acquisition
Miller, et. al., A Randomized Trial of Methods to Help Clinicians Learn Motivational Interviewing. Journal of Consulting and Clinical Psychology (2004)
Four Months Later
• Conditions– Manual Only– 14 hour Workshop– Workshop + Feedback– Workshop + Coaching– Workshop + Feedback + Coaching
Only condition that maintained benefit after 4 months
Could not detect
who had training
Is this true with cognitive-behavioral programming?
• Conditions– Manual Only– Manual + Web-based Training (40 hours)– Manual + Training + Supervision
(observation and feedback)
Sholomskas, et. al., We don’t Train in Vain: Three Strategies of Training Clinicians in CBT. Journal of Consulting and Clinical Psychology (2005)
Cognitive-Behavioral Intervention
• Conditions– Manual Only– Manual + Web-based Training – Manual + Training + Supervision
No Transfer
Modest Transfer
Proficient
Sholomskas, et. al., We don’t Train in Vain: Three Strategies of Training Clinicians in CBT. Journal of Consulting and Clinical Psychology (2005)
The Importance of Fidelity
Every major study shows a strong relationship between program integrity
and recidivism
INTEGRITY
RECIDIVISM
What Do We Know About Fidelity?
• Fidelity is related to successful outcomes (i.e., recidivism reductions)
• Poor fidelity can lead to null effects or even iatrogenic effects
• Fidelity can be measured and monitored
• Fidelity cannot be assumed
Some Lessons Learned from the Research
• Who you put in a program is important – pay attention to risk
• What you target is important – pay attention to criminogenic needs
• How you target offenders for change is important – use behavioral approaches
• Program Integrity makes a difference - Service delivery, training/supervision of staff, support for program, QA, evaluation, etc.
CQI – What Is It?
• A method of continuously examining processes and making them better
• Key principles:– Use of data and team approaches to improve
decision making– Involvement of entire organization to improve quality– Strong focus on customers– Continuous improvement of all processes and
outcomes
CQI versus QA
QA: QUALITY ASSURANCE
•Retrospective review process•Emphasis on regulatory and contract compliance•Catching people being bad leads to hide and seek behavior
21
CQI versus QACQI: CONTINUOUS QUALITY ASSURANCE
•A prospective process
•Holds quality as a central priority within the organization
•Focus on customer needs; relies on feedback from internal and external customers
•Emphasizes systematic use of data
•Not blame-seeking
•Trust, respect, and communication
•Move toward staff responsibility for quality, problem solving and ownership of services 22
Benefits of CQI
• Proof of effective services• Maintain or secure funding
• Improve staff morale and retention
• Educate key stakeholders about services
• Highlights opportunities for
improvement
• Data to inform quality improvement
initiatives
Benefits of CQI
• Establish/enhance best practices
• Monitor/ensure treatment fidelity
• Evidence that programs utilizing
CQI/evaluation processes demonstrate
better outcomes
• Engaging in CQI helps organizations to
achieve their mission
CQI process for Group Interventions
Feedback and Coaching
Feedback and Coaching
• Place emphasis on learning process – don’t try evaluation until a pre-planned learning curve period
• Place emphasis on improving, rather than criticizing
Fidelity to Model
• Not a discussion of RIGHT or WRONG
• Key is to identify what is CONSISTENT OR INCONSISTENT with the model
Process of Supervision
1. Direct observation of sessions
2. Structured feedback about adherence and competence
3. Coaching to improve implementation with proficiency
4. Continue evaluation with frequency until reach proficiency
Coaching Format• Staff should be afforded the opportunity
to share their impressions of the session
• Coach should provide direct and specific feedback - Show them the rating form- Review strengths- Areas for improvement, give specific examples
• Concrete plans should be developed to address areas needing improvement
Rating Form• Should include facilitator ratings in the
following general areas:– Knowledge of cognitive-behavioral
interventions and ability to explain new skills and concepts
– Adherence to curriculum– Group behavior management– Ability to effectively communicate and build
rapport with participants
One example…
• Let’s review the Group Facilitator Evaluation Form
Practice Activity
• Watch the following group demonstration and use your Group Facilitator Evaluation Form to record observations
Designing an Ongoing CQI Process
Develop Proficient Staff
• Define proficiency– Provide information to staff
• Once rated as proficient, decrease observation frequency
For Non-Proficient Staff
• Define proficiency– Provide information to staff
• Work on change planning until proficiency is reached
Change Planning
• Plan of correction• Proactive approach to problem solving• Empower staff• Use objective data to inform decision
making
Use of PDSA Cycles
• PLAN – analyze a process, determine what changes are needed, and establish a plan for making the changes.
• DO – implement the changes on small scale.
• STUDY – check to see whether the changes are working.
• ACT – if changes are working, implement changes on larger scale. If not, begin the cycle again.
Strategies for Improvement• Role play• Trial and error• Written vignettes – if client says . . . you respond
_______.• Role play and make common mistakes – have them
identify mistakes and how to improve upon those• Coach develops a pictogram or flow chart of methods• Project client responses• Rewards and consequences
Developing your CQI team
Effective CQI Leadership
• Who are the leaders?• What makes them good leaders?• What is their role in the CQI process?• Management vs. Leadership
What Can You Do?
• Know the mission of your organization and your role in helping to achieve that mission
• Know the evidence-based practices for your client population and try to apply them with fidelity every day
What Can You Do?
• Become an active member of the CQI process
• Communicate with others about the value of CQI
• Learn to look for available data prior to engaging in problem-solving
• Understand that acknowledging that the agency can improve its performance on a particular measure does not mean that the agency is saying that staff are not competent at their jobs.
• Adopt a personal philosophy that you want to be better at your job tomorrow than you are today.
Summary
• The relationship between strong fidelity and decreased recidivism is proven
• Group interventions can be supervised using CQI methods
• The goal is to guide facilitators to proficiency
