521: The Quality Service Review Process. The Pennsylvania Child Welfare Resource Center Day 1 - Agenda Training Overview Overview of the QSR Process Overview

521: The Quality Service Review Process

521: The Quality Service Review ProcessThe Pennsylvania Child Welfare Resource Center

Day 1 - Agenda

• Training Overview• Overview of the QSR Process• Overview of the Indicators• Roll-Up Sheet• The QSR Interviews• Child/Youth and Family Status Indicators

2


Day 2 – Agenda

• Practice Performance Indicators• Feedback Session with the

Caseworker/Supervisor• Case-Specific Team Debriefing• The QSR Written Case Review Summary• Closing and Evaluation

3


Learning Objectives

• Identify the components of the QSR Process;• Recognize the QSR Indicators;• Apply the interview techniques relative to the

protocol;• Assign reliable status and performance

indicator scores;• Demonstrate a Caseworker/Supervisor

Feedback Session;• Identify the steps to prepare for the Case-

Specific Debriefing; and• Recognize the components of the QSR Written

Case Review Summary.4


PA CQI

Positive Change

QSR

QSR’s Connection to CQI

5


DAPIMTM and “Learning by Doing”

Implement

Work products – team activities, action plans, charters for working teams, communication efforts

Plan

Work products- quick wins, mid- and long-term improvement plans, communication and capacity plans

Define

Work products – baseline surveys, alignment notes, initial feelings, ground rules, defined areas for improvements, desired future state

Performance & Capacity

Monitor

Work products – monitoring quick wins and other changes, evaluations and measures of progress and impact

Assess

Work products – findings: strengths and gaps, root causes and general remedies

6


Purposes of the QSR Process

• To appraise the current status of the focus child/youth and family and the performance of key practice areas;

• To assure that the actions of a team of child welfare professionals are guided by clear and comprehensive policies and are rooted in Pennsylvania’s Practice Model/Standards;

• To improve the quality and usefulness of information needed to evaluate the safety, permanence and well-being of children and youth;

7


Purposes of the QSR Process (continued)

• To assure the quality and effectiveness of services for children, youth and families by regularly reviewing programs and practices;

• To support the efforts of child welfare agencies to maintain an environment that encourages learning and program improvement; and

• To stimulate actions and next steps, to build on strengths and support efforts to improve outcomes.

8


What is the QSR?

The QSR is designed to conduct an in depth review of front line practice in specific locations and points in time.

It is used for:

• Appraising the current status of a child/youth in key life areas;

• Appraising the current status of the parent/caregiver; and

• Appraising the performance of key practices for the same child/youth/family.

9


How is the QSR Different From Other Quality Assurance Methods?Traditional Review Were visits timely?

Is the child’s IEP inthe file?

Was there a referralto mental health?

Quality Service Review

Are children currentlysafe?

Is the child progressing well academically?

Is the child functioning well emotionally?

10


How is the QSR Different From Other Quality Assurance Methods? (continued)

Traditional Review

Is there a current plan in the file?

Did the parents sign the plan?

Is there a written assessment?

Quality Service Review

Is the plan likely to lead to permanence?

Do the parents feel that the plan is their own?

Does the assessment accurately identify underlying needs?

11


Skills and Abilities

Discuss with your partner the following questions:

• Which three challenges or needs are most important for you to overcome in order to become a successful QSR Reviewer?

• On a scale of 1-10, with one being a minimal challenge or need and 10 being a major challenge or need, how would you rate your three identified challenges or needs?

• What will it take for you to decrease your rating 1-2 points?

12


The QSR Process

1. Review Team Pairings2. Sample Selection3. Focus Groups and Stakeholder Interviews4. Case Reviews5. Feedback Session with the

Caseworker/Supervisor6. Case-Specific Debriefing7. QSR Written Case Review Summary8. Exit Conference (optional for reviewers)9. Final Report10.Next Steps Meeting11.County Improvement Plan

13


What Happens Prior to the Onsite Review?• Identification and pairing of internal and

external reviewers;

• Random sample of a county’s cases are selected and stratified by a service area;

• Family permission obtained to participate in the review and consents/releases obtained; and

• Key interviews in each case are identified and appointments are scheduled.

14


Focus Groups and Stakeholder Interviews

Purpose:– Obtain feedback specific to systemic

issues rather than case-specific information; and

– Provide a forum for open dialogue about

challenges and solutions within the system.

15


Focus Groups and Stakeholder Interviews

Invited to…

Focus Groups:• Caseworkers, Supervisors, Managers, Service

Providers, and Resource Parents

Stakeholder Interviews:• Judges, Attorneys, Police, Probation Officers,

County Commissioners, etc.

16


Exit Conference – What Happens?

• A few reviewers will be asked to present the highlights from their case review.

• The Local and State Site Leads deliver the aggregate findings from all the cases reviewed along with trends discovered from the focus groups and stakeholder interviews.

• Key strengths and areas needing improvement are identified.

17


Final Report – What Happens?

• Final Report which includes analysis of demographic information, quantitative analysis of indicator scores, and information from the County Specific Data Packet is shared.

• The county will use the data and analysis to develop a County Improvement Plan.

18


Next Steps Meeting – What Happens?

• Approximately 6-8 weeks after the onsite review;

• Led by the Local and State Site Leads;

• Lasts approximately two hours;

• Discuss the recommendations for next steps in working toward an action plan for improving frontline practice and system performance; and

• Begins the next step in the CQI process.19


What is Your Role?Each pairing of reviewers will:

• Conduct case reviews:

Case file review - overview of the case record;

Interviews - meet with key players involved in the case;

Roll-Up Sheet - demographic information and scoring;

• Provide feedback to the caseworker (and supervisor) - immediate feedback including suggestions and potential next steps for the case;

• Present information gathered from case review and feedback on the family story at the Case-Specific Team Debriefing;

• Complete a QSR Written Case Review Summary; and

• Reviewers may be invited to attend the Exit Conference.

20


Child/Youth and Family Status Indicators

• Safety: Exposure to Threats of Harm;• Safety: Risk to Self/Others;• Stability;• Living Arrangement;• Permanency;• Physical Health;• Emotional Well-Being;• Early Learning and Development;• Academic Status;• Pathway to Independence; and• Parent or Caregiver Functioning.

21


Practice Performance Indicators• Engagement Efforts;• Role and Voice;• Teaming;• Cultural Awareness and Responsiveness;• Assessment and Understanding;• Long-Term View;• Child/Youth and Family Planning Process;• Planning for Transitions and Life Adjustments;• Efforts to Timely Permanence;• Intervention Adequacy and Resource Availability;• Maintaining Family Relationships; and• Tracking and Adjusting.

22


What Time Period is Considered in Scoring?• Child/Youth and Family Status Indicators

look at the past 30 days– Exceptions are: Stability and Academic Status

• Practice Performance Status Indicators look at the past 90 days– Exceptions are: Planning for Transitions and Life

Adjustments and Efforts to Timely Permanence

NOTE: Recent performance improvements have some influence on scores

23


How are Cases Scored?

Each indicator is scored on a scale of 1 – 6• 6 – Optimal• 5 – Substantial Acceptable Range• 4 – Fair• 3 – Marginal• 2 – Poor Unacceptable Range• 1 – Adverse

Maintenance Zone: 6-5Refinement Zone: 4-3

Improvement Zone: 2-1

24


Distinction Between 3 and 4

The Groundhog Rule

If this case were frozen in time as it is

today, would performance or status be

considered acceptable or unacceptable?

25


The Central Scoring RulesRules Include:

• Concentrate on the Focus Measure and Core Concepts in each indicator.

• Read each indicator’s instructions: – Some indicators score results, others score effort or intent.– Not Applicable rating criteria are identified as appropriate.– Score each sub-category within each indicator, as appropriate.

• Stay within the time-based observation windows associated with each indicator.

• Rate indicators based on events that have occurred or conditions that were present within the time-based observation window.

26


Roll-Up Sheet Purpose

Purpose

• Gather Data and Monitor Practice Model Implementation:

– Demographic information– Case information– Indicator scores

27


Roll-Up Sheet Completion

• Basic case information is filled out by the caseworker prior to onsite review.

• Remaining information is gathered through case file review and interviews.

• Reviewers will submit a validated Roll-Up Sheet electronically to Site Leads on a flash drive on the final day of onsite review.

28


Roll-Up Sheet Components

1. General Review Information2. Child/Youth’s Demographic Information3. Child/Youth’s Current Placement4. Child/Youth’s Educational Information5. Case Information6. Child/Youth and Family Status Domain7. Practice Performance Status Domain

29


Planning QSR Interviews

• Know the case history;

• Plan relevant questions for each respondent;

• Have accurate directions and be punctual; and

• Organize the interview to help the respondent participate.

30


Prior to the Interview

• Review the interview schedule and begin preparation by identifying potential questions;

• Identify the lead interviewer and notetaker for each interview;

• Remember that you are a reviewer and not a caseworker during the QSR process; and

• Learn something about the interests of the child/youth to better engage him/her.

31


During the Interview• Take notes to document the interview;• Take every opportunity to recognize family strengths;• Listen more than talk;• Use the list of QSR questions to keep you on track;• When needed, ask questions on the Roll-Up Sheet;

32


During the Interview (continued)

• Always ask about the child’s safety (if safety threats are identified, call site lead immediately);

• Be patient with family stories;• Gather information, do not identify

solutions to the case’s or family’s concerns;

• Develop the relationship before inquiring about sensitive family issues; and

• Avoid leading questions.

33


After the Interview

• Do not share information gathered during one interview to the next interview;

• Resist reaching conclusions prior to the completion of all interviews;

• Contact your site lead or next interview if you are running a few minutes behind;

• Note areas needing confirmation or follow-up; and

• Contact your site lead if interviews lead to other key people in the child/youth’s life that you feel would be important to talk to regarding the case review.

34


The QSR Interview – Exploring Questions for Providers

For clinicians - What are your goals for this child/youth/family?

What are the child/youth/family’s goals? What are the strengths of this child/youth/family? What worries you most about this case? Tell me what you are working on now. Who do you see as members of this family’s team? What’s the next big hurdle for this child/youth/family? What will success look like? On a scale of 1-10, how likely do you think success will

be achieved? Ask follow-up questions. Where do you see this case in six months?

35


Interviewing Practice – Sara’s Case

• Both reviewers read the handout, Facts about the Fisher Family.

• Reviewer playing the role of the mother reads the handout, Interview with Cynthia Fisher.

• Reviewer playing the role of reviewer considers, What do you want to learn in the interview with Sara’s mother?

• Conduct the interview.

36


Child/Youth and Family Indicators• With your partner, review Handouts #6-10.

• Summarize the indicator, score the indicator and identify two good questions for each indicator you are assigned.

Group 1: 5 and 1aGroup 2: 2 and 1bGroup 3: 4 and 5Group 4: 3 and 6Group 5: 7b and 4Group 6: 6 and 9

• Be prepared to join other teams in a group discussion about the scoring experience.

37


Child/Youth and Family Indicators• With your partner, review Handouts #6-10.


Group 1: 1a and 9Group 2: 2 and 1bGroup 3: 4 and 5Group 4: 3 and 6Group 5: 7b and 4

Be prepared to join other teams in a group discussion about the scoring experience.

38


Practice Performance Indicators• With your partner, review Handouts #6-10.


Group 1: 1a and 5Group 2: 9 and 11Group 3: 2 and 10Group 4: 3 and 7Group 5: 4 and 8Group 6: 1b and 6


39


Practice Performance Indicators• With your partner, review Handouts #6-10.


Group 1: 1a and 5Group 2: 6, 9, and 11Group 3: 2 and 10Group 4: 1b, 3, and 7Group 5: 4 and 8


40


Caseworker/Supervisor Feedback Session• Thank the caseworker and supervisor for their time.• Review the QSR purpose. • Outline the strengths of the case.• Remind caseworker of the high QSR standards. • Very briefly summarize family story – ensure facts are

correct.• Share new information, if applicable.• Ask caseworker for ideas for next steps.• Discuss the strengths and challenges of the

indicators.• Share reviewer ideas for next steps.• Thank the caseworker/supervisor and emphasize that

they had a meaningful role in making a difference for children, their families and the system.

41


Feedback Session Practice• Please complete Handout #13, Notes for Providing

Feedback to Caseworkers.• When completed, one partner will be the caseworker; the

other partner, the reviewer.• Practice a feedback session providing the caseworker

feedback on Sara’s case. • Compare your notes with your partner from Handout

#13, did you identify similar:– Strengths/Accomplishments;– Practice Challenges/Limiting Conditions;– Opportunities for Advancing Practice/ Improving Local

Conditions (Next Steps).

42


Case-Specific Team Debriefing

• Inter-rater reliability check for consistency between case description and scores.

• Identification of themes for feedback to the county and written report.

• Preliminary communication about review findings to leadership.

43


Case-Specific Team Debriefing Outline

Case Presentation Outline – 15 minutes• Case Summary (2 minutes)• Child/Youth and Family Status Indicators –

Strengths and Challenges (4 minutes)• Practice Performance Indicators – Strengths

and Challenges (4 minutes)• Next Steps (2 minutes)• Group Questions/Feedback (3 minutes)

44


The QSR Written Case Review Summary - Purpose

• Explain the “story” behind the data;

• Provide feedback to the caseworker and local leadership; and

• Be utilized as an effective training tool within the county agency.

45


The QSR Written Case Review Summary- Additional Guidelines• Should not tell the case story/history; but rather include

information you gathered to support the ratings of the indicators;

• Written about the ratings and why the factors contributing to the indicators are either favorable or unfavorable;

• Practical and helpful recommendations; and • Make a note of your due date:

Three day Review – Due prior to the conclusion of the 3rd day

Two day Review – Due approximately 7-10 days following the onsite QSR

NOTE: There will be an editing process done by the Site Lead Team, so you may be asked to edit your Written Case Review Summary and/or provide more supportive information. 46


CQI Continues…

Remember QSR is NOT a one-time event. It’s part of the bigger CQI process that leads to

improved outcomes of Safety, Permanence and Well-Being for children, youth and families.

47

Documents

521: The Quality Service Review Process. The Pennsylvania Child Welfare Resource Center Day 1 - Agenda Training Overview Overview of the QSR Process Overview