Child and Adolescent Characteristics, Outcomes, and Wraparound Fidelity:

Results from Eight California Programs

Abram Rosenblatt, Michelle Coufal, Kate Cordell, Elisha Heruty,

Catherine Aspiras, Mary Ann WongEMQ FamiliesFirst

2

Presentation Overview

• Youth Profile and Core Outcomes• Child and Adolescent Needs and Strengths

(CANS) Outcomes• Wraparound Fidelity Index (WFI-4) Profile and

Outcomes• Longitudinal Outcomes

3

Number of Youth Served

Total Number of Youth Served

in Wraparound: 4,43242%

24%

21%

3%

11%

Bay Area Capital Central Inland EmpireLos Angeles

4

Number of Youth Served in CY11

Total Number of Youth Served in

Wraparound in CY11: 1,158

28%

19%

6%

24%

24%

Bay Area Capital Central Inland EmpireLos Angeles

5

Youth’s Profile• Average Age at Admission: 14 years• 61% Male; 39% Female• Youth’s Ethnicity

Native American 1%

Other 2%

African American21%

Asian American/Pacific Islander

3%

Latin American30%

Caucasian 40%

6

Youth’s Profile in CY11• Average Age at Admission: 14 years• 60% Male; 40% Female• Youth’s Ethnicity

Native American

1%Other

1%

African American

17% Asian American/

Pacific Islander

2%

Latin American46%

Caucasian 34%

7

Primary Diagnosis at Admission

6%

48%

35%

2%9%

0%

20%

40%

60%

80%

100%

Adjustment MoodDisorder

DisruptiveBehavior

Psychotic Other

8

Primary Diagnosis at Admission for Youths Served in CY11

9%

39%44%

1%7%

0%

20%

40%

60%

80%

100%

Adjustment MoodDisorder

DisruptiveBehavior

Psychotic Other

9

Living Situation at Admission

55%

16%

0.2% 2%

24%

1% 2%0%

20%

40%

60%

80%

100%

Home

Foster Family

Hospital

Justice

Residential

Shelter/Homeless

Other

10

Living Situation at Admission for Youths Served in CY11

60%

23%

0% 2%15%

0.3% 0.3%0%

20%

40%

60%

80%

100%

Home

Foster Family

Hospital

Justice

Residential

Shelter/Homeless

Other

11

Outcomes for Discharged Youth

12

Profile of Discharged Youth

• Number of Youth Discharged from Wraparound:

3,893• Average Length of Stay: 12 months

13

Profile of Youth Discharged in CY11

• Number of Youth Discharged from Wraparound:

607• Average Length of Stay: 11 months

14

Living Situation: Admit vs. Discharge

67% 73%

29%23%

3% 3% 0.3% 1%

0%

20%

40%

60%

80%

100%

Community Facility Other Unknown

Admit Discharge

15

Living Situation: Admit vs. Discharge for Youth Discharged in CY11

82%80%

17% 17%

1% 1% 0.4% 2%

0%

20%

40%

60%

80%

100%

Community Facility Other Unknown

Admit Discharge

16

Core Outcomes for Discharged Youth

80% 82% 82%

0%

20%

40%

60%

80%

100%

In Home In School Out of Trouble

17

Core Outcomes for Youth Discharged in CY11

81% 83% 84%

0%

20%

40%

60%

80%

100%

In Home In School Out of Trouble

18

Longitudinal Core Outcomes

77%85%

76%

0%

20%

40%

60%

80%

100%

In Home In School Out of Trouble

19

Why the CANS?

• Item Level Tool• Items translate immediately into action levels • It is about the youth, not about the service • Cultural and developmental contexts are considered

before establishing action levels • It is about the ‘what’, not about the ‘why’

• 30 day window for rating unless otherwise specified

20

CANS Overall Reliable Change Index (RCI)

21

CANS Overall RCI by Program

22

CANS Life Domain Functioning Domain RCI by Program

23

CANS Child Strengths Domain RCI by Program

24

CANS Caregiver Strengths and Needs Domain RCI by Program

25

CANS Child Behavior and Emotional Needs Domain RCI by Program

26

CANS Child Risk Behaviors Domain RCI by Program

27

What is the WFI-4?

• Measures implementation of wraparound process• Set of interviews – Facilitator, Caregiver, and Youth

– 40 items for Facilitator and Caregiver– 32 items for Youth

• Confidential interviews w/multiple respondents unique perspectives

28

Interview Detail:

• 671 facilitators interviewed

– 40.3 minutes average time

• 522 caregivers interviewed

– 39.1 minutes average time

• 305 youth interviewed

– 34.3 minutes average time

Summary of Respondents

Administration Time Frame: January 2008 – January 20121,498 interviews from 671 families

8 traditional wraparound programs in 7 counties:

•Santa Clara•Sacramento•Nevada•Yolo•Fresno•San Bernardino•Los Angeles

29

Demographics of WFI Youth• Average Age at Admission: 13.5 years• 57% Male; 43% Female• Youth’s Ethnicity

Caucasian 38%

Latin American41%

Asian American/Pacific Islander

3%

African American17%

Other 1%

Native American 1%

30

Primary Diagnosis at Admission for WFI Youth

10%

45%39%

1%6%

0%

20%

40%

60%

80%

100%

Adjustment MoodDisorder

DisruptiveBehavior

Psychotic Other

31

Living Situation at Admission for WFI Youth

54%

19%

0% 1%

24%

0.3% 0.3%0%

20%

40%

60%

80%

100%

Home

Foster Family

Hospital

Justice

Residential

Shelter/Homeless

Other

32

Outcomes for Discharged WFI Youths

33

Living Situation: Admit vs. Discharge for WFI Youth

70%81%

29%

17%

1% 1% 0.3% 1%

0%

20%

40%

60%

80%

100%

Community Facility Other Unknown

Admit Discharge

34

EMQ FF vs National Data: Overall Fidelity

83%77%

87%83% 80%

75% 78%73%

0%

20%

40%

60%

80%

100%

Combined Facilitator Caregiver Youth

EMQ FF National Mean

35

EMQ FF vs National Data : Fidelity Scores by Phase

82%76%

86%76%

88%81%

76%69%

0%

20%

40%

60%

80%

100%

Engagement Plan Development Implementation Transition

EMQ FF National Mean

36

EMQFF Phase Scores by Program

81%86%

89%

78% 80%

87%90%

78%84%85%87%

76%

85%83%87%

76%81%

89%90%

75%78%

87%92%

80%84%

87%87%

75%81%

78%83%

70%

0%

20%

40%

60%

80%

100%

Program 1 Program 2 Program 3 Program 4 Program 5 Program 6 Program 7 Program 8

Engagement PlanningImplementation Transition

37

EMQFF Principle Scores by Program

91%

70%70%

94%89%

94%

72%73%

95%

82%

92%

80%

67%

92%

79%

90%

78%

65%

92%

79%

94%

72%72%

96%

82%

90%

76%

82%

93%87%

91%

76%

67%

92%

82%

91%

72%

64%

89%

69%

0%

20%

40%

60%

80%

100%

Program 1 Program 2 Program 3 Program 4 Program 5 Program 6 Program 7 Program 8

Family Voice and Choice Team-BasedNatural Supports CollaborationCommunity-Based

38

EMQFF Principle Scores by Program continued

97%

78%

91%

82%79%

97%

78%

90%

81%83%

95%

78%

91%85%

72%

96%

74%

86%84%

76%

98%

80%

89%

82%83%

95%

78%

89%86%

76%

96%

76%

90%85%

76%

92%

70%

81%86%

66%

0%

20%

40%

60%

80%

100%

Program 1 Program 2 Program 3 Program 4 Program 5 Program 6 Program 7 Program 8

Cultural Competence IndividualizedStrength-Based PersistenceOutcome- Based

39

Average Total WFI Scores By Program

EMQ Wraparound Program Average Total WFI Score

Program 1 84%

Program 2 85%

Program 3 83%

Program 4 82%

Program 5 85%

Program 6 85%

Program 7 83%

Program 8 78%

40

Core Outcomes for Discharged WFI Youth

80%84%

90%

0%

20%

40%

60%

80%

100%

In Home In School Out of Trouble

41

Longitudinal Core Outcomes for WFI Youth

85% 88% 83%

0%

20%

40%

60%

80%

100%

In Home In School Out of Trouble

42

Data Implications

• Further studies employing the WFI and CANS on Outcomes

43

WFI and CANS Outcomes

• Is our fidelity associated with our outcomes?

• Which fidelity elements are more strongly associated with which outcomes?

44

WFI & CANS Outcomes

• 324 clients with two CANS Outcomes at least 6 months apart and a WFI at 6 months of service

45

WFI & Life Domain Functioning CANS

• Improved or Declined CANS groups determined by Reliable Change Index (RCI)

46

WFI & Child Strengths CANS

• Improved or Declined CANS groups determined by Reliable Change Index (RCI)

47

WFI & Caregiver Strengths/Needs CANS

• Improved or Declined CANS groups determined by Reliable Change Index (RCI)

48

WFI & Child Behav. & Emo. Needs CANS

• Improved or Declined CANS groups determined by Reliable Change Index (RCI)

49

WFI & Child Risk Behaviors CANS

• Improved or Declined CANS groups determined by Reliable Change Index (RCI)

50

WFI and CANS Outcomes

• Which source of fidelity scores are more strongly associated with improved outcomes?

51

WFI-Facilitator Source & CANS

• Facilitator fidelity scores are strongly associated with CANS outcomes scores.

52

WFI-Caregiver Source & CANS

• Caregiver fidelity scores are lower than facilitator fidelity scores.

53

WFI-Youth Source & CANS

• Youth fidelity scores are lower than facilitator scores and exhibit reversal of pattern in some elements for association between fidelity and outcomes.

54

WFI and CANS Outcomes

• How is fidelity at different phases of the program associated with CANS outcomes?

55

WFI by Phase & CANSThe group of clients who only

declined in CANS domains had

significantly lower fidelity scores at

the:

• Planning (p=0.04),

• Implementation (p=0.005)

• Transition (p=0.001)

phases of the Wraparound program

as compared to the group of clients

who improved in at least one CANS

domain.

56

WFI and CANS Outcomes

• Focused Quality Improvement:– In what fidelity elements should we focus our energies in order

to maximize improvement in Child Behavioral and Emotional Needs outcomes?

• Measurable Quality Improvement– How much improvement (i.e., what increased percentage of kids

would have reliable improvement) would we expect to see if we brought all of these elements in our programs up to ‘high fidelity’

57

Improving Fidelity and CANS Outcomes • There were 317 clients with a

CANS Child Behavioral and Emotional Needs score which could be reliably improved from Time 1 (GTE RCI of 2.2).

• Overall, 45.1% of clients improved between Time 1 and Time 2.

• Programs with higher fidelity in certain elements resulted in better outcomes.

Order of elements determined by classification and regression tree (CART).

45.1 %40.1 %14.8 %

High fidelity (>=85%) for Element 5 (Community Based) = YES

47.2 %38.1 %14.8 %

High fidelity (>=85%) for Element 6 (Culturally Competent) = YES

48.8 %36.9 %14.3 %

High fidelity (>=85%) for Element 9 (Persistent) = YES

52.3 %36.7 %11.0 %

High fidelity (>=85%) for Element 10 (Outcomes Based) = YES

56.5 %37.0 %6.5 %

Child Behavior and Emotional Needs

176

Improved

No Change

Declined

Child Behavior and Emotional Needs

168

Improved

No Change

Declined

Child Behavior and Emotional Needs

109

Improved

No Change

Declined

Child Behavior and Emotional Needs

46

Improved

No Change

Declined

Improved

No Change

Declined

Child Behavior and Emotional Needs

317

58

WFI and CANS Outcomes Summary

• Our fidelity scores are associated with our CANS outcomes

• Facilitator fidelity scores are more strongly associated with CANS outcomes

• The data suggests that focused quality improvement in fidelity will result in a measurable improvement in CANS outcomes

59

Challenges

New staff New Measure Implementations

Difficult populations Language barriers

Ratio of certified interviewers to interviewees Relatively low response rates

60

Lessons Learned

Program buy-in Work with Wrap teams Use EMQ FF language certified employees

61

Future Directions

62

Future Directions

• Build on Initial Analyses• Translate to Clinical Staff• Consider Fidelity Intervention Options• Ongoing WRAP Fidelity Feedback (Bruns)

• Build Ongoing Reporting Mechanisms• Link to Quality of Care and Key Performance Indicators

63

Q & A

64

References

• Bruns, E. (nd). Wraparound Fidelity Assessment System. Retrieved from Wrap Info website: http://depts.washington.edu/wrapeval/WFI.html

• Lyons, J. Child and Adolescent Needs and Strengths. Retrieved from Praed Foundation website: http://www.praedfoundation.org/About%20the%20CANS.html

• Wraparound Evaluation & Research Team. The Wraparound Process. Retrieved from http://depts.washington.edu/wrapeval/approach.html

65

Contact Information

Abram Rosenblatt arosenblatt@emqff.org (408) 364-4016