SUPERVISOR CORE: MANAGING FOR RESULTS · example, minor revisions are indicated by the numbered sequence Version 1.1, 1.2, 1.3, etc., while major revisions are delineated by a sequence

California Com

mon Core Curricula for Child W

elfare Supervisors

SUPER

VISOR CO

RE: M

ANAG

ING

FOR R

ESULTS

Trainer’s Guide

Version 1.1 | July 2009

California Common Core Curricula for Child Welfare Supervisors

SUPERVISOR CORE: MANAGING FOR RESULTS Trainer’s Guide


This Page Intentionally Left Blank

TAB 1

Title Page


California Common Core Curricula for Child Welfare Supervisors

SUPERVISOR CORE: MANAGING FOR RESULTS Trainer’s Guide



TAB 2

Acknowledgments


Acknowledgments California’s Common Core Curricula for Supervisors is the result of the invaluable work and guidance of a great many people throughout the child welfare system in California and across the country. It would be impossible to list all of the individuals who contributed, but some groups of people will be acknowledged here. The Statewide Training and Education Committee (STEC) provided overall guidance for the development of the curricula. Convened by the California Social Work Education Center (CalSWEC) and the California Department of Social Services (CDSS), STEC has a wide membership that includes Regional Training Academy (RTA) representatives, county representatives, university-based Title IV-E Project Coordinators, the Inter-University Consortium in Los Angeles (IUC), and other key stakeholders. A subcommittee of STEC, the Content Development Oversight Group (CDOG), provided oversight and approval for the research, writing, and revising performed by curriculum developers. Additional review and guidance for the curricula was provided by a Content Advisory Team (CAT) composed of representatives from California’s regional training entities, universities, and county development programs. As with many large curriculum projects in public child welfare, significant portions of the Supervisor Core were adapted from existing curricula. Along the way, many other people provided their insight and hard work, attending pilots of the trainings, reviewing sections of curricula, or providing other assistance. California’s child welfare system greatly benefits from this collaborative endeavor, which helps our workforce meet the needs of the state’s children and families. The curriculum is developed with public funds and is intended for public use. For information on use and citation of the curriculum, please refer to:

http://calswec.berkeley.edu/CalSWEC/CCCCA_Citation_Guidelines.doc

FOR MORE INFORMATION on California’s Core Curricula, as well as the latest version of this curriculum, please visit the California Social Work Education Center (CalSWEC) website: http://calswec.berkeley.edu

TAB 3

Background and Context


C A L I F O R N I A S O C I A L W O R K E D U C A T I O N C E N T E R

U N I V E R S I T Y O F C A L I F O R N I A , B E R K E L E Y S C H O O L O F S O C I A L W E L F A R E

1

BACKGROUND AND CONTEXT COMMON CORE CURRICULA FOR CHILD WELFARE SUPERVISORS

Curriculum Development The Common Core Curricula is the result of a multi-year statewide collaborative effort to develop standardized curricula for California’s newly hired child welfare supervisors and child welfare workers. Development and implementation of the Common Core Curricula was mandated by California’s Program Improvement Plan (PIP) as part of the 2003 federal Child and Family Services Review (CFSR). Although in-service core training had historically been provided by the Regional Training Academies (RTAs), the IUC and county staff development departments, the PIP stipulated that the California Department of Social Services (CDSS) “develop a common core curriculum for all new child welfare workers and supervisors that is delivered by all training entities statewide.” The goal of this mandate was defined as the implementation of “a common core curriculum…in every county to train all new child welfare workers and child welfare/probation supervisors.” The Statewide Training and Education Committee (STEC) developed the initial series of Common Core Curricula in FY2004/2005, marking the first implementation of supervisor and new worker training that was standardized for the entire state. The purpose of statewide standardization is to achieve consistency and equity in the application of best and evidence-based practice in all 58 California counties. Each of the content areas of the Common Core has a set of measurable learning objectives for knowledge, skills and values essential to the provision of excellent service to families and children who participate in California’s county child welfare programs. Values Underlying the Development of Common Core Training STEC used the following underlying values in developing recommendations for common core training:

▪ Common core training is grounded in social work values and ethics.1

1 The National Association of Social Workers (NASW) Code of Ethics states, “Social work administrators and supervisors should take reasonable steps to provide or arrange for continuing education and staff development for all staff for whom they are responsible. Continuing education and staff development should address current knowledge and emerging developments related to social work practice and ethics…” (NASW Code of Ethics, 1996, revised 1999, Section 3.08). CalSWEC’s Standards and Values support the use of ongoing training as a form of best practices: “Standards of practice are by their nature subject to change. In view of shifting societal standards, as well as advancing knowledge

2

▪ Common core training builds upon, but is not limited to, new worker training currently underway in California, and utilizes existing training structures.

▪ Standards encourage flexibility in the way counties meet identified training needs. ▪ Standards encourage the application of best practices aimed at improving outcomes

for children and families, by training strategies that progress from knowledge acquisition to building and demonstrating skills.

▪ Standards endorse training delivery methods for common core training that yield measurable learning objectives and that provide the basis for evaluation of knowledge, skills, and attitude acquisition in order to promote positive outcomes for children and families.

▪ Standards are consistent with those endorsed by California’s Title IV-E university programs for the bachelor’s and master’s degrees in social work.

▪ Common core training encourages inclusion of community partners, whenever possible, in order to share responsibility for child safety, permanency, and well-being.

Levels of Standardization STEC determined that content areas of the Common Core Curricula for Child Welfare Supervisors would vary in their level of standardization: One content area has standardized information and standard delivery, statewide:

▪ Casework Supervision Five other content areas have standardized information, with detailed instructions on delivery:

▪ Child Welfare Policy and Practice for Supervisors ▪ Evidence-based Practice ▪ Fiscal Essentials ▪ Educational Supervision ▪ Managing for Results

Completion of the Common Core All content areas of the Common Core Curricula for Child Welfare Supervisors must be completed within twelve months from the date of hire or promotion.

about children, human behavior, and human ills, standards must be subject to continuous reflection and review.” (CalSWEC Website) http://calswec.berkeley.edu/CalSWEC/CalSWEC_Standards_Revised.html and http://calswec.berkeley.edu/CalSWEC/CalSWEC_Values_Revised.html

3

Foundational Themes to Guide Training and Practice Five foundational themes were identified for the Common Core Curricula series. The themes and their working definitions are: Fairness and Equity A principle that promotes equivalent opportunity for all children and families to achieve positive outcomes from child welfare interventions. Fairness and equity issues often concern reducing disparities in service access across population groups and diminishing overrepresentation of ethnic minorities in the child welfare system. Policies, procedures, and/or practices based on fairness and equity principles consider the unique sociocultural context of each individual and frequently involve the expansion of culturally responsive community resources. Family & Youth Engagement Practices and strategies congruent with relevant sociocultural dynamics that effectively engage parents, youth and extended family members in a respectful and collaborative manner in the assessment, intervention and case planning processes. Strength-based Practice Practice that identifies strengths in an individual, family, or system, and the formulation of service arrays and interventions that acknowledge and build on those strengths. A strength-based approach honors and respects the dignity of family members and incorporates the family’s collective knowledge about the resources and strengths in their family system. Strength-based practice involves joining with the family to reach goals for improvement in family functioning. Outcomes-Informed Practice Practice that supports and is informed by federal and state outcomes. All training in California supports the federal outcomes of Safety, Permanency and Well-Being. California also has developed state-specific performance measures. These performance measures are referenced in the curricula where they apply. For more information on the performance measures in California, please refer to the website for the Child Welfare Dynamic Report System at the Center for Social Sciences Research (CSSR) at UC, Berkeley: http://cssr.berkeley.edu/ucb_childwelfare/. Evidence-based Practice (“EBP”) The application to service delivery of research evidence related to child welfare, integrated with clinical expertise and client values. The existing body of research reflects varying levels of methodological rigor and efficacy, and differences in applicability to child welfare practice. Where available, research on child welfare practice is integrated into the common core.

4

Training Evaluation The evaluation components of the Common Core Curricula for Child Welfare Supervisors were developed concurrently with the creation of the curricula content. They are based on the Training Evaluation Framework Report developed by CalSWEC in FY 04/05 in response to the Program Improvement Plan (PIP). In addition to evaluating trainee satisfaction with the entire Common Core, Casework Supervision features an embedded skill evaluation. By this method of evaluation, trainees practice the skill of casework supervision and are evaluated on the basis of their responses to case scenarios presented during the course of the training. Analysis of the satisfaction and skill evaluation data is used to assist training evaluation experts to improve the training. More information on the Training Evaluation Framework, as well as on training evaluation generally, can be found at: http://calswec.berkeley.edu/CalSWEC/CWTraining.html. Revision Process Major revisions to the Common Core Curricula are conducted every three years or sooner, based on developments critical to effective social work practice. Minor revisions occur systematically as needed, to reflect current practice and changes in policy and legislation. Each common core curriculum is delineated by a version number (e.g., Version 1.0, 1.1, 1.2, 2.0, etc.). The type of version number indicates whether the version contains minor revisions or major revisions. For example, minor revisions are indicated by the numbered sequence Version 1.1, 1.2, 1.3, etc., while major revisions are delineated by a sequence of integer version numbers, e.g., Version 1.0, 2.0, 3.0, etc. The latest version of each curriculum is posted on the CalSWEC website. Principles and Values of the Revision Process The following principles and values are applied to the design of the revision process:

▪ Content will reflect “state of the art” knowledge and applications ▪ Content will apply transfer of learning principles and strategies ▪ Content will support and expand upon the competencies established in the Title IV-E

bachelor’s- and master’s-level social work programs ▪ The revision process will draw upon the combined expertise of practitioners and

university partners


TAB 4

How to Use This Guide


1

CALIFORNIA COMMON CORE CURRICULA

FOR CHILD WELFARE SUPERVISORS

HOW TO USE THE TRAINER’S AND TRAINEE’S GUIDES

Please read carefully as a first step in preparing to train this curriculum.

IMPORTANT NOTE: Each curriculum within the Common Core series is mandated and standardized for all new child welfare supervisors in the state of California. It is essential that all trainers who teach any of the Common Core Curricula in California instruct trainees using the standardized Training Content as provided. The training of standardized content also serves as the foundation for conducting standardized testing to evaluate and improve the effectiveness of new supervisor training statewide.

GENERAL INFORMATION

The Common Core Curricula model is designed to define clearly the content to be covered by the trainer. Each curriculum consists of a Trainee’s Guide and a Trainer’s Guide. The Trainee’s Guide contains the standardized information which is to be conveyed to trainees. The Trainer’s Guide includes guidance to assist the trainer in presenting the standardized information. Casework Supervision requires a standardized delivery to support the embedded skill evaluation contained in the curriculum, while the other five modules in the Common Core Curricula for Child Welfare Supervisors allow some flexibility in the delivery of the content. The list of components of the Trainer’s and Trainee’s Guides described under the subheadings below vary slightly for each module of the Supervisor Common Core. It is recommended that trainers first review the Background and Context, Agenda, and Suggested Lesson Plan. Trainers can then read the Training Content and Tips in the Trainer’s Guide and the Training Content in the Trainee’s Guide in order to become thoroughly familiar with each topic and the suggested training activities. The curricula are developed with public funds and intended for public use. For information on use and citation of the curricula, please refer to the Guidelines for Citation:

http://calswec.berkeley.edu/CalSWEC/CCCCA_Citation_Guidelines_Rev_040909.doc

2

Please note that each individual curriculum within the Common Core Curricula is subject to periodic revision. The curricula posted on the CalSWEC website are the most current versions available. For questions regarding the curricula, contact Melissa Connelly [email protected] or Phyllis Jeroslow [email protected], or call CalSWEC at 510-642-9272.

COMPONENTS OF THE TRAINER’S AND TRAINEE’S GUIDES Background and Context The Background and Context describes how and why the Common Core was developed, as well as the values, levels of standardization, completion requirements, and revision process associated with the Common Core series. As an additional resource for trainers, the Background and Context also provides working definitions of the foundational themes that are interwoven throughout the Common Core, and basic information about the use of embedded skill testing for evaluating the effectiveness of the training. Competencies and Learning Objectives The Competencies and Learning Objectives serve as the basis for the Training Content that is provided to the trainees. All the Competencies and Learning Objectives for each curriculum are listed in a separate tab in both the Trainer’s and Trainee’s Guides. The Learning Objectives are subdivided into three categories: Knowledge, Skills, and Values. They are numbered in series beginning with K1 for knowledge, S1 for skills, and V1 for values. The Learning Objectives for each segment of the curriculum are also indicated in the suggested Lesson Plan. Competencies are defined as broad indicators of essential and best practices. Typically, several Learning Objectives support the development of each Competency. The Learning Objectives are more specific than the Competencies and are designed to provide measurable indicators of learning. Knowledge Learning Objectives entail the acquisition of new information and often require the ability to recognize or recall that information. Skill Learning Objectives involve the application of knowledge and frequently require demonstration of the skill. Values Learning Objectives describe attitudes, ethics, and desired goals and outcomes for practice. Generally, Values Learning Objectives do not easily lend themselves to measurement, although values acquisition may sometimes be inferred through other responses elicited during the training process. Agenda (most modules) The Agenda is a simple, sequential outline indicating the order of events in the training day, including the coverage of broad topic areas, training activities, evaluation (if applicable), lunch, and break times.

3

Suggested Lesson Plan (most modules, Trainer’s Guide only) The Suggested Lesson Plan is a mapping of the structure and flow of the training. It presents each topic in the order recommended and indicates the duration of training time for each topic. The Suggested Lesson Plan is offered as an aid for organizing the training. The Suggested Lesson Plan is divided into major sections by Day 1 and Day 2 of the training, as applicable, and contains three column headings: Topic, Methodology, and Learning Objectives. Some Lesson Plans may have a fourth column for key Learning Points to emphasize. The Topic column is divided into training segments. The Methodology column indicates suggested training activities that accompany each segment, and the Learning Objectives column reflects the specific objectives that are covered in each segment. Evaluation Protocols (Trainer’s Guide; for Casework Supervision only) It is necessary to follow the step-by-step instructions detailed in this section concerning the embedded skill evaluation in order to preserve the integrity and consistency of the training evaluation process. Additionally, trainers should not allow trainees to take away or make copies of any test materials so that test security can be maintained. Training Content and Tips (Trainer’s Guide only) The Training Content and Tips section is the main component of the Trainer’s Guide. It includes detailed descriptions, guidance, and step-by-step tips for preparing, presenting, and processing the activities. The descriptions also indicate the Training Content in the Trainee’s Guide and the PowerPoint slides that accompany each activity. Thumbnails of the slides are also provided. Training Content (Trainee’s Guide only; can be inserted into the Trainer’s Guide) The Training Content in the Trainee’s Guide contains the standardized text of the curriculum that accompanies the trainer’s lectures and other training activities. Supplemental Handouts Supplemental Handouts refer to additional handouts not included in the Training Content tab of the Trainee’s Guide. For example, Supplemental Handouts include PowerPoint printouts that accompany in-class presentations or worksheets for training activities. Some documents in the Supplemental Handouts are placed there because their size or format requires that they be printed separately. Glossary A glossary may be provided. It defines words and acronyms commonly used in child welfare practice that appear in the training content or supplemental information. The glossary is provided to help trainees learn language and terms specific to the field.

4

Bibliography (most modules, in one or both guides) The Bibliography indicates the sources that were reviewed by the curriculum designers to prepare the main and supplemental content information, training tips, training activities and any other information conveyed in the training materials. It may also include additional resources that apply to a particular content area. In certain curricula within the Common Core series, the Bibliography may be subdivided by topic area. Materials Checklist (Trainer’s Guide only) In order to facilitate the training preparation process, a Materials Checklist may be provided. The Materials Checklist indicates all the materials needed for the entire training. Multi-media materials may include such items as videos, CDs, posters, and other audiovisual aids.

TAB 5

Competencies and Learning Objectives


Supervisor Core | Managing for Results: Trainer’s and Trainee’s Guides | Version 1.1, July 2009

1

SUPERVISOR CORE:

MANAGING FOR RESULTS - TRAINER’S AND TRAINEE’S GUIDES -

CORE COMPETENCY AND LEARNING OBJECTIVES

COMPETENCY The trainee will understand and be able to incorporate outcome-based management into his or her role as child welfare supervisor.

LEARNING OBJECTIVES Knowledge K1. The trainee will be able to recognize key concepts and terminology for outcome

measurements (e.g., median, point in time, entry cohort, measures, rolling year). K2. The trainee will be familiar with the outcome measures on the quarterly, county-

specific reports created as part of the C-CFSR process (i.e., the difference between the federal and state-enhanced measures).

K3. The trainee will be able to explain how to navigate the Child Welfare Dynamic

Report System website of the UC Berkeley Center for Social Services Research http://cssr.berkeley.edu/cwscmsreports/), and understand that all data on the C-CFSR quarterly, county-specific reports (as well as much more detailed information) can be located there.

K4. The trainee will be able to explain the interrelated nature of the outcome measures

for safety, permanency and well-being that are tracked on the quarterly county-specific outcome reports.

K5. The trainee will be able to identify different methods and limits of interpreting

data, and how to apply these methods and limits to the supervisory role.

Supervisor Core | Managing for Results: Trainer’s and Trainee’s Guides | Version 1.1, July 2009

2

Skills S1. The trainee will be able to demonstrate the ability to ask policy and practice

questions related to trends identified in the C-CFSR outcome measures. S2. The trainee will be able to develop a plan to improve the delivery of child welfare

services in his or her agency through monitoring outcomes with the C-CFSR process.

Values V1. The trainee will value the need to supervise and monitor in ways to achieve ASFA

outcomes. V2. The trainee will value ongoing recognition of trends identified in the outcome

measures that enhance culturally sensitive and responsive practice by helping target services appropriately to ethnic, age, and gender groups.

V3. The trainee will value incorporating various practice issues into supervision:

A. Outcomes vs. process B. Fairness and equity C. Incorporating data

TAB 6

Daily Agenda


. Supervisor Core | Managing for Results: Trainer’s and Trainee’s Guides | Version 1.1, July 2009

1

SUPERVISOR CORE:

MANAGING FOR RESULTS - TRAINER’S AND TRAINEE’S GUIDES -

AGENDA

Part 1

I. Welcome, Introductions, and Session Agenda

II. Review Learning Objectives

III. Background and Concepts for Outcomes in Child Welfare

Data-Informed Practice -- What Do I Care? CFSR AB 636

Key Concepts to Understand Child Welfare Outcomes

Longitudinal Data Analysis (entry cohorts versus point in time, etc.) Fun Exercise with Medians and Means The Cycle of Experiences/Inter-related Nature of Outcomes

Break

Part 2

IV. Using Outcome Data in Child Welfare Practice

The AB 636 Quarterly County Data Report

What Is It and What's on It Federal versus State Enhanced Outcome Measures What the ¡ ! Is a "Rolling Year" ?

V. Taking Your First Steps Toward Data Analysis

Introduction to the Child Welfare Dynamic Report System website, UC

Berkeley Center for Social Services Research (CSSR)

. Supervisor Core | Managing for Results: Trainer’s and Trainee’s Guides | Version 1.1, July 2009

2

Part 3

VI. Outcome-Based Management—a Hands-On Exercise

VII. Report Back and Wrap-Up

TAB 7

Lesson Plan


Supervisor Core | Managing for Results: Trainer’s Guide | Version 1.1, July 2009

1

SUPERVISOR CORE:

MANAGING FOR RESULTS - TRAINER’S GUIDE -

LESSON PLAN

Topic Methodology Learning Objective Learning Point

PART 1 1 hour 8:30 – 9:30 a.m. 1. Welcome,

Introductions, and Session Agenda

2. Review

Learning Objectives

3. Background

and Concepts for Child Welfare Outcomes

Data-Informed

Practice—What do I Care?

Key Concepts

in Child Welfare Outcomes

Welcome and introduction of trainer/trainers, and review of session topics.

Brief review of

learning objectives for the module.

Lecture with

PowerPoint slides: Data-informed practice -- why is it important?

Flip Chart listing

of trainees' thoughts on barriers to using data.

Lecture with

PowerPoint slides: Key concepts in child welfare outcomes—particularly bias

K1. The trainee will be able to recognize key concepts and terminology for outcome measurements (e.g., median, point in time, entry cohort, measures, rolling year). K4. The trainee will be able to explain the interrelated nature of the outcome measures for safety, permanency, and well-being that are tracked on the quarterly county-specific outcome reports. K5. The trainee will be able to identify different methods and limits of interpreting data, and how to apply these methods and limits to the supervisory role.

Historically, the child welfare system did not do well at evaluating its performance.

Child welfare

policy and practices need to be guided by clear and specific goals, and progress toward those goals requires good data.

Overcoming

skepticism about the usefulness of data is crucial for better outcomes--and for improving the lives of children and families.

Monitoring child

welfare agency performance is no longer a good idea—it's the law.

Point in time


2


of snapshots versus entry cohort data.

Class

participation exercise on differences between medians and means.

Lecture with

PowerPoint slide and discussion of the "Cycle of Experiences" in the child welfare system.

V1. The trainee will value the need to supervise and monitor in ways to achieve ASFA outcomes.

snapshots and exit cohorts have inherent biases.

Medians are useful

when talking about measures such as length of stay that may contain extreme data points.

There is no "gold

standard" for success on any outcome. Any measure, viewed in isolation, tells you nothing useful about county performance.

15 min 9:30 – 9:45 a.m. Break PART 2 30 min 9:45 – 11:15 a.m. 4. Using

Outcome Data in Child Welfare Practice

• The California

Outcomes and Accountability Quarterly County Data Report

Brief review of the outcome measures contained on the California Outcomes and Accountability Quarterly report.

Lecture with

PowerPoint visual aid on difference between federal and state-enhanced measures.

K2. The trainee will be familiar with the outcome measures on the quarterly, county-specific reports created as part of the C-CFSR process (i.e., the difference between the federal and state-enhanced measures.) K3. The trainee will be able to explain how to navigate the Child Welfare Dynamic Report System website of the UC Berkeley Center for Social

The California Outcomes and Accountability quarterly report contains information on CWS participation rates along with the three federal child welfare “domains” of safety, permanency, and well-being.

The federal

standards were created based on the limitations of


3


5. Taking your First Steps Toward Data Analysis

• The UCB

Child Welfare Dynamic Report System website introduction

Lecture with PowerPoint visual aid on rolling year concept.

Pop quiz on

rolling year concept.

Online

demonstration and introduction to the UCB Child Welfare Dynamic website.

Services Research, and understand that all data on the C-CFSR quarterly, county-specific reports (as well as much more detailed information) can be located there.

available federal data (i.e., AFCARS) and have serious flaws.

The AB 636 "state

enhanced" measures use entry cohort analysis to supplement the federal standards and provide more comprehensive information.

A rolling year

examines an outcome for a year-long cohort, but the year will not always be equal to the January-December calendar year.

All the data on the

California Outcomes and Accountability quarterly report are posted on the websites of UCB & CDSS

15 min 11:15 - 11:30 a.m. Break


4


PART 3 1 hour 11:30-12:30 p.m. 6. Hands-on Data

Exercise for Small Groups

7. Report Back &

Wrap Up

Small groups of 5-7 people will work together to examine and interpret real outcome data.

Trainer will

circulate among groups to help with process.

Class will

reconvene as one group after 45 minutes to share results of their work

K4. The trainee will be able to explain the interrelated nature of the outcome measures for safety, permanency, and well-being that are tracked on the quarterly county-specific outcome reports. S1. The trainee will be able to demonstrate the ability to ask policy and practice questions related to trends identified in the C-CFSR outcome measures. S2. The trainee will be able to develop a plan to improve the delivery of child welfare services in his or her agency through monitoring outcomes with the C-CFSR process. V2. The trainee will value ongoing recognition of trends identified in the outcome measures that enhance culturally sensitive and responsive practice by helping target services appropriately to ethnic, age, and gender groups. V3. The trainee will

Opportunity to apply information covered in Parts 1 & 2 and to begin to grapple with examining outcome data, interpreting results, and formulating plans to affect targeted outcomes.

Determine which

outcomes (and possibly for which subgroups) are “high priority” and “high performance” areas.

Examine trends in

the outcomes over time and by different subgroups (age, ethnicity, gender, placement type).

Consider the

relationships between outcome measures--recalling the "cycle of experiences" discussion and handout.

Identify factors that

may be influencing performance.

Identify linkages

between performance


5


value incorporating various practice issues into supervision: A. Outcomes vs. process B. Fairness and equity C. Incorporating data

outcomes and practice processes.

Determine areas of

potential strength and those in need of improvement.


TAB 8

Training Content and Tips



1

SUPERVISOR CORE:

MANAGING FOR RESULTS - Trainer’s Guide -

TRAINING TIPS AND CONTENT PART 1 1. Introduction Take several minutes for welcome, introduction of trainer, and trainees (depending on size of group), and brief overview of session agenda. Child welfare services agencies are required by the C-CFSR to monitor the impact of their programming efforts for children, youth, and families through performance on a series of outcome measures. This training module provides a basic understanding of: outcome evaluation concepts, specific measures tracked by the C-CFSR process, and useful approaches for applying outcome information to monitor, assess, and improve practice. 2. Learning Objectives Knowledge K1. The trainee will be able to recognize key concepts and terminology for outcome

measurements (e.g., median, point in time, entry cohort, measures, rolling year). K2. The trainee will be familiar with the outcome measures on the quarterly, county-

specific reports created as part of the C-CFSR process (i.e., the difference between the federal and state-enhanced measures).

K3. The trainee will be able to explain how to navigate the Child Welfare Dynamic

Report System website of the UC Berkeley Center for Social Services Research, and understand that all data on the C-CFSR quarterly, county-specific reports (as well as much more detailed information) can be located there.

K4. The trainee will be able to explain the interrelated nature of the outcome

measures for safety, permanency, and well-being that are tracked on the quarterly county-specific outcome reports.


2

Skills S1. The trainee will be able to demonstrate the ability to ask policy and practice

questions related to trends identified in the C-CFSR outcome measures. S2. The trainee will be able to develop a plan to improve the delivery of child

welfare services in his or her agency through monitoring outcomes with the C-CFSR process.

Values V1. The trainee will value the need to supervise and monitor in ways to achieve

ASFA outcomes. V2. The trainee will value ongoing recognition of trends identified in the outcome

measures that enhance culturally sensitive and responsive practice by helping target services appropriately to ethnic, age, and gender groups.

V3. The trainee will value incorporating various practice issues into supervision:

A. Outcomes vs. process B. Fairness and equity C. Incorporating data

3. Background and Concepts for Outcomes in Child Welfare (Trainer's Slides PowerPoint [PPT] Slide Show) Slide 1

Supervisor Core Training: Managing for Results

Original presentation was created for Version 1.0 by Daniel Webster, Barbara Needell, Wendy Piccus, Aron Shlonsky, & Lynn Usher.

Revised for Version 1.1 by Shared Vision Consultants and CalSWEC to include outcome updates and slides available from presentations posted on the CSSR website.

Data-Informed Practice—What do I Care? The child welfare system has traditionally not done a very good job of monitoring and evaluating its performance.


3

Slide 2

Background and Concepts for Outcomes in Child Welfare:Data Informed Practice -- What do I Care?

Part 1:

Slide 3 The Current Placement System*

(Highly Simplified)

Child In Child OutA bunch of stuff happens

*adapted from Lyle, G. L., & Barker, M.A. (1998) Patterns & Spells: New approaches to conceptualizing children’s out of home placement experiences. Chicago: American Evaluation Association Annual Conference

Current Placement System—Highly Simplified Child welfare policy and practices need to be guided by clear and specific goals, and progress toward those goals requires good performance data. Yet—in spite of the large quantity of information often collected about children in out-of-home care—child welfare agencies are often unable to provide quick and reliable responses to questions posed by policymakers, administrators, and the public.

Slide 4 Data Skepticism

Data Skepticism ACTIVITY: Ask trainees to call out reactions they have to the idea of using data in their work, reasons that they may not like data, etc. Use flip chart to record answers. Sentiment among child welfare agency staff is that, while they may be inundated by data, this information is not useful for planning, evaluation, or other decisions. Their experience has often been that data systems used by the agency were designed for other purposes and cannot provide answers to questions that are pertinent to their practice. Further, they likely feel that entering data into CWS/CMS is a waste of time and takes away valuable time that they could be doing actual casework. This attitude is largely due to a legacy of state data systems like CWS/CMS as being “roach motels” of data—workers enter information in, but virtually nothing ever comes out. Or, when data does come out, it is not


4

Slide 4 (continued) Data Skepticism

presented in a way that is easily accessible or useful. A first step, then, in overcoming this aversion to data is to make the connection for workers between the information they input into CWS/CMS about their cases, and how successful their agency is helping its clients. To do so requires analyzing data that has been put into CWS/CMS and sharing it—on an ongoing basis—in a format that workers can understand. When workers hear the word “data,” they immediately think of “MATH” or “STATISTICS” and can be intimidated or turned off. A common reaction is “I hate math! I became a social worker so that I could help people—not work with numbers!” Another likely reaction is, “I've been a social worker for 25 years, and I know very well what area most needs work in our agency—it’s placement disruptions (or reunifying kids more quickly, or cutting down on the number of kids reentering care, etc.).” These typical responses are not unreasonable— given that workers have probably never seen examples of how data can be useful. If supervisors can be shown that information drawn regularly from CWS/CMS is a form of feedback on the impact of their efforts (as evidenced by performance on important goals such as preventing recurrence of maltreatment, or achieving reunification for children, etc.), their distrust of the usefulness of data will decrease. It is important to recognize that each worker’s practice experience provides a very real and valid perspective of their agency’s areas of success and failure. This validation can be used to point out to supervisors that each of them is actually a data person whether they realize it or not. That is, their experience is their data source from which they draw conclusions and make decisions about what areas have greater or lesser importance. Extrapolating from each worker's experience with their specific caseload to the information on all workers’ caseloads that is captured in CWS/CMS, provides a more comprehensive picture of where


5

Slide 4 (continued) Data Skepticism

a child welfare agency needs to focus its efforts. Clearly, improvements could be made in all parts of the child welfare system—but limited time and resources demand that efforts be made in the areas of greatest need. As a supervisor, it is critical to know where your attention would best be focused—and to keep on top of trends because the area of focus changes over time. In the ER unit—are referrals and/or substantiations going up or down? For which age or ethnic groups? Are these trends reflected in entries to care? In the permanency unit—are reunifications increasing or not? If they are, is our re-entry rate the same or are many of the kids we send home later coming back into care? Drawing on information in CWS/CMS will help supervisors to identify these trends and keep track of progress toward addressing them. Regularly updating staff in unit meetings—and celebrating any progress or success—can positively reinforce and motivate staff. Fancy statistics are not necessary. Simple line or pie charts can easily point out those areas where the agency is succeeding best and where it most needs to focus its efforts to help families. Supervisors will begin to understand that access to timely data and the ability to analyze this information can help agency staff to identify and respond to changing population needs—and enable workers to monitor the results of their work. Overcoming skepticism regarding the usefulness of data is therefore a crucial first step toward better outcomes—and thus for improved lives of children and families. Ask for ways that might make using data and outcomes measures more accessible for supervisors and workers. Record suggestions on flip chart. If training staff from only one or two counties, ask them to tell you general trends in child welfare in their county (e.g., how many children enter care each year, how long do children generally stay in care, what proportion of children reunify after 12 months in care, etc.).


6

Record these answers and check them against actual trends for the counties during the live demo of the UC Berkeley website. Then underscore the necessity to make these efforts to be aware of key trends, keep staff informed about areas of need, and to track progress, because… monitoring child welfare agency performance is no longer simply a good idea—it's the law.

Slide 5

The California Child Welfare Outcomes & Accountability System

• Quarterly distribution of county specific outcome indicators data

• Peer Quality Case Review

• County Self Assessment

• County Self Improvement Plan

• Continuous monitoring of outcomes

California’s Child Welfare Outcomes & Accountability System The Children and Family Services Review (CFSR) is an ongoing nationwide process conducted by the U.S. Department of Health and Human Services. The first stage of the CFSR examined service delivery by child protection, foster care, adoption, family preservation, family support, and independent living programs in all states. The CFSR examines seven outcomes for children and families as well as seven systemic factors. States were measured against national standards. During the first round of the CFSR, no state achieved substantial conformity on all measures. Thus, all states are implementing a Program Improvement Plan (PIP). The CFSR reviews all states periodically to follow progress and may impose fiscal penalties on those states that fail to improve. In 2003, California did not achieve conformity on any of the seven outcomes, and passed only one of seven systemic factors. The state faced $18 million in penalties if it had failed to meet federal requirements by 2005. In January 2008, the federal government levied a nearly $9 million penalty against the state for failing to meet the outcome goals regarding re-entry and placement stability. Loss of this funding during times of fiscal hardship will likely result in loss of services to children and families. Assembly Bill 636, the Child Welfare Outcomes and Accountability Act of 2001, was developed


7

Slide 5 (continued)







as a means to carry out California's PIP. The bill provides a legislative framework for monitoring and assessing county performance to ensure the safety, health and well-being of the state’s children. Since the passing of this legislation, California was instrumental in influencing the federal government to review and subsequently revise their federal outcome measures in 2007. Although many of the revised federal measures mirror the previously enhanced California measures, there are still a few vital outcome areas that California chooses to track in the absence of federal mandates. Those measures include the “Timely Response” of social workers regarding Immediate and 10-day referrals and “Timely Social Worker Visits.” Once again, California is leading the way in the creation of well-being measures for children who are in foster care. As of June 2009, the measures for health, dental, and educational outcomes for children in care were under development. Each county has created a self-assessment that identifies strengths and areas for improvement, as well as a system improvement plan (SIP) to outline actions needed to improve county performance with respect to the outcomes.

Key Concepts to Understand Child Welfare Outcomes Before examining the specific outcomes tracked under The California Child Welfare Outcomes and Accountability System, it is important to know the background about the reasons (i.e., sound methodological basis) why many of the measures were chosen.


8

Slide 6 Trainee Handout # 1

Caseload Snapshots Versus Entry Cohorts

Jan. 1, 2008

Jan. 1, 2009Jan. 1, 2007

Longitudinal Data Analysis—The Advantage of Using Entry Cohorts Caseload Snapshots Versus Entry Cohorts1 (Handout #1) A limitation of traditional child welfare research is that the data used are often comprised of periodic snapshots of the caseload of children in care at a given point in time. This information is important for providing basic management accountability; however, it does not accurately capture the experience of all children who come in contact with the child welfare system, and can be misleading when examining issues such as length of stay or placement moves in care. Point in time estimates are biased. They over-represent children who have the worst experiences in care—that is, those who tend to stay in care for long periods. Slide #6 illustrates this phenomenon: the x-axis represents a timeline from January 1, 2007, up to January 1, 2009, and the 10 numbered horizontal lines represent the duration in care of 10 children in foster care. A point in time snapshot taken of the caseload on January 1, 2008, will only observe about half the children (line numbers 1, 2, 3, 5, 8 and 10 from the top). The other children will not be detected by this snapshot. Therefore, any mean or median measure of length of time spent in care will be biased by the children observed in the snapshot who have longer lengths of stay. Placement stability estimates would be similarly biased. Using snapshot data, staff may not recognize the opportunity to effectively focus resources by assuming that most children in the system are poorly served.

1 This discussion adapted from: Webster, D., Needell, B., & Wildfire, J. (2002). Data are your friends: Child welfare agency self-evaluation in Los Angeles County with the Family to Family initiative. Children and Youth Services Review, 24(6/7), 471-484.


9

Slide 6 (continued) Caseload Snapshots Versus Entry Cohorts

Jan. 1, 2008

Jan. 1, 2009Jan. 1, 2007

ACTIVITY: Referring to PPT slide #6, ask trainees to call out how many children would comprise the calendar year 2007 entry cohort (Answer: 7). Then ask trainees to call out how many children would comprise the calendar year 2008 entry cohort (Answer: 3).

Fun Exercise with Medians and Means

“How long do children stay in care in your county?” is an important question that people often ask. The following brief exercise illustrates the need to reflect on the approach taken to answer the question about length of stay. ACTIVITY: Prior to class, trainer cuts out rows (make more as needed) from the following table and distributes one row each to student at the beginning of this exercise. (The trainee given the “Bill Gates” row is asked to keep his/her identity secret).

WORK TITLE INCOME UCB Ph.D., Consultant Roughly minimum wage

(let’s say, $10 K) Administrative Assistant $20 K

Administrative Assistant $20 K

Social Worker I $40 K


Social Worker II $50 K


Supervisor $60 K

Supervisor $60 K

Branch Manager $75 K


Bill Gates, CEO Microsoft

$3 Billion


10

Slide 6 (continued) Caseload Snapshots Versus Entry Cohorts

Jan. 1, 2008

Jan. 1, 2009Jan. 1, 2007

Ask students (except for Bill Gates) to state their work title and income, and then to line themselves up from highest to lowest income. The instructor points out that the line up is another way to talk about a “DISTRIBUTION” and asks the class to identify the MEDIAN income from the lineup. The class will see that the median income is $50 thousand. The instructor then asks Bill Gates to identify himself and to join the lineup, and asks the class to identify the median income now that a new person has been added. The class will see that the median income will still be $50 thousand. Thus the students will see that that the median income is a reasonable description of the middle point of the incomes—even with an extreme observation. Finally, the instructor will ask Bill Gates to step away, and for the class to identify the MEAN income from the lineup minus Bill Gates. The students will find that that the average or mean income is about $46 thousand. When Bill Gates rejoins the group and the class is asked to re-compute the mean income, the average income will be over $250 million—a poor description of the individuals’ incomes (i.e., much too high for everyone except Bill Gates, for whom the description is much too low). The instructor can use the exercise to point out how medians are a much more reasonable concept to use when talking about measures (particularly length of stay) that may contain extreme, “outlier” data points. The Cycle of Experiences in the Child Welfare System As a last word before the break, it is important to underscore the interrelated nature of interpreting outcome data. This is a main reason why the federal “standards” are inherently misleading—i.e., if one “achieves conformity” on a given measure, that is considered success. However this “success” may adversely affect other outcomes and therefore cannot be viewed in isolation.


11

Slide 7 Trainee Handout # 2

The Cycle of Experiences in the Child Welfare SysThe Cycle of Experiences in the Child Welfare Systemtem

CounterbalancedCounterbalancedIndicators ofIndicators of

SystemSystemPerformancePerformance

PermanencyPermanencyThroughThrough

Reunification,Reunification,Adoption, orAdoption, or

GuardianshipGuardianship

ShorterShorterLengthsLengthsOf StayOf Stay

StabilityStabilityOf CareOf Care

Rate of Referrals/Rate of Referrals/Substantiated ReferralsSubstantiated Referrals HomeHome--BasedBased

Services vs.Services vs.OutOut--ofof--HomeHome

CareCare

Maintain Positive Maintain Positive AttachmentsAttachmentsTo Family,To Family,

Friends, andFriends, andNeighborsNeighbors

Use of LeastUse of LeastRestrictiveRestrictive

Form of CareForm of Care

Source: Usher, C.L., Wildfire, J.B., Gogan, H.C. & Brown, E.L. (2002). Measuring Outcomes in Child Welfare. Chapel Hill: Jordan Institute for Families,

Reentry to CareReentry to Care

The Cycle of Experiences (Handout #2) There is no “gold standard” for determining success on any outcome. Any particular measure, viewed in isolation, tells you nothing useful about county performance. Data tells a story, and the relationship between outcomes must be taken into account when evaluating performance. For example, while we may seek to reduce the number of children entering care—if/when we succeed in doing so, we must be aware of how the smaller population entering care will likely be more difficult to work with (i.e., we have excluded more families from oversight by the agency—but those families that do require intervention have more challenging circumstances). Thus a reduction in entries may affect other measures such as time to permanency or placement stability. The supervisor needs to understand, therefore, that monitoring and drawing conclusions about any given outcome must be done with respect to the relationships between outcomes. (Other potential relationships—reduction in entries could lead to higher recurrence of maltreatment; increase in reunification could lead to increase in reentry; increase in guardianship or adoption could lead to decrease in reunification.) Also, other factors outside of child welfare, such as an economic downturn, etc., may lead to an increase in entries even in spite of best practice. By being aware of the interrelated nature of outcomes, the supervisor can make a more realistic assessment of the effect their work is having on their clients. They will not be overly pessimistic, for example, when the number of children reunified during a period of time drops—if they are also monitoring entries to care and note that a smaller number of children with greater needs is coming into care.


12

Slide 7 (continued) The Cycle of Experiences in the Child Welfare SysThe Cycle of Experiences in the Child Welfare Systemtem










CareCare







By the same token, they will not prematurely think that they don’t need to continue efforts to improve reunification services just because an increasing number of children are going home from the agency. They will also know to also track the proportion of those reunifying who later return to care—to ensure that the apparent, successful reunification events are not being followed by an unintended negative outcome (re-entry). Finally, the supervisor will be able to use this understanding of relationships between the outcomes to more clearly, forcefully, and accurately “tell the story” of how well children and families are being helped—whether the supervisor is motivating staff in a unit meeting, communicating needs for practice change to a child welfare director, or being “accountable” to relevant members of the community (e.g., judges, child advocates, legislators, board of supervisors). A couple of initiatives in California that focus on using outcome data: Family to Family, an initiative developed by the Annie E. Casey Foundation and also supported by the Stuart Foundation, is being implemented in 22 counties that represent 84 % of the children in foster care. The four core components are: community partnerships; recruitment, training and support of foster parents; team decision-making; and self-evaluation. Self-evaluation, the use of data, allows community partners to focus on neighborhoods with the highest concentration of CPS referrals and entries into foster care. Using Family to Family strategies, children who must be removed from the home can be placed with foster families in their neighborhoods. http://www.aecf.org/initiatives/familytofamily. California Permanency for Youth Project, a three-year program supported by the Stuart Foundation and the Walter S. Johnson Foundation, is dedicated to ensuring that all youth leave the child welfare system with a permanent lifelong connection with a caring adult. The project collaborates with the courts, group homes, foster families, adoption agencies and the child welfare system to help them achieve permanency for foster youth in Alameda, Monterey,


13

Slide 7 (continued) The Cycle of Experiences in the Child Welfare SysThe Cycle of Experiences in the Child Welfare Systemtem










CareCare







San Mateo and Stanislaus counties. The project also works to improve public policy, increase training and raise public awareness on the issue. It tracks its success by utilizing data on older youth in foster care and on exits from care. http://www.cpyp.org. Summarize points supporting why data is important for supervisors:

It is critical to know where your limited time would best be focused—information on all workers’ caseloads captured in CWS/CMS provides a more comprehensive picture of where a child welfare agency is performing well and where it needs improvement.

Regularly updating staff in unit meetings—

and celebrating any progress or success—can positively reinforce and motivate workers.

Regular use of data and understanding

relationships between the outcomes “tells the story” of how well children and families are being helped—whether supervisors are motivating staff in a unit meeting, communicating needs for policy and practice change to a child welfare director, or being “accountable” to relevant members of the community (e.g., judges, child advocates, legislators, board of supervisors).

Break


14

Slide 8

Using Outcome Data in Child Welfare Practice

Part 2:

PART 2 4. Using Outcome Data in Child Welfare Practice This part of the seminar will introduce the specific outcome measures tracked under the California Outcomes and Accountability System, the quarterly data report, and the UC Berkeley website. The UC Berkeley website demonstration is a “live” online session that requires a high-speed Internet connection. The California Quarterly Outcomes and Accountability County Data Report What is it, and what’s on it? The California Outcomes and Accountability quarterly report provides summary-level federal and state outcome measures that are intended to be the basis for ongoing county self-assessment of its performance over time. Each county is expected to use data in the report for the self-assessment process—that is, to determine reasons for current performance as well as to help plan for measurable improvement. The California Outcomes and Accountability quarterly report contains information on the following areas: CWS participation rates along with the three federal child welfare “domains” of safety, permanency, and well-being. A condensed version of the report layout is included in trainees’ handout packets. The most recent version of the complete quarterly report for all California counties and the state as a whole is available at http://cssr.berkeley.edu/ucb_childwelfare/ccfsr.aspx California Quarterly Outcomes & Accountability County Data Report (Trainee Handout #3) Child Welfare Services Participation Rates: provides data on the number, and number per 1,000 children in the county/state, for key child welfare indicators. It is intended as background information to assist your county in analyzing your county’s performance by the outcome indicators.


15

Safety Outcomes: designed to reflect the effectiveness of efforts to protect children from abuse/neglect by reporting instances of abuse and neglect at various stages of child welfare services and process, measures which reflect the frequency of social worker contact with children and the speed of face-to-face investigation of abuse/neglect allegations. Permanency Outcomes: designed to reflect the number of foster care placements for each child, the length of time a child is in foster care, and the rate that children re-enter foster care after they have returned home or other permanent care arrangements have been made. Child and Family Well-Being Outcomes: designed to reflect the degree to which children in foster care retain relationships with the family and extended communities with whom they were associated at the time of their removal from their parents. The county data report is produced on a quarterly basis—that is, every three months. Each quarter, a data extract from CWS/CMS is queried to create the reports. Counties are allowed a two-month window after the close of each quarter to comply with CWS/CMS data entry requirements. Thus, taking the 2007 Quarter 2 extract from CWS/CMS as an example, information—about any activity within a child’s case history that occurred prior to June 30, 2007—could be entered into CWS/CMS up until August 31, 2007, and still be included in the 2007 Quarter 2 data extract. Slide 9

Federal Versus State-Enhanced MeasuresIs Performance getting better or worse?

Adoption within 24 Months

Data Source: Multi-State Data Archive, Chapin Hall Center for Children

State A

0%

5%

10%

15%

20%

25%

30%

1990

1991

1992

1993

1994

1995

1996

1997

Ado

pted

ExitsEntries

State A

0%

5%

10%

15%

20%

25%

30%

1990

1991

1992

1993

1994

1995

1996

1997

Ado

pted

ExitsEntries

Federal Versus State-Enhanced Measures

Unlike point-in-time counts that contain information on children who are still in care at a given point in time, or exit cohorts that examine children leaving care and are potentially confounding by mixing long- and short-stayers in care, longitudinal data systems capture information on all children who enter foster care and follow them for the entire time they are in care. Information on a cohort of children as they move through out-of-home care does not systematically exclude children from the study population if they leave care. As Fred Wulczyn of the Chapin Hall Center for Children has observed, “if you were trying to evaluate a cancer therapy or HIV treatment—would you look only at patients who survived (or only at those who died) to determine the effectiveness of the intervention—or would you follow all patients from the time the treatment


16

started?” The answer is obvious. Reliable information on children’s child welfare experiences requires following children from first entry to care through placements within and exits from the system, and, for some children, subsequent re-entries to care. Supervisors, workers, and administrators need to evaluate performance and base decisions on the most comprehensive and accurate information possible. The AB 636 “state enhanced” measures, therefore, use entry cohort analysis to supplement the federal standards and provide this information. Many of the new measures adopted in 2006 by the Administration for Children and Families (ACF) were part of California’s enhanced measures adopted through the AB 636 legislation. As mentioned above, California continues to monitor and track data regarding social worker timely response and monthly visits to children even though there is not a mandate to do so. Other measures enhanced by California include the well-being measures regarding sibling placements, least restrictive placements, ICWA/cultural considerations and the post foster care outcomes. What the ¡ ! is a "Rolling Year" ?2 Use graphics on PowerPoint slides to explain this topic. Use flip chart if further explanation is needed.

2 Part of this discussion based on: Hurley, D. (2004). Completing the county self-assessment, Module 3: Analyzing child welfare outcome indicators. California Social Work Education Center: Berkeley, California.


17

Slide 10

2008 2009

Jan. 1

Mar. 1 Mar. 1

Dec. 31

Jul. 7 Jul. 7

Jan. 1 Dec. 31

Dec. 31 Dec. 31

Tracking an Entry Cohort for 1 Year

Slide 11

Rolling Years:Quarter 1 Extract

Jan. 1

2008 2009

Dec. 31Mar. 31 Mar. 31

Jan. 1 Dec. 31Jan. 1

2007

Extract Cut-off No data after

April 1

Last day of data prior to cut-off

March 31, 2009

ENTRY COHORT

Apr. 1

April 1, 2007 – March 31, 2008

Last date able to track entry for

full 12 monthsprior to cut-off.

(Trainee Handout #4) Slide 12


Jan. 1

2008 2009

Dec. 31Jun. 30 Jun. 30


2007


July 1

Last day of data prior to cut-offJune 30, 2009

ENTRY COHORT

Jul. 1

July 1, 2007 -- June 30, 2008

Last date able to track entry for full 12 monthsprior to cut-off.

Rolling Years

Each quarter, most measures are available using the most recent data available for that measure using “ROLLING YEARS.” A rolling year examines an outcome for children in a year-long cohort, but the year being tracked will not always be equal to the January–December calendar year. A rolling year can be April 1 (of one year) to March 31 (of the following year); other rolling year periods can be July 1(of one year) to June 30 (of the following year), or October 1 to September 30. The January 1 to December 31 calendar year is also a rolling year. There are 4 Rolling Year Time Periods (corresponding with the 4 quarterly data extracts from CWS/CMS):

Q1: April 1–March 31

(extract cut-off: April 1st)

Q2: July 1–June 30 (extract cut-off: July 1st)

Q3: October 1-September 30

(extract cut-off: October 1st)

Q4: January 1–December 31

(extract cut-off: January 1st)

The rolling year used depends on the outcome being examined. For example, using 2009 Quarter 2 data, UC Berkeley can look at first entries to care for children who enter July 1, 2008–June 30, 2009, reunifications within 12 months for children who entered July 1, 2007–June 30, 2008, and adoptions within 24 months for children who entered July 1,


18

Slide 11


Jan. 1

2008 2009

Dec. 31Mar. 31 Mar. 31


2007


April 1

Last day of data prior to cut-off

March 31, 2009

ENTRY COHORT

Apr. 1

April 1, 2007 – March 31, 2008



Slide 12


Jan. 1

2008 2009

Dec. 31Jun. 30 Jun. 30


2007


July 1


ENTRY COHORT

Jul. 1

July 1, 2007 -- June 30, 2008


2006–June 30, 2007. As new data become available each quarter with the most recent extract from CMS, these measures can be “rolled” forward to track a 365-day cohort up to the most recent study cut-off date. For example, for 2009 Quarter 3, information on October 1, 2008, to September 30, 2009, will be available. This gives users of the website the ability to monitor trends over time without the problems associated with quarterly measures (e.g., seasonal effects, very little data in small counties). Pop Quiz on Rolling Years (Trainee Handout #4) ACTIVITY: Pop Quiz. Ask trainees to take 3 minutes to answer the questions (listed below) on Handout #4. After 3 minutes, ask trainees to call out correct answers to questions. Review rolling years concept if necessary. Pop Quiz on Rolling Years: When the 2009 Quarter 3 data are available (i.e., last day of data: September 30, 2009), the next progressive quarter’s worth of information can be added in tracking outcome measures. Question 1: In the 2009 Q3 data extract, what is

the most recent time period used to measure first entries to care?

Answer: The time period is October 1, 2008,

to September 30, 2009. Question 2: In the 2009 Q3 data extract, what is

the most recent time period used to measure the percent of children reunified within 12 months?


to September 30, 2008, since this is the most recent cohort of children for whom each can be followed for


19

an entire 12 months to end up in the September 30, 2009, data extract.

Question 3: In the 2009 Q3 data extract, what is

the most recent time period used to measure the percent of children adopted within 24 months?


to September 30, 2007, since this is the most recent cohort of children for whom each can be followed for an entire 24 months to end up in the September 30, 2009, data extract.

5. Taking Your First Steps Toward Data Analysis UC Berkeley Dynamic Website Introduction The introduction of the UC Berkeley Child Welfare Dynamic Report System Website requires a high-speed Internet connection. The trainer must be very familiar with the layout and information available on the UC Berkeley Child Welfare Dynamic Report System Website. Screen shots are provided in the Trainee’s PowerPoint slides (slide numbers are indicated below) strictly as points of reference for organizing the training. Slide 13

Child Welfare Dynamic Web Site

UC Berkeley Child Welfare Dynamic Report System Website All the data on the California Outcomes quarterly report are posted on the websites of the Center for Social Services Research at University of California, Berkeley (http://cssr.berkeley.edu/ucb_childwelfare/), or on the California Department of Social Services site (http://www.dss.cahwnet.gov/research/). In addition to the summary-level numbers provided on the county data reports, data on the UC Berkeley Child Welfare Dynamic Report System website include much more detailed information for each of the California Outcomes and Accountability measures (i.e., measures can be examined by age group, ethnicity, gender, and, in most cases, placement type) as well as for other outcomes. Be sure to click on and read the methodology link for any outcome analysis that you are examining. The


20

Slide 14 CWS Quarterly

Outcomes Report

Slide 15

National Standards

• In California, the Center for Social Services Research attempts to replicate each of the measures and composite scores, break them out by child welfare and probation agencies, and report/update quarterly.

• The goal is to improve State performance on all measures (every improvement reflects a better outcome for children)

Source: Center for Social Services Research, UC Berkeleyhttp://cssr.berkeley.edu/ucb_childwelfare

methodology description provides important detail about how the analysis was conducted. The quarterly data report includes a number of tabbed pages which allow the user to compare the latest outcome data with an established baseline or another period of time. The report also provides a unique view of the composite measures. The composite viewer tab shows the outcome performance relative to the national standard. The goal is to improve state performance on all measures. There is also a tab which details the methodology for each measure and provides a simple explanation for each outcome indicator. There are also tabs that show the past dates for the retrieval of past outcome data. Finally, there are graphs that depict the outcome performance over time at a one-shot glance. Other important features to explore on the UC Berkeley Child Welfare Dynamic Report System website:

All Available Outcomes by Specified County—provides all reports available on the UC Berkeley Report System website pertaining to a county that a user specifies. Select your county from a drop-down menu for county-specific information. With the dynamic site you can also choose different variables to review over time. http://cssr.berkeley.edu/ucb_childwelfare/

Data Highlights—are one-page documents

about statewide trends and developments in child welfare as interpreted from CWS/CMS data. A useful exercise would be to substitute your own county data for the statewide data. An index of available highlights is available at http://cssr.berkeley.edu/CWSCMSreports/Highlights/


21

Slide 16 Measure Contributions to

Composites

C1.1 (22%)

C1.2 (21%)

C1.3 (12%)

C1.4 (46%)

0%

20%

40%

60%

80%

100%

Composite 1

Reunification Within 12 Months (Exit Cohort)

Median Time To Reunification (Exit Cohort)

Reentry Following Reunification (Exit Cohort)

Reunification Within 12 Months (Entry Cohort)

Note: Measures may not sum to exactly 100% due to rounding.Source: Center for Social Services Research, UC Berkeley

http://cssr.berkeley.edu/ucb_childwelfare

Slide 17

Measure Contributions to Composites

C2.1 (15%)

C2.2 (19%)

C2.3 (22%)

C2.4 (18%)

C2.5 (26%)

0%

20%

40%

60%

80%

100%

Composite 2

Adoption Within 24 Months (Exit Cohort)

Median Time To Adoption (Exit Cohort)

Adoption Within 12 Months (17 Months In Care)

Legally Free Within 6 Months (17 Months In Care)

Adoption Within 12 Months (Legally Free)



Slide 18


C3.1 (33%)

C3.2 (25%)

C3.3 (42%)

0%

20%

40%

60%

80%

100%

Composite 3

Exits to Permanency (24 Months In Care)

Exits to Permanency (Legally Free At Exit)

In Care 3 Years Or Longer (Emancipated/Age 18)



Slide 19


C4.1 (33%)

C4.2 (34%)

C4.3 (33%)

0%

20%

40%

60%

80%

100%

Composite 4

Placement Stability (8 Days To 12 Months In Care)

Placement Stability (12 To 24 Months In Care)

Placement Stability (At Least 24 Months In Care)



Composite Viewer—shows the outcome performance relative to the national standard for:

Safety Measures:

S1_1: No Recurrence of Maltreatment S2_1: No Maltreatment in Foster Care

and Permanency Composites: Composite 1: Reunification Composite Composite 2: Adoption Composite Composite 3: Long-term Care Composite Composite 4: Placement Stability

Composite

The series of slides #16 through #20 depict how the individual measures of each permanency composite are weighted. These slides correspond to the Trainee Supplement New Federal Measures Listed with National Standards. For trainees who have great interest in this subject, the Trainee Supplement Federal Outcome Descriptors describes the computations for each outcome measure, as well as the rates for referrals, substantiations, entries, and ‘in care’, which are also found on the CSSR website. Slide #21 shows the overall structure for the relationships among measures, components, composites, and federal goals.

Mailing List Link—for email updates from the

UC Berkeley site. You will be notified of changes, updates, and additions to the site. To sign up for email updates go to: http://cssr.berkeley.edu/ucb_childwelfare/mailinfo.aspx

Presentations—Presentations by UC Berkeley child welfare researchers are available on the website and can provide supplemental information for understanding and reading data found on the website.


22

Slide 20 Measure Contributions to

Composites



C4.1 (33%)C3.1 (33%)C2.1 (15%)C1.1 (22%)

C4.2 (34%)C3.2 (25%)

C2.2 (19%)C1.2 (21%)

C4.3 (33%)C3.3 (42%)

C2.3 (22%)C1.3 (12%)

C2.4 (18%)

C1.4 (46%)

C2.5 (26%)

0%

20%

40%

60%

80%

100%

Composite 1 Composite 2 Composite 3 Composite 4

Slide 21

SafetyIndicator 1

S1.1

Indicator 2 S2.1

Permanency

Composite 1:Reunification

Composite 2: Adoption

Composite 3: Long-Term

Composite 4: Stability

Component A Component B

Component A Component B Component C


Measure 1 Measure 2 Measure 3 Measure 4

Measure 1 Measure 2 Measure 3 Measure 4 Measure 5

Measure 1 Measure 2 Measure 3


http://cssr.berkeley.edu/cwscmsreports/presentations/ Static Site—Information is still available on the static website, although the new dynamic website is more user-friendly. http://cssr.berkeley.edu/cwscmsreports/defaultStatic.asp In conjunction with explaining the CWS/CMS dynamic website, county-specific data tools should also be discussed. For example, Safe Measures reports or Business Objects reports are also useful tools for monitoring and assessing performance outcomes. When all tools are used in conjunction, supervisors can make the link between the macro picture of the federal outcome measures and standards and state, county, unit and worker outcomes and performance. It may be helpful to select one outcome area to demonstrate. For example, a supervisor could compare “visitation with parents” with “reunification within 12 months” in their unit and in their county. The data could assist the supervisor to address visitation and reunification issues with staff.

Break

PART 3 6. Outcome-Based Management—A Hands-On Exercise People learn best by doing. This final portion of the seminar will give trainees the opportunity to apply some of information covered in Parts 1 & 2 and to begin to grapple with looking at outcome data, interpreting results, and formulating plans to positively affect targeted outcomes. About 45 minutes should be spent working with the data; and the remaining 15 minutes can be used for reporting back.


23

Slide 22

Outcomes Based Management—a Hands-on Exercise

Part 3:

The following slides are provided courtesy of:Center for Social Services ResearchUniversity of California, Berkeleyhttp://cssr.berkeley.edu/ucb_childwelfare

Outcomes Based Management—A Hands-on Exercise (Handouts #5-16) The trainer should break the trainees into groups of 5–7 people. Each group will work together on interpreting the outcome data in their handout packet. The trainer should circulate between the groups to answer questions and help keep them on task. After forming the different groups, the trainer should underscore suggestions on what to think about as they review the data (these are listed on Handout #5). Not all the following points are applicable to this in-class exercise, but should be borne in mind whenever they work with outcome information in their county. Things to keep in mind as you examine outcome data3: Begin your analysis by determining which

outcomes (and possibly for which subgroups) are “high priority” and “high performance” areas within your county.

Examine trends in the outcomes over time (e.g.,

has the number and rate of entries changed over the past several years? Has the proportion of children reunified within 12 months changed?).

Also remember that sometimes data may not

answer “why” in relation to practice questions. For example, in the first round of the CSFR California failed to meet the required standards in the PIP regarding placement stability, among other outcomes. The data was able to show there weren’t improved outcomes in those areas, however it did not tell the story of “why” that was so. Data is needed to help focus the questions and discussion to drill-down to the

3 Some of these points adapted from: Hurley, D. (2004). Completing the county self-assessment, Module 3: Analyzing child welfare outcome indicators. California Social Work Education Center: Berkeley, California.


24

Slide 23 Trainee Handout #6

California:Ethnicity and the Path through the Child Welfare System 2007

(Missing Values & Other Race Excluded from % Calculations,18 years of Age)

6.214.9 14.6 18.6

26.319.9

32.728.5 27.2

26.225.2

27.7

50.452.9

48.0

10.3 3.0

44.751.8 50.2

2.43.9 4.1 3.50.5 0.8 1.2 1.6 1.4 1.3

0%

20%

40%

60%

80%

100%

Population(10,007,501)

Allegations(492,764)

Substantiations(107,524)

Entries (36,077)

In Care (72,221)

Exits* (36,190)

NativeAmerican

Asian/PI

Hispanic

White

Black



July 1, 2008California:

Race/Ethnicity by Caseload Case Service %

Components (Missing Values & Other Race Excluded from Calculations, Children of all Ages Included)

44.558.5

45.538.2

31.046.5

23.9

19.4

24.625.7

23.0

30.2

11.4 11.18.4

16.7 10.6

13.4

8.0

11.3

19.315.211.3

30.7

4.34.03.43.52.93.4

0%

20%

40%

60%

80%

100%

Total (104,698)

Black (24,367)

White (25,248)

Hispanic (50,007)

Asian/PI (3,408)

Native American(1,455)

Permanent Placement Family ReunificationFamily Maintenance Post-Placement Family Maintenance Pre-PlacementEmergency Response


<1 yr 1-2 yrs

3-5 yrs 6-10 yrs

11-15 yrs16-17 yrs

Black (113.2*) ALL

(49.2*) White (41.1*) Hispanic

(48.5*) Asian/PI(17.9*)

171

123130

121

102

81

66

495052

47

41

54

4042

4340

34

60

4749

52

4641

1815

1920

1815

2007California:

Referrals per 1,000 by Age and Ethnicity

*Series Total

“why”. In the case of the first round CSFR, the data outcomes have generated new questions.

Analyze age and especially ethnic disparities

(slides #23–28/Trainee Handouts 6–11). Look at differences between placement types

(particularly kinship versus non-kinship care) in slide #29/Trainee Handout 12.

Examine the differences in exits to permanency

(slide #30/ Trainee Handout 13). Comment on the difference between the national

goal and the data for Measure C3.3: In Care 3 Years of Longer (slide #31/Trainee Handout 14).

Note the measures for Permanency Composite

4: Placement Stability (slide #32/Trainee Handout 15).

Note the trend towards improvement for the

percentage of all siblings placed together as reflected in slide #33/Trainee Handout 16 .

Consider the relationships between outcome

measures. Recalling the “cycle of experiences” discussion and handout, a change in one performance measure may have a relationship with other outcomes. For example, diverting children from out-of-home care would decrease entry rates—but the children who still require placement may be more troubled, and less likely to be reunified, affecting reunification rates.

Identify factors that may be influencing

performance. Some examples are: • Management information system issues

(e.g., data cleanup, data entry, or interpretation issues that may have influenced performance measures.

• Staff and provider training needs • Case review system • Service array and agency collaborations • Recruitment, licensing and retention of


25


<1 yr 1-2 yrs

3-5 yrs 6-10 yrs

11-15 yrs16-17 yrs

Black (25.4*) ALL


(11.3*) Asian/PI(4.4*)

71

3330

25

20

13

25

1311

119

7

23

1110

97

5

24

1312

11

97

74

55

4

3

2007California:

Substantiated Referrals per 1,000 by Age and Ethnicity

*Series Total


<1 yr 1-2 yrs

3-5 yrs 6-10 yrs

11-15 yrs16-17 yrs

Black (11.2*) ALL


(3.7*) Asian/PI(1.3*)

40

1513

99

6

13

5

43

32

12

43

22

2

12

54

33

2

3

11

11

1

2007California:

Entries to Foster Care per 1,000 by Age and Ethnicity

*Series Total


<1 yr 1-2 yrs

3-5 yrs 6-10 yrs

11-15 yrs16-17 yrs

Black (31.7*) ALL


(6.7*) Asian/PI(1.7*)

2632

2725

3642

88

6689

77

55

67

89

76

67

22

21

22

2007California:

Children in Foster Care per 1,000 by Age and Ethnicity

*Series Total


0

10,000

20,000

30,000

40,000

50,000

60,000

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Point in Time

Plac

emen

t Fr

eque

ncy

1998 to 2008California:

Foster Care Caseload by Placement Type

Group/Shelter

Kinship

FFA

Foster

TOTAL Caseload

68,475

104,300

foster and adoptive parents • Quality assurance system

Identify any county policies, regulations, or

practices that may contribute to your performance on a given indicator.

Identify linkages between performance

outcomes and practice processes. Think of other outcome information that you

may want to have in order to get a more complete picture of performance (e.g., no re-entry data is given in the handouts, but this information would be helpful to know with respect to children reunifying from care).

Conclude analysis by determining areas of

strength and those in need of improvement. Summary Points for Hands-on Exercise Handouts Black children are referred for maltreatment

proportionally more than any other group. Black children make up 6.2% of the total child population, but account for 14.9% of referrals. (Slide #2/Trainee Handout #6).

Once referred, children are equally likely to

have substantiation. Across racial groups approximately 22–29% of children who are referred are substantiated.

Black children with open cases are the least

likely to be served at home and the most likely to be in long term foster care. Only 8 % of the Black children with open cases are in the Family Maintenance Program, compared to 11.1% of White Children, and 13.4% of Hispanic children. (Slide #24/Trainee Handout 7 , for July 1, 2008)

Black children are the most likely to enter foster

care. Black children enter care at a rate of 11.2 per 1,000—compared to 3.0 per 1,000 for White children, and 3.7 per 1,000 for Hispanic


26



C3.1: Exits to Permanency (24m In Care), by Exit Type

0

3

6

9

12

15

18

21

24

27

30

33

36

2002 2003 2004 2005 2006 2007

% E

xiti

ng

% Reunification

% Guardianship

% Adoption

% Exits to Permanency

TOTAL # in Care 24m+

27,696

44,435


0

10

20

30

40

50

60

70

80

90

100

2002 2003 2004 2005 2006 2007 2008

%


C3.3: In Care 3 Years Or Longer (Emancipated Or Age 18)

National Goal (37.5%)

% In Care 3+ Yearsgoal

# Emancipating or Age 18 in Care 4,6244,250



C4.1,2,3: Placement Stability

0

10

20

30

40

50

60

70

80

90

100

2002 2003 2004 2005 2006 2007

% W

ith

2 or

Few

er P

lace

men

ts

24+ Months In Care

12 to 24 Months In Care

8 Days to 12 Months in Care



Sibling Placements

0

10

20

30

40

50

60

70

80

90

100

2002 2003 2004 2005 2006 2007 2008

% All Siblings Placed Together

Some or All Siblings Placed Together

Children. (Jan. 1, 2007–Dec. 31, 2007, data) Black children are the most likely to be in foster

care. The rate of Black children in foster care is 29.1 per 1,000—compared to 5.2 per 1,000 for White children, and 6.0 per 1,000 for Hispanic Children. (July 1, 2008, count day)

Black children stay in care longer than children

from other racial groups. The median length of stay is 553 days for Black children, 432 days for White children, and 487 days for Hispanic children. (Jan. 1, 2001–Dec. 31, 2006)


27

7. Report Back and Session Wrap-Up The trainer should close the session by underscoring that managing for results necessitates creating a flow of information to support mid-course corrections and continuous improvements in outcomes. Some helpful ideas to accomplish this are:

Recognize that evaluation is a process, not an isolated event or a particular report.

Regularly monitor progress (e.g., expand management team meetings monthly to include regular updates on outcomes, create specific data/outcomes tracking committee that regularly meets and reports to management team and all agency staff).

Cultivate an office culture where everyone owns the outcomes: • Prominently display outcomes throughout the agency • Regularly update all staff on progress towards achieving outcomes

Finally, a homework assignment is provided in their Trainee Content (Handout #18). This series of steps is meant to review some of the key points covered in the seminar, and to reinforce the notion that supervisors can take what they have learned back to the office and begin applying it immediately. They can locate/download the latest copy of their county’s Quarterly Outcomes report, and refer to the UC Berkeley Child Welfare Dynamic Report System website for detailed underlying data (e.g., trends and relationships between outcome measures). Through the Dynamic Report System, they can track their agency's successes and areas needing improvement to help children and families more effectively.


TAB 9

Trainer’s Supplements


Trainer & Trainee Supplement

1 of 3

Federal Measures Listed with National Standards

Federal Measures National Standards

Safety Measure 1 –

S1.1 ‐ No Recurrence Of Maltreatment 94.6

Safety Measure 2 –

S2.1 ‐ No Maltreatment In Foster Care 99.68

Permanency Composite 1 – Reunification 122.6

Component A: Timeliness of reunification (3 measures)

C1.1 ‐ Reunification within 12 months (exit cohort) 75.2

C1.2 ‐ Median time to reunification (exit cohort) 5.4

C1.3 ‐ Reunification within 12 months (entry cohort) 48.4

Component B: Permanency of reunification

C1.4 ‐ Re‐entry following reunification (exit cohort) 9.9

Permanency Composite 2 – Adoption 106.4

Component A: Timeliness of adoptions of children discharged from foster care

C2.1 ‐ Adoption within 24 months (exit cohort) 36.6

C2.2 ‐ Median time to Adoption (exit cohort) 27.3

Component B: Progress toward adoption for children who meet ASFA Time‐in‐care Requirements

C2.3 ‐ Adoption within 12 months (17 months in care) 22.7 C2.4 ‐ Legally free within 6 months (17 months in care) 10.9

Component C: Progress toward adoption for children who are legally free for adoption

C2.5 ‐ Adoption within 12 months (legally free) 53.7


2 of 3

Permanency Composite 3 – Long Term Care 121.7

Component A: Achieving Permanency for Children in Foster Care for Extended Periods of Time

C3.1 ‐ Exits to Permanency (24 months in care) 29.1

C3.2 ‐ Exits to Permanency (legally free at exit) 98

Component B: Children Emancipated Who Were in Foster Care for Extended Periods of Time

C3.3 ‐ In care 3 years or longer (emancipated or age 18 in care) 37.5

Permanency Composite 4 – Placement Stability 101.5

Component A: Placement Stability

C4.1 ‐ Placement Stability (8 days to 12 months in care) 86

C4.2 ‐ Placement Stability (at least 12 months in care, but less than 24 months) 65.4

C4.3 ‐ Placement Stability (at least 24 months in care) 41.8

Well‐Being1

Siblings • All siblings placed together • Some or all siblings placed together

Not Applicable

Least Restrictive Placement • First Placement in care least restrictive (Relative Home, Foster

Home, FFA, Group/Shelter, Other)

ICWA Eligible/Cultural considerations • ICWA Eligible placed with: (Relative, Non‐Relative, Non‐Indian

Family Home, Indian/Non‐relative Indian Family, Non‐Indian/Non‐Relative)

• Placement with SCP of same or different ethnicity

Authorized for Psychotropic Medication

1 As of June 2009, specific well‐being measures including health, dental and education for children in care are under development.


3 of 3

Youth Aging Out/Post Foster Care Outcomes/Chaffee Requirements • High School Diploma • Enrolled in College/Higher Education • Received ILP Services • Completed Vocational Training • Employed or Other means of support


Trainer Supplement: Work Title/ Income Activity

Supervisor Core | Managing for Results | Version 1.1, July 2009

Work Title Income

UCB Ph.D., Consultant Roughly minimum wage

(let's say, $10 K)

Work Title Income


Work Title Income


Work Title Income


Work Title Income


Work Title Income


Trainer Supplement: Work Title/ Income Activity


Work Title Income


Work Title Income

Supervisor $60 K

Work Title Income

Supervisor $60 K

Work Title Income


Work Title Income


Work Title Income

Bill Gates, CEO Microsoft

$3 Billion

Managing for Results, Supervisory Core

Supervisor Core Training: Managing for Results

Original presentation was created for Version 1.0 by Daniel Webster, Barbara Needell, Wendy Piccus, Aron Shlonsky, & Lynn Usher.

Revised for Version 1.1 by Shared Vision Consultants and CalSWEC to include outcome updates and slides available from presentations posted on the CSSR website.

Background and Concepts for Outcomes in Child Welfare:Data Informed Practice -- What do I Care?

Part 1:

The Current Placement System*(Highly Simplified)

Child In Child OutA bunch of stuff happens

*adapted from Lyle, G. L., & Barker, M.A. (1998) Patterns & Spells: New approaches to conceptualizing children’s out of home placement experiences. Chicago: American Evaluation Association Annual Conference


Data Skepticism







Caseload Snapshots Versus Entry Cohorts

Jan. 1, 2008

Jan. 1, 2009Jan. 1, 2007


The Cycle of Experiences in the Child Welfare SysThe Cycle of Experiences in the Child Welfare Systemtem










CareCare







Using Outcome Data in Child Welfare Practice

Part 2:

Federal Versus State-Enhanced MeasuresIs Performance getting better or worse?

Adoption within 24 Months

Data Source: Multi-State Data Archive, Chapin Hall Center for Children

State A

0%

5%

10%

15%

20%

25%

30%

1990

1991

1992

1993

1994

1995

1996

1997

Ado

pted

ExitsEntries

State A

0%

5%

10%

15%

20%

25%

30%

1990

1991

1992

1993

1994

1995

1996

1997

Ado

pted

ExitsEntries


2008 2009

Jan. 1

Mar. 1 Mar. 1

Dec. 31

Jul. 7 Jul. 7

Jan. 1 Dec. 31

Dec. 31 Dec. 31

Tracking an Entry Cohort for 1 Year


Jan. 1

2008 2009

Dec. 31

Mar. 31 Mar. 31Jan. 1 Dec. 31Jan. 1

2007


April 1

Last day of data prior to cut-offMarch 31, 2009

ENTRY COHORT

Apr. 1

April 1, 2007 – March 31, 2008




Jan. 1

2008 2009

Dec. 31

Jun. 30 Jun. 30Jan. 1 Dec. 31Jan. 1

2007


July 1


ENTRY COHORT

Jul. 1

July 1, 2007 -- June 30, 2008



Child Welfare Dynamic Web Site

CWS QuarterlyOutcomes Report

National Standards

• In California, the Center for Social Services Research attempts to replicate each of the measures and composite scores, break them out by child welfare and probation agencies, and report/update quarterly.

• The goal is to improve State performance on all measures (every improvement reflects a better outcome for children)




C1.1 (22%)

C1.2 (21%)

C1.3 (12%)

C1.4 (46%)

0%

20%

40%

60%

80%

100%

Composite 1

Reunification Within 12 Months (Exit Cohort)

Median Time To Reunification (Exit Cohort)

Reentry Following Reunification (Exit Cohort)

Reunification Within 12 Months (Entry Cohort)




C2.1 (15%)

C2.2 (19%)

C2.3 (22%)

C2.4 (18%)

C2.5 (26%)

0%

20%

40%

60%

80%

100%

Composite 2

Adoption Within 24 Months (Exit Cohort)

Median Time To Adoption (Exit Cohort)

Adoption Within 12 Months (17 Months In Care)

Legally Free Within 6 Months (17 Months In Care)

Adoption Within 12 Months (Legally Free)




C3.1 (33%)

C3.2 (25%)

C3.3 (42%)

0%

20%

40%

60%

80%

100%

Composite 3

Exits to Permanency (24 Months In Care)

Exits to Permanency (Legally Free At Exit)

In Care 3 Years Or Longer (Emancipated/Age 18)





C4.1 (33%)

C4.2 (34%)

C4.3 (33%)

0%

20%

40%

60%

80%

100%

Composite 4

Placement Stability (8 Days To 12 Months In Care)

Placement Stability (12 To 24 Months In Care)

Placement Stability (At Least 24 Months In Care)






C4.1 (33%)C3.1 (33%)C2.1 (15%)C1.1 (22%)

C4.2 (34%)C3.2 (25%)

C2.2 (19%)C1.2 (21%)

C4.3 (33%)C3.3 (42%)

C2.3 (22%)C1.3 (12%)

C2.4 (18%)

C1.4 (46%)

C2.5 (26%)

0%

20%

40%

60%

80%

100%

Composite 1 Composite 2 Composite 3 Composite 4

SafetyIndicator 1

S1.1

Indicator 2 S2.1

Permanency

Composite 1:Reunification

Composite 2: Adoption

Composite 3: Long-Term

Composite 4: Stability


Component A Component B Component C


Measure 1 Measure 2 Measure 3 Measure 4

Measure 1 Measure 2 Measure 3 Measure 4 Measure 5




Outcomes Based Management—a Hands-on Exercise

Part 3:

The following slides are provided courtesy of:Center for Social Services ResearchUniversity of California, Berkeleyhttp://cssr.berkeley.edu/ucb_childwelfare

California:Ethnicity and the Path through the Child Welfare System 2007

(Missing Values & Other Race Excluded from % Calculations,18 years of Age)

6.214.9 14.6 18.6

26.319.9

32.728.5 27.2

26.225.2

27.7

50.452.9

48.0

10.3 3.0

44.751.8 50.2

2.43.9 4.1 3.50.5 0.8 1.2 1.6 1.4 1.3

0%

20%

40%

60%

80%

100%

Population(10,007,501)

Allegations(492,764)

Substantiations(107,524)

Entries (36,077)

In Care (72,221)

Exits* (36,190)

NativeAmerican

Asian/PI

Hispanic

White

Black


July 1, 2008California:

Race/Ethnicity by Caseload Case Service %

Components (Missing Values & Other Race Excluded from Calculations, Children of all Ages Included)

44.558.5

45.538.2

31.046.5

23.9

19.4

24.625.7

23.0

30.2

11.4 11.18.4

16.7 10.6

13.4

8.0

11.3

19.315.211.3

30.7

4.34.03.43.52.93.4

0%

20%

40%

60%

80%

100%

Total (104,698)

Black (24,367)

White (25,248)

Hispanic (50,007)

Asian/PI (3,408)

Native American(1,455)

Permanent Placement Family ReunificationFamily Maintenance Post-Placement Family Maintenance Pre-PlacementEmergency Response


<1 yr 1-2 yrs

3-5 yrs 6-10 yrs

11-15 yrs16-17 yrs

Black (113.2*) ALL


(48.5*) Asian/PI(17.9*)

171

123130

121

102

81

66

495052

47

41

54

4042

4340

34

60

4749

52

4641

1815

1920

1815

2007California:

Referrals per 1,000 by Age and Ethnicity

*Series Total

<1 yr 1-2 yrs

3-5 yrs 6-10 yrs

11-15 yrs16-17 yrs

Black (25.4*) ALL


(11.3*) Asian/PI(4.4*)

71

3330

25

20

13

25

1311

119

7

23

1110

97

5

24

1312

11

97

74

55

4

3

2007California:

Substantiated Referrals per 1,000 by Age and Ethnicity

*Series Total

<1 yr 1-2 yrs

3-5 yrs 6-10 yrs

11-15 yrs16-17 yrs

Black (11.2*) ALL


(3.7*) Asian/PI(1.3*)

40

1513

99

6

13

5

43

32

12

43

22

2

12

54

33

2

3

11

11

1

2007California:

Entries to Foster Care per 1,000 by Age and Ethnicity

*Series Total


<1 yr 1-2 yrs

3-5 yrs 6-10 yrs

11-15 yrs16-17 yrs

Black (31.7*) ALL


(6.7*) Asian/PI(1.7*)

2632

2725

3642

88

6689

77

55

67

89

76

67

22

21

22

2007California:

Children in Foster Care per 1,000 by Age and Ethnicity

*Series Total

0

10,000

20,000

30,000

40,000

50,000

60,000

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Point in Time

Plac

emen

t Fr

eque

ncy


Foster Care Caseload by Placement Type

Group/Shelter

Kinship

FFA

Foster

TOTAL Caseload

68,475

104,300


C3.1: Exits to Permanency (24m In Care), by Exit Type

0

3

6

9

12

15

18

21

24

27

30

33

36

2002 2003 2004 2005 2006 2007

% E

xiti

ng

% Reunification

% Guardianship

% Adoption

% Exits to Permanency

TOTAL # in Care 24m+

27,696

44,435


0

10

20

30

40

50

60

70

80

90

100

2002 2003 2004 2005 2006 2007 2008

%


C3.3: In Care 3 Years Or Longer (Emancipated Or Age 18)

National Goal (37.5%)

% In Care 3+ Yearsgoal

# Emancipating or Age 18 in Care 4,6244,250


C4.1,2,3: Placement Stability

0

10

20

30

40

50

60

70

80

90

100

2002 2003 2004 2005 2006 2007

% W

ith

2 or

Few

er P

lace

men

ts

24+ Months In Care

12 to 24 Months In Care

8 Days to 12 Months in Care


Sibling Placements

0

10

20

30

40

50

60

70

80

90

100

2002 2003 2004 2005 2006 2007 2008

% All Siblings Placed Together

Some or All Siblings Placed Together




FEDERAL OUTCOME DESCRIPTORS Number Measure Description Direction Weight

PR Referral Rates Computed by dividing the unduplicated count of children with an abuse or neglect allegation by the child population and multiplying by 1,000. (Age 0 to 17 years.)

‐

PR Substantiation Rates Computed by dividing the unduplicated count of children with a substantiated allegation by the child population and multiplying by 1,000. (Age 0 to 17 years.)

‐

PR Entry Rates Computed by dividing the unduplicated count of children with entering foster care by the child population and multiplying by 1,000. This rate looks at all entries to foster care. Both children first entering care and reentering care are included. (Age 0 to 17 years.)

‐

PR In Care Rates Computed by dividing the Point In Time count of children in child welfare supervised foster care by the child population and then multiplying by 1000. (Age 0 to 17.)

‐

S1.1 No Recurrence of Maltreatment

The percentage of children who were victims of a substantiated or indicated child maltreatment allegation within a specified 6‐month period for whom there was no additional substantiated maltreatment allegation during the subsequent 6 months. (Age 0 to 20 years.)

+

S2.1 No Maltreatment in Foster Care

The percentage of children who were not victims of a substantiated maltreatment report by a foster parent or facility staff while in out‐of‐home care. (Age 0 to 20 years.)

‐

C1 Reunification Composite

This is a child welfare supervised estimate of the composite score used in the CFSR2.

+



Number Measure Description Direction Weight

C1.1 Reunification within 12 Months (exit cohort)

The percentage of children discharged to reunification within 12 months of removal. The denominator is the total number of children who exited foster care to reunification during the specified year; the numerator is the count of exiting children who were reunified in less than 12 months.

+ 22%

C1.2 Median Time to Reunification (exit cohort)

The median length of stay (in months) for children discharged to reunification (excludes children in care < 8 days).

‐ 21%

C1.3 Reunification within 12 Months (entry cohort)

The percentage of children reunified within 12 months of removal for a cohort of children first entering foster care. The entry cohort is comprised of children entering foster care for the first time during a 6‐month period. (Age 0 to 17 years.)

+ 12%

C1.4 Reentry Following Reunification (exit cohort)

The percentage of children reentering foster care within 12 months of a reunification discharge. T (Age 0 to 17 years.)

‐ 46%

C2 Adoption Composite This is a child welfare supervised estimate of the composite score used in the CFSR2.

+

C2.1 Adoption within 24 Months (exit cohort)

The percentage of children adopted within 24 months of removal. (Age 0 to 17 years.)

+ 15%

C2.2 Median Time to Adoption (exit cohort)

The median length of stay (in months) for children discharged to adoption. (Age 0 to 17 years.)

‐ 19%

C2.3 Adoption within 12 Months (17 Months in Care)

The percentage of children in foster care for 17 continuous months or longer on the first day of the year, who were then adopted within 12 months. (Age 0 to 17 years.)

+ 22%




C2.4 Legally Free within 6 Months (17 Months in Care)

The percentage of children who were in foster care for 17 continuous months or longer and not legally free for adoption on the first day of the period, who then became legally free for adoption within the next 6 months. (Age 0 to 17 years.)

+ 18%

C2.5 Adoption within 12 Months (Legally Free)

The percentage of children discharged from foster care to adoption within 12 months of becoming legally free. (Age 0 to 17 years.)

+ 26%

C3 Long Term Care Composite


+

C3.1 Exits to Permanency (24 Months in Care)

The percentage of children discharged to a permanent home by the last day of the year and prior to turning 18, who had been in foster care for 24 months or longer. (Age 0 to 17 years.)

+ 33%

C3.2 Exits to Permanency (Legally Free at Exit)

The percentage of legally free children who were discharged to a permanent home prior to turning 18. (Age 0 to 17 years.)

+ 25%

C3.3 In Care 3 Years or Longer (Emancipated/Age 18)

The percentage of children in foster care for 3 years or longer who were then either discharged to emancipation or turned 18 while still in foster care. (Age 0 to 18 years.)

‐ 42%

C4 Placement Stability Composite


+




C4.1 Placement Stability (8 Days to 12 Months in Care)

The percentage of children with two or fewer placements who have been in foster care for 8 days or more, but less than 12 months. Time in care is based on the latest date of removal from the home. (Age 0 to 17 years.)

+ 33%

C4.2 Placement Stability (12 to 24 Months in Care)

The percentage of children with two or fewer placements who have been in foster care for at least 12 months, but less than 24 months. Time in care is based on the latest date of removal from the home. (Age 0 to 17 years.)

+ 34%

C4.3 Placement Stability (at least 24 Months in Care)

The percentage of children with two or fewer placements who have been in foster care for 24 months or more. Time in care is based on the latest date of removal from the home. (Age 0 to 17 years.)

+ 33%

Documents

SUPERVISOR CORE: MANAGING FOR RESULTS · example, minor revisions are indicated by the numbered sequence Version 1.1, 1.2, 1.3, etc., while major revisions are delineated by a sequence