Slide 1 Monitoring and Sustaining System Change Moderator: Alessandra Meyer, Senior Program...

Slide 1

Monitoring and Sustaining System Change

Moderator: Alessandra Meyer,

Senior Program Associate, Vera’s Center on Youth Justice

April 21, 2023

January 6, 2015

Slide 2 • April 21, 2023

The Status Offense Reform Center

• Vera’s Status Offense Reform Center (SORC)

• Funded and supported by the MacArthur Foundation, as part of the Resource Center Partnership

• www.statusoffensereform.org

• Mission

• To help policymakers and practitioners create effective, community-based responses for keeping youth who commit status offenses out of the juvenile justice system and safely in their homes and communities.

• What does SORC provide?

• A range of tools, resources, and information to help guide system transformation

Slide 3 • April 21, 2023

How to Bring About System Change

• Featured Resource: A Toolkit for Status Offense System Change

• Module 1: Structuring System Change• Module 2: Using Local Information to Guide System

Change• Module 3: Planning and Implementing System

Change• Module 4: Monitoring and Sustaining System

Change

Slide 4 • April 21, 2023

Monitoring and Sustaining System Change

• Step 1: Monitor Performance

• Step 2: Conduct a Process Evaluation

• Step 3: Conduct an Outcome Evaluation

• Step 4: Sustain Positive Change

Slide 5 • April 21, 2023

Our Presenters

Julie Revaz

Manager of Administrative Services, Judicial Branch,

State of CT, Court Support Services Division

Tara Grieshop-Goodwin

Chief Policy Officer, Kentucky Youth Advocates

Dane Bolin Jeff Vander

Assistant Administrative Administrator Analyst

Calcasieu Parish Policy Jury

Slide 6 • April 21, 2023

Connecticut: Monitoring performance

Slide 7 • April 21, 2023

An Overview of Reform Approach

• Child and Youth Family Support Centers (CYFSCs) are one-stop service centers for youth and families needing intensive services

Immediate Access / Intake / ServiceCognitive Behavioral GroupsEducational AdvocacyFamily Mediation and EBP Referrals

Slide 8 • April 21, 2023

Continuous Quality Improvement – Results-Based Accountability

Results Based Accountability

...is a framework adopted by the CT Legislature to answer: How much do we do? How well do we do it? Is anyone better off?

…requires the development of indicators of success, the creation of goals based on

benchmarks

…favors inclusion and transparency

Slide 9 • April 21, 2023

RBA continued

• Stakeholder focus groups brainstorm “how you’d know if the miracle occurred….”

• Stakeholders vote on measures to select 3-5 based on their communication, data and proxy power

• One year of baseline data is assembled and annual goals are typically set 1% above or below the baseline

• Contractors have real time access to data

• Data is reviewed locally quarterly, by administration bi-annually, and with all parties together annually

Slide 10 • April 21, 2023

Other Quality Assurance Efforts

• Evidence-based models have prescribed QA expectations

• Quality Assurance coaches review videotapes of assessments, cognitive behavioral groups, and some case management sessions, and provide monthly feedback to program staff

• Clients provide quarterly input via Working Alliance Inventory surveys (WAIs)

• CT Judicial Branch contract staff act as problem-solvers and TA coaches with program providers

Slide 11 • April 21, 2023

Status Offender Program Completion

Slide 12 • April 21, 2023

Q1 Group Completion Rates

Slide 13 • April 21, 2023

Tips for Performance Monitoring

1. Teach a man to fish

2. The story behind the numbers

3. Transparency of outcomes

4. Unanticipated consequences of “what gets measured gets done”

5. So then what?

6. Sometimes less is more

7. Performance-based contracting

Slide 14 • April 21, 2023

Status Offenders Referred to SCJM

Slide 15 • April 21, 2023

Court Commitments 1999 - 2013

Slide 16 • April 21, 2023

Juvenile Court Intake 2008-2013

Slide 17 • April 21, 2023

Calcasieu Parish, LA:Conducting process and outcome evaluations

Slide 18 • April 21, 2023

An Overview of Reform Approach

The center is a collaborative project of the Calcasieu Parish Children and Youth Planning Board Member agencies.

The mission of the MARC is to provide a single entry point and coordinated approach to juvenile services for the youth and families of Calcasieu Parish.

MARC Video: http://youtu.be/59T1CMiBecQ

Slide 19 • April 21, 2023

MARC Goals MARC Criteria• Provide youth and families with a single access point.

• Provide on-site screening and assessments of youth.

• Reduce Law Enforcement processing times.

• Reduce time between arrest and intake.

• Promote public safety and wellbeing.

• Walk-in cases (Families requesting information or assistance)

• Alleged Status cases

• Alleged Delinquent cases

MARC Goals/Criteria

Slide 20 • April 21, 2023

Process Evaluation

• Evaluation Structure

Partnership with Vera (outside evaluator)

Vera had prior knowledge of process and site development

• Qualitative in nature

Interviews with staff

Pre/Post-MARC opening data sets

• Timing of the evaluation

Slide 21 • April 21, 2023

MARC Outcome Research Questions

• Are youth referred to MARC more likely to be successfully diverted compared to prior system youth?

• Does MARC produce a net-widening effect?

• How well are youth assessed and referred to the appropriate services based on their needs?

• Did the implementation of MARC increase the speed of service delivery and case processing of youth?

Slide 22 • April 21, 2023

Learning and Improving from the Process Evaluation

Slide 23 • April 21, 2023

Learning and Improving from the Process Evaluation

• Evaluation identified: JIFF goals nominated were not matched directly to services.

Follow-up assessments required a continuation of services and follow-up documentation.

• The Agency: Increased documentation of cross-referrals

Instituted random quality assurance audits

Developed a follow-up procedure of high-risk cases.

Developed a walk-in Mental and Substance Abuse assessment clinic.

Developed a work flow application to ensure follow-up of services.

Slide 24 • April 21, 2023

Reduced processing times

Outcome Evaluation

Slide 25 • April 21, 2023

• Reduced time to re-referrals Pre-MARC days to new referral- 125

Post-MARC days to new referral- 159

Outcome Evaluation

Slide 26 • April 21, 2023

Diversion vs. Formal Petitioned

Outcome Evaluation

Slide 27 • April 21, 2023

Tips for Conducting an Evaluation

• Create a concrete set of research questions in the beginning.

• Data, Data, Data... Pre and Post

• Identify Staff that can provide feedback and assist in data collection to Evaluators.

• Set-up status calls throughout, be prepared for bumps in the road.

Slide 28 • April 21, 2023

Kentucky:Sustaining reform through legislation

Slide 29 • April 21, 2023

Kentucky context

50 statesamong

Data source: Coalition for Juvenile Justice

4th highest

for youth locked up for status offenses

Slide 30 • April 21, 2023

Youth in secure detention in Kentucky

Data source: Department of Juvenile Justice and Louisville Metro Youth Detention Services 

Slide 31 • April 21, 2023

An Overview of Reform Approach

• Statewide reforms many years in the making

• Building will to reform system

• Power of local model

Slide 32 • April 21, 2023

Key components of legislative reforms

• Creates team to review cases that aren’t successful with diversion process

• Some specific limits on detention of youth for running away

• Graduated response in diversion

• Creates fiscal incentive program

• Improves data collection

• Establishes Juvenile Justice Oversight Council

Slide 33 • April 21, 2023

Sustaining Reform through Legislation

• Building buy-in for statewide reforms

• Elevate the issue with public

• Engage lawmakers

• Strong communication

Slide 34 • April 21, 2023

• Determining the “how”

• Power of local model

Deciding on reforms

Campbell County

Slide 35 • April 21, 2023

Implementation

• Two phases

• FAIR team pilots underway

• Revisions to rules of law

• State agencies implementing changes

Slide 36 • April 21, 2023

Tips for Reforming through Legislation

• Share the numbers

• Create a strong message

• Engage media

Slide 37 • April 21, 2023

Questions?

Slide 38 • April 21, 2023

To learn more about Vera’s work in this area, contact:Alessandra Meyer, ameyer@vera.org

To access the Toolkit for Status Offense System Change and other resources, visit the Status Offense Reform

Center at:www.statusoffensereform.org

Find us on twitter: @SOreformcenter

Slide 39 • April 21, 2023

Audience Poll

How helpful did you find this webinar?

A.Very Helpful

B.Somewhat Helpful

C. Not Helpful