Slide 1 Monitoring and Sustaining System Change Moderator: Alessandra Meyer, Senior Program Associate, Vera’s Center on Youth Justice October 14, 2015

Slide 1

Monitoring and Sustaining System Change

Moderator: Alessandra Meyer,

Senior Program Associate, Vera’s Center on Youth Justice

April 21, 2023

January 6, 2015

Slide 2 • April 21, 2023

The Status Offense Reform Center

• Vera’s Status Offense Reform Center (SORC)

• Funded and supported by the MacArthur Foundation, as part of the Resource Center Partnership

• www.statusoffensereform.org

• Mission

• To help policymakers and practitioners create effective, community-based responses for keeping youth who commit status offenses out of the juvenile justice system and safely in their homes and communities.

• What does SORC provide?

• A range of tools, resources, and information to help guide system transformation


How to Bring About System Change

• Featured Resource: A Toolkit for Status Offense System Change

• Module 1: Structuring System Change• Module 2: Using Local Information to Guide System

Change• Module 3: Planning and Implementing System

Change• Module 4: Monitoring and Sustaining System

Change


Monitoring and Sustaining System Change

• Step 1: Monitor Performance

• Step 2: Conduct a Process Evaluation

• Step 3: Conduct an Outcome Evaluation

• Step 4: Sustain Positive Change


Our Presenters

Julie Revaz

Manager of Administrative Services, Judicial Branch,

State of CT, Court Support Services Division

Tara Grieshop-Goodwin

Chief Policy Officer, Kentucky Youth Advocates

Dane Bolin Jeff Vander

Assistant Administrative Administrator Analyst

Calcasieu Parish Policy Jury


Connecticut: Monitoring performance


An Overview of Reform Approach

• Child and Youth Family Support Centers (CYFSCs) are one-stop service centers for youth and families needing intensive services

Immediate Access / Intake / ServiceCognitive Behavioral GroupsEducational AdvocacyFamily Mediation and EBP Referrals


Continuous Quality Improvement – Results-Based Accountability

Results Based Accountability

...is a framework adopted by the CT Legislature to answer: How much do we do? How well do we do it? Is anyone better off?

…requires the development of indicators of success, the creation of goals based on

benchmarks

…favors inclusion and transparency


RBA continued

• Stakeholder focus groups brainstorm “how you’d know if the miracle occurred….”

• Stakeholders vote on measures to select 3-5 based on their communication, data and proxy power

• One year of baseline data is assembled and annual goals are typically set 1% above or below the baseline

• Contractors have real time access to data

• Data is reviewed locally quarterly, by administration bi-annually, and with all parties together annually


Other Quality Assurance Efforts

• Evidence-based models have prescribed QA expectations

• Quality Assurance coaches review videotapes of assessments, cognitive behavioral groups, and some case management sessions, and provide monthly feedback to program staff

• Clients provide quarterly input via Working Alliance Inventory surveys (WAIs)

• CT Judicial Branch contract staff act as problem-solvers and TA coaches with program providers


Status Offender Program Completion


Q1 Group Completion Rates


Tips for Performance Monitoring

1. Teach a man to fish

2. The story behind the numbers

3. Transparency of outcomes

4. Unanticipated consequences of “what gets measured gets done”

5. So then what?

6. Sometimes less is more

7. Performance-based contracting


Status Offenders Referred to SCJM


Court Commitments 1999 - 2013


Juvenile Court Intake 2008-2013


Calcasieu Parish, LA:Conducting process and outcome evaluations



The center is a collaborative project of the Calcasieu Parish Children and Youth Planning Board Member agencies.

The mission of the MARC is to provide a single entry point and coordinated approach to juvenile services for the youth and families of Calcasieu Parish.

MARC Video: http://youtu.be/59T1CMiBecQ


MARC Goals MARC Criteria• Provide youth and families with a single access point.

• Provide on-site screening and assessments of youth.

• Reduce Law Enforcement processing times.

• Reduce time between arrest and intake.

• Promote public safety and wellbeing.

• Walk-in cases (Families requesting information or assistance)

• Alleged Status cases

• Alleged Delinquent cases

MARC Goals/Criteria


Process Evaluation

• Evaluation Structure

Partnership with Vera (outside evaluator)

Vera had prior knowledge of process and site development

• Qualitative in nature

Interviews with staff

Pre/Post-MARC opening data sets

• Timing of the evaluation


MARC Outcome Research Questions

• Are youth referred to MARC more likely to be successfully diverted compared to prior system youth?

• Does MARC produce a net-widening effect?

• How well are youth assessed and referred to the appropriate services based on their needs?

• Did the implementation of MARC increase the speed of service delivery and case processing of youth?


Learning and Improving from the Process Evaluation


Learning and Improving from the Process Evaluation

• Evaluation identified: JIFF goals nominated were not matched directly to services.

Follow-up assessments required a continuation of services and follow-up documentation.

• The Agency: Increased documentation of cross-referrals

Instituted random quality assurance audits

Developed a follow-up procedure of high-risk cases.

Developed a walk-in Mental and Substance Abuse assessment clinic.

Developed a work flow application to ensure follow-up of services.


Reduced processing times

Outcome Evaluation


• Reduced time to re-referrals Pre-MARC days to new referral- 125

Post-MARC days to new referral- 159

Outcome Evaluation


Diversion vs. Formal Petitioned

Outcome Evaluation


Tips for Conducting an Evaluation

• Create a concrete set of research questions in the beginning.

• Data, Data, Data... Pre and Post

• Identify Staff that can provide feedback and assist in data collection to Evaluators.

• Set-up status calls throughout, be prepared for bumps in the road.


Kentucky:Sustaining reform through legislation


Kentucky context

50 statesamong

Data source: Coalition for Juvenile Justice

4th highest

for youth locked up for status offenses


Youth in secure detention in Kentucky

Data source: Department of Juvenile Justice and Louisville Metro Youth Detention Services



• Statewide reforms many years in the making

• Building will to reform system

• Power of local model


Key components of legislative reforms

• Creates team to review cases that aren’t successful with diversion process

• Some specific limits on detention of youth for running away

• Graduated response in diversion

• Creates fiscal incentive program

• Improves data collection

• Establishes Juvenile Justice Oversight Council


Sustaining Reform through Legislation

• Building buy-in for statewide reforms

• Elevate the issue with public

• Engage lawmakers

• Strong communication


• Determining the “how”

• Power of local model

Deciding on reforms

Campbell County


Implementation

• Two phases

• FAIR team pilots underway

• Revisions to rules of law

• State agencies implementing changes


Tips for Reforming through Legislation

• Share the numbers

• Create a strong message

• Engage media


Questions?


To learn more about Vera’s work in this area, contact:Alessandra Meyer, [email protected]

To access the Toolkit for Status Offense System Change and other resources, visit the Status Offense Reform

Center at:www.statusoffensereform.org

Find us on twitter: @SOreformcenter


Audience Poll

How helpful did you find this webinar?

A.Very Helpful

B.Somewhat Helpful

C. Not Helpful

Documents

Slide 1 Monitoring and Sustaining System Change Moderator: Alessandra Meyer, Senior Program Associate, Vera’s Center on Youth Justice October 14, 2015