NJ - 1

Performance Measurement

Reporting

Development Services Group, Inc.Don Johnson

For more information contactDevelopment Services Group, Inc.

7315 Wisconsin Avenue, Suite 800E, Bethesda, MD 20814 Phone: 301-951-0056 Fax: 301-951-3324 Toll-free: 1-877-5-TITLE-5

www.dsgonline.com

NJ - 2

Why Use Performance Measurement?

New requirement from OJJDP, similar to other federal funding sources – comply with GRPA

Gives national standards for a move toward accountability

Provides an additional program monitoring tool for PPB

Priority for Evidence-Based Programs Priority for Evidence-Based Programs and Increased Accountability Title II of the JJDP Act of 2002 states, “designated

state agency will, to the extent practicable, give priority in funding to programs and activities that are based on rigorous, systematic, and objective research that is scientifically based.”

Title II also states that the state agency shall “not expend funds to carry out a program if such program, during the preceding 2-year period, fails to demonstrate, before the expiration of such 2-year period, that such program achieved substantial success in achieving the goals specified.”

Priority for Evidence-Based Programs Priority for Evidence-Based Programs and Increased Accountability

Title V of the JJDP Act of 2002 states the Administrator shall give priority to “developing data-driven prevention plans, employing evidence-based prevention strategies, and conducting program evaluations to determine impact and effectiveness.”

What Is Performance Measurement?

Performance measurement is a system of tracking progress in accomplishing specific goals, objectives, activities and outcomes.

Performance measurement: Is directly related to program goals and objectives Measures progress quantitatively – outputs and

outcomes Is not exhaustive of all possible measures Provides a “temperature” reading—it may not tell

you everything you want to know but provides a quick and reliable gauge of selected results

NJ - 5

Evaluation is a more formal process, often done by an experienced researcher outside the program.

Performance Measurement is less formal and can be a component of a formal evaluation.

Evaluation versus Performance Measurement

NJ - 6

Title V Program Areas

Funds for Title V Programs must be for:

At-risk youth to prevent them from entering the juvenile justice system, or

Early intervention programs for juveniles with first-time and non-serious offenses to keep them out of the juvenile justice system.

Applications under Title V must propose to use Title V funds for these populations only.

NJ - 7

Title V Program Areas

Select the Program Area that most closely aligns with your proposed Strategy/ Program based on the prioritized risk factors and problem indicators.

If funded, Juvenile Justice Commission staff will help verify the correct

Program Area to use for Performance Reporting.

There are 18 program areas, as listed in Section VIII of the NOAF, and described on the Title V web page http://dsgonline.com/ Program_Logic_Model/titlev_pm.htm

NJ-8

Title V Program Areas

Delinquency Prevention – directed at youth “at risk” of becoming delinquent. Program

types: parent education, anger management, conflict resolution, youth development, improving social skills, etc.

School Programs – designed to prevent truancy, suspension and expulsion; including school safety programs.

Gangs – directed at reducing gang-related activities.

Program Areas most likely to be selected

NJ - 9

Outputs Versus OutcomesOutputs are products of program implementation/activities (volume of work accomplished – numbers).

Outcomes are benefits or changes that result from the program. Related to changes in Behavior, Attitudes, Skills or Knowledge (BASK).

               2 types of outcomes:

Short-term Outcomes: those that occur during a youth’s involvement with the program or by the time of program completion.

Long-term Outcomes: those that occur 6 months to 1 year after a youth completes the program.

NJ - 10

NJ - 11

Output and Outcome Quiz

Individually

Review the hand out

Record your own answers

At tables/In small groups

Compare answers

Be prepared to report out

NJ - 12

A graphic representation that clearly identifies and

lays out the logical relationships between program

objectives, activities, outputs, and outcomes.

Objectives Activities Outputs Outcomes• Short-term• Long-term

Logic Model

NJ - 13

Clarifies assumptions and relationships between program efforts and expected results

Supports development of effective proposals

Helps specify what to measure to verify progress and impact

Guides assessment of underlying assumptions and self-correction

Benefits of a Logic Model

NJ - 14

Sample Grid Chart: Delinquency PreventionOutput Performance Measures

NJ- 15

Sample Grid Chart: Delinquency PreventionOutcome Performance Measures

NJ - 16

NJ - 17

NJ - 18

Logic Models and Logic Models and GridsGrids

These logic models and all of the logic models for each of the 18 Title V program areas can be downloaded from the OJJDP performance measurement web site at http://dsgonline.com/Program_Logic_Model/titlev_pm.htm.

NJ - 17

Reporting RequirementsFrom the applicable Program Area, successful

applicants will report the following:

Output Measures It is required to report on all of the mandatory output measures, and at least two of the non- mandatory output measures.

Outcome Measures It is required to report on all of the mandatory short-term and long-term outcome measures, and at least two of the non-mandatory outcome measures (either short- or long-term).

NJ - 20

1. Amount of funds awarded (state reports)2. Number of program youth served: An unduplicated count of the number of youths

served by the program during the reporting period. Definition of the number of youth served for a reporting period is the number of program youth carried over from previous reporting period, plus new admissions during the reporting period.  

A spreadsheet tool is available on the Performance Measurement web site (hard copy in packet).

Mandatory Output Measure

NJ - 21

Number and percent of youth completing program requirements:

The number and percent of program youth who have successfully fulfilled all program obligations and requirements. Program requirements will vary by program but should be a predefined list of requirements or obligations that clients must meet prior to program completion. Program records are the preferred data source.

Mandatory Short-term Outcome Measure

NJ - 22

Number and percent of program youth exhibiting desired change in targeted behaviors (select one or more that apply to your program):

Substance use School attendance Antisocial behavior Family relationships Pregnancies

Mandatory Outcome MeasureShort-term and Long-term

NJ - 23

Logic Model Example

NJ - 24

Logic Model Exercise

NJ - 25

NJ - 26

Elements of the data collection plan MEASURES: Specify observable and measurable characteristics. Include all performance measures

DEFINITION: Specify what you will use to describe or represent each measure (indicators)

FREQUENCY OF COLLECTION: Specify regular intervals when data will be collected

Write Data Collection Plan

NJ - 27

Elements of the data collection plan RESPONSIBLE FOR COLLECTION: Specify who will

collect the data

INSTRUMENT/DATA SOURCE: Specify the data you will use to measure each indicator

LOCATION OF DATA: Specify where data will come from

HOW PROCESSED OR RETRIEVED: Data collection method to be used – MOU Required?

Write Data Collection Plan (cont’d)

Data Collection Example

NJ - 28

Data Collection Form Exercise

NJ - 29

NJ - 30

Performance Measurement Information

Additional information on OJJDP Performance Measures for Title V available at:

www.dsgonline.com/performance_measures.htm

NJ - 31

1. Identify an evidence-based program to be implemented to address your problem

2. Incorporate performance measurement into your proposal

Select a program area (corresponding to your proposed program/strategy)

Construct program logic model Review mandatory output and outcome

measures Select non-mandatory output and

outcome measures Develop data collection plan

Next Steps