1 Facilitators: Lorna McCue and Pam Kinzie, HC Link Guest Presenter: Andrew Taylor, Taylor Newbury...

1

Facilitators: Lorna McCue and Pam Kinzie, HC Link

Guest Presenter: Andrew Taylor, Taylor Newbury Consulting

Results-Based Accountability for community organizations and networks

Session 2: Performance Accountabilityfor Programs, Agencies and Service Systems

Web/Teleconference Tips Synchronize your voice with the webconnection

Web/Teleconference Tips

Mute/Un-mute: *1

ON HOLD

Web/Teleconference Tips Private and group chat

Web/Teleconference Tips Size of screen

If the whole page is not displayed an image of the the page will appear for navigation

Disconnected? See instructions Screen saver or Power saver

Contact Us

www.hclinkontario.ca

416-847-1575 Toll-free: 1-855-847-1575Fax: 416-408-4843

info@hclinkontario.ca

Introductions

Who’s Online?

Facilitators:• Pam Kinzie, HC Link• Lorna McCue, HC Link• Andrew Taylor,

Taylor Newbury Consulting

Please indicate your:• Name• Organization

Session 2 Agenda1. Welcome and Introductions

2. Learning Objectives & Agenda Review

3. Recap from Session 1

4. Performance Measures – 4 Quadrants• Choosing Headline Measures• Comparing Performance• Turn the Curve Report: Performance• Performance accountability questions

5. Performance Measures – Real life Examples

6. “Homework” Assignment

7. Q&A

8. Wrap-Up

Learning Objectives

After participating in this webinar you will be able to:

• Define performance accountability;• Describe performance measures in each of the 4

quadrants• Identify how performance accountability may be

useful to your organizations• Take the next steps to find out more about RBA

Results Accountabilityis made up of two parts:

Performance Accountabilityabout the well-being of

CLIENT POPULATIONS

For Programs – Agencies – and Service Systems

Population Accountabilityabout the well-being of

WHOLE POPULATIONSFor Communities – Cities – Counties – States - Nations

“All performance measures that have ever existed for any

program in the history of the universe involve answering two sets of interlocking questions.”

HowMuchdid we do?

( # )

HowWelldid we do it?

( % )

Quantity QualityPerformance Measures

EffortHow hard did we try?

EffectIs anyone better off?

Performance Measures

HowMuch

HowWell

Performance Measures

Effort

Effect

Quantity

How much did we do?

Education

Quality

How well did we do it?

Effe

ct

Ef

fort

Is anyone better off?

How much service did we deliver?

How welldid we deliver it?

How much change / effect did we produce?

What quality of change / effect did we produce?

Quantity

How much did we do?

Education

Quality

How well did we do it?

Effe

ct

Ef

fort

Number ofstudents

Student-teacher ratio

Number ofhigh schoolgraduates

Percent ofhigh schoolgraduates

Is anyone better off?

Quantity

How much did we do?

Education

Quality

How well did we do it?

Effe

ct

Ef

fort

Number ofstudents

Student-teacher ratio

Is anyone better off?Number of 9th graders who graduate on time and enter college or employment after graduation

Percent of 9th graders who graduate on timeand enter college oremployment after graduation

Lay

Definition

All Data Have Two Incarnations

Technical

Definition

HS Graduation Rate % enrolled June 1 who graduate June 15

% enrolled Sept 30 who graduate June 15

% enrolled 9th grade who graduate in 12th grade

RBA Categories Account for All Performance Measures (in the history of the universe)

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Product Output Impact

Process Input

Effe

ctEf

fort

Cost

TQM

Effectiveness

Efficiency

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Process Input

Effe

ctEf

fort

Cost

TQM

Product Output Impact

RBA Categories Account for All Performance Measures (in the history of the universe)

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Process Input

Effe

ctEf

fort

Cost

TQM

1. Did we treat you well?

2. Did we help you with your problems?

*

Product Output Impact

RBA Categories Account for All Performance Measures (in the history of the universe)

* World’s simplest completecustomer satisfaction survey

Not All Performance Measures Are Created Equal

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Process Input

Effe

ctEf

fort

Cost

TQM

Product Output Impact

How much did we do?

Not All Performance Measures Are Created Equal

How well did we do it?

Is anyone better off?

LeastImportant

Quantity Quality

E

ffect

E

ffort

MostImportant

Least

Most

AlsoVery Important

How much did we do?

The Matter of Control

How well did we do it?

Is anyone better off?

Quantity Quality

Effe

ct

Effo

rt

LeastControl

PARTNERSHIPS

MostControl

The Matter of Use

1. The first purpose of performance measurement is to improve performance.

2. Avoid the performance measurement equals punishment trap.•Create a healthy organizational environment.

•Start small.

•Build bottom-up and top-down simultaneously.

1. To Ourselves Can we do better than our own history?

2. To Others When it is a fair apples/apples comparison.

3. To Standards When we know what good performance is.

Comparing Performance

2. To Others When it is a fair apples/apples comparison.

3. To Standards When we know what good performance is.

1. To Ourselves First

Can we do better than our own history?

Using a BaselineCHART ON THE WALL

Comparing Performance

1. To Ourselves First Can we do better than our own history?

2. To Others When it is a fair apples/apples comparison.

Reward?

Punish?

3. To Standards When we know what good performance is.

Comparing Performance

1. To Ourselves First Can we do better than our own history?

2. To Others When it is a fair apples/apples comparison.

Comparing Performance

3. To Standards When we know what good performance is.

The Matter of Standards

Quantity

Effe

ctEf

fort1. Quality of Effort Standards aresometimes WELL ESTABLISHED

• Child care staffing ratios• Application processing time• Handicap accessibility• Child abuse response time

2. Quality of Effect Standards arealmost always EXPERIMENTAL

• Hospital recovery rates • Employment placement and

retention rates• Recidivism rates

3. Both require aLEVEL PLAYING FIELD and an ESTABLISHED RECORDof what good performance is.

BUT

AND

Advanced Baseline Display

Your Baseline

Comparison Baseline

Goal (line)Target or Standard

Instead:Count anything better than baseline as progress.

Avoid publicly declaringtargets by year if possible.

●

Create targets only when they are:FAIR & USEFUL

How much did we do?

Separating the Wheat from the ChaffTypes of Measures Found in Each Quadrant

How well did we do it?

Is anyone better off?

# Clients/customers served

# Activities (by type of activity)

% Common measurese.g. client staff ratio, workload ratio, staffturnover rate, staff morale, % staff fully trained, % clients seen in their own language,worker safety, unit cost

% Skills / Knowledge (e.g. parenting skills)

#

% Attitude / Opinion (e.g. toward drugs)

#

% Behavior (e.g.school attendance)

#

% Circumstance (e.g. working, in stable housing)

#

% Activity-specific measures

e.g. % timely, % clients completing activity, % correct and complete, % meeting standard

Point in Time vs. Point to Point Improvement

How much did we do?

Choosing Headline Measures and the Data Development Agenda

How well did we do it?

Is anyone better off?

Quantity Quality

E

ffect

E

ffort

# Measure 1 ----------------------------

# Measure 2 ----------------------------

# Measure 3 ----------------------------

# Measure 4 ----------------------------

# Measure 5 ----------------------------

# Measure 6 ----------------------------

# Measure 7 ----------------------------

#1 Headline

#2 Headline

#3 Headline

#1 DDA

#2 DDA

#3 DDA% Measure 8 ----------------------------

% Measure 9 -----------------------------

% Measure 10 ---------------------------

% Measure 11 ---------------------------

% Measure 12 ---------------------------

% Measure 13 ---------------------------

% Measure 14 ---------------------------

# Measure 15 ----------------------------

# Measure 16 ----------------------------

# Measure 17 ----------------------------

# Measure 18 ----------------------------

# Measure 19 ----------------------------

# Measure 20 ----------------------------

# Measure 21 ----------------------------

% Measure 15 ----------------------------

% Measure 16 ----------------------------

% Measure 17 ----------------------------

% Measure 18 ----------------------------

% Measure 19 ----------------------------

% Measure 20 ----------------------------

% Measure 21 ----------------------------

Program: _______________Performance Measure

PerformanceMeasureBaseline

Story behind the baseline

Partners

Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. --------- No-cost / low-cost

One Page Turn the Curve Report: Performance

4. --------- Off the Wall

Performance Accountability

1. Who are our customers?2. How can we measure if we are delivering services

well?3. How can we measure if our customers are better off?4. How are we doing on the most important of these

measures?5. Who are the partners that have a role to play in doing

better?6. What works, what could work, to do better?7. What do we propose to do?

for programs, agencies and service systems