Performance 2.0: From Measurement to Management

Jon DesenbergPolicy Director

The Performance Institute 800 15th Street NW

Third FloorWashington DC 20005Phone: 202-739-9642

www.performanceweb.org Jon.Desenberg@PerformanceInstitute.org

Why measure?

To Plan?

To Comply?

To Manage?

To Optimize?

To Innovate?

What gets measured gets done.

–Peter Drucker

Long-Term Move Towards Performance

Performance management is not a new phenomenon—50 years of work in the making to link resources with results

– Budget Accounting Procedures Act (BAPA) of 1950

– Planning-Programming-Budgeting System (PPBS), 1965-1971

– Management by Objectives (MBO) 1973-1974– Zero-Based Budgeting (ZBB), 1977-1981

Finally a Move in the Right Direction

• GPRA—(1993) Government Performance Results Act

• PMA—(2001) President’s Management Agenda

• PART—(2002) Program Assessment Rating Tool

• Mandates focusing on performance and accountability—bottom line results

More Timely Reporting for Better Real-Time Decisions

OMB has eliminated monthly reporting requirements for the foreseeable future...given the continued demand for timely reporting.

The reporting frequency and detail may change slightly over time, reflecting the information needs of the administration and the public, Agencies should begin exploring the accounting and process changes required to update obligations and gross outlays more frequently than a weekly report.

Transparency = Performance Management:

• Recovery Act Funds are awarded and distributed in a prompt, fair, and reasonable manner;

• The recipients and uses of all funds are transparent to the public, and the public benefits of these funds are reported clearly, accurately, and in a timely manner;

• Funds are used for authorized purposes and potential for fraud, waste, error, and abuse are mitigated;

• Projects funded under this Act avoid unnecessary delays and cost overruns; and

• Program goals are achieved, including specific program outcomes and improved results on broader economic indicators.

Federal Performance : New Developments

• A re-focused Process in development

• The Federal Chief Performance Officer

• Weekly Recovery Act Reporting

• Data.gov

Understanding Current Performance Implementation

• Four GAO Studies – 1994-2007

– 1994: 19% Outcome Measures

– 2007: 50% Outcome Measures

• PMA and PART: Checklist or Results?

– “Costly and Burdensome”

Effective

Ineffective

Results NotDemonstrated

Performance Management:Have We Forgotten “Why?”

• Do the same problems still need our attention? And what are their causes?

• Is the strategy effective? – Can it be replicated?– Can we speed adoption?

• Can we improve real communication?– To the White House, Congress and

(even) appropriators– State, Local, Non-profit and other

partners

“We have run out of money, now we must think” – Winston Churchill - 1943

Peak Deficits (% GDP) Predicted Soon

-4-202468

10121416

USA France Germany

2008Peak

UK Japan RussiaThe Economist June 2009

Performance / Productivity Gains Increasingly Seen as Critical

$0

$20

$40

$60

$80

$100

$120

$140

USA Japan UK France

5%15%

OECD.org –May 2009

Understanding Logic Models

What is a logic model?1. Logical chain of events providing blueprint for mission

achievement2. Graphic representation that illustrates the rationale

behind a program or organization3. Depicts causal relationships between activities,

strategies, and end results4. Contains goals and performance measures 5. Integrates various program activities into a cohesive

whole6. Vehicle for dialogue, planning, program management

and evaluation

What does a logic model look like? • Graphic display of boxes and arrows;

vertical or horizontal

– Relationships, linkages

• Any shape

– Circular, dynamic

– Cultural adaptations, storyboards

• Level of detail

– Simple

– Complex

• Multiple models

Logic modeling is based on mapping and defining linkages between what we do and why we do it .

I Work Out for One Hour

Each Day

I Will Burn More

Calories Than I

Consume

Lose Fat and Build

Muscle

Improve My Looks and Health

Have Better Image,

Feel Better & Live Longer

IF THEN IF THEN IF THEN IF THEN

INPUTS OUTPUTS OUTCOMES

Series of If-Then Relationships

Assumptions: improving looks = better self image

Factors: Health History

InputsPeople and resources required to achieve outcomes

Activities/OutputsWhat the inputs produce

Immediate and Intermediate OutcomesChanges required to achieve end outcome

End Outcome

End goal or ultimate benefit

Clarifying the terms

Assumptions: beliefs or evidence that supports your IF-THEN logic

Factors: external influences beyond control that effect IF-THEN relationships

Logic Model “V”

Bottom-Line Investment

Top-Line Return

Alignment

Linkage

Mea

sure

men

t

Logic Model “V” Performance Dimensions

Distribute program grants

Distribute program grants

Child violence and abuse can be prevented and

detected

Child violence and abuse can be prevented and

detected

Child health and development can be

protected and maintained

Child health and development can be

protected and maintained

Children can grow into productive

citizens and attain their intended

impacts on society

Children can grow into productive

citizens and attain their intended

impacts on society

“So That”

“So That”

“So That”

Degree of Influence by Department

Ultimate Program IntentOutput

Intermediate Outcome

Intermediate Outcome

High-Level Outcome

Value Chain Diagram

Early Outcomes Later OutcomesLater ActivitiesEarly Activities

If we do…

Outreach

Screening

ID of elevated kids

And we do…

Case mgmt of EBLL kids

Refer EBLL kids for medical treatment

Train family in in-home detection techniques

Assess environment of EBLL child

Refer environment for clean-up

Then….

EBLL kids get medical treatment

Family performs in-home techniques

Lead source identified

Environment gets cleaned up

Lead source removed

And then…

EBLL reduced

Developmental slide stopped

Quality of life improves

Global Logic Model: Childhood Lead Poisoning Program

Definition:

EBLL – Elevated Blood Lead

LevelsModule Three: Using Logic Models

Inputs Activities OutputsEnd

Outcomes

Most logic models incorporate the following elements.

Intermediate Outcomes

Attitudes Behaviors Conditions

WHY?

HOW

EFFECT

CONTROL

Performance 2.0 Abroad

• Desired productivity gains translated into specific input and output targets for every agency

Finland

New Zealand

• National outcome dashboard for Hospital Waiting Time, Crime, Test Scores, other.

U.K.

• Deliberate Move Away from Short Term Output Measures towards fewer, more “nuanced” outcomes

The British Model

• UK Center for the Measurement of Government Activity– Output=Input Formula eliminated in 1993– Quality adjustment advancements on outputs

• Through academic and peer review

• Health & Education

– Management SWOT Teams– New Calls for True Independent Center

Important Next Steps

• Defining “Value” – What Do People really want?

• What are the relative preferences for outcomes?

• Match with metrics used in regulatory process

Resources and Opportunities

Capability

Service Offerings

Functioning

Utility

Draft Framework: Outcomes for Adult Care

Connecting Congress with Program Results:

1. CompetitionTransparency and Pressure to Perform

Cross-Government Results

Purchase Agreements for Appropriators

2. InnovationOMB Examiners: “Beyond Triage Mode”

3. Dispersion

4. Enhanced Productivity

The Guidestar Model

• Independent Access to Metrics and Budget Information

• GAO / CBO both reluctant to play this role

• OMB not deemed objective

• Meet Growing Demand

Taking Performance to the Next Level

• Incremental Improvements: No Longer Acceptable?

• Senate Bill 3521 – Is the One Year Budget Cycle Preventing Performance?

• Should Front Line Managers keep their Savings?

• Breaking Out of the “Performance Office” to:– Understand the issue’s

“drivers”

– Discover why performance levels vary

– Find different interventions for different circumstances

– Enhance Knowledge Sharing across programs

Discussion & Diagnostics

Connecting Both Ends of Pennsylvania Avenue

• To Monitor and Control the White House?

• Ensuring Each District’s Interests are Secured?

• To Regularly Evaluate & Re-authorize?

• To Sunset Programs?• Naming and Shaming?• To Begin Using More Data?

Key Members of CongressSenator Claire McCaskill

Former State Auditor

New Subcommittee Chair

Senator Tom Coburn “Waste of the Day” “Dr. No” / USA Spending.gov

Cong. Dennis Moore

Blue Dog Policy Chair

PAYGO Emphasis

Budget Committee

Senator John Cornyn

Transparency and Oversight emphasis

Why is Congress Unhappy with Current Reporting?

“This site is not providing data that the public can understand”

H.R.2142 - Government Efficiency, Effectiveness, and Performance Improvement

Act• Each agency head should consult with the congressional

committees with jurisdiction over the agency each fiscal year regarding the agency performance plan.

• Each agency head, in collaboration with OMB, to conduct an assessment of each agency program at least once every five fiscal years:– assessing the program’s purpose, – strategic plan and objectives, – organizational design, – management, – efficiency, and – effectiveness in achieving its performance objectives and identifying

strengths and weaknesses and factors affecting program success.

(1) make available, by May 1 of each year, a list of programs to be assessed and the criteria and methodology that will be used;

(2) submit in a report to Congress, at the same time the President submits the annual federal budget, assessment results, including an identification of program best practices for allocating resources and recommendations for modifications focusing on opportunities for consolidation and integration of programs and authorities; and

(3) develop an improvement plan, which the agency head shall implement, that responds to the assessment report and identifies follow-up actions to improve program performance.

Rep. Henry Cuellar

HR 2412 Would Require OMB To:

Views from The Hill

• “The Information is buried in thousands of p ages that we end up throwing out.”

• “I pick up the phone and call GAO, PART is political.”

• “They act as if we appropriate by goals. We want information by program area.”

• The Bush Administration redesigned the budget process so that we couldn’t find

core information.”

Does Anyone Want This Job?

How Do We Evaluate Performance

Management?• Past decisions are viewed as

rational if the outcome was good.

• If performance management is always ex post and subjective, honest evaluation of intelligence becomes difficult.

The Knowledge – Data Gap

1. Lessons are learned inadequately,

2. Memories are recalled incompletely,

3. The futures is predicted inaccurately,

4. Data rarely overcomes

preconceived notions.

When The Data isn’t enough

Possible Solutions:

• Rich histories

• Multiple observers and multiple interpretations

• Hypothetical histories

A Data Driven Culture Embraces Experimentation

Think of:

• Trial programs, pilot studies, and small experiments, and about the inferences that can be drawn from them.

• Does the Government embrace experimentation?

How to Get Fired at Harrah’s

“Institute a program without first running a data driven experiment.”-Gary Loveman

A Learning Organization

• An average of 300 data “experiments” per day

• Increased Retention by 87%

• Lowered the cost of a new account by 83%

• Stock price outpaced

S & P by a factor of ten

TestingLearning

Acting

Analytical Expertise:How do we get there?

0%10%20%30%40%50%60%70%80%90%

100%

1997 1999 2005 2008

371 Large and Medium Sized CompaniesJeanne G. Harris Institute

Measurement Maturity Index

• In order to have a good idea of your organization’s measurement progress the index will give a score up to a maximum of 300 in 4 key areas.

• Continue to assess your organizations over time to understand their progress and help them understand where to focus.

Four Maturity Themes

1. Interaction Discussion, Feedback, Pilots, Learning, Actionable

2. Integration Cross-Functional Understanding, Casual Relationships Explored

Four Maturity Themes

3. Focus Results-focus, Critical and Strategic The routine measures are eliminated for the high-

leverage

4. Context Widely Used, Valued, Trusted No Fear, Positive and Motivating

Level One

Basic Performance Management

Systemic Negative/

Neutral

Context

Programmatic

Specialized

Ad Hoc

Level Two

Transformational Performance Management

Transforming Positive Context

Collaborative

Level Three

300 Score

Measurement Maturity Index

Re-emphasizing theLogic Model “V”

Bottom-Line Investment

Top-Line Return

Alignment

Linkage

Mea

sure

men

t

Exit Opportunity

Proven Success

ResourcesAvailable

IMPORTANCE

PERFORMANCE

4

3

2

1

1 2 3 4

a

Factors

a = I4, P2

b = I3, P3

c = I2, P1

d = I1, P4

b

c

d

Attention Needed

Selecting Your MeasuresThe Program Performance Assessment Window™