THE ROLE OF LEADERSHIP IN PERFORMANCE MANAGEMENT

Donald P. Moynihan, La Follette School of Public Affairs, University of Wisconsin-MadisonPresentation to Chicago Federal Leadership Forum

Have you encountered?

Strategic planning Performance measures Performance contracts Pay for performance

The role of leadership

• During my 20 years in the private sector as a CEO and advisor to CEOs, I found that leadership, measurement, and a motivated workforce create the foundation for good performance. I am confident that the same is true in government

Jeff Zients – Chief Performance Officer, 2009

Outline

Defining terms Era of governance by performance

management From Bush to Obama How do we use performance systems? What fosters use of performance data? Summary points

Defining terms

Performance management

A system that generates performance information through strategic planning and performance measurement routines, and connects this information to decision venues,

Performance regimes

Performance tools create unprecedented pressure on public actors to perform, in a context where performance is defined by quantitative indicators

Purposes of Performance Information Promote: How can I convince political

actors, stakeholders and the public that my agency is doing a good job?

Celebrate: What accomplishments are worthy of the important ritual of celebrating success?

Learn: What is what working or not working?

Improve: What exactly should who do differently to improve performance?

Purposes of Performance Information Evaluate: how well is my agency

performing? Control: how can I ensure that my

subordinates are doing the right thing? Budget: on what program’s, people, or

projects should my agency spend the public’s money?

Motivate: how can I motivate employees and collaborators to improve performance?

ERA of governance by performance management

Era of Governance by Performance Management

The rise of a doctrine Not new, but more influential than

before Must justify actions in terms of outputs

and outcomes Basis for holding new structural forms

accountable

Doctrinal logic for change

Government Performance and Results Act 1993 Mandated:

5 year strategic plans, updated every 3 years

Specific goals and objectives Annual performance reviews and plans

From Bush to Obama

Bush approach

Presidents Management Agenda“everyone agrees that scarce federal resources should be allocated to programs that deliver results”

Wanted to integrate performance data into budget process

Congressional Justifications

Center around performance goals Pushback from Appropriations Committees

Veteran’s Administration told; “to refrain from incorporating ‘performance-based’ budget documents”; later told: “If the Department wishes to continue the wasteful practice of submitting a budget structure that will not serve the needs of the Congress, the Congress has little choice but to reject that structure and continue providing appropriations that serve its purposes.”

Two budgets required

Congressional Justifications

Department of Transportation told: “agencies are directed to refrain from including substantial amounts of performance data within the budget justifications themselves, and to instead revert to the traditional funding information previously provided. Performance-related information may be submitted under separate cover.”

Negative consequences were promised for agencies that ignored this directive: “If the Office of Management and Budget or individual agencies do not heed the Committee’s direction, the Committee will assume that individual budget offices have excess resources that can be applied to other, more critical missions.”

Program Assessment Rating Tool (PART)

5 year summary by OMB of evidence on program performance for 1016 programs 18 percent are Effective 31 percent are Moderately Effective 29 percent are Adequate 3 percent are Ineffective 19 percent are Results Not Demonstrated

PART as Evidence-based Dialogue Third-party program review with a clear opinion Greater emphasis on performance The standard of proof for program performance

can only be satisfied by positive evidence of results

The burden of proof for performance rests on agencies

Entire programs are evaluated on a regular basis

The routine nature of PART creates an incentive to engage

Obama: A Pragmatic approach “The question we ask today is not whether our

government is too big or too small, but whether it works -- whether it helps families find jobs at a decent wage, care they can afford, a retirement that is dignified. Where the answer is yes, we intend to move forward. Where the answer is no, programs will end. And those of us who manage the public's dollars will be held to account, to spend wisely, reform bad habits, and do our business in the light of day, because only then can we restore the vital trust between a people and their government”

Example: Pedometer challenge!

Voluntary Belief that transparent performance

numbers will change behavior, create a sense of competition and raise performance

Early evidence on Obama

Performance measurement will be important “The President is creating a focused team

within the White House that will work with agency leaders and the OMB to improve the results and outcomes for Federal Government programs while eliminating waste and inefficiency”

Chief performance officer Continue to maintain agency level performance

positions

What happens to PART?

Not clear Criticized as ideological, as too broad, as

a data collection exercise Analysis remains in place, but new

PARTs have not started OMB have offered agencies funds for

better evaluations

New emphasis on leadership Focusing leaders on what matters – key

goals Accelerating results – Performance

Improvement Council; data driven meetings

Style: focused collaboration

New focus on information use Will be a central aspect of the Obama

administration’s performance initiatives Jeff Zients: “The ultimate test of our

performance management efforts is whether or not the information is used”

Shelly Metzenbaum: “the key performance management challenge facing the Obama administration is to use—not just produce—performance goals and measures”

How do we use performance systems?

Why care about use?

For reforms to succeed, implies that data is used

Provides a tractable means of studying the impact of results-based reform

Public organizations have devoted significant time and resources into creating routines to collect and disseminate data

Almost no attention to creating routines of use How do you use performance data?

Types of responses: 4 Ps

Passive Perverse Political Purposeful

Passive use of data

Passive: Do the minimum to comply with

requirements Do not actually use data Correlated with cynicism about reforms

Perverse use of data

Effort Substitution: Reducing effort on non-measured dimensions

Cherry picking/Cream-skimming: Focusing effort on subgroups of clients most likely to provide greatest impact on performance measures while effectively denying services to others.

Measure selection: Selecting metrics or data to measure that will offer the most favorable portrayal of a service

Hiding numbers: Declining to present performance measures that may exist

Perverse use of data

Output distortion: Manipulating measurement processes to improve measured performance.

Ratchet effects: Curbing productivity in one time period to avoid the setting of more challenging targets in another.

Churning: Frequently adopting different targets or measures to prevent comparison across time.

Cheating: Simply making up numbers, though rare, does occur.

Responding to perversity

Add new/additional measures

Change existing measures

Rely/cultivate intrinsic norms to limit misbehavior

Avoid high-powered incentives

Political uses of data

Process of selecting measures means shaping a program narrative

“Understand that measuring policy is not a science. It is an art. It is words, and pictures and numbers. And you create impressions, beliefs, understandings and persuasions.”

Political uses of data

Data tells us what happened Program officials still need to interpret and

explain: why performance did or did not occur; the context of performance; how implementation occurred; an understanding of outside influences on

performance; and how to choose which program measure is a

priority. Exploit ambiguity and subjectivity of data

Political: Ambiguity of data

Examine same programs, but disagree on data

Agree on data, but disagree on meaning Agree on meaning, but not on next

action steps/resources

Political: Subjectivity of data

Actors will select and interpret performance information consistent with institutional values and purposes

Evidence of Ambiguity in PART Ambiguity of terms:

E.g.: Program purpose, quality evaluation, ambitious, having made progress

How to interpret results? Multiple logics from experiment: Argue that ratings are unreliable Cut poorly managed programs Raise funding for programs with positive assessments Parity: Raise funding because program with similiar

assessment received more Delay cuts because progress being made Clear relationship between resources, need and program

delivery Stakeholder and congressional views

Evidence of Subjectivity with PART OMB using PART to expand influence in

performance management/policy OMB can define programs, goals,

measures, agency responsibility Disagreement with agencies/Congress

on meaning/relevance of PART Experimental evidence:

UW students significantly more likely to disagree with OMB, and to argue for higher assessments and resources

Implications for Decisionmaking Performance information use reflects

political process, does not replace it Performance information use does not

lead to clarity Ability to structure dialogue tied to

power

Purposeful use of data

Use data to improve program performance

Goal-based learning efficiency improvements better targeting of resources more informed strategic decisions, tying indicators to rewards/sanctions in

contract arrangements

Purposeful use of data

Use of performance information for problem-solving more likely to occur in intra-institutional settings Reduces competing interpretations

Problem of neglect rarely do anything with information

Learning forums

Routines specifically focused on solution-seeking, where actors collectively examine information, consider its significance and decide how it will affect future action

What measures are useful for agency officials?

What other ways can we encourage learning forums?

What fosters performance information use?

The Right Context

Simple function that is easy to measure Clear link between measures of actions,

and measures of outcomes One-dimensional – relatively few

measures that do not conflict with one another

Stakeholder support – clear agreement about purpose

Other factors

Learning forums Mission-based culture/supportive culture Resources Administrative stability Administrative capacity

Quantitative approach

3 studies using survey-based data Self-reported performance information

use Results from Moynihan and Pandey (in

press) and Moynihan, Wright and Pandey (2009; 2010)

Variable Hypothesized direction

Result

Individual beliefsPublic service motivation Positive ***

Job attributesReward expectation Positive --Generalist leader Negative ***Task-specific experience Positive --

Organizational factorsInformation availability Positive ***Developmental culture Positive ***Flexibility Positive *Budget staff take adversarial role Positive/negative --

External factorsCitizen participation Positive/negative +Professional influence Positive +*** = significant at .001; ** = .01; * = .05 +=.10 (two tailed tests)Controls: region, income per capita, government size, population size, population homogeniety

Study 1: Ordinal regression of reported performance information use for decisions

Intrinsic vs. extrinsic motivation

Sense of public service motivation mattered

Possibility of extrinsic reward did not create an incentive to use data

Implication: performance information use as extra role behavior

Organizational factors

Information availability Supply-side approach Use increases with better information, and

when information is tied to management systems

Organizational factors

Demand side approach Culture matters

Previous work focuses on whether culture welcomed performance management reforms

What about broader measures of culture? Developmental culture (adaptability, readiness,

growth) Flexibility – unlikely to use data if cannot

apply insights

Specialist vs. generalist leaders Task-specific knowledge provides

context in which to interpret and apply data

Leadership role Task-specific leaders more likely to use

data than generalist leaders

Other evidence of leadership Support/commitment Provision of resources Participation

What other ways can leadership matter?

Study 2: Transformational leadership Approach to leadership consistent with

performance: Articulate an appealing vision of the

organization’s mission and future Model behavior consistent with vision,

inspiring role model Challenge old assumptions

Propositions

Transformational leadership behaviors will have an indirect, positive effect on performance information use through its influence on goal clarity

Transformational leadership behaviors will have an indirect, positive effect on performance information use through its influence on organizational culture.

Key measures

Transformational leadership Asked department heads/assistant city

managers on extent to which city manager demonstrates transformational leadership:

articulates his/her vision of the future. leads by setting a good example challenges me to think about old problems in

new ways says things that make employees proud to be

part of the organization. as a clear sense of where our organization

should be in five years.• Aggregated responses by organization

Structural Equation Model

0.32*

0.60*

0.63

0.17*

0.52

0.26*

Performance

Use

EE

0.13*

Goal

Clarity

Developmental

Culture

TransformationalLeadership

All paths reported as standardized coefficient

*p < 0.05

0.66*

0.33*Information

Performance

AvailabilityInformation

0.57

E

0.89

E

Implications

Leadership and management Indirect effects are important “Setting the table” as long-term

leadership strategy

Study 3: Perceived social impact Individuals who see their work as helping

others more likely to use performance information

Some evidence that individuals who perceive greater social impact are more motivated

Why should it relate to performance information use?

Key measures

Perceived social impact I feel that my work makes a positive

difference in other people’s lives. I am very aware of the ways in which my

work is benefiting others. I am very conscious of the positive impact

my work has on others. I have a positive impact on others in my

work on a regular basis

Purposeful and political use

Purposeful I regularly use performance information to make decisions. I use performance information to think of new approaches

for doing old things. I use performance information to set priorities. I use performance information to identify problems that

need attention. Political I use performance information to communicate program

successes to stakeholders. I use performance information to advocate for resources to

support program needs. I use performance information to explain the value of the

program to the public.

Table 2: OLS Regression of Performance Information Use

Independent variables Purposeful Use Political

Perceived social impact .43 (.12)*** .52 (.11)***

Goal clarity .19 (.10)* .25 (.09)**

Centralization .05 (.07) .10 (.08)

Political support .14 (.07)* .06 (.07)

Red tape .04 (.03) .02 (.02)

Years in position .01 (.007) .006 (.008)

Supervisory level .03 (.05) .02 (.04)

Educational level .05 (.07) .09 (.06)

Sector -.06 (.15) .08 (.14)

*** = .001;**= .01; *=.05 one-tailed test N=184; R2= .25 N= 186; R2= .31

Study 4: Experimental approach How does performance information matter

to decisions? How does the framing of performance

information affect decisions? Respondents given surveys with scenario –

make budget recommendations Series of vignettes for different programs Half vignettes are control, half are treatment

Theoretical background

Research on decision frames from psychology and behavioral economics

Performance information is strategically selected and presented – does this work?

Does the Addition of Performance Data Matter?

Control: no data; treatment: addition of data without clear correlation to resources

The Department of Land and Water Resources is responsible for monitoring and maintaining the water quality of lakes in the county, including two major lakes that are popular for swimming and other water sports during the summer. Estimates of water quality are based on pH levels, pesticides, nitrates and other chemicals in the water.

2007 2008 2009

Program funding 174,000 179,000 182,000

Number of days water quality of major lakes deemed unsafe

12 14 11

Does the Addition of Performance Data Matter?

Control: no data; treatment: addition of data with clear relationship to resources

The Department of Social Services delivers a program called the Home Downpayment Initiative. Using a mix of federal, state, and local resources, the program seeks to increase the homeownership rates among low-income and minority initiatives. To do so, it provides financial assistance to first-time homebuyers for downpayment and closing costs.

2007 2008 2009

Home Downpayment funding 723,000 747,000 769,000

Number of families purchasing homes 36 38 46

Is outcome data more powerful than output?

Control: output data; treatment: outcome data The Department of Health Services offers a program called

Health Check, which is a preventive health check-up program made available for anyone under the age of 21 who is currently enrolled in Medicaid. Health Check provides a head-to-toe medical exam, immunizations, eye exam, lab tests, growth and development check, hearing check, nutrition check, and teen pregnancy services. The goal of the program is to prevent the incidence of more serious and more expensive health situations. 2007 2008 2009

Health Check funding 232,000 244,000 269,000

Clients treated 1232 1401 1325

Estimated savings due to preventive care 383,000 402,000 389,000

Threshold effects

Treatment: performance data pass a memorable threshold (200)The County Tourism Board seeks to increase visits from those who live outside the county, and to increase the use of recreational and cultural opportunities by both locals and outsiders. It collects data from local hotels, restaurants, and other businesses that depend on tourists. In the last number of years, the number of tourists visiting the county has stayed relatively flat at about 100,000, and the Board has focused its marketing budget on “quality, not quantity,” by increasing the dollar amount that each tourist spends.

2007 2008 2009

Program budget 120,898 124,490 131,289Average daily dollar amount spent by tourists 178 184 195

2007 2008 2009Program budget 124,294 127,196 135,329Average daily dollar amount spent by tourists 183 188 201

Including equity measures

Treatment: addition of equity measure that aligns with mission

The Department of Social Services funds the Early Intervention Program, which provides services for children three and under with developmental delays and disabilities. The mission statement for the Early Intervention Program is: “Our mission is to provide access to therapies that improve child developmental outcomes.” The program is administered by a non-profit, and employs therapists to work with children and families in the home environment. 2007 2008 2009

Program budget329,677 333,451 341,386

Number of hours of contact per year8901 8977 9011

Number of children served212 233 221

Percent of eligible children below the poverty line that access services 42% 47% 51%

Summary points: what to do

Move beyond passive and limit perverse use

Focus on political use What is the narrative of your program? What goals are meaningful and telling?

How do they relate to the narrative? What goals are essential to explaining

program purpose and achievement? How do you frame and communicate

measures? Who is your audience?

Summary points: what to do

Focus on purposeful use Provide resources, be involved, make clear

that it is important Encourage right context for use

Foster goal clarity Encourage supportive culture

Create and support learning forums Appeal to intrinsic motivation

Focus on demonstrating significance of measures

Questions/Comments

dmoynihan@lafollette.wisc.edu

http://www.lafollette.wisc.edu/facultystaff/moynihan-donald.html

The Dynamics of Performance Management Georgetown University Press