Has the Government Performance and Results Act Improved the Availability and

Use of Performance Information?

Jerry ElligSenior Research Fellow

Government Performance and Results Act of 1993 One motivation: “Federal managers are seriously

disadvantaged in their efforts to improve program efficiency and effectiveness, because of insufficient articulation of program goals and inadequate information on program performance.”

Requirements:

Strategic plans outlining goals (including outcomes) and planned program evaluations

Annual performance plans establishing quantitative goals Annual performance reports beginning fiscal year 1999

Possible research questions

Has GPRA increased the availability and use of performance information in agencies?

Has GPRA increased the use of performance information by policymakers?

Has increased use of performance information affected government’s efficiency, effectiveness, scope, or size?

Have GPRA-induced changes in government affected human welfare?

Why might GPRA improve availability or use of information? Tullock, Downs theories of bureaucracy:

Individuals advance by doing what their superiors want.

How do agency officials know policymakers want performance management?

GPRA is clearly the law Presidents have shown consistent interest Congress has shown less consistent interest

Why might GPRA fail?

GPRA skeptics Analytical and data difficulties Outcomes harder to measure for some kinds of programs

(eg, grants, R&D) US political system hampers agreement on outcome goals

Agency theory Align agency incentives with performance budgeting Align managers’ incentives with performance contracts Give mangers greater discretion over use of resources Reform civil service

The empirical analysis

Dependent variable: Periodic Government Accountability Office (GAO) surveys ask federal managers in 24 agencies if they have various kinds of performance information and use it for various purposes

Independent variable: Annual Mercatus Center Scorecard measures quality of the same 24 agency GPRA performance reports

Are these 2 things correlated?

GAO surveys

Asks federal managers if they have various kinds of performance measures for their programs or projects

Asks federal managers if they use performance information for various specified purposes in their programs or projects

2000 and 2007 had enough responses to calculate valid percentages for each agency

Use % of managers responding “to a great extent” or “to a very great extent”

Average of agency responses

Figure 1: Average % of managers with various types of performance measures

0 10 20 30 40 50 60

Output

Efficiency

Customer satisfaction

Quality

Outcome

2000

2007

Average of agency responses

Figure 2: Average % of managers who use performance information for these purposes

0 10 20 30 40 50 60

Setting priorities

Allocating resources

Adopting new approaches

Coordinating with external orgs.

Refining program measures

Setting performance goals

Setting job expectations

Rewarding employees

Managing contracts

2000

2007

The Mercatus Scorecard 1-5 rating scale

3 Categories

TransparencyPublic BenefitsLeadership

4 criteria in each category

Criteria tightened each year to reflect previous year’s best practices

Total score can range from 12 to 60

5 Sets a standard for best practice

4 Shows innovation and creativity

3 Satisfactory

2 Partially complete

1 Fails to meet expectations

Figure 3: Mercatus Scorecard scores

0

10

20

30

40

50

60

2000

2007

2 econometric specifications

Pooled 2000 and 2007 data 46 observations Control variables

Year, score*year Leadership commitment (survey response) % of budget spent on competitive grants, block grants,

R&D, regulation

2007-2000 differences 22 observations Leadership control variable sometimes significant

GPRA linked to availability of information(results significant at 90% level or higher) A one point change in an agency’s Scorecard score is

associated with between one-third and one-half of a percentage point change in the number of managers stating that they have measures of Outcomes Outputs Efficiency

An agency producing a GPRA report with an average Scorecard score of 34 would have about 10 percent more managers reporting that they have outcome, output, or efficiency measures.

40-60 percent of managers in various agencies said they had outcome, output, or efficiency measures.

GPRA and use of performance information(results significant at 90% level or greater)

Score of agency GPRA reports is positively correlated with five different uses of performance information:

allocating resources setting priorities coordinating with external agencies establishing measures setting goals

How big is Scorecard score’s effect on uses of performance information? Coefficients range between 0.3 and 0.6.

An agency producing a GPRA report with an average Scorecard score of 34 would have 10-20 percent more managers reporting that they use performance information for these purposes.

25-50 percent of managers said they use performance information for various purposes.

Influence on use of information may be wider Scorecard score may affect other uses of

information

Existence of outcome measures is correlated with Scorecard scores

Every use of performance information is correlated with existence of outcome measures

Scorecard scores may have an indirect effect on every use of performance information

All of these regressions only measure effects that are attributable to differences in quality of agencies’ GPRA initiatives

Other interesting results

Leadership control variable often has a big positive influence

More competitive grants => more managers with outcome measures

More block grants => lower percentages of managers claiming they have efficiency measures

More regulation => lower percentages of managers that say they have outcome measures or use performance information to allocate resources

More R&D => lower percentages of managers reporting they have output measures, efficiency measures, or use performance information for many purposes

What can we conclude?

GPRA had an effect: Quality of agencies’ GPRA initiatives has an effect on availability and use of performance information

Tullock was right: Passage of GPRA, executive branch guidance, and commitment of agency leadership motivated improvement without wholesale revision to bureaucratic incentive structure

Skeptics were right about some things: Some kinds of performance measurement and use of info seems less likely in agencies utilizing more block grants, regulation, or R&D programs