LESSONS LEARNED FROM OTHER NATIONS

PAUL L. POSNERGEORGE MASON UNIVERSITY

2

PERFORMANCE AND THE MANAGEMENT CYCLE

• DETERMINING GOALS AND TARGETS• PROVIDING MANAGERS WITH FLEXIBILITY TO

ACHIEVE GOALS• MEASURING AND REPORTING ON PERFORMANCE• USING PERFORMANCE DATA TO INFORM DECISIONS

• PROGRAM DESIGN• RESOURCE ALLOCATION• PERSONNEL EVALUATIONS

3

PERFORMANCE REFORMS IN OTHER NATIONS

• PLANS• TARGETS AND ASSESSMENTS• BUDGETING• COST ACCOUNTING• REPORTS AND MONITORING• AUDITING• EVALUATION• CONTRACTS AND GRANTS

4

IMPROVING PERFORMANCE

5

HISTORY OF OECD NATIONS’ INVOLVEMENT

6

OECD NATIONS USE OF MEASURES

• OUTPUTS AND OUTCOMES

7

MOST SIGNIFICANT PROBLEMS

• MEASUREMENT AND DATA CHALLENGES

8

OVERCOMING CHALLENGES

• IMPROVEMENTS OVER PAST FIVE YEARS

9

PERFORMANCE BUDGETING

• PERFORMANCE PART OF BUDGET PROCESS

Note: Includes responses from 27 out of 28 countries.

10

PERFORMANCE BUDGETING

• ROLE OF PERFORMANCE IN BUDGET

11

Decisionmaking Process

• EFFECTS OF PERFORMANCE ON DECISIONS

12

A U.S. Historical Perspective

1900……………….1930s 1940………………1950s1960……………..

Budget and Accounting Budget and Accounting GPRA 1993Act of 1921 Procedures Act of 1950

Focus: Dollars Transactions ProgramsPeople Activities OutputsAccounts Functions

Outcomes

Impact

Emphasis: Resources WorkPurpose

13

HISTORY OF U.S. REFORMS

• HOOVER COMMISSION – 1950’S

• PLANNING-PROGRAMMING-BUDGETING-- 1960’S

• MANAGEMENT BY OBJECTIVES --1970’S

• ZERO-BASED BUDGETING --1970’S

• GOVERNMENT PERFORMANCE AND RESULTS ACT --1990’S

• PROGRAM ASSESSMENT RATING TOOL --2000’S

14

GPRA: What it Is and What it’s Not

GPRA melds the best features, and avoids the worst, of its predecessors

Required connections to budget presentations…………………………… reflect the influence of BAPA.Emphasis on cross-agency comparisonsand performance measurement………. harkens back to PPBS.Focused on outcomes, but allowing arange of performance measures……… emphasizes the premise of MBO.Absence of mechanistic approaches and formulaic quantification…………… avoids major problems of PPBS and

ZBB.Explicitly defined roles for executiveand legislative branches………………. removes “executive initiative” tag

of PPBS and ZBB.Defined a phased and iterativeimplementation approach……………… not the “ready/fire/aim” method of

PPBS and ZBB.

15

IMPACT OF GPRA

• SUSTAINED FOR OVER 10 YEARS UNDER PRESIDENTS CLINTON AND BUSH

• AGENCIES DEFINE MEASURES WITH OVERSIGHT FROM PRESIDENT AND CONGRESS

• AGENCIES TAKE OWNERSHIP AND USE IN INTERNAL MANAGEMENT

• USEFUL TO THEIR OWN MANAGEMENT• GAIN COMPETITIVE ADVANTAGE IN PRESTIGE

16

IMPACT ON PERFORMANCE

• EFFECTS ON PERFORMANCE

• WEATHER SERVICE TORNADO WARNINGS

• COAST GUARD BOATING ACCIDENTS

• VETERANS’ HEALTH CARE FACILITIES

• NATIONAL PARK SERVICE

17

Performance Budgeting Challenges

Key Elements to “Successful” Performance Budgeting

• Defining expectations clearly

• Addressing structural alignment between plans, budgets and total costs

• Increasing the supply of credible outcomes, measures, and information

• Promoting demand for information used by actors with different needs

18

Defining Expectations

• Whether we deem performance budgeting a success or failure largely depends on our expectations

• There are several models for what performance budgeting might look like, including:

• Mechanical – funding levels directly tied to performance• Managerial – consensus on broadly defined goals; little

oversight• Incentives – marginal programmatic changes

reward/punish performance• Agenda – changes the decision-making process; not

necessarily the decisions themselves

19

Different Orientations

PERFORMANCE STATEMENT OF NET

BUDGET PLANNING COSTS

Agency General Goal Agency

Budget Account Strategic Objective Responsibility Segment

Program Activity Performance Goal Segment Output

Source: GAO.

20

HUD’s Fiscal Year 2001 Performance Plan     General Goals

Selected examples of accounts or program activities

BudgetRequest

Increase availability of decent, safe,

and affordable housing in American

communities

Ensure equal opportunity in housing for all

Americans

Promote self-sufficiency and asset

development of families and

individuals

Improve community, quality of life

and economic

vitality

Restore the public trust

in HUD

Public Housing Capital Fund

$2,955 $2,069 $443 $148 $295 --

Community Development Block Grants

$4,900 $1,470 $490 $980 $1,960 --

FHA: GI/SRI$456 $456 -- -- -- --

Change in HUD’s Presentation of Performance Plan-Budget Linkages, Fiscal Years 2000 and 2001

21

BUSH ADMINISTRATION MANGEMENT REFORM STRATEGY

• Centralization – management goals and assessments

• Public scorecard and approbation

• Linkage to budget

• Contesting congressional control

22

http://www.whitehouse.gov/omb/budget/fy2006/pdf/budget/effective.pdf

23

http://www.whitehouse.gov/omb/budget/fy2006/pdf/budget/effective.pdf

24

http://www.whitehouse.gov/omb/budget/fy2006/pdf/budget/effective.pdf

25http://www.whitehouse.gov/results/agenda/standards.pdf

26

PART

OMB’s Program Assessment Rating Tool (PART) has the potential to prompt demand

• Promote demand for credible performance information in the budget formulation process

• Increase the attention paid to evaluation & performance information among federal agencies & third parties

• Prompt a reexamination of the base for selected programs

• Allows for a non-incremental look at selected programs

27

Background

What is PART?• ~ 30 questions applied to 20 percent of programs annually• Most questions applied to all programs w/ subset of

questions by program type (Competitive grants, Block/formula grants, Capital assets & service acquisition, Credit, Regulatory-based, Direct federal, R&D)

• Four weighted sections: (1) program purpose/design (20%); (2) strategic planning (10%); (3) program management (20%); & (4) program results (50%)

28

PART: OMB’s Program Assessment Rating Tool

Program Purpose & Design (weight = 20 %)

Strategic Planning (weight = 10%)

Program Management (weight = 20%)

Program Results (weight = 20%)

• To assess whether the program’s purpose & design are clear & sound.

• To assess whether the agency has established valid long-term & annual measures & targets for the program.

• To rate agency management of the program, including financial oversight & program improvement efforts.

• To rate program performance on measures & targets reviewed in the strategic planning section through other evaluations

29

PART: Program Purpose & Design

• Is the program purpose clear?

• Does the program address a specific & existing problem, interest or need?

• Is the program designed so it is not redundant or duplicative of any other federal, state, local or private effort?

• Is the program design free of major flaws that would limit program’s effectiveness or efficiency?

• Is the program design effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program’s purpose directly?

30

PART: Strategic Planning

• Does the program have a limited number of specific long-term performance measures that focus on outcomes & meaningfully reflect the program purpose?

• Does the program have ambitious targets & timeframes for its long-term measures?

• Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program’s long-term goals?

• Does the program have baselines & ambitious targets for its annual measures?

• Do all partners (grantees, sub-grantees, contractors, cost-sharing partners, government partners) commit to & work toward annual &/or long-term goals?

• Are independent evaluations of sufficient scope & quality conducted on a regular basis or as needed to support program improvements & evaluate effectiveness & relevance to the problem, interest, or need?

• Are budget requests explicitly tied to accomplishment of the annual & long-term performance goals, & are the resource needs presented in a complete & transparent manner in the program’s budget?

• Has the program taken meaningful steps to address strategic planning deficiencies?

31

PART: Program Management

• Does the agency regularly collect timely & credible performance information, including information from key program partners, & use it to manage the program & improve performance?

• Are Federal managers & program partners (including grantees, sub-grantees, contractors, cost-sharing partners, government partners) held accountable for cost, schedule & performance results?

• Are funds (Federal & partners’) obligated in a timely manner & spent for the intended purpose?

• Does the program have procedures (i.e., competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure & achieve efficiencies & cost effectiveness in program execution?

• Does the program collaborate & coordinate effectively with related programs?

• Does the program use strong financial management practices?

• Has the program taken meaningful steps to address its management deficiencies?

32

PART: Program Results

• Has the program demonstrated adequate progress in achieving its long-term performance goal(s)?

• Does the program (including program partners) achieve its annual performance goals?

• Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

• Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose & goals?

• Do independent evaluations of sufficient scope & quality indicate that the program is effective & achieving results?

33

PART : Program Type

Competitive Grants

Block/Formula Grants

Regulatory-Based

Capital Assets & Service Acquisition

Credit

Direct Federal

R&D

• Programs that distribute funds to state, local & tribal governments, organizations, individuals & other entities through a competitive process.

• Programs that distribute funds to state, local & tribal governments & other entities by formula or block grant.

• Programs that employ regulatory action to achieve program & agency goals through rulemaking that implements, interprets or prescribes law or policy, or describe procedures or practice requirements.

• Programs where the primary means to achieve goals is the development & acquisition of capital assets or the purchase of services from a commercial source.

• Programs that provide support through loans, loan guarantees & direct credit.

• Programs in which support & services are provided primarily by federal employees.

• Programs that focus on creating knowledge or applying it toward the creation of systems, devices, methods, materials or technologies.

34

PART Scores Results

• For FY 2004 and 2005• 11% rated “Effective”• 26% rated “Moderately Effective”• 21% rated “Adequate”• 5% rated “Ineffective”• 37% rated “Results Not Demonstrated”

• Inadequate performance goals/measures or data

• Of the 7 program types, R&D programs had the highest median scores (63.5 out of 100) & Block/Formula Grant programs the lowest (54.7) for FY 04

Source: GAO analysis of OMB data.

35

Findings of GAO PART Review

36

PUBLIC TRANSPARENCY

• Reporting Principles• Provide a coherent and balanced picture of performance

that is brief and to the point. • Focus on outcomes, not outputs. • Associate performance with earlier commitments, and

explain any changes. • Set performance in context. • Link resources to outcomes. • Explain why the public can have confidence in the

methodology and data used to substantiate performance

37

Potential Barriers

• Unrealistic expectations • Multiple and unclear objectives • Defining measures of performance • Balancing measures • Data systems and information• Excessive measures and complexity • Cultural change – • Organizational and technical capacity

38

Strategies for Sustainability

• A longer term perspective • A phased in implementation• Consistent support by leadership • Support of necessary expertise • Auditing and validation of data• Careful integration with budgeting and personnel • Development of “nonpunitive” incentives • Training and capacity building • Public transparency• Supplementing and not supplanting other controls• Selective application

39

• Constitutional Backdrop• Political Trends• Oversight Tools and Strategies• Congressional Roles and Venues• Congressional Support Agencies: The General Accounting

Office• Congress and Management Reform• Challenges to Oversight

OVERSIGHT: AN IMPETUS FOR REFORM

40

Congressional Oversight Tools and Strategies

• Hearings• Investigations• Support agencies audits/evaluations• Authorizations and reauthorizations• Approval of nominations• Transaction approval/veto• Budget and appropriations• Organization of executive agencies• Executive management reforms• Reports

41

Congressional Support Agencies

• The Government Accountability Office• Congressional Research Service• Congressional Budget Office• Inspectors General

42

Roles of the GAO

• A support agency for the Congress• Reports and testimonies addressed to congressional

members and committees• Institutional knowledge

• Oversight of executive agencies• Recommendations to agency heads• Rulings on procurements and spending

• Public reporting promotes transparency and accountability

43

GAO Roles: Oversight, Insight, Foresight

• Oversight – determine that government agencies are complying with laws and preventing waste, fraud and abuse

• The “High Risk” series identifies most vulnerable activities and programs such as

• Medicare• Loans• Tax administration

44

Role for GAO

• Insight – improving the economy, efficiency and effectiveness of programs

• Examples include• Do federal food safety programs adequately protect the

public?• Are voting places accessible to the disabled?• How well are military academies educating and training

young men and women? www.gao.gov/cgi-bin/getrpt?GAO-03-1001

45

Supporting Management Reform

• Financial Management• Information systems• Performance Management• Human Capital• Acquisition Management