Upload
alexander-pring
View
218
Download
1
Tags:
Embed Size (px)
Citation preview
1
Budgeting for Performance in the U.S.Using the
Program Assessment Rating Tool
J. Kevin Carroll
U.S. Office of Management and BudgetJuly 2008
2
Agenda
• Overview of U.S. R&D spending
• Budget Players and Process in the U.S.
• Why PART Was Created
• PART Basics
• How OMB Uses PART Results
• Supplemental Information
3
R&D as a Share of Discretionary SpendingIt’s approximately constant over the last 30 years!
0%
5%
10%
15%
20%
25%
30%
1962 1967 1972 1977 1982 1987 1992 1997 2002
R&D/ Discretionary, Civilian Civilian R&D share, excluding ApolloR&D/ Discretionary, Total Total R&D share, excluding Apollo
4
5
0
10
20
30
40
50
60
70
80
90
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
EastWestNorth
6
Agenda
• Overview of U.S. R&D spending
• Budget Players and Process in the U.S.
• Why PART Was Created
• PART Basics
• How OMB Uses PART Results
• Supplemental Information
7
The Administration
• 15 Cabinet-level Departments• > 100 agencies, boards, & commissions• Executive Office of the President
8
• Office of the Vice President*• Chief of Staff*• Council of Economic Advisers• Council on Environmental Quality• Domestic Policy Council• National Economic Council• National Security Council• Office of Homeland Security• Office of Management and Budget*• Office of National Drug Control Policy* • Office of Science & Technology Policy• Office of the United States Trade Representative*• Etc…
The Executive Office of the President
* Cabinet rank members
9
What does OMB do?• Leads or participates in the development
and resolution of all budget, policy, legislative, regulatory, procurement, and management issues on behalf of the President.
• Oversees the implementation, coordination, and management of agency programs.
10
Congress
"No money shall be drawn from the treasury, but in consequence of appropriations made by law; and a regular statement and account of receipts and expenditures of all public money shall be published from time to time."
– U.S. Constitution, Article 1, Section 9
~250 Committees and Subcommittees
About a dozen appropriations subcommittees in both the
House and the Senate
11
FY 2008FY 2007
OMBpreparesFY 2008Budget
CongressionalAction
On FY 2008 Budget
We are here
Agenciesprepare FY 2009budget requests
OMB assessesFY 2009requests
OMBpreparesFY 2009Budget
CongressionalAction
On FY 2009 Budget
Overlapping Budget ProcessesOct 1, 2006 Oct 1, 2007
OMB oversees agency budget execution and evaluates performance
12
Agenda
• Overview of U.S. R&D spending
• Budget Players and Process in the U.S.
• Why PART Was Created
• PART Basics
• How OMB Uses PART Results
• Supplemental Information
13
The Challenge in 2001
– Agency performance information inadequate, inconsistent or not available
– Performance measures not outcome-oriented and not tied to agency mission
– Performance results not integrated into agency decisions and budget requests.
• How do we incorporate program performance into funding and management decisions?
14
Budget – Performance Integration
• One of five government-wide initiatives of the President’s Management Agenda
• PART is a tool for assessing agency performance in this initiative– The “stoplight” scoring system used to assess
agency progress in the BPI initiative includes criteria related to PART
15
Agenda
• Overview of U.S. R&D spending
• Budget Players and Process in the U.S.
• Why PART Was Created
• PART Basics
• How OMB Uses PART Results
• Supplemental Information
16
PART Basics (1)
The PART has four sections:I. Purpose and Design (20 percent)II. Strategic Planning (10 percent)III. Management (20 percent)IV.Results (50 percent)
• Each section has 5 to 10 questions.
• Sections I, II, and III allow only Yes or No responses.
• Section IV allows partial credit.
17
PART Basics (2)
• PART Guidance document – elements of a “Yes” response – acceptable documentation of evidence
• Products posted to ExpectMore.gov: • program score (0 to 100)• effectiveness rating • summary of findings • Improvement plan / follow-up actions
18
PARTs Tailored to Program Type• The PART has seven versions, one for each
program “type”:– credit - block/formula grant– competitive grant - direct federal– regulatory - research and development– capital assets & service acquisitions
• Questions in the “Direct Federal” version are common to all PARTs
• Other versions include extra questions tailored for program type
• Question weighting can be adjustedIssue: Definition of a Program
19
Section I: Purpose & Design Highlights• Clear purpose?• Address a specific and existing problem?• Not redundant or duplicative?• Free of major design flaws that limit the
program’s effectiveness or efficiency?• Resources effectively targeted?
20
Section II: Strategic Planning
Highlights:• Long-term and annual performance measures?• Baselines and ambitious targets?• Plan for independent evaluations of sufficient
scope, quality, and frequency?• Budget requests tied to accomplishment of
goals?• Address strategic planning deficiencies?
21
Section III: Program Management
Highlights:• Collect and use performance info?• Federal managers and contractors held
accountable for cost, schedule, performance?• Funds spent for intended purpose?• Effective collaboration and coordination?• Strong financial management practices?• Address management deficiencies?
22
Section IV: Program Results/Accountability
Highlights:• Meeting long-term and annual goals?• Demonstrate improved efficiency?• Results of independent evaluations?• Scoring: partial credit allowed
– Yes– Large Extent– Small Extent– No
23
Agenda
• Overview of U.S. R&D spending
• Budget Players and Process in the U.S.
• Why PART Was Created
• PART Basics
• How OMB Uses PART Results
• Supplemental Information
24
What Do We Get Out of PART?1. Overall program “effectiveness” ratings, based
on banding of scores that can range from 0 to 100:
– Effective - Ineffective– Moderately Effective - Results Not Demonstrated– Adequate
2. Specific follow-up actions aimed at improving performance
3. Data to inform decision making processes, including both budget and management decisions
25
“Effectiveness Ratings”(cumulative number of programs assessed)
50%38%
29% 24% 22% 19%
5%
5%
4%4% 3%
3%
15%
20%
26%28%
28%29%
24%26%
26% 29%30% 31%
6%11% 15% 15% 17% 18%
0%
20%
40%
60%
80%
100%
2002 (234) 2003 (407) 2004 (607) 2005 (793) 2006 (977) 2007 (1,004)
Results Not Demonstrated Ineffective Adequate Moderately Effective Effective
26
(a) Data for Budget Decision Making
Budget Process
Agency Requests
Funding Guidance
Administration Priorities
Programmatic trends/external
factorsPART
Departments/agencies-budget proposals-actions to improve performance
Director’s Review -funding decisions-management recommendations-legislative proposals
27
(b) Data for Budget Decision Making
• PART informs budget decisions; not sole basis for decisions.– A good PART score does not necessarily
mean more funding.– A bad PART score does not necessarily mean
less funding.
• PART helps identify needed planning and management improvements.
28
Agenda
• Overview of U.S. R&D spending
• Budget Players and Process in the U.S.
• Why PART Was Created
• PART Basics
• How OMB Uses PART Results
• Supplemental Information
29
Portion of Discretionary funds spent on R&D by OMB division
0%
20%
40%
60%
80%
100%
Nat. Res. Hum. Res. Gen. Govt. Natl. Sec.
% B
A,
FY
200
4
Non-R&DF S&T "D" Non-R&D
30
Total funds spent on R&D by OMB Division
0%
20%
40%
60%
80%
100%
Nat. Res. Hum. Res. Gen. Govt. Natl. Sec.
% $
, FY
200
4
F S&T "D" Non-R&D
31
Implementation Issues
• Definition of a “program”
• Reassessment frequency
• Tracking PART follow-up actions
• Quality control
• Data collection, analysis, and presentation
• Congressional and public interest
• Increasing transparency: ExpectMore.gov
32
R&D Specific Questions
• 2RD2) If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?
• 2RD2) Does the program use a prioritization process to guide budget requests and funding decisions?
• 3RD1) For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?
33
The PART makes the assessment approach consistent across programs; performance measures make assessment appropriate for each program.•Performance measures are key to:
– assessing program effectiveness– incentivizing the right behavior
•Performance measures should be:– Salient and meaningful– Capture the most important aspects of a program’s
purpose and priorities
Performance Measures
34
Goal = performance measure + target
• Targets should be quantitative and trendable over time
• Quantitative baseline required.
Goals, Measures, and Targets
35
Outcomes and Outputs
Performance goals may focus on outcomes or outputs.
• Outcomes: Intended result, effect, or consequence of program. Public benefit should be clear. External factors influence.
• Outputs: What the program produced or provided.
• The PART strives for measures of outcomes. – Output or process measures should clearly tie to outcomes.
36
Consistency Check Purpose
• Ensure OMB examiners are applying PART guidance consistently– PART response vs. PART guidance – NOT PART response vs. other PART
response
• Assessing quality and content of Explanation and Evidence
• Generally no review of evidentiary documents
37
Available at www.omb.gov/part
• PART guidance
• Link to ExpectMore.gov
• Supporting materials on performance measurement and evaluation
• Frequently asked questions