© 2009 The MITRE Corporation. All rights reserved Approved for Public Release. Case #09-3713. September 11, 2009 Kevin Buck and Michele Steinbach Applying

© 2009 The MITRE Corporation. All rights reservedApproved for Public Release. Case #09-3713.

Approved for Public Release. Case #09-3713.

September 11, 2009Kevin Buck and Michele Steinbach

Applying Best Practices for Performance Management

Improvement



Agenda Purpose of this briefing

The challenge: delivering on the strategic vision

Case study: the challenges for a federated Research and Development (R&D) enterprise

A performance management improvement decision framework

Case study: performance management recommendations

So why focus on performance management?

2



Purpose

3

The purpose of this briefing is to describe how a government enterprise that is multi-faceted and multi-cultural can tailor and apply performance management principles to better manage their portfolios and achieve their vision.



Delivering on the Strategic Vision Federal agencies are under increasing pressure to

demonstrate:– Delivery to stakeholders– A solid plan for achieving strategic goals

Articulating the strategic vision is not enough– Agency leaders must actively steer the enterprise toward its desired

end-state Moving from a description of the future to making that future

a reality involves:– Mapping out the transition plan, – Executing that plan, and – Monitoring plan execution to ensure the enterprise arrives at the

correct destination

4



Case Study: Federated R&D EnterpriseKey Performance Management Challenges

Limited span of direct performance control– Some aspects of the portfolio are directly within the financial control and

authority of portfolio manager, while some aspects of the portfolio are notDecentralized portfolio

– The portfolio spans numerous organizational boundaries and performance management cultures

Constrained information flow– Information security requirements, the classified nature of specific data, and

different cultures constrain information flow across the enterprisePerformance management resources are constrainedNumerous reporting requirements

– Enterprise is subject to many performance-management related regulations and other compliance requirements

Effectiveness/efficiency of data collection/analysis systems

5



Key AssumptionsMany government organizations wish to improve

performance– Application of performance management techniques can

improve performance

There is no “one size fits all” solutionPerformance management solutions must be tailored to a

government enterprise’s specific goals and circumstancesThere are cost, risk, and benefit tradeoffs that must be

considered in determining if, when, and how performance management improvements should be undertaken

6



Purpose of a Performance Management Improvement Decision FrameworkFramework objective:

– Support performance improvement by reducing or eliminating challenges that will hinder the ability of a government enterprise to improve as effectively and efficiently as possible.

7

To achieve this objective, the framework links:– Performance to performance management

– Improvement recommendations to a best-of-breed performance management process

– Improvement recommendations to an enterprise’s specific goals and circumstances



8

TailorTailor

DiagnoseDiagnose:Compare

improvement goals to current

performance management

maturity, challenges, and enablers

RecommendRecommend:

Identify customized

recommendations and prioritize next

steps

Key Framework Elements



9

Why the Need to “Diagnose”?

Improvement LandscapeImprovement Readiness

Improvement Goals

Performance Management Maturity

Improvement Fertile GroundChange Capacity

Processes, People, Technology

Stakeholder Engagement

Shaped by Regulatory, Political, and Competitive Environment

Past Failures and Successes

Enablers and Constraints

Integration of Supporting Processes

Policies and Regulations



Your improvementgoals

Enablers and Constraints

Improvement Readiness

Prioritization ofimprovement

recommendations

Performance Management Maturity

Diagnostic Elements



11

Category Ad Hoc Repeatable Defined Managed Optimizing

Enterprise Goals and Priorities

Enterprise goals not clearly defined.

Enterprise goals established but not clearly mapped to mission and vision.

Enteprise goals mapped to mission and vision. There is some traceability of strategic goals to some stakeholder needs.

Effectiveness of goals measured and reviewed. Key stakeholder needs are directly linked to strategic goals.

Enteprise goals prioritized, refined and improved based on results. Stakeholder needs are prioritized and directly influence enterprise goals.

MetricsMetrics defined after the fact or ad hoc.

Metrics are defined and measurable but primarily represent what can, rather than what should, be measured.

Metrics are clearly defined, measurable, and accepted. Some metrics can be directly traced to some stakeholder needs.

Effectiveness of metrics measured and reviewed. Key stakeholder needs are directly linked to metrics.

Metrics refined and improved based on effectiveness. Stakeholder needs are prioritized, key stakeholder needs are directly linked to metrics, and metric effectiveness is regularly reassessed by rigorously revisiting stakeholders and their needs.

TargetsPerformance targets not defined.

Performance targets defined.

Performance targets well established and accepted. Some targets are influenced by some stakeholder needs.

Performance targets established based on rigorous benchmarking process. Key stakeholder needs are directly linked to targets.

Performance targets refined and improved based on results. Targets are consistently reassessed based on interaction with stakeholders and a rigorous revisiting of stakeholder needs and priorities.

Strategic Alignment

Metrics not aligned to strategic goals

Metrics loosely aligned to strategic goals

Metrics clearly aligned to strategic goals. Some metrics and strategic goals are influenced by some stakeholder needs.

Metrics measured and reviewed for alignment to strategic goals. Review includes consideration of key stakeholder needs.

Metrics / goals refined and improved based on alignment reviews. Consistent interaction with key stakeholders influences refinement of metrics/goals.

Activity Alignment

Metrics do not directly measure program activities.

Metrics loosely aligned to program activities.

Metrics clearly aligned to program activities. Some activities are influenced by some stakeholder needs.

Metrics measured and reviewed for alignment to program activities. Review includes consideration of key stakeholder needs.

Metrics / program activities refined and improved based on alignment reviews and interaction with key stakeholders and reassessment of stakeholder priorities.

MeasurementMeasurement of metrics is ad hoc and inconsistent.

Measurement of metrics is conducted on a routine basis.

Measurement of metrics is institutionalized. Some efforts are undertaken to assess stakeholder satisfaction.

Measurement effectiveness tracked and measured. Stakeholder satisfaction/needs are consistently evaluated.

Measurement of metrics is constantly refined and improved, with an objective of more directly measuring stakeholder satisfaction.

Leadership

Leadership not actively engaged in performance management.

Leadership routinely engaged in performance management.

Leadership views performance management as standard business practice. Leadership acknowledges the importance of stakeholder needs to performance management.

Leadership applies formalized governance process to measure and monitor effectiveness of performance management efforts. Leadership actively promotes the alignment of stakeholder needs to performance management.

Leadership refines and improves performance management efforts. Leadership proactively promotes the direct engagement with key stakeholders as an intrinsic aspect of performance management.

Infrastructure (IT, personnel, guidance, training)

No infrastructure in place to track and manage performance management efforts.

Infrastructure in place to support performance management efforts.

Infrastructure institutionalized and sustained.

Infrastructure measured for effectiveness and efficiency.

Infrastructure refined and improved based on effectiveness. Centralized performance management knowledge repository.

Performance Management Maturity Model (PM3)

Sample Diagnostic – PM3 In Aggregate



12

Sample Diagnostic – PM3 SnapshotCategory Ad Hoc Repeatable Defined Managed Optimizing

Metrics

Metrics defined after the fact or ad hoc.

Metrics are defined and measurable but primarily represent what can, rather than what should, be measured.

Metrics are clearly defined, measurable, and accepted. Some metrics can be directly traced to some stakeholder needs.

Effectiveness of metrics measured and reviewed. Key stakeholder needs are directly linked to metrics.

Metrics refined and improved based on effectiveness. Stakeholder needs are prioritized, key stakeholder needs are directly linked to metrics, and metric effectiveness is regularly reassessed by rigorously revisiting stakeholders and their needs.



13

Sample Diagnostic – Enablers/Challenges Factor Group A: Enabling Environment

– Factor 1: Commitment to improvement?– Factor 2: Organizational Politics?– Factor 3: Organizational Stability?– Factor 4: Communication?

Factor Group B: Management Resources– Factor 5: Skills/experience?– Factor 6: Organizational design?– Factor 7: Funds for managing?– Factor 8: Training opportunities?

Factor Group C: Tools/Techniques– Factor 9: Standardized taxonomy?– Factor 10: Application of decision-making techniques/analyses?– Factor 11: Application of management evaluation techniques?– Factor 12: Data collection/sharing systems?

Factor Group D: Processes/Practices– Factor 13: Structured processes for performance management?– Factor 14: Structured processes for performance and mgmt improvement?– Factor 15: Compliance with regulations and policies?– Factor 16: Application of benchmarking for improvement?



Sample Diagnostic – Enablers/Challenges

Factor Group A:

Enabling Environment

Factor Group B

Management Resources

Factor Group C

Tools/Techniques

Factor Group D:

Processes/Practices

Factor 1

Factor 9

Fac

tor 1

3

More vulnerable to failureLarger Capability Gaps

14

* Adapted from R. Stevens, The MITRE Corp., “Systems Engineering in the Information Age: The Challenge of Mega-Systems”, 12Jan2005Factor 1 Factor 2

Factor 3Factor 4

Facto

r 5

Fac

tor

6

Facto

r 7

Factor 8

Factor 9

Factor 10

Factor 11

Facto

r 12

Fac

tor

13F

acto

r 14

Fact

or 1

5

Factor 16



15

Key Framework Elements - RecommendationsImprovement Goals

• Identify “gotchas” to avoid

•Provide detailed guidance on adopting/implementing performance management process steps

• Identify techniques and tools to apply

• Identify how to communicate the need and value of performance management to stakeholders

•Determine how to align performance management efforts with Presidential direction and regulations

Priorities

??

??

??

??

??



16

Step 4: Develop process for monitoring and managing performance on an on-going basis

Step 1: Define strategic outcomes

Step 2: Describe current performance

Step 3: Describe the future state

Step 5: Define and implement a periodic performance management process evaluation

Key Framework Elements - RecommendationsPerformance management process

– Steps do indeed “fit most”, but …

Where do I enter this process?

Where should I focus attention and resources?



Why focus on Performance Management? Strategic Planning

Business/Operations Planning and Management

Financial Management and Budgeting

Cost Management

Risk Management

Transition Planning and Change Management

Investment Decision-Making Processes and Management

Innovation Management

Supply Chain Management

Performance Engineering

Human Capital Management

Enterprise Architecture and Systems Engineering

17

Performance Management

Enterprise Management

Portfolio Management

Program/Project Management

Acquisition Management

Vision and Strategic Outcomes



Managing Portfolio Performance ImprovementWe are here now

X

http://www.albrightstrategy.com/framework.html

A good road map is dynamic and evolves over time as conditions change and new discoveries are made.

Alternate paths must always be considered.

Roadmap Type Critical Activities

“Know Why” “Know What” “Know How” “To Do”

Technology • Mission analysis• Codify organizing

logic for business processes and infrastructure (e.g., EA “as is”)

• Capabilities analysis

• Performance baselining and benchmarking

• Specify “To Be” environment

• Strategic planning• Root cause

analysis.• Gap analysis

(e.g., SWOT )• Dependencies

analysis• Portfolio analysis

• Functional analysis

• Systems engineering

• Performance analysis and engineering

• Transition planning

• Analysis of Alternatives

• Portfolio analysis and optimization

• Change management

• Investment management

• Portfolio management

• Acquisition management

• Resource management

Process

People

Application or System

Roadmap elements adapted from:

X

We need to be here by xx/yy/zz

18

PfM a key element



Role of PfM: Managing the Transition

19

FUTURE STATE

TODAY

Goal: Invest in more experimental technologies for greater innovative opportunities; focus more on long-term needs.

This is a “build” slide best viewed in

slide show view



20

FUTURE STATE

TODAYEnterprise Goals

Technology

Maturity

Short Term Long Term

Experimental

Mature I

I

I

III

Baseline or Current State: Portfolio dominated by mature technologies and short-term focus

Role of PfM: Managing the TransitionThis is a “build”

slide best viewed in slide show view

I = Investment



21

FUTURE STATE

TODAY

Interim State: Portfolio shifting toward longer-term focus, with higher-maturity technology still predominating.

Enterprise Goals

Technology

Maturity


Experimental

Mature

I I

I

I

I

I



I = Investment



22

FUTURE STATE

TODAY

Enterprise Goals

Technology

Maturity


Experimental

Mature

II

I

I

II

I

End State: Investments concentrated in more experimental technologies supporting long-term enterprise goals.



I = Investment



Time

Resources

Project Phases

1.Start up

2.Development

3.Implementation

4.MaintenanceS

DI

M

Today

Future

Many

Few S

I

D

I M

RISK

Role of PfM: Optimizing Across Multiple Dimensions

Investment timing

Prioritizing varying objectives, investment purposes, and stakeholder needs that may be unrelated or interdependent

RISK

23

Capability Gap #1

Capability Gap #2

Capability Gap #5

Investment AInvestment BInvestment CInvestment DInvestment EInvestment FInvestment GInvestment Z

Stakeholders



Case Study: Federated R&D EnterpriseAdopt a Performance Management PhilosophyThe portfolio delivers if the enterprise achieves outcomes and fulfills

stakeholder needs within expected costs and schedule– Performance management involves people, processes, and

technology that enable successful enterprise performance and its effective communication to stakeholders

Performance management success drivers:

Link performance measuring/monitoring to stakeholder needs and outcomes

Proactively focus on leading indicators of potential success/failure

Only measure/monitor what is important for characterizing progress in achieving outcomes and fulfilling stakeholder needs

Motivate desired performance by selecting measures that clearly signal what is important to the enterprise

Strike the right balance between qualitative/quantitative measures and among enterprise priorities

24



Summary

To achieve its goals, the enterprise must – Develop a sound strategy that supports stakeholder needs;– Create a roadmap that outlines the path toward its strategic

goals; and,– Execute that roadmap, through sound application of portfolio

and performance management principles and processes.

A performance management improvement decision framework can help the enterprise to tailor those processes to the organization’s strengths, weaknesses, resource constraints, and organizational enablers.

25

Documents

© 2009 The MITRE Corporation. All rights reserved Approved for Public Release. Case #09-3713. September 11, 2009 Kevin Buck and Michele Steinbach Applying