Page 1

NASA’s Focus on Improving PerformancePresentation to the NDIA

May 21, 2009

Sandra SmalleyActing Director, Engineering and Program/Project Management Policy

Office of the Chief Engineer

ssmalley@nasa.gov

Page 2

Big Picture

• NASA's mission is to pioneer the future in space exploration, scientific discovery, and aeronautics research.

• NASA employs a strategic management approach requiring all organizations to manage requirements, schedule, and budget according to a program/project management method built on NASA’s Governance Model and NASA’s Core Values and Guiding Principles.

Page 3

Multi-Tiered approach to cost, schedule and performance assessment

• Enhanced Policy– New Agency policy for Acquisition

• Strengthened strategic integration

• Application of Joint Confidence Levels

– Existing policy for Program/project management

• Provides framework and requirements for Program and Project Management

• Established standard WBS and common project life cycle for Space Flight Projects

• Currently being revised to flow down new acquisition policy

• Enhanced Review process– Engages all levels (Project, Program, Mission Directorate, Agency)

• Key Decision Points

– Reviewed Independently

• Periodic reviews (monthly, quarterly, annually)

Page 4

Enhanced PolicyImplemented New Acquisition Policy

Maintain and enhance Existing Program/Project Management Policy

Page 52-18-09

New Agency Policy for Acquisition

Defines acquisition as the process for obtaining the systems, research, services, construction, and supplies that the Agency needs to fulfill its mission.

Acquisition—which may include procurement (contracting for products and services)—begins with an idea or proposal that aligns with the NASA Strategic Plan and fulfills an identified need and ends with the completion of the program or project or the final disposition of the product or service.

Page 6

Goal – Document Agency’s best practices and improve the Agency’s performance against external commitments for space flight and Information Technology programs/projects

Strengthened the Acquisition Process:• Added early involvement by senior management

– Acquisition Strategy Planning (ASP) meeting• Ensures consistency with Vision, Strategic Plan, and

Agency budget request• Assigns program/project to a Center • Directs major partnerships

– Acquisition Strategy Meeting (ASM), early in Formulation• Reviews make-or-buy decisions• Approves acquisition strategy

• Retained Procurement Strategy Meetings prior to release of major RFPs/contracts

• Complies with all FAR and NASA FAR requirements

Highlights of the new Acquisition Policy

Page 72-18-09

Highlights of the new Acquisition Policy Continued…

Improve estimating and performance:

• Programs and projects in Implementation are to be baselined or rebaselined and budgeted based on a joint cost and schedule probabilistic analysis.

• Programs are to have a confidence level of 70% or the level approved by the decision authority of the responsible Agency-level management council.

• As a minimum, projects are to be funded at a level that is equivalent to a confidence level of 50% or as approved by the decision authority of the responsible management council.

• Requires consistency among the approved cost estimates, funding strategy, the NASA budget, and external commitments

Page 8

8

Joint Confidence Level Overview

Policy Overview:• All space flight and information technology programs

shall develop a joint cost and schedule probabilistic analysis and be baselined or rebaselined and budgeted such that there is a 70 percent probability of achieving the stated life cycle cost and launch schedule

• Applicable decision authorities may approve a different joint confidence level

• Projects are to be baselined or rebaselined and budgeted at confidence level consistent with the program’s confidence level

• At a minimum, projects are to be funded at a level that is equivalent to a confidence level of 50 percent, or as approved by the applicable decision authority

• Programs or project’s proposed cost and schedule baselines are to be assessed by an independent review team

• Mission Directorates or Mission Support Offices are to confirm that program and projects’ life cycle cost estimates and annual budget submissions are consistent

•JCL is a…- Joint (cost and schedule) Confidence Level is the probably that cost will be equal or less than the targeted cost AND schedule will be equal or less than the targeted schedule date- Process and product that helps inform management the likelihood of a projects’ programmatic success- Process that combines a projects’ cost, schedule, and risk into a complete picture

•JCL is not a…- Specific methodology (e.g. resource loaded schedule)- Product from a specific tool (e.g. @RISK)

JCL Defined

Page 9

Existing Policy for Program/Project Management

Three program and project management policy documents governing the following portfolios of projects:

• Space Flight

• Information Technology and Institutional Infrastructure

• Research and Technology

Highlights include:

• Standardized project life cycle for Space Flight Projects

• Established key decision points

• Established review structure (dependent and independent)

Page 10

Standard Program and Project Life Cyclefor Space Flight Projects

• Provides a uniform life cycle for human and robotic missions

– Common process flow, uniform phases, and KDPs

– Disciplined review structure for technical requirements and implementation plans

• 5 key elements in execution of the life cycle:– Key Decision Points

– Decision Authority role as gatekeeper

– Required independent reviews

– Required life cycle gate products

– Center Management Councils and Governing Program Management Councils role in life cycle process

Page 11

Project Life CycleSimplified

Key Decision Points

FORMULATION IMPLEMENTATION

Major Reviews

A C D E

Project

PhasesConcept

Studies

Concept &

Technology

Development

Preliminary

Design &

Tech. Comp.

Final

Design &

Fabrication

Sys. Assembly,

Test, & Launch

CloseoutOps. &

Sustainment

A B

B

C

F

D E FPre-A

Mission Concept Review

Systems Requirements Review

Mission Definition Review

Critical Design Review

Systems Integration Review

Operational Readiness Review

Flight Readiness Review

Post Launch Assessment

ReviewDecommissioning

Review

Preliminary Design Review

Page 12

Project Assessments(Independent Life Cycle Reviews and

Periodic Reviews)

Key Decision Points

FORMULATION IMPLEMENTATION

A C D E

ProjectPhases Concept

StudiesConcept &TechnologyDevelopment

PreliminaryDesign & Tech. Comp.

FinalDesign & Fabrication

System Assembly,Test, & Launch

CloseoutOps. &Sustainment

A B

B

C

F

D E FPre-A

Independent Life Cycle Reviews

(includes independent

standing review board assessment)

Periodic Performance Assessments

(assessed monthly/quarterly/

annually at all levels: Project, Program, Center, Mission

Directorate, Agency)

Page 13

Independent Life Cycle Review Overview

The review of programs and projects at each life cycle milestone by competent individuals who are not dependent on or affiliated with the program/project to objectively assess: • The adequacy and credibility of the technical approach

(including but not limited to: requirements, architecture, and design),

• Schedule,

• Resources,

• Cost,

• Risk, and

• Management approach;

• Progress against the Program/Project Plan;

• Readiness to proceed to the next phase; and

• Compliance with NPR 7120.5 and 7123.1 requirements.

Page 14

Independent Life Cycle Assessments Process

Decision AuthorityStanding

Review Board (SRB)

Establish

Convene

Provide Terms of Reference

Program/ProjectInternal Reviews,

Baselines, Summary Packages

Report

Program/Project

Other Relevant Individuals

(e.g., Technical Authorities, Program Analysis & Evaluation)

Center Management

Council(CMC)

Decision Authority

Decision on Readiness for Next Phase

Assessment Disposition of

Findings

Program Management

Councils (PMC)

Recommendations

Page 15

Why Have A Life Cycle Independent Review Process?

To provide:

• The program/project with a credible, objective assessment of how they are doing.

• NASA senior management with an understanding of whether

– The program/project is on the right track,

– Is performing according to plan, and

– Externally-imposed impediments to the program/project’s success are being removed.

• A credible basis for a decision to proceed into the next phase.

– The independent review also provides additional assurance to external stakeholders that NASA’s basis for proceeding is sound.  

Page 16

Project

Periodic Performance Reviews

Baseline Performance Review (BPR)

Mission Directorate, Program, Project, and Institutional

Mission Directorate (MD) and Center

Program and Project

Contractor Performance

Monthly(Assess cost, schedule, technical, risk,

…etd)

Agency

Level

Center Level

Project Level

Monthly/QuarterlyPerformance

(Assess cost, schedule,Technical, risk and

impact to porfolio/program/center…etc.)

Monthly/Quarterly Review

(Assess cost, schedule, technical, risk, cross cutting

technical, cross cutting institutional,…etc)

Page 1717

DRAFT V3

“Top-Level” BPR Process

Programsand

Projects Input:

TemplateQuestions,

Data and reviews

BPR Meeting

IndependentAssessment of

Programs, Projects& Mission Directorates

Actions/ Feedback

Cross CuttingTechnical &

Non-TechnicalIssues

AAPre-Brief and

Post-Brief

1st of the monthApproximately 3 weeks…

Mission Directoratesand Centers

Input:Template

Questions,Data and reviews

MissionSupportInput:

TemplateQuestions,

Data and reviews

Multiple Perspectives

Mission SupportAssessment of

Programs & ProjectsAnd Functional Area

CenterAssessment of

Programs & ProjectsAnd Functional Area

MissionDirectorate& ProjectResponse

SpecialTopics

Page 18

Mission Directorate OverviewMission Directorate OverviewMay 2009May 2009

Status: – 5 projects rated Red overall: Project Names would be provided here– 1 project rated YellowYellow overall: Project name would be provided here– Project A successfully underway; Project B successfully launched

Programmatic

TechnicalProgram A: Project Y Observatory Cryo-Thermal

performance issues.Program B : Inability to manufacture replacement boards for Project X’s S/C Single Board Computer. Project says:

"Considerable residual concern remains."Program C : Project Z actuators.

All: Continued difficulties developing instruments.

ScheduleProgram A : Project A milestones slipping; Project Z slack

margin and instrument beta and gama deliveries.Program C: Project D Instrument Alpha has consumed all schedule margin; Project X launch delay & potential for

more from SBC issues.

CostAll: Access to Space continues to be an issue: Rocket B

replacement; Rocket A failure, & impact on Project X; Very crowded launch manifest in fall 2011. Conflicts for launch processing facilities.

$47M in Project X APA reserves, needed for 70% Confidence level, was moved from FY'10 & '11 to after

Program A : Funding of Project Y slip limiting division's Program B : Project A & Project B FY'09 reserves low.

Program C : Overall division budget issues.

Watch list :Items • Project A - Difficult cost vs risk tradeoff choices. • Project C -- Cost impact on project if ULA launch delay is realized.

Y

April May

Y

EXAMPLE

Page 1919

DRAFT V3

PHASE

2009 Month COMMENTS

Program Alpha G G G

Project A Y Y Y On-orbit payload issue delayed launch to NET May 12 2015.

Project B Y Y Y Flow Control Valve flew successfully on spacecraft. Additional mitigation actions being studied for future flights.

Project C G G G

Project D G G G

Project E G G G

Project F G G G

Project G Y Y G Retention of workforce risk improving

Project X G G G

Project Y G G G

Project Z G G Y Incomplete TPA input, delay in scheduled activities

Program Beta G R R

Project H Y R R Project A successfully launched on 3/6/09. Project B/C launch delayed to NET 6/2/10 due to Partner WGS delay. Launch vehicle on Project Lon 2/24/09 failed.

Project I G G G

Program Gamma Y Y Y

Project J G G G

Project K Y Y Y PWD bacteria, SARJ Anomalies, UPA failure, ELC schedule delays, Partner ballistic re-entry (No issue could be found with returned Partner pyro bolt).

Project L G G G

Project M G G G

Project N Y Y Y Cargo transport uncertainty post Program Gamma

Formulation Implementation

PROGRAM/ PROJECT

Mission Directorate Programs and ProjectsMission Directorate Programs and ProjectsA

Feb Mar AprB

Feb Mar AprC

Feb Mar AprD

Feb Mar Apr EFeb Mar Apr

EXAMPLE

Page 20

Technical: Schedule:

Cost: Programmatic:

Status: In Formulation

Project Status:Project Status:Project XProject X YY

Y

R Y G G

• Working Thrust Oscillation concerns: active controller dropped, passive system baselined but Project

considering additional isolator (between the upper stage and service module, which could impact the

stack height) (Y)• First Stage has implemented 100% tangential x-ray

NDE on castings• Project XTower Recontact/Plume Damage (Y)

Feb Mar

Y G G• AB-1 motor test schedule moving toward August 2009

due to FSM 17 review• Delta PDR scheduled for Thrust Oscillation in Fall 2009

• Unclear as to how the Integrated PlanningMtg09 results will affect cost confidence level (Y)

• Additional $16M was provided by HQ to resolve the CR issue. Improved cost from Red to Yellow

• A number of contract actions are in worked to align elements with program guidance. Planned for

execution beginning in the March 09 timeframe• Partner rayon has become available and NASA has an

opportunity to procure the material

EXAMPLE

Page 21

BPR Benefits/ Results

21

• Focused NASA Senior Leaders as an “Execution Board of Directors”

– Quick response to emerging issues– Course corrections in implementation of PMC, SMC decisions– Identify cross-cutting issues and action plans e.g., software complexity, instrument acquisition, etc.– Resolution of organizational communication issues (Field Centers/ Programs/ Mission Directorates/ etc.)– Identify mission support issues e.g. procurement workforce

Page 22

Conclusion

NASA recognizes the following as key elements for continued performance enhancement:

– Clear policy and standards

– Engaged leadership and team members at all levels

– Review processes for periodic status checks

– Variety of tools to assess performance and address issues

– Knowledge and experience to know when to apply the appropriate tool and how to interpret the results