Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
NDIA SE Conference-SW Initiatives 10/29/09 Page-1 UNCLASSIFIED
Overview of Department of Defense (DoD) Software Engineering Initiatives
Mr. Scott LuceroDeputy Director, Software Engineering
Systems Engineering DirectorateOffice of the Director, Defense Research and Engineering
12th Annual NDIA Systems Engineering ConferenceOctober 29, 2009
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-2
Elements of a DoD Strategy for Software Engineering
• Support Acquisition Success – Ensure effective and efficient software solutions across the
acquisition spectrum of systems, SoS and capability portfolios
• Improve the State-of-the-Practice of Software Engineering– Advocate and lead software initiatives to improve the state-of-the-
practices through transition of tools, techniques, etc.
• Leadership, Outreach and Advocacy– Implement at Department and National levels, a strategic plan for
meeting Defense software requirements
• Foster Software Resources to meet DoD needs– Enable the US and global capability to meet Department software
needs, in an assured and responsive manner
Promote World-Class Leadership for Defense Software Engineering
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-3
NDIA Top Software Issues September 2006
1. The impact of requirements upon software is not consistently quantified and managed in development or sustainment. “SW Requirements”
2. Fundamental system engineering decisions are made without full participation of software engineering. “SE/SW Integration”
3. Software life-cycle planning and management by acquirers and suppliers is ineffective. “SW Sustainment”
4. The quantity and quality of software engineering expertise is insufficient to meet the demands of government and defense industry. “Human Capital”
5. Traditional software verification techniques are costly and ineffective for dealing with the scale and complexity of modern systems. “SW Testing”
6. There is a failure to assure correct, predictable, safe, secure execution of complex software in distributed environments. “SW Assurance”
7. Inadequate attention is given to total lifecycle issues for COTS/NDI impacts on lifecycle cost and risk. “SW COTS / NDI / Reuse”
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-4
Top Software Issues - 2006 vs. Software Systemic Findings - 2008
N ationa l D efense Industria l Associa tion (N D IA)Top 7 Softw are Issues
August 2006
D U SD (A& T) SSE D irectora te P rogram R eview Softw are System ic Analysis F ind ings
Softw are Requirem ents
System s/Softw are Integration
Softw are Hum an Capital
Softw are AssuranceSoftw are Assurance
Softw are Engineering M anagem ent
� Pro ject P lann ing� M anagem ent O versight� Softw are C onfiguration M anagem ent
Softw are M etrics � Softw are M etrics� EVM
Know ledge Sharing � Process� R eporting
Softw are Requirem ents� Engineering� M anagem ent� Acquisition S tra tegy
Softw are Developm ent� Softw are Testing*� Softw are Susta inm ent/M ain tenance*� Softw are C O TS/N D I*� Technology R eadiness� Softw are A rch itecture
Softw are Sustainm ent
Softw are CO TS/NDI
Softw are Testing
Softw are Hum an Capital � R esources� Q uality Leve l
System s/Softw are Integration
� System s of System s� In teroperab ility� Tech R efresh
National Defense Industrial Association (NDIA)Top 7 Software Issues
September 2006
ODDRE/SE Systemic Analysis ofProgram Support Review Findings
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-5
Current Software Engineering Initiatives
• Program Support– Provide software support for acquisition program reviews.
Develop independent schedule and defect estimates.
• Human Capital– Software Acquisition Training and Education Workgroup:
Establish SW competencies across the acquisition career fields– Reference Curriculum for Graduate Study of Software Engineering:
Version 1.0 completed this month, to be sustained by IEEE and ACM.
• Advance the State of the Practice– Software Sustainment, NDIA Software T&E Summit/Workshop
• Policy and Guidance– Earned Value Management, Military Handbook for Work Breakdown
Structures: MIL-HDBK-881.– Oversight of Services’ SW Acquisition Process Improvement
Programs.
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-6
Notional Example of Schedule Feasibility Analysis
Likelihood of delivery to current schedule: less than 1%
Current Plan(Dec 2013)
Review Team Projection(50th percentile Dec 2014)
End Date Cum Probability (%)1/26/2014 13/17/2014 3
5/6/2014 86/25/2014 138/14/2014 2210/3/2014 31
11/22/2014 411/11/2015 54
3/2/2015 654/21/2015 756/10/2015 837/30/2015 909/18/2015 9511/7/2015 99
12/27/2015 100
Current Plan (Dec 2013)
Review Team Projection (50th percentile - Dec 2014)
12/7/2013 6/25/204 1/11/2015 7/30/2015
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-7
Software and Systems Reliability
• DoD has renewed emphasis on systems reliability and lifecycle costs of shortfalls
– DDRE effort underway to consolidate software reliability guidance
• Starting to use parametric models to project numbers of latent software defects and discovery rates
– Used to support:– Development of satellite launch plans– Aircraft production decisions– Operational test readiness reviews
• Gauging software reliability using Mean Time to Defect (MTTD) discovery
System Testing DRs / Test Hr(DT events to IOT&E)
Software Block Maturity – Nov 2008
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-8
Software Human Capital Efforts
• Software Acquisition Training and Education (SATEWG)– Chartered February 2008 by USD(AT&L) to add software competencies to
DoD’s 13 acquisition career fields– Recent accomplishments:
− Developed software competency framework, − Established SPRDE software competencies− Gap analysis of SATEWG competency framework and DAU’s
Software Acquisition Management courses− Current focus is on PM, Contracting and Test career fields
• Graduate Software Engineering Reference Curriculum (GSwERC)– Partnership with Industry and Academia– Version 1.0 completed September 2009– Transitioned to IEEE and ACM for sustainment
Software Industrial Base Study – July 2007There is a choke-point in availability of top-tier software
managers, architects, and domain experts.Supply of sufficiently trained SW developers is inadequate near-term.
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-9
Better planning needed to partition software work among multiple developers and increase competition
*AIAA Infotech Conference 2009, Software Sustainment Challenges in Defense Acquisition
Software Sustainment Challenges
• Software intensive systems encourage*:– Build-a-little, test-a-little, field-a-little risk reduction– Incremental and spiral development efforts– Concurrent planning, development and sustainment activities
• No longer a natural ‘break point’ where software development can be transitioned to a sustainment organization– Technical capability of Government sustainment organizations
reduced due to acquisition reform
• Planning for software sustainment now a lost art– Acquisition programs no longer produce MIL-HDBK-347 Computer
Resource Life Cycle Management Plans
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-10
NDIA Software Test and Evaluation Summit/Workshop – Sep 2009
• Purpose: “Recommend policy and guidance changes to emphasize robust software T&E approaches in Defense acquisition.”
• Speakers from Government, Industry and Academia
• Conducted workshops on:– How much software T&E is enough– Software T&E involvement across the lifecycle– Emerging paradigms: SOA, SoS, Security
• Workshops specifically addressed:– Policy & guidance, Human capital, RFP language,
SW T&E tools
• NDIA Software Experts and DT&E sub-committee to produce white paper by December 2009
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-11
Software Measurement and Analysis Improvement Areas
Integrate software guidance into proven management techniques
Concepts - Requirements - Arch/Design - Development - Maintenance
Determine better methods of
obtaining cost estimating data Improve estimation
tools, techniques, & practices
Generate software appropriate WBS
Find best Earned Value Management (EVM)
practices for SW
Link quality indicators to EVM
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-12
Software Earned Value Management (EVM) Study/Pilot
• Develop methods to combine EVM and software metrics to predict cost and schedule overruns
• Piloted on a 5-year ACAT 1D software development program
• Pilot indicator shows estimate-at-completion (EAC) forecasts for:– Existing program management plans– Milestone-based EVM measures– Software metrics, i.e, growth profile of size, effort, defects
Equivalent EAC forecasts provide an increased confidence in project plans
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-13
Estimates at Completion (EAC) for Metrics, Earned Value, Program Plans
CPMO Spiral - Sample SummaryPerformance Measurement Baseline
$71
$478
$706$792
$941
$1,051
$1,145
$1,318
$1,420
$47
$228$291
$338
$565715
$1,032
$1,251
$1,370
$1,471$1,559 $1,579
$0
$200
$400
$600
$800
$1,000
$1,200
$1,400
$1,600
$1,800
Oct07 Nov07 Dec07 Jan08 Feb08 Mar08 Apr08 May08 Jun08 Jul08 Aug08 Sep08 Oct08 Nov08 Dec08
TIME (Months)
DO
LLA
RS
($K
BAC BCWS (PV) BCWP (EV) ACWP (AC) ETC
BAC = $1420
Data Date
EV MetricsCalculated Schedule Range:09/08/08 - 10/12/08
Calculated Estimate at Complete Range:$1,797K - $2,515K
Cumulative CPI = 0.79SW MetricsCalculated Schedule Range:05/14/08 - 06/18/08
Calculated Estimate at Complete Range:$1 570K $1 710K
SW MetricEAC
Program Plan
Completion
EV EAC
Confidence increases as EACs overlapMultiple measures reaching the same conclusion
UNCLASSIFIEDNDIA SE Conference-SW Initiatives 10/29/09 Page-14 14
Questions/Discussion
Contact Information:Don Scott Lucero
Deputy Director, Software Engineering Systems Engineering Directorate, Defense Research and Engineering