22
S ystem s Engineering D ivision 1 NDIA Software Industry Experts Panel 14 July 2008 NDIA Software Industry Experts Panel Paul R. Croll, Chair NDIA Systems Engineering Division

NDIA Software Industry Experts Panel Paul R. Croll, Chair

  • Upload
    avon

  • View
    40

  • Download
    1

Embed Size (px)

DESCRIPTION

NDIA Software Industry Experts Panel Paul R. Croll, Chair. NDIA Systems Engineering Division. Who We Are. The NDIA Software Industry Experts Panel acts as a “voice of industry” in matters relating to DoD software issues - PowerPoint PPT Presentation

Citation preview

Page 1: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

1NDIA Software Industry Experts Panel14 July 2008

NDIASoftware Industry Experts Panel

Paul R. Croll, Chair

NDIA Systems Engineering Division

Page 2: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

2NDIA Software Industry Experts Panel14 July 2008

Who We Are

● The NDIA Software Industry Experts Panel acts as a “voice of industry” in matters relating to DoD software issues

● The Panel helps identify and resolve software acquisition and development issues facing the industry and its Government customer base

● The Panel may also, from time to time, identify for investigation certain technologies or practices that promise to improve industry responsiveness to DoD needs.

● Members– Paul Croll, CSC, Chair– JoAn Ferguson, General Dynamics– Gary Hafen, Lockheed Martin– Blake Ireland, Raytheon– Al Mink, SRA International– Ken Nidiffer, SEI– Shawn Rahmani, Boeing– Rick Selby, Northrop Grumman

Page 3: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

3NDIA Software Industry Experts Panel14 July 2008

What We Do

● Investigate, analyze, and develop recommendations concerning software issues, in response to NDIA and Government requests

● Provide industry comments on Government positions, initiatives, or work products

● Develop industry white papers and position papers● Reach out to relevant stakeholders through NDIA

conferences and other venues as appropriate

Page 4: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

4NDIA Software Industry Experts Panel14 July 2008

NDIA Top Software Issues Workshop 24-25 August 2006

● Identify Top 5 Software Engineering problems or issues prevalent within the defense industry– Document issues

• Description and current state

• Rationale and SW impacts

– Develop recommendations (short term and long term)– Generate task report– Submit to OSD

Page 5: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

5NDIA Software Industry Experts Panel14 July 2008

Top Software Issues

1. The impact of requirements upon software is not consistently quantified and managed in development or sustainment

2. Fundamental system engineering decisions are made without full participation of software engineering

3. Software life-cycle planning and management by acquirers and suppliers is ineffective

4. The quantity and quality of software engineering expertise is insufficient to meet the demands of government and the defense industry

5. Traditional software verification techniques are costly and ineffective for dealing with the scale and complexity of modern systems

6. There is a failure to assure correct, predictable, safe, secure execution of complex software in distributed environments

7. Inadequate attention is given to total lifecycle issues for COTS/NDI impacts on lifecycle cost and risk

Page 6: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

6NDIA Software Industry Experts Panel14 July 2008

Defense Software Strategy Summit18-19 October 2006

● Keynote Address, the Honorable Dr. James I. Finley, Deputy Under Secretary of Defense (Acquisition and Technology)

● Program Executive Officer and Service/Defense Agency panels on software related acquisition issues and initiatives

● Plenary Session Topics -- NDIA Top Software Issues, Software Industrial Base Study, and Software Producibility

● Workshops – Software Acquisition and Sustainment

• Mr. Mike Nicol, Air Force Aeronautical Systems Center• Mr. Lawrence T. Osiecki, US Army, Armament Software Engineering Center

– Policy• Mr. Jim Clausen, DoD CIO, Office of Commercial IT Policy • Col Peter Sefcik, Jr., USAF Chief, Air Force Engineering Policy and Guidance Team• Lt Col Mark Wilson, SAF/AQR Systems & Software Engineering

– Human Capital• Dr. Kenneth E. Nidiffer, Fellow, Systems and Software Consortium• Mr. George Prosnik, Defense Acquisition University E&T Center

– Software Engineering Practices• Mr. Grady Campbell, Software Engineering Institute• Mr. Paul R. Croll, CSC, Industry Co-Chair NDIA Software Committee

Page 7: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

7NDIA Software Industry Experts Panel14 July 2008

DoD Software Summit Issues

Software Acquisition and Sustainment● Software issues not addressed early in lifecycle● Software requirements not well defined at program start● Management has limited visibility into software

development processes and status● Risk areas – single point failures not adequately

addressed, e.g., single software providers, incomplete data rights, key personnel stability, life cycle support of COTS

● Acquirers do not adequately address software sustainment and total life cycle early in the program

● Some agencies contract before engineering is complete, prior to system design and development

Policy● PMs need assistance with software policy

and analysis ● Arbitrary separation of weapon and

information technology software policies ● Policy implementation guidance and

follow-up monitoring is limited● Department needs software group with good

expertise to oversee and implement policy● Need capability to share policy and

guidance information

● Human Capital● Experienced system & software engineers seem

missing from key DoD leadership positions ● Shortage of highly experienced software

managers, architects, domain and technical experts

● Eroding depth and breath of experience for personnel in DoD

● Young people may consider system and software engineering as a career dead end

● Emerging skill set may be needed for future complex DoD systems, e.g., systems of systems

● Software Engineering ● Weak linkage between software

requirements and capabilities/portfolios● System development methods do not

properly leverage software ability to rapidly field new capability

● Systems and software engineering lifecycles not always consistent or harmonized

● Software considerations not consistently addressed in architectures

● Inadequate software estimating methods, e.g., COTS/NDI; best practices not applied

Reaffirmed NDIA Top Software IssuesReaffirmed NDIA Top Software Issues

Page 8: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

8NDIA Software Industry Experts Panel14 July 2008

Software Issues/GapsWorkshop Findings

Primary Software Focus Areas*

Software Development Techniques

*based on NDIA Top SW Issues, OSD Program Support Reviews, and DoD Software Summit findings

Data andMetrics

SW Metrics – A, OSW Cost – OSW EVM – DCMASW Estimation - GAP

Human Capital

Education Sources – N, ALeadership Training – A, SEISETA Quals – GAPSW Human Cap Strategy – GAPIndustrial Base – OUniversity Curriculum – OWorforce Survey - AF

KnowledgeSharing

DAU Software ACC – DAUBest Practices Clearinghouse – DAU, OSW Inventory – LMRLifecycle Guides – M, NRoot Cause Analysis – OLocal Knowledge Portals - N

Agile – O, SEIArchitecture – A, SEICOTS – SEIOpen Source – AFSustainment – GAPSW Interoperability – GAPSW Test - GAP

SW & SE Integration

Requirements – GAPSE/SW Process Int – OSW Council – NSW Dev Plan – NSW in SEP – NSW in Tech Reviews – NSW Quality Attributes - GAP

SoftwareAcquisition

ManagementStandards – O, NDAG Ch 4/7 – O, AFProg Spt – O, AllContract Language – A, M, NSW Estimation – GAPLifecycle Policy – AFRisk Identification - GAP

OngoingInitiative Owners

O – OSD/SSAA – ArmyN – NavyAF – Air ForceM – MDASEIDCMADAUL&MRGAP – No activity

Source: Kristen Baldwin, Deputy Director, Software Engineering and System Assurance, OUSD(AT&L), April 18, 2007

Page 9: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

9NDIA Software Industry Experts Panel14 July 2008

DoD Software Gaps

● Estimation● Risk Identification● Sustainment● Interoperability● Test● Requirements● Quality Attributes● SETA Qualifications● Human Capital Strategy

Source: Kristen Baldwin, Deputy Director, Software Engineering and System Assurance, OUSD(AT&L), April 18, 2007

Page 10: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

10NDIA Software Industry Experts Panel14 July 2008

DoD Software Gaps

● Estimation● Risk Identification● Sustainment● Interoperability● Test● Requirements● Quality Attributes● SETA Qualifications● Human Capital Strategy

Source: Kristen Baldwin, Deputy Director, Software Engineering and System Assurance, OUSD(AT&L), April 18, 2007

Page 11: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

11NDIA Software Industry Experts Panel14 July 2008

Software Industry Experts PanelAction Plan

● Software Points of Influence list– bi-directional commitment

● Software interested parties list– information awareness

● Supporting resolution of identified DoD Software Gaps– Human Capital, Requirements– Risk, Quality Attributes– Test, Estimation– Sustainment– Interoperability, SETA

Page 12: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

12NDIA Software Industry Experts Panel14 July 2008

Overall Workshop Objectives

● Three Workshop Panels– Software Requirements– Software Risk and Estimation– Software Quality Attributes

● Workshop Objectives For Each Panel– Define a specific plan to crystallize concrete progress

within the next 6-18 months– Define work products and a plan to develop them over 6,

12 and 18 month periods– Identify stakeholders relevant to each of these work

products

Page 13: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

13NDIA Software Industry Experts Panel14 July 2008

Software In Acquisition Workshop

Tuesday, October 16, 2007 Wednesday, October 17, 2007

0800-0815 Welcome: Dr Finley - DUSD(A&T) 0800-1500 Workshops - continued

0815-0900 Keynote: Carl Siel – Navy Chief SE

1515-1600 Keynote: Dr Myers, DoD Deputy CIO

0900-1015 Industry Presentation – NDIA Software Experts Panel

1600-1645 Workshop Outbriefs (3)

1025-1445 Presentation Tracks (12 Presentations)

1645-1700 Closing Remarks

1500-1700 Focused Workshops

Attendance: - 100+ attendees- Services, Agencies, Industry, Academia, FFRDC, NASA

Workshop Topics: - Software Requirements- Software Estimation / Software Risk- Software Quality Attributes

Page 14: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

14NDIA Software Industry Experts Panel14 July 2008

Requirements Workshop Recommendations

● Define an effective “software portfolio” management framework– Protect the continuity of systems/software and requirement

engineering throughout the software life cycle

● Implement the techniques we know will work and identify any shortcomings

● Find ways to leverage the malleability of software – Software has the ability to adapt to changing requirements

● Change our view/perspective of “sustainment” to “continuous evolution”

● Establish a research program

Page 15: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

15NDIA Software Industry Experts Panel14 July 2008

Software Estimation/Risk Recommendations

● Establish Work Breakdown Structure guidance to better highlight Software Engineering activity

● Developing and evolving an integrated software data repository and related tools

● Conduct Root Cause analysis studies to understand the problems in software estimation and the use of estimates in the acquisition process

● Develop and implement an incremental acquisition approach (as well as the overall acquisition framework) that accommodates the uncertainty associated with early software estimates and allows for adjustment and refinement over time

● Establish policy, related guidance, and recommended implementation approaches for software data collection and analysis across all DoD acquisition programs

Page 16: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

16NDIA Software Industry Experts Panel14 July 2008

Software Quality AttributePriority Recommendations

● Develop engineering guidance on quantitatively identifying, predicting, evaluating, verifying, and validating Quality Attributes– Address tie-in to KPPs and TPMs– Identify methods for predicting quality attribute outcomes for the

delivered system, throughout the life cycle● Improving OSD/Service-level acquisition policy regarding Quality

Attributes– Identify benefits of addressing software quality attributes as part

of an acquisition risk reduction strategy– Address gaps in SEP, TEMP, JCIDS, DAG, RFP language– Define expectations for Quality Attribute review during

Acquisition Milestone Reviews (e.g. PDR)● Develop taxonomy of software quality attributes and how they are

related● Develop Program Manager guidance on Introduction to Software

Architectural Evaluation of Quality Attributes● Develop Collaboration site for collecting data, sharing work

products, facilitating on-going discussion

Page 17: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

17NDIA Software Industry Experts Panel14 July 2008

Software In Acquisition Spring Workshop 2008

● The purpose of this workshop was to be a “touch point” for the actions that resulted from the SSA annual software workshop in October of last year

● Review of issues and recommendations in each of the areas covered under the October workshop: – Requirements– Risk/Cost– Quality Attributes

● Three Working Groups– Software Requirements– Software Risk and Estimation– Software Quality Attributes

● Objectives For Each Working Group– Task Statements– Deliverables– Schedule

Page 18: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

18NDIA Software Industry Experts Panel14 July 2008

Outcome

● Knit Together Selected “Big Ideas” into Unified Proposed Initiative:

● Leveraging Competitive Prototyping through Acquisition Initiatives in Integrated Systems and Software Requirements, Risk, Estimation, and Quality

Page 19: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

19NDIA Software Industry Experts Panel14 July 2008

Task Definition

● Task 1: Conduct surveys and interviews of leading software professionals (government, industry, academia) to gather ideas, assess impacts, and sense expectations for Competitive Prototyping

● Task 2: Provide amplification of Competitive Prototyping memo for integrated SE/SW, [including where in the lifecycle there are opportunities for Competitive Prototyping [and how they can be contractually achieved]

● Task 3:Identify first adopters of Competitive Prototyping and facilitate and gather insights on effective usage, including collecting and analyzing data

● Task 4: Develop guidance for early selection and application of integrated SE/SW quality systems for Competitive Prototyping [for RFP-authors]

● Task 5: Develop Software Engineering Handbook for Competitive Prototyping including material explicitly targeted to different audiences (acquirer, supplier, etc.).

● Task 6: Develop training assets (materials, competencies, skill sets, etc.) that capture best-of-class ideas/practices for Competitive Prototyping

Page 20: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

20NDIA Software Industry Experts Panel14 July 2008

Competitive Prototyping Survey

● Purpose– “To gather recommendations, assess impacts, and sense

expectations for Competitive Prototyping" from key members of government, industry, and academia

● Conducted by the Center for Systems and Software Engineering (CSSE), at the University of Southern California (USC)

● The domain of interest for the survey and interviews comprise those projects that are considered large-scale "software-intensive systems" (SiS)– Systems for which software is a principal and defining component– Where software likely represents the key source of technological

and programmatic risk in its development– The software development costs of projects of interest should be

valued at $100 million, or higher

Page 21: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

21NDIA Software Industry Experts Panel14 July 2008

OUSD(AT&L)/SSE-USC/CSSE CP Workshop

● OUSD(AT&L)/SSE-USC/CSSE Workshop on Integrating Systems and Software Engineering under Competitive Prototyping with the Incremental Commitment Model– Washington, DC, July 14-17, 2008

● Discussion of the way forward from the survey through the remainder of the tasks

Page 22: NDIA Software Industry Experts Panel Paul R. Croll, Chair

Systems Engineering Division

22NDIA Software Industry Experts Panel14 July 2008

For More Information . . .

Paul R. CrollCSC17021 Combs DriveKing George, VA 22485-5824

Phone: +1 540.644.6224Fax: +1 540.663.0276e-mail: [email protected]