Process Maturity Profile - DTIC · Process Maturity Profile: Software CMM 2005 Mid-Year Update 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER ... Based on U.S. organizations

Software Engineering InstituteCarnegie Mellon

Software CMM - CBA IPI, SPA and SCAMPI Appraisal Results

September 2005 Process Maturity Profile - Page 1© 2005 by Carnegie Mellon University


Process Maturity Profile

Software CMM®

2005 Mid-Year Update

September 2005

We could not produce this report without the support of the organizations and lead appraisers who reported their appraisal results to the SEISM.

Our many thanks for their continuing cooperation with our data collection and analysis efforts.

CMMI Appraisal Program

The Software Engineering Institute is a federally funded research and development center sponsored by the U.S. Department of Def ense and operated by Carnegie Mellon® University

® CMM and Carnegie Mellon are registered in the U.S. Patent and Trademark Office by Carnegie Mellon UniversitySM SEI is a service mark of Carnegie Mellon University

© 2005 by Carnegie Mellon University

Report Documentation Page Form ApprovedOMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.

1. REPORT DATE SEP 2005 2. REPORT TYPE

3. DATES COVERED 00-00-2005 to 00-00-2005

4. TITLE AND SUBTITLE Process Maturity Profile: Software CMM 2005 Mid-Year Update

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) 5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Carnegie Mellon University,Software Engineering Institute,Pittsburgh,PA,15213

8. PERFORMING ORGANIZATIONREPORT NUMBER

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)

11. SPONSOR/MONITOR’S REPORT NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited

13. SUPPLEMENTARY NOTES

14. ABSTRACT

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT Same as

Report (SAR)

18. NUMBEROF PAGES

31

19a. NAME OFRESPONSIBLE PERSON

a. REPORT unclassified

b. ABSTRACT unclassified

c. THIS PAGE unclassified

Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18




OutlineIntroduction

Current Status

Community Trends

Organizational Trends

Summary

This briefing is based solely on the number of appraisals reported to the SEI. It cannot be used as a source for an exact indicator of all organizations in the world using SEI models or appraisal methods nor can it be used for certification or verification purposes.

The SEI does not certify organizations. The SEI only licenses SEI partners and SEI authorized lead assessors to conduct appraisals. Neither the SEI nor any other organization is a “certifying authority” of the results from an appraisal. Therefore requests to the SEI to provide information to be used for this purpose can not be fulfi lled. Information provided by the SEI, such as this briefing, is to demonstrate use of its products and services only.

Organizations performing source selection or verification should consider performing an evaluation appraisal. For more information on appraisal methods and for a directory of authorized lead appraisers who can perform them, visit: http://www.sei.cmu.edu/appraisal-program/




Introduction - 1: Purpose and SourceCharacterize the process maturity of the software community

This briefing uses information from appraisals using the SW-CMM in:

• CMM®-Based Appraisals for Internal Process Improvement (CBA IPIs) and

• Software Process Assessments (SPAs) and

• Standard CMMI® Appraisal Method for Process Improvement (SCAMPISM), v1.1

Please visit:

http://www.sei.cmu.edu/appraisal-program/profile/about.html

for additional information or to answer questions you may have about this briefing




This briefing includes three primary sections:

1. Current Status- Snapshot of the software community based on the most

recent appraisal, since 2001, of reporting organizations

2. Community Trends- Global distribution of appraisals- Growth in the number of appraisals performed - Shifts in the maturity profile over time

3. Organizational Trends- Analysis of Key Process Area (KPA) satisfaction - Time to move up in maturity based on change in maturity

level and time between appraisals

Introduction - 2: Briefing Contents




Section 1: Current StatusAppraisals conducted from throughand reported to the SEI by

• organizations• participating companies• projects• Non-USA organizations

Please refer to: Terms Used in this Report on page 29

1,613860

7,99165.3%

June 2005July 20052001




Based on most recent appraisal, since , of organizations reporting a maturity level rating

Maturity Profile by All Reporting Organizations

% o

f Org

aniz

atio

ns

1,6132001

6.3%

41.7%

34.2%

8.1%9.8%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Initial Repeatable Defined Managed Optimizing




Based on organizations reporting organization categories

Reporting Organization Categories

3.0%

16.8%

80.2%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Military/Federal

DoD/Fed Contractor

Commercial/In-house

% of Organizations

1,613




Based on most recent appraisal, since , of

Maturity Profile by Reporting Organization Categories

% o

f Org

aniz

atio

ns

2001 1,613

6.1%

41.9%

32.8%

8.4%10.8%

6.3%

36.5%

44.6%

6.3% 6.3%

10.2%

63.3%

14.3%

8.2%

4.1%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Commercial/In-house DoD/Fed Contractor Military/Federal

organizations reporting categories and a maturity level rating




Based on organizations reporting SIC code. For more information visit: http://www.osha.gov/oshstats/sicser.html

Organization TypeBased on Primary Standard Industrial Classification (SIC) Code

574

Chemicals And Allied Products0.5%

Fabricated Metal Products0.5%

Printing And Publishing0.3%

Primary Metal Industries0.7%

Rubber And Misc Plastics Products

0.2%

Furniture And Fixtures0.2%

Finance, Insurance and Real Estate15.0%

Health Services0.2%

Other Services0.2%

Amusement & Recreation Services

0.3%

Public Administration (Including Defense)

8.5%

Transportation, Communication, Electric, Gas

and Sanitary Services3.7%

Wholesale Trade0.5%

Retail Trade0.9%

Industrial Machinery And Equipment

4.0%

Electronic & Other Electric Equipment

4.7%

Instruments And Related Products

5.9%

Transportation Equipment6.4%

Engineering & Management Services10.1%

Business Services37.1%

Manufacturing23.5%

Services47.9%




2000+1.1%

1001 to 20001.7%501 to 1000

3.8%

301 to 5007.0%

26 to 5018.5%

51 to 7514.3%

76 to 1009.8%

201 to 3008.9%

101 to 20021.6%

25 or fewer13.3%

Based on organizations reporting size data

Organization SizeBased on the total number of employees within the area of the organization that was appraised

201 to 2000+22.5%

1 to 10055.9%

1,585




10.0

%

5.5% 8.

8%

5.2%

4.7% 5.7%

6.3% 7.4%

5.9%

66.4

%

52.2

%

42.3

%

36.1

%

36.3

%

30.5

%

22.5

%

18.0

% 22.2

%

23.5

%

20.4

%

32.8

%

32.6

% 38.7

%

36.8

%

37.6

% 43.2

%

44.3

%

51.9

%

29.4

%

0.9%

6.5% 7.0% 10

.3%

12.9

%

8.5%

7.2%

13.1

%

3.7%

11.8

%

2.4% 3.1%

9.3%

9.7%

9.4%

17.7

%

20.7

% 24.6

%

14.8

%

29.4

%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

25 or fewer 26 to 50 51 to 75 76 to 100 101 to 200 201 to 300 301 to 500 501 to 1000 1001 to 2000 2000+


Maturity Profile by Organization SizeBased on the total number of employees within the area of the organization that was appraised

Based on organizations reporting size data and a maturity level rating

The number of organizations for the 2000+ category is small whic h inflates the maturity level bars. Please see page 9 and take this into account. The purpose of this chart is to indicate that all sizecategories contain most, if not all, maturity levels.

1,585

% o

f Org

aniz

atio

ns




USA and Non-USAReporting Organization Categories

5.0%

60.3%

3.0%

11.8%

19.8%

0% 10% 20% 30% 40% 50% 60% 70%

Military/Federal

DoD/Fed Contractor

Commercial/In-house

% of Organizations

USA

Non-USA

Based on U.S. organizations and Non-USA organizations reporting organization categories1053560




Based on U.S. organizations and Non-USA organizations reporting categories and a maturity level rating

Maturity Profile by All Reporting USA and Non-USA Organizations

% o

f Org

aniz

atio

ns

11.1%

51.3%

29.6%

4.5% 3.6%3.7%

36.6% 36.7%

10.0%13.1%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


USA Non-USA

1053560




CBA IPIs , SPAs and SCAMPI v1.1 using the SW-CMM conducted since 1987 through and reported to the SEI by

• appraisalsCBA IPIsSPAsSCAMPIs

• organizations• participating companies• reappraised organizations• projects


Section 2: Community Trends

June 2005July 2005

3,7283,151

2,8591,107

68515,634

48493




Countries Where Appraisals have been Performed and Reported to the SEI

Argentina Australia Austria Barbados Belgium Brazil CanadaChile China Colombia Costa Rica Cyprus Czech Republic DenmarkEgypt Finland France Germany Greece Hong Kong HungaryIndia Indonesia Ireland Israel Italy Japan Korea, Republic ofLatvia Lithuania Malaysia Mauritius Mexico Morocco NetherlandsNew Zealand Norway Pakistan Peru Philippines Poland PortugalPuerto Rico Russia Saudi Arabia Singapore South Africa Spain Sri LankaSweden Switzerland Taiwan Thailand Turkey Ukraine United Arab EmiratesUnited Kingdom United States Uruguay Venezuela Vietnam




Number of Appraisals and Maturity Levels Reported to the SEI by Country

The organization maturity profile, page 6, can be created for certain countries using the Interactive Maturity Profile availablein the Software Engineering Information Repository Web site. Please visit: http://seir.sei.cmu.edu. If you are not already a member of the SEIR web site, please register to join. Registration and use of the site are free.

CountryNumber of Appraisals

Maturity Level 1

Reported

Maturity Level 2

Reported

Maturity Level 3

Reported

Maturity Level 4

Reported

Maturity Level 5

Reported CountryNumber of Appraisals

Maturity Level 1

Reported

Maturity Level 2

Reported

Maturity Level 3

Reported

Maturity Level 4

Reported

Maturity Level 5

Reported

Argentina 10 or fewer Mauritius 10 or fewer

Australia 36 Yes Yes Yes No Yes Mexico 30 Yes Yes Yes Yes Yes

Austria 10 or fewer Morocco 10 or fewer

Barbados 10 or fewer Netherlands 25 Yes Yes Yes No Yes

Belgium 14 Yes Yes Yes No No New Zealand 10 or fewer

Brazil 29 Yes Yes Yes Yes Yes Norway 10 or fewer

Canada 81 Yes Yes Yes Yes Yes Pakistan 10 or fewer

Chile 20 Yes Yes Yes No Yes Peru 10 or fewer

China 284 Yes Yes Yes Yes Yes Philippines 12 No Yes Yes No Yes

Colombia 10 or fewer Poland 10 or fewer

Costa Rica 10 or fewer Portugal 10 or fewer

Cyprus 10 or fewer Puerto Rico 10 or fewer

Czech Republic 10 or fewer Russia 10 or fewerDenmark 10 or fewer Saudi Arabia 10 or fewer

Egypt 10 or fewer Singapore 23 Yes Yes Yes No Yes

Finland 10 or fewer South Africa 10 or fewer

France 146 Yes Yes Yes Yes Yes Spain 26 No Yes Yes No Yes

Germany 70 Yes Yes Yes No No Sri Lanka 10 or fewer

Greece 10 or fewer Sweden 10 or fewer

Hong Kong 10 Switzerland 10 or fewer

Hungary 10 or fewer Taiwan 10 or fewer

India 399 Yes Yes Yes Yes Yes Thailand 17 Yes Yes Yes No No

Indonesia 10 or fewer Turkey 10 or fewer

Ireland 11 Yes Yes Yes No No Ukraine 10 or fewer

Israel 31 Yes Yes Yes No NoUnited Arab Emirates 10 or fewer

Italy 36 Yes Yes Yes No Yes United Kingdom 140 Yes Yes Yes No No

Japan 161 Yes Yes Yes Yes Yes United States 1982 Yes Yes Yes Yes Yes

Korea, Republic of 77 Yes Yes Yes Yes Yes Uruguay 10 or fewer

Latvia 10 or fewer Venezuela 10 or fewer

Lithuania 10 or fewer Vietnam 10 or fewer

Malaysia 10 or fewer




Based on appraisals conducted through and reported to the SEI by

Number of SW-CMM Appraisals Reported to the SEI by Year

0

100

200

300

400

500

600

1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

First Appraisal Reappraisal

3,728 June 2005 July 2005

Num

ber

of A

ppra

isal

s




Based on appraisals reporting organization categories

Number of SW-CMM Appraisals Reported to the SEI by Organization Categories and Year

0

50

100

150

200

250

300

350

400

450

500

1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Commercial/In-house DoD/Fed Contractor Military/Federal

Num

ber

of A

ppra

isal

s

3,726




80.3

%

12.1

%

7.6%

64.4

%

21.9

%

12.0

%

1.4%

0.3

%

37.3

%

33.9

%

20.2

%

4.5%

4.2%

32.1

% 35.8

%

20.9

%

5.9%

5.2%

27.5

%

37.1

%

23.1

%

5.9% 6.4%

24.3

%

37.0

%

25.7

%

6.2% 6.8%

23.3

%

37.1

%

26.5

%

6.1% 7.0%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


1987-19911992-19961997-20012002200320042005

Trends in the Community Maturity Profile%

of O

rgan

izat

ions

Based on a cumulative view of the most recent appraisals of organizations up through the year indicated. This accounts for the difference from the figures on page 5.

Year Orgs132635

1,6601,9972,4202,7602,859




Section 3: Organizational TrendsAppraisals conducted through and reported to the SEI by

• Key Process Area (KPA) profiles- satisfaction of KPAs by maturity level for organizations

appraised at levels 1 and 2

• reappraised organizations- accounting for appraisals- although some organizations conducted multiple

reappraisals, only the first and latest appraisals were used in creating the charts on pages 23 & 24


685

1,895

June 2005

1,564

July 2005




Based on CBA IPI and SCAMPI with SW-CMM appraisals reporting a maturity level rating

Key Process Area Profiles - 1Organizations Appraised at Level 1

% of Appraisals

578

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

PCM

TCM

DP

SQM

QPM

PR

IC

SPE

ISM

TP

OPD

OPF

SCM

SQA

SSM

PTO

SPP

RM

FullySatisfiedRated

Repeatable

Defined

Managed

Optimizing

Software Subcontract Management (SSM) is not applicable/not rated in many assessments. Please take that into account when interpretingits Fully Satisfied rating.




Key Process Area Profiles - 2Organizations Appraised at Level 2

Based on CBA IPI and SCAMPI With SW-CMM appraisals reporting a maturity level rating1317

% of Appraisals

Defined

Managed

Optimizing

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

PCM

TCM

DP

SQM

QPM

PR

IC

SPE

ISM

TP

OPD

OPF

FullySatisfied

Rated




Maturity Level of First and Latest Appraisal

% o

f Org

aniz

atio

ns

40.4% 39.6%

11.2%7.9%

0.9%

7.3%

25.8%

41.3%

9.2%

16.4%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


First Latest

Based on reappraised organizations using their first and latest appraisal685




No Change12.6%

Moved Down2.9%

Level 4 to 57.0%

Level 3 to 55.7%

Level 3 to 43.1%

Level 1 to 42.0%

Level 1 to 311.4%

Level 2 to 52.2%

Level 1 to 221.0%

Level 2 to 43.6%

Level 1 to 50.7%

Level 2 to 327.7%

ReappraisalsChange in Maturity Level

Based on reappraised organizations using their first and latest appraisal685

Level 1 to 1 5.3%Level 2 to 2 4.4%Level 3 to 3 1.8%Level 4 to 4 0.4%Level 5 to 5 0.7%




Time to Move Up

Number of months to move to nextmaturity level

Largest observed value that is not an outlier

75th Percentile

25th PercentileMedian

Smallest observed value that is not an outlier

Recommended time between appraisals {

100

30

18

0

50

All (1987 to Present)

1 to 2 2 to 3 3 to 4 4 to 5200 274 65 64

1992 to Present

1 to 2 2 to 3 3 to 4 4 to 5175 262 63 64

Pre-1992

1 to 2 2 to 325 12

75

1923

39 40

Time Period of Initial Appraisal

LevelOrgs

2520 20

25

13 13




Summary of Current Status

Commercial and in-house organizations reporting most appraisals

Of U.S. organizations, the services and manufacturing industries reported most appraisals

Half of the organizations reporting size have 100 or fewer software personnel

More Non-USA appraisals reported than U.S. in this 5 year snapshot




Summary of Organizational Trends - 1

The total number appraisals for 2004 is less than 2003. It is the first time decrease since 1996

Software Subcontract Management is the least frequently satisfied level 2 KPA among organizations* appraised at level 1

Integrated Software Management is the least frequently satisfiedlevel 3 KPA among organizations* appraised at level 2

Higher maturity has been reached among those organizations reporting reappraisals




Summary of Organizational Trends (continued) - 2

For organizations that began their CMM-based SPI effort in 1992 or later, the median time to move from:

• maturity level 1 to 2 is 19 months• maturity level 2 to 3 is 20 months• maturity level 3 to 4 is 25 months• maturity level 4 to 5 is 13 months

Note: The median time to move from maturity level 1 to 2 dropped from 20 month to 19 month




Terms Used in this ReportAppraisal - The appraisal methods used in this report are the Software Process

Assessment (SPA), CMM-Based Appraisal for both Internal Process Improvement (CBA IPI) and Standard CMMI® Appraisal Method for Process Improvement (SCAMPI). We do request and receive other Software CMM-based appraisals such as Software Capability Evaluations (SCE) and Interim Profiles. As our sampling size of these other methods increase, they will be reported here.

Company - Parent of the appraised entityA company can be a commercial or non-commercial firm, for-profit or not for-profit business, a research and development unit, a higher education unit, a government agency, or branch of service, etc.

Organization - a.k.a. Appraised entity The organization unit to which the appraisal results apply. An appraised entity can be the entire company, a selected business unit, units supporting a particular product line or service, etc.

Non-USA - Appraised entity whose geographic location is not within the United States. The parent of the appraised entity may or may not be based within the United States.




Report your Appraisal Results to the SEIThe briefing is only possible due to the cooperation of organizations and individuals sending in their appraisal results to the SEI

In order to provide this information and service in the future, it will depend on this continued cooperation

Please visit:


for forms, information, and instructions on how to report appraisals to the SEI




Contact InformationPlease visit:


and review the information provided before contacting:

SEI Customer Relations (412) 268-5800SEI FAX number (412) 268-5758

Internet [email protected]

Mailing AddressCustomer RelationsSoftware Engineering InstituteCarnegie Mellon UniversityPittsburgh, PA 15213-3890

Documents

Process Maturity Profile - DTIC · Process Maturity Profile: Software CMM 2005 Mid-Year Update 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER ... Based on U.S. organizations