Process Maturity Profilencoulter/cen6070/handouts/... · Characterize the process maturity of the software community This briefing uses information from appraisals using the SW-CMM

Carnegie Mellon UniversitySoftware Engineering Institute Software CMM - CBA IPI,SPA and SCAMPI Appraisal Results

March 2004 Process Maturity Profile - Page 1© 2004 by Carnegie Mellon University

Software Engineering InstituteCarnegie Mellon University

Process Maturity Profile

Software CMM®

2003 Year End Update

March 2004

We could not produce this report without the support of the organizations and lead appraisers who reported their appraisal results to the SEISM.

Our many thanks for their continuing cooperation with our data collection and analysis efforts.

Software Engineering Measurement and Analysis Initiative

The Software Engineering Institute is a federally funded research and development center sponsored by the U.S. Department of Def ense and operated by Carnegie Mellon® University

® CMM and Carnegie Mellon are registered in the U.S. Patent and Trademark Office by Carnegie Mellon UniversitySM SEI is a service mark of Carnegie Mellon University

© 2004 by Carnegie Mellon University



OutlineIntroduction

Current Status

Community Trends

Organizational Trends

Summary

This briefing is based solely on the number of appraisals reported to the SEI. It cannot be used as a source for an exact indicator of all organizations in the world using SEI models or appraisal methods nor can it be used for certification or verification purposes.

The SEI does not certify organizations. The SEI only licenses and authorizes lead appraisers to conduct appraisals. Neither the SEI nor any other organization is a “certifying authority” of the results from an appraisal. Therefore requests to the SEI to provi de information to be used for this purpose can not be fulfilled. Information provided by the SEI, such as this briefing, is to demonstrate use of its products and services only.

Organizations performing source selection or verification should consider performing an evaluation appraisal. For more information on appraisal methods and for a directory of authorized lead appraisers who can perform them, visit: http://www.sei.cmu.edu/managing/app.directory.html



Introduction - 1: Purpose and SourceCharacterize the process maturity of the software community

This briefing uses information from appraisals using the SW-CMM in:

• CMM®-Based Appraisals for Internal Process Improvement (CBA IPIs) and

• Software Process Assessments (SPAs) and

• Standard CMMI® Appraisal Method for Process Improvement (SCAMPI), v1.1

Please visit:

http://www.sei.cmu.edu/sema/profile_about.html

for additional information or to answer questions you may have about this briefing



Introduction - 2: Briefing ContentsThis briefing includes three primary sections:

1. Current Status- Snapshot of the software community based on the most

recent appraisal, since , of reporting organizations

2. Community Trends- Global distribution of appraisals- Growth in the number of appraisals performed - Shifts in the maturity profile over time

3. Organizational Trends- Analysis of Key Process Area (KPA) satisfaction - Time to move up in maturity

1999



Section 1: Current StatusAppraisals conducted from throughand reported to the SEI by

• organizations• participating companies• projects• Non-USA organizations

Please refer to: Terms Used in this Report on page 29

1,593686

8,18554.7%

December 20031999January 2004



Based on most recent appraisal, since , of organizations reporting a maturity level rating

Maturity Profile by All Reporting Organizations

% o

f Org

aniz

atio

ns

1,5931999

12.3%

43.3%

27.4%

7.8% 9.2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Initial Repeatable Defined Managed Optimizing



Based on organizations reporting organization categories

Reporting Organization Categories

3.7%

20.5%

75.8%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Military/Federal

DoD/Fed Contractor

Commercial/In-house

% of Organizations

1,593



Based on most recent appraisal, since , of

Maturity Profile by Reporting Organization Categories

% o

f Org

aniz

atio

ns

1999 1,593

12.4%

45.0%

23.8%

8.4%10.3%9.5%

35.3%

42.9%

5.8% 6.4%

25.4%

50.8%

15.3%

6.8%

1.7%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


Commercial/In-house DoD/Fed Contractor Military/Federal

organizations reporting categories and a maturity level rating



Based on organizations reporting SIC code. For more information visit: http://www.osha.gov/oshstats/sicser.html

Business Services35.8%

Instruments And Related Products

6.1%

Transportation Equipment6.0%

Electronic & Other Electric Equipment

5.4%

Industrial Machinery And Equipment

4.9%

Retail Trade0.8%Wholesale Trade

0.7%

Transportation, Communication, Electric, Gas

and Sanitary Services6.5%

Public Administration (Including Defense)

8.0%

Engineering & Management Services10.8%

Services, Nec0.3%

Health Services0.3%

Finance, Insurance and Real Estate12.1%

Furniture And Fixtures0.1%

Food And Kindred Products0.1%

Fabricated Metal Products1.1%

Primary Metal Industries0.3%

Printing And Publishing0.5%

Chemicals And Allied Products0.3%

Organization TypeBased on Primary Standard Industrial Classification (SIC) Code

Manufacturing25.1%

Services47.4%

738



2000+1.0%

1001 to 20001.9%

501 to 10004.9%

301 to 5008.2%

26 to 5017.8%

51 to 7512.5%

76 to 1009.3%

201 to 30010.0%

101 to 20022.4%

25 or fewer12.0%

Based on organizations reporting size data

Organization SizeBased on the total number of employees within the area of the organization that was appraised

201 to 2000+26.0%

1 to 10051.3%

1,561



15.4

%

14.0

%

12.8

%

14.5

%

10.6

%

10.3

%

9.4%

5.3%

13.3

%

12.5

%

66.5

%

54.3

%

45.6

%

39.3

%

37.8

%

37.8

%

32.0

%

23.7

%

13.3

% 18.8

%

15.4

%

23.0

% 28.7

%

26.2

%

33.2

%

27.6

%

34.4

%

35.5

%

33.3

%

25.0

%

1.1%

5.4%

5.6%

11.0

%

10.3

%

9.0%

7.8%

14.5

%

10.0

%

12.5

%

1.6% 3.2%

7.2% 9.

0%

8.0%

15.4

%

16.4

% 21.1

%

30.0

%

31.3

%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

25 or fewer 26 to 50 51 to 75 76 to 100 101 to 200 201 to 300 301 to 500 501 to 1000 1001 to 2000 2000+


Maturity Profile by Organization SizeBased on the total number of employees within the area of the organization that was appraised

Based on organizations reporting size data and a maturity level rating

The number of organizations for the 2000+ category is small whic h inflates the maturity level bars. Please see page 9 and take this into account. The purpose of this chart is to indicate that all sizecategories contain most, if not all, maturity levels.

1,561

% o

f Org

aniz

atio

ns



USA and Non-USAReporting Organization Categories

5.7%

49.0%

3.7%

14.8%

26.8%

0% 10% 20% 30% 40% 50% 60%

Military/Federal

DoD/Fed Contractor

Commercial/In-house

% of Organizations

USA

Non-USA

Based on U.S. organizations and Non-USA organizations reporting organization categories872721



Based on U.S. organizations and Non-USA organizations reporting categories and a maturity level rating

Maturity Profile by All Reporting USA and Non-USA Organizations

% o

f Org

aniz

atio

ns

18.4%

48.0%

26.1%

4.2% 3.3%7.2%

39.3%

28.6%

10.8%14.1%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


USA Non-USA

872721



CBA IPIs , SPAs and SCAMPI v1.1 using the SW-CMM conducted since 1987 through and reported to the SEI by

• appraisalsCBA IPIsSPAsSCAMPI

• organizations• participating companies• reappraised organizations• projects


Section 2: Community Trends

December 2003January 2004

3,1572,663

2,401829598

13,266

48410



Countries Where Appraisals have been Performed and Reported to the SEI

Argentina Australia Austria Barbados Belgium Brazil Canada ChileChina Colombia Costa Rica Cyprus Denmark Egypt Finland FranceGermany Greece Hong Kong Hungary India Ireland Israel ItalyJapan Korea, Republic of Latvia Malaysia Mexico Netherlands New Zealand NorwayPakistan Peru Philippines Poland Portugal Puerto Rico Russia Saudi ArabiaSingapore South Africa Spain Sweden Switzerland Taiwan Thailand TurkeyUkraine United Arab Emirates United Kingdom United States Uruguay Venezuela Viet nam



Number of Appraisals and Maturity Levels Reported to the SEI by Country

The organization maturity profile, page 8, can be created for certain countries using the Interactive Maturity Profile availablein the Software Engineering Information Repository Web site. Please visit: http://seir.sei.cmu.edu. If you are not already a member of the SEIR web site, please register to join. Registration and use of the site is free.

CountryNumber of Appraisals

Maturity Level 1

Reported

Maturity Level 2

Reported

Maturity Level 3

Reported

Maturity Level 4

Reported

Maturity Level 5

Reported CountryNumber of Appraisals

Maturity Level 1

Reported

Maturity Level 2

Reported

Maturity Level 3

Reported

Maturity Level 4

Reported

Maturity Level 5

ReportedArgentina 10 or fewer Malaysia 10 or fewerAustralia 34 Yes Yes Yes No Yes Mexico 22 Yes Yes Yes Yes YesAustria 10 or fewer Netherlands 21 Yes Yes Yes No YesBarbados 10 or fewer New Zealand 10 or fewerBelgium 10 Norway 10 or fewerBrazil 23 Yes Yes Yes Yes No Pakistan 10 or fewerCanada 67 Yes Yes Yes No Yes Peru 10 or fewerChile 12 Yes Yes Yes No No Philippines 10China 152 Yes Yes Yes Yes Yes Poland 10 or fewerColombia 10 or fewer Portugal 10 or fewerCosta Rica 10 or fewer Puerto Rico 10 or fewerCyprus 10 or fewer Russia 10 or fewerDenmark 10 or fewer Saudi Arabia 10 or fewerEgypt 10 or fewer Singapore 18 Yes Yes Yes No YesFinland 10 or fewer South Africa 10 or fewerFrance 124 Yes Yes Yes Yes Yes Spain 13 No Yes Yes No YesGermany 49 Yes Yes Yes No No Sweden 10 or fewerGreece 10 or fewer Switzerland 10 or fewerHong Kong 10 or fewer Taiwan 10 or fewerHungary 10 or fewer Thailand 13 Yes Yes Yes No NoIndia 330 Yes Yes Yes Yes Yes Turkey 10 or fewerIreland 10 or fewer Ukraine 10 or fewer

Israel 30 Yes Yes Yes No NoUnited Arab Emirates 10 or fewer

Italy 30 Yes Yes Yes No Yes United Kingdom 133 Yes Yes Yes No NoJapan 113 Yes Yes Yes Yes Yes United States 1838 Yes Yes Yes Yes Yes

Korea, Republic of 60 Yes Yes Yes Yes Yes Uruguay 10 or fewerLatvia 10 or fewer Venezuela 10 or fewer

Vietnam 10 or fewer



Based on appraisals conducted through and reported to the SEI by

Number of Appraisals Reported to the SEIby Year

0

100

200

300

400

500

600

1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003

First Appraisal Reappraisal

3,157 Dec. 2003 Jan 2004

*

*

Num

ber

of A

ppra

isal

s



Based on appraisals reporting organization categories

Number of Appraisals Reported to the SEIby Organization Categories and Year

0

50

100

150

200

250

300

350

400

450

1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003

Commercial/In-house DoD/Fed Contractor Military/Federal

Num

ber

of A

ppra

isal

s

3,155



80.3

%

12.1

%

7.6%

64.4

%

21.9

%

12.0

%

1.4%

0.3

%

37.3

%

33.9

%

20.2

%

4.5%

4.2%

32.2

% 35.7

%

20.9

%

5.9%

5.3%

27.7

%

37.1

%

22.8

%

6.0%

6.4%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


1987-19911992-19961997-200120022003

Trends in the Community Maturity Profile%

of O

rgan

izat

ions

Based on a cumulative view of the most recent appraisals of organizations up through the year indicated. This accounts for the difference from the figures on page 8.

Year Orgs132634

1,6601,9962,401



Section 3: Organizational TrendsAppraisals conducted through and reported to the SEI by

• Key Process Area (KPA) profiles- satisfaction of KPAs by maturity level for organizations

appraised at levels 1 and 2

• reappraised organizations- accounting for appraisals- although some organizations conducted multiple

reappraisals, only the first and latest appraisals were used in creating the charts on pages 23 & 24


598

1,656

December 2003

1,364

January 2004



Based on CBA IPI and SCAMPI with SW-CMM appraisals reporting a maturity level rating

Key Process Area Profiles - 1Organizations Appraised at Level 1

% of Appraisals

562

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

PCM

TCM

DP

SQM

QPM

PR

IC

SPE

ISM

TP

OPD

OPF

SCM

SQA

SSM

PTO

SPP

RM

FullySatisfiedRated

Repeatable

Defined

Managed

Optimizing

Software Subcontract Management (SSM) is not applicable/not rated in many assessments. Please take that into account when interpretingits Fully Satisfied rating.



Key Process Area Profiles - 2Organizations Appraised at Level 2

Based on CBA IPI and SCAMPI With SW-CMM appraisals reporting a maturity level rating1094

% of Appraisals

Defined

Managed

Optimizing

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

PCM

TCM

DP

SQM

QPM

PR

IC

SPE

ISM

TP

OPD

OPF

FullySatisfied

Rated



Maturity Level of First and Latest Appraisal

% o

f Org

aniz

atio

ns

43.6%

38.0%

10.0%7.7%

0.7%

7.7%

27.3%

41.1%

8.0%

15.9%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


First Latest

Based on reappraised organizations using their first and latest appraisal598



No Change12.0%

Moved Down2.7%

Level 4 to 57.0%

Level 3 to 55.4%

Level 3 to 42.8%

Level 1 to 41.8%

Level 1 to 312.5%

Level 2 to 52.2%

Level 1 to 222.7%

Level 2 to 42.8%

Level 1 to 50.8%

Level 2 to 327.1%

ReappraisalsChange in Maturity Level

Based on reappraised organizations using their first and latest appraisal598

Level 1 to 1 5.7%Level 2 to 2 4.2%Level 3 to 3 1.2%Level 4 to 4 0.5%Level 5 to 5 0.5%



Time to Move Up

Number of months to move to nextmaturity level

Largest observed value that is not an outlier

75th Percentile

25th PercentileMedian

Smallest observed value that is not an outlier

Recommended time between appraisals

{

100

30

18

0

50

All (1987 to Present)

1 to 2 2 to 3 3 to 4 4 to 5188 234 49 55

1992 to Present

1 to 2 2 to 3 3 to 4 4 to 5163 222 47 55

Pre-1992

1 to 2 2 to 325 12

75

2224

39 40

Time Period of Initial Appraisal

LevelOrgs

25

19 2025

13 13



Summary of Current Status

Commercial and in-house organizations reporting most appraisals

Of U.S. organizations, the services and manufacturing industries reported most appraisals

Half of the organizations reporting size have 100 or fewer software personnel

More Non-USA appraisals reported than U.S. in this 5 year snapshot



Summary of Organizational Trends - 1

Software Quality Assurance is the least frequently satisfied level 2 KPA among organizations* appraised at level 1

Integrated Software Management is the least frequently satisfiedlevel 3 KPA among organizations* appraised at level 2

Higher maturity has been reached among those organizations reporting reappraisals

*Adjusted for number of organizations rating the KPA



Summary of Organizational Trends (continued) - 2

For organizations that began their CMM-based SPI effort in 1992 or later, the median time to move from:

• maturity level 1 to 2 is 22 months• maturity level 2 to 3 is 19 months• maturity level 3 to 4 is 25 months• maturity level 4 to 5 is 13 months



Terms Used in this ReportAppraisal - The appraisal methods used in this report are the Software Process

Assessment (SPA), CMM-Based Appraisal for both Internal Process Improvement (CBA IPI) and Standard CMMI® Appraisal Method for Process Improvement (SCAMPI). We do request and receive other Software CMM-based appraisals such as Software Capability Evaluations (SCE) and Interim Profiles. As our sampling size of these other methods increase, they will be reported here.

Company - Parent of the appraised entityA company can be a commercial or non-commercial firm, for-profit or not for-profit business, a research and development unit, a higher education unit, a government agency, or branch of service, etc.

Organization – a.k.a. Appraised entity The organization unit to which the appraisal results apply. An appraised entity can be the entire company, a selected business unit, units supporting a particular product line or service, etc.

Non-USA - Appraised entity whose geographic location is not within the United States. The parent of the appraised entity may or may not be based within the United States.



Report your Appraisal Results to the SEIThe briefing is only possible due to the cooperation of organizations and individuals sending in their appraisal results to the SEI

In order to provide this information and service in the future, it will depend on this continued cooperation

Please visit:

http://www.sei.cmu.edu/sema/report.html

for forms, information, and instructions on how to report appraisals to the SEI



Contact InformationPlease visit:

http://www.sei.cmu.edu/sema/profile_about.html

and review the information provided before contacting:

SEI Customer Relations (412) 268-5800SEI FAX number (412) 268-5758

Internet [email protected]

Mailing AddressCustomer RelationsSoftware Engineering InstituteCarnegie Mellon UniversityPittsburgh, PA 15213-3890

Documents

Process Maturity Profilencoulter/cen6070/handouts/... · Characterize the process maturity of the software community This briefing uses information from appraisals using the SW-CMM