Analysis of Process Maturity and Productivity with SRDR Data

Analysis of Process Maturity and Productivity with SRDR Data

USC CSSE Annual Research ReviewApril 29 – May 1, 2014

Anandi Hira, Jo Ann Lane

Outline• Motivation• Explanation of the SRDR Data Repository• Data Processing• Analysis Procedure• Results of Analyses per Taxonomy and Comparison

– COCOMO II Comparisons– Application Domains– Productivity Types

• Factors to Consider in Productivity Analysis -> Future Work• Parameter Suggestions for Future Research and Analyses• Questions/Suggestions

Motivation

• Improve Productivity

• Invest resources to improve processes

• Process Certification Productivity ?

• SRDR Data

SRDR Data• Software Resources Data

Reporting (SRDR)• Quantitative data and

associated parametric project characteristics

• DoD software-intensive system development projects

• Data analysis and trends research

• Relevant Parameters provided:– Total Effort (hours)– Equivalent Total SLOC– SLOC Counting Method– CMM/CMMI Levels

• Unused Parameters– Effort distribution per

phase, Programming language(s), Personnel Experience

Data Processing

• Filtering Data– Remove outliers– Remove points

without relevant parameters

– Projects < 10 EKSLOC– Levels 2 and 4

• Normalizing Data– Logical SLOC– Counting adjustment

factors– Non-comment: 0.66 *

SLOC– Physical: 0.34 * SLOC

Analysis Procedure

1 • Combine CMM and CMMI Levels 3 and 5

2• Compare productivity to

parameter ratings of COCOMO II™

3• Categorize data by Application

Domains• ANOVA to test significance

4• Categorize data by

Productivity Types• ANOVA to test significance

COCOMO Comparisons

Size Range

Mean Level 3

MeanLevel 5

% Increase (Prod)

% Increase/ Level

COCOMO II

10-50 EKSLOC

158.47 168.88 6.57% 3.28% 5% for 30 EKSLOC

50 – 100 EKSLOC

278.19 267.09 -3.99% -1.995% 7% for 75 EKSLOC

> 100 EKSLOC

261.33 350.45 34.10% 17.05% 9.5% for 300 EKSLOC

87 EKSLOC

215.33 242.50 12.62% 6.31% 7% for 75 EKSLOC

Application Domains

Application Domains – ANOVA Test Results

Application Domain F-value

P-value

Result

Command & Control 0.564 0.464 RejectedCommunications 0.767 0.387 Rejected

Productivity Types

Productivity Types – ANOVA Test Results

Productivity Type F-value P-value ResultCommand & Control (C & C) 0.564 0.464 RejectedMission Processing (MP) 3.070 0.091 RejectedReal Time Embedded (RTE) 0.274 0.603 RejectedTelecommunications (TEL) 0.745 0.393 RejectedVehicle Payload (VP) 0.031 0.863 Rejected

Findings and Conclusions

• COCOMO Comparisons– Data broken up by sizes do not closely correspond to

parameter ratings – Average of all data corresponds to parameter rating

• Application Domains and Productivity Types– Inconsistence with regards to productivity

increase/decrease from Level 3 to Level 5– Difference in productivity ranges of Level 3 and 5 are

statistically insignificant

Future Work – Factors to Consider• Counting methods not

standard and may skew analysis

• Code reuse gains factored and normalized in data

• Analysis of trends of productivity over time

• IDPD

• Cost drivers and parameters that effect productivity not provided and random with respect to time– Staff experience– Tool support– Code reuse– Improved architecting

and risk resolution

Productivity Over Time

Future Work – Parameter Suggestions

• (Relative) Time of Project Implementation– Other data points– Adopting process

maturity levels

• Equivalent Metric for Non-Development Effort

• Equivalent Output Metric per Phase/Activity

• Rework SLOC and Effort• Volatility• Complexity

Questions and Suggestions