MetricsMetrics

A Good Manager A Good Manager MeasuresMeasures

measurementmeasurement

What do weWhat do weuse as ause as abasis?basis? • • size?size? • • function?function?

project metricsproject metrics

process metricsprocess metricsprocessprocess

productproduct

product metricsproduct metrics

Why Do We Measure?Why Do We Measure?

assess the status of an ongoing project

track potential risks

uncover problem areas before they go “critical,”

adjust work flow or tasks,

evaluate the project team’s ability to control quality of software work products.

Measures, Metrics and Measures, Metrics and IndicatorsIndicators

A A measuremeasure provides a quantitative indication of provides a quantitative indication of the extent, amount, dimension, capacity, or size the extent, amount, dimension, capacity, or size of some attribute of a product or processof some attribute of a product or process

The IEEE glossary defines aThe IEEE glossary defines a metricmetric as “a as “a quantitative measure of the degree to which a quantitative measure of the degree to which a system, component, or process possesses a system, component, or process possesses a given attribute.”given attribute.”

An An indicatorindicator is a metric or combination of metrics is a metric or combination of metrics that provide insight into the software process, a that provide insight into the software process, a software project, or the product itself software project, or the product itself

Process MeasurementWe measure the efficacy of a software process We measure the efficacy of a software process indirectly. indirectly.

That is, we derive a set of metrics based on the outcomes that That is, we derive a set of metrics based on the outcomes that can be derived from the process. can be derived from the process.

Outcomes include Outcomes include measures of errors uncovered before release of the softwaremeasures of errors uncovered before release of the softwaredefects delivered to and reported by end-usersdefects delivered to and reported by end-userswork products delivered (productivity)work products delivered (productivity)human effort expendedhuman effort expendedcalendar time expendedcalendar time expendedschedule conformanceschedule conformance other measures.other measures.

We also derive process metrics by measuring the We also derive process metrics by measuring the characteristics of specific software engineering tasks. characteristics of specific software engineering tasks.

Process Metrics Guidelines

Use common sense and organizational sensitivity when Use common sense and organizational sensitivity when interpreting metrics data.interpreting metrics data.Provide regular feedback to the individuals and teams who Provide regular feedback to the individuals and teams who collect measures and metrics.collect measures and metrics.Don’t use metrics to appraise individuals.Don’t use metrics to appraise individuals.Work with practitioners and teams to set clear goals and Work with practitioners and teams to set clear goals and metrics that will be used to achieve them.metrics that will be used to achieve them.Never use metrics to threaten individuals or teams.Never use metrics to threaten individuals or teams.Metrics data that indicate a problem area should not be Metrics data that indicate a problem area should not be considered “negative.” These data are merely an indicator considered “negative.” These data are merely an indicator for process improvement.for process improvement.Don’t obsess on a single metric to the exclusion of other Don’t obsess on a single metric to the exclusion of other important metrics.important metrics.

Software Process Improvement

SPI

Process model

Improvement goals

Process metrics

Process improvementrecommendations

Process MetricsProcess Metrics

Quality-relatedQuality-related focus on quality of work products and deliverablesfocus on quality of work products and deliverables

Productivity-relatedProductivity-related Production of work-products related to effort expendedProduction of work-products related to effort expended

Statistical SQA dataStatistical SQA data error categorization & analysiserror categorization & analysis

Defect removal efficiencyDefect removal efficiency propagation of errors from process activity to activitypropagation of errors from process activity to activity

Reuse dataReuse data The number of components produced and their degree of The number of components produced and their degree of

reusabilityreusability

Project Metrics

used to minimize the development schedule by making the used to minimize the development schedule by making the adjustments necessary to avoid delays and mitigate adjustments necessary to avoid delays and mitigate potential problems and riskspotential problems and risks

used to assess product quality on an ongoing basis and, used to assess product quality on an ongoing basis and, when necessary, modify the technical approach to improve when necessary, modify the technical approach to improve quality.quality.every project should measure:every project should measure:

inputsinputs—measures of the resources (e.g., people, tools) required to —measures of the resources (e.g., people, tools) required to do the work.do the work.

outputsoutputs—measures of the deliverables or work products created —measures of the deliverables or work products created during the software engineering process.during the software engineering process.

resultsresults—measures that indicate the effectiveness of the —measures that indicate the effectiveness of the deliverables.deliverables.

Typical Project MetricsTypical Project Metrics

Effort/time per software engineering taskEffort/time per software engineering task

Errors uncovered per review hourErrors uncovered per review hour

Scheduled vs. actual milestone datesScheduled vs. actual milestone dates

Changes (number) and their characteristicsChanges (number) and their characteristics

Distribution of effort on software Distribution of effort on software engineering tasksengineering tasks

Typical Size-Oriented Typical Size-Oriented MetricsMetrics

errors per KLOC (thousand lines of code)errors per KLOC (thousand lines of code)defects per KLOCdefects per KLOC$ per LOC$ per LOCpages of documentation per KLOCpages of documentation per KLOCerrors per person-montherrors per person-monthErrors per review hourErrors per review hourLOC per person-monthLOC per person-month$ per page of documentation$ per page of documentation

Typical Function-Oriented Typical Function-Oriented MetricsMetrics

errors per FP (thousand lines of code)errors per FP (thousand lines of code)

defects per FPdefects per FP

$ per FP$ per FP

pages of documentation per FPpages of documentation per FP

FP per person-monthFP per person-month

Comparing LOC and FPComparing LOC and FPProgramming LOC per Function point

Language avg. median low high

Ada 154 - 104 205Assembler 337 315 91 694C 162 109 33 704C++ 66 53 29 178

COBOL 77 77 14 400Java 63 53 77 -JavaScript 58 63 42 75Perl 60 - - -PL/1 78 67 22 263Powerbuilder 32 31 11 105SAS 40 41 33 49Smalltalk 26 19 10 55SQL 40 37 7 110Visual Basic 47 42 16 158

Representative values developed by QSM

These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 14

Software Project Software Project PlanningPlanning

The overall goal of project planning is to The overall goal of project planning is to establish a pragmatic strategy for establish a pragmatic strategy for controlling, tracking, and monitoring a controlling, tracking, and monitoring a complex technical project.complex technical project.

Why?Why?

So the end result gets done on So the end result gets done on time, with quality!time, with quality!

These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 6/e and are provided with permission by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005 15

Steps in EstimatingSteps in Estimating

Estimate cost and effortEstimate cost and effort Decompose the problemDecompose the problem Develop two or more estimates using size, Develop two or more estimates using size,

function points, process tasks or use-casesfunction points, process tasks or use-cases Reconcile the estimatesReconcile the estimates

16

EstimationEstimation

Estimation of resources, cost, and schedule Estimation of resources, cost, and schedule for a software engineering effort requires for a software engineering effort requires experienceexperience access to good historical information (metricsaccess to good historical information (metrics the courage to commit to quantitative the courage to commit to quantitative

predictions when qualitative information is all predictions when qualitative information is all that existsthat exists

Estimation carries inherent risk and this risk Estimation carries inherent risk and this risk leads to uncertaintyleads to uncertainty

17

Project EstimationProject Estimation

Project scope must be Project scope must be understoodunderstood

Elaboration (decomposition) Elaboration (decomposition) is necessaryis necessary

Historical metrics are very Historical metrics are very helpfulhelpful

At least two different At least two different techniques should be usedtechniques should be used

Uncertainty is inherent in the Uncertainty is inherent in the processprocess

18

Estimation TechniquesEstimation Techniques

Past (similar) project experiencePast (similar) project experience

Conventional estimation Conventional estimation techniquestechniques task breakdown and effort task breakdown and effort

estimatesestimates size (e.g., FP) estimatessize (e.g., FP) estimates

Empirical modelsEmpirical models

Automated toolsAutomated tools

19

Estimation AccuracyEstimation Accuracy

Predicated on …Predicated on … the degree to which the planner has properly estimated the degree to which the planner has properly estimated

the size of the product to be builtthe size of the product to be built the the ability to translateability to translate the size estimate into human the size estimate into human

effort, calendar time, and dollars (a function of the effort, calendar time, and dollars (a function of the availability of reliable software metrics from past availability of reliable software metrics from past projects)projects)

the degree to which the project plan reflects the the degree to which the project plan reflects the abilities abilities of the software teamof the software team

the the stability of product requirementsstability of product requirements and the and the environment that supports the software engineering environment that supports the software engineering effort.effort.

20

Functional DecompositionFunctional Decomposition

functional functional decompositiondecomposition

StatementStatementofof

ScopeScopePerform a Perform a

Grammatical “parse”Grammatical “parse”

21

Conventional Methods:Conventional Methods:LOC/FP ApproachLOC/FP Approach

compute LOC (lines of code)/FP compute LOC (lines of code)/FP (function points) using estimates of (function points) using estimates of information domain valuesinformation domain values

use historical data to build estimates use historical data to build estimates for the projectfor the project

22

Example: LOC Example: LOC ApproachApproach

Average productivity for systems of this type = 620 Average productivity for systems of this type = 620 LOC/pm. LOC/pm.

Burdened labor rate =$8000 per month, the cost per Burdened labor rate =$8000 per month, the cost per line of code is approximately $13. line of code is approximately $13.

Based on the LOC estimate and the historical Based on the LOC estimate and the historical productivity data, the total estimated project cost is productivity data, the total estimated project cost is $431,600 and the estimated effort is 54 person-$431,600 and the estimated effort is 54 person-months.months.

Function PointsFunction Points

Number of external inputs – from user or another Number of external inputs – from user or another applicationapplication

Number of external outputsNumber of external outputs

Number of external inquiries – request from user that Number of external inquiries – request from user that generates an on-line outputgenerates an on-line output

Number of internal logical files (maintained by system)Number of internal logical files (maintained by system)

Number of external interface files (provides data but not Number of external interface files (provides data but not maintained by system)maintained by system)

FP = count-total X [0.65 + 0.01 X FP = count-total X [0.65 + 0.01 X SumSum (F (Fii)])]

23

Count-TotalCount-Total

Count-Total = sum(number X weight)Count-Total = sum(number X weight)

Where weights are:Where weights are:

Simple Simple Average Average ComplexComplex

External InpExternal Inp 33 44 66

External OutExternal Out 44 55 77

External InqExternal Inq 33 44 66

Internal FilesInternal Files 77 1010 1515

External FilesExternal Files 55 77 1010

24

Calculation of sum(FCalculation of sum(F ii))Total of 1-5 rating of following 14 questions:Total of 1-5 rating of following 14 questions:

Does the system require reliable back-up/recovery?Does the system require reliable back-up/recovery? Are specialized data communications required?Are specialized data communications required? Are there distributed processing functions?Are there distributed processing functions? Is performance critical?Is performance critical? Will run in heavily utilized operating environment?Will run in heavily utilized operating environment? On-line data entry required?On-line data entry required? For on-line data entry, will it require multiple screens?For on-line data entry, will it require multiple screens? Are ILF’s updated on-line?Are ILF’s updated on-line? Are input, output, files, or inquiries complex?Are input, output, files, or inquiries complex? Is the internal processing complex?Is the internal processing complex? Is the code designed to be reusable?Is the code designed to be reusable? Are conversion and installation included?Are conversion and installation included? Is the system designed for installation in different organizations?Is the system designed for installation in different organizations? Is the application designed to facilitate change and Is the application designed to facilitate change and ease of use?ease of use?

25

26

Example: FP ApproachExample: FP Approach

The estimated number of FP is derived:The estimated number of FP is derived:FPFPestimatedestimated = count-total X [0.65 + 0.01 X = count-total X [0.65 + 0.01 X SumSum (F (Fii)])]

FPFPestimatedestimated = 375 = 375

organizational average productivity = 6.5 FP/pm. organizational average productivity = 6.5 FP/pm. burdened labor rate = $8000 per month, the cost per FP is burdened labor rate = $8000 per month, the cost per FP is approximately $1230. approximately $1230. Based on the FP estimate and the historical productivity data, Based on the FP estimate and the historical productivity data, the total the total estimated project cost is $461,000 and the estimated effort is 58 person-estimated project cost is $461,000 and the estimated effort is 58 person-monthsmonths..

Software SizingSoftware Sizing

Software Project EstimationSoftware Project Estimation

Software Sizing:Software Sizing: Fuzzy logic sizing:Fuzzy logic sizing: establish establish

software’s magnitudesoftware’s magnitude Function point sizingFunction point sizing Standard component sizingStandard component sizing

# of generic components# of generic components

Historical project sizeHistorical project size Change sizingChange sizing

# of changes required# of changes required

Type of change required e.g. Type of change required e.g. add, change, deleteadd, change, delete

Software Project EstimationSoftware Project Estimation

Expected value of Size can be measured as:Expected value of Size can be measured as: S=(SS=(Soptopt+4S+4Smm+S+Spesspess)/6)/6 Where Where SSopt = opt = Optimistic Optimistic

SSmm =Most Likely =Most Likely

SSpesspess=Pessimistic=Pessimistic

LOC Based EstimationLOC Based Estimation

You are required to establish estimates for Virtual Class You are required to establish estimates for Virtual Class room System. For first increment you have to provide room System. For first increment you have to provide automation of course registration, Attendance, lectures.automation of course registration, Attendance, lectures.LOC estimates for each Functionality are provided:LOC estimates for each Functionality are provided:

User screens 3000User screens 3000 Course registration 1550Course registration 1550 Attendance 1800Attendance 1800 Lectures 3000Lectures 3000 Student query 800Student query 800

LOC Based EstimationLOC Based Estimation

Suppose productivity of your Organization is Suppose productivity of your Organization is 500 LOC/pm500 LOC/pm

Labor rate is 8000Rs/pmLabor rate is 8000Rs/pm

Calculate Effort and Total cost of this Calculate Effort and Total cost of this increment.increment.

Total LOCTotal LOC

User screens 3000User screens 3000 Course registration 1550Course registration 1550 Attendance 1800Attendance 1800 Lectures 3000Lectures 3000 Student query 800Student query 800

= Total LOC=10150 = Total LOC=10150 LOCLOC

EffortEffort

LOC/ ProductivityLOC/ Productivity

=10150/500=10150/500

=20.3 pm or 21 pm=20.3 pm or 21 pm

Total CostTotal Cost

Total cost=Rate * pmTotal cost=Rate * pm

= 8000 * 21= 8000 * 21

= 168000Rs.= 168000Rs.

Cost per LOCCost per LOC=168000/10150=168000/10150

=16.55 Rs.=16.55 Rs.

FP based EstimationFP based Estimation

For above mentioned systemFor above mentioned system Student can be registeredStudent can be registered Courses can be added/updated.Courses can be added/updated. Student can view coursesStudent can view courses Student can inquire his/her registration statusStudent can inquire his/her registration status

Count Count Simple Simple Average ComplexAverage Complex

External InpExternal Inp 3 *3 * 33 4 4 6 = 9 6 = 9

External OutExternal Out 4 *4 * 44 5 5 7 =16 7 =16

External InqExternal Inq 1 *1 * 33 4 6 = 3 4 6 = 3

Internal FilesInternal Files 4 *4 * 77 10 15 =28 10 15 =28

External FilesExternal Files0 *0 * 55 7 7 10 = 0 10 = 0

Count Total= Count Total= =56 =56

Calculation of sum(FCalculation of sum(F ii))Total of 0-5 rating of following 14 questions:Total of 0-5 rating of following 14 questions:

Does the system require reliable back-up/recovery? 3Does the system require reliable back-up/recovery? 3 Are specialized data communications required? 2Are specialized data communications required? 2 Are there distributed processing functions? 0Are there distributed processing functions? 0 Is performance critical? 2Is performance critical? 2 Will run in heavily utilized operating environment? 1 Will run in heavily utilized operating environment? 1 On-line data entry required? 4On-line data entry required? 4 For on-line data entry, will it require multiple screens? 5For on-line data entry, will it require multiple screens? 5 Are ILF’s updated on-line? 5Are ILF’s updated on-line? 5 Are input, output, files, or inquiries complex? 2Are input, output, files, or inquiries complex? 2 Is the internal processing complex? 3Is the internal processing complex? 3 Is the code designed to be reusable? 0Is the code designed to be reusable? 0 Are conversion and installation included? 1Are conversion and installation included? 1 Is the system designed for installation in different organizations? 0Is the system designed for installation in different organizations? 0 Is the application designed to facilitate change and Is the application designed to facilitate change and ease of use? 1ease of use? 1

37

The estimated number of FP is derived:The estimated number of FP is derived:FPestimated = count-total X [0.65 + 0.01 X Sum FPestimated = count-total X [0.65 + 0.01 X Sum

(Fi)](Fi)]FPestimated = 56 * (0.65+.01*29)FPestimated = 56 * (0.65+.01*29)

=52.64 or 53=52.64 or 53If organizational average productivity = 5 FP/pm. If organizational average productivity = 5 FP/pm. labor rate = 8000Rs. per month,labor rate = 8000Rs. per month,Cost = labor rate * Person MonthCost = labor rate * Person MonthBased on the FP estimate and the historical productivity Based on the FP estimate and the historical productivity data, data, the total estimated project cost is 84800Rs. and the the total estimated project cost is 84800Rs. and the estimated effort is 10.6 person-months.estimated effort is 10.6 person-months.

ReferenceReference

Pressman 15, 22,23Pressman 15, 22,23 Section: 15.2.1,15.3, 22.2,23.6Section: 15.2.1,15.3, 22.2,23.6