THE VALUE OF COSMIC SOFTWARE SIZE MEASUREMENT …psmo.pl/wp-content/uploads/2017/06/PoSMA2017-COSMIC_Achievements.pdfTHE VALUE OF COSMIC SOFTWARE SIZE MEASUREMENT FOR PROJECT ESTIMATING

THE VALUE OF COSMIC SOFTWARE SIZE

MEASUREMENT FOR PROJECT ESTIMATING

AND CONTROL

Polish Software Measurement Association Conference

June 2016, Warsaw

Charles Symons

Agenda 2

Objectives of software size measurement:

‘Functional Size Measurement’ (FSM)

methods

The COSMIC method – key features

Evidence that the COSMIC method has

achieved its goals

Conclusions Copyright: COSMIC 2017. All rights

reserved

Objective 1: Software managers want to

control and compare performance of project

activities 3

Delivery to budget & time: • Actual vs. Estimated Cost • Actual vs. Estimated Duration

Project productivity • Size / Effort

Project speed • Size / Duration

Product scope and quality • Size • Functional (e.g. business needs) • Technical (e.g. maintainability, post-delivery defects density)

Also consider the performance of on-going maintenance and enhancement activities

Copyright: COSMIC 2017. All rights reserved

Objective 2: ….and to use performance data

for estimating future projects 4

Establish benchmarks

Project data repository

Measure and track software

requirements size

Measure project performance

Estimating budgeting, etc.

Copyright: COSMIC 2017

A simple approach to effort estimation -

for a project, iteration, etc

Productivity = Software size

Project effort

New project ‘typical’ effort = Estimated software size

Benchmark project productivity

New project estimated effort =

Adjustments for project-specific

factors

Estimated software size

Benchmark project productivity x

(Establish benchmarks from data for several projects)


COSMIC (& IFPUG) define three types of

requirements for a software project, or

iteration, etc. 6

Functional User Requirements (FUR) • what the software must do Non-Functional Requirements (NFR) • quality, technical and environmental constraints, etc. Project Requirements and Constraints (PRC) • targets, processes & tools, resources, dependencies, etc.


Functional Size Measurement (FSM) methods

can help achieve both objectives 7

Sizing method options:

Counts of lines of code:

X Cannot be estimated until software designed X Technology-dependent, no standards Accounts for all requirements that are delivered

Functional size: International standard methods Technology-independent What about ‘Non-Functional’ Requirements?

Other sizing methods: e.g. UCP, OOP, Story Pts. (?) etc:

X No reliable standards; only local benchmark data possible

X What about ‘Non-Functional’ Requirements? X Early total effort estimation?


However, there are several methods of

Functional Size Measurement 8

1980 1985 1990 1995 2000

COSMIC FFP v. 2.0

Allan Albrecht

FPA IFPUG 4.0

IFPUG 4.1

MkII FPA

MkII FPA v.1.3

Full FP’s v.1

3-D FP’s

Feature Points

ISO ‘FSM’ Standard

14143

COSMIC v. 4.0.1

2016

IFPUG 4.3

‘First Generation’ FSM Methods

Nesma IFPUG


‘First Generation’ FSM Methods share several

weaknesses 9

Designed to measure ‘whole’ business applications

Difficult to use with modern software requirements engineering and development methods (e.g. Agile)

Sizes

No well-defined unit of measure

Mostly limited, closed size ranges


Traditional FP vs. COSMIC FP measurement

scale – a key difference 10

Function Points (FP)

3 FP 4 FP

6 FP

COSMIC Function Points - CFP

1 CFP 2

3 4

5 6

7 8

9 10

11 …

No abitrary max

A single CFP exists and is well defined

10


Agenda 11


FSM methods



achieved its goals

Conclusions


Goals of the COSMIC method for sizing

software 12

A size measure of Functional User Requirements

based on fundamental software engineering principles

for business, real-time and infrastructure software

independent of technology or processes used for the software or project

usable for both our software measurement objectives

‘open’, freely available (via www.cosmic-sizing.org )


http://www.cosmic-sizing.org/



The COSMIC method has a well-defined 3-

stage Measurement Process

FUR

Measurement

Strategy

Generic Software Model

Mapping

PhaseFUR in the form of the

Generic Software Model

Measurement

Phase

Functional size of

the software in

units of CFP

Definition of each piece of

software to be measured and

of the required measurement

Input from measurement sponsor

Software Context Model

FUR

13


In the Strategy phase, we must define the

software to be measured, its users, the

layers, etc. 14

Copyright: COSMIC and authors, 2016

Software

being

measured

Persistent Storage

Functional Users? • Humans • Hardware devices • Other software

Layers?

Level of decomposition?

Measurement Phase: software FUR define a

number of ‘Functional Processes’

Functional User Requirements

Functional Processes

Data Manipulation

Data Movement

Sub-processes

Theory: Functional User Requirements

Functional Processes

Data Movements (account for

associated data manipulation)

1

1

n

2 - n

In practice:

15


A software functional process must respond to

an event detected or generated by a

functional user

cau

ses

a

Boundary

that is moved into a

FP

Functional Process

Functional User

to g

ener

ate

a

Data Group

16


Event

A functional process can have four types of

‘Data Movement’ sub-processes

Software being

measured

Boundary

Functional Users • Hardware devices, • Other software or • Humans

Entries

Exits

Reads Writes

Persistent storage

The ‘Data Movement’ is the unit of measure: 1 CFP (COSMIC Function Point)

17


The Measurement Phase: Size = the total of

data movements of all functional processes

A functional process must have at least 2 CFP

A triggering Entry

An ‘outcome’ – i.e. a Write or an Exit

Largest observed functional processes?

In banking ~ 65 CFP

In avionics >100 CFP

18


What about non-functional requirements (NFR)? 19

The COSMIC FFuunnccttiioonnaall SSiizzee MMeeaassuurreemmeenntt MMeetthhoodd

VVeerrssiioonn 44..00..11

Guideline on Non-Functional & Project

Requirements

How to consider non-functional and project requirements in software project performance measurement, benchmarking and estimating

Version 1.

November 2015

Glossary of terms for Non-Functional Requirements

and Project Requirements used in

software project performance measurement, benchmarking and

estimating

VERSION 1.0

September 2015

A joint COSMIC/IFPUG study defined NFR and developed a Glossary of NFR and Project Requirement terms A COSMIC Guideline advises how to deal with NFR in projects


Studies show that system NFR may evolve into

software FUR, that COSMIC can measure 20

Outline

Functional

Requirements

Outline

(System) NFR

Project

Requirements

& Constraints

Require-

ments

Analysis

Definition

& Design

Build, Test

&

Implement

A

r

c

h

i

t

e

c

t

u

r

e

Imple-

mented

software

system

or

software

product

Approxi-

mate

FUR

Detailed

FUR

Detailed

NFR


Agile: sizes can be measured at all levels and

aggregated up to the system size

21

COSMIC sizes may be used for: • early total software sizing and

effort estimation; • US, Sprint, etc, sizing and

estimation; • progress control at any level.

Sprint

Iteration

Release

System

User Story (new &/or re-work)


Agenda 22


FSM methods



achieved its goals

Conclusions


Case: Renault 1) achieves cost estimation

accuracy from its ECU designs (automatically!) 23

Cost vs size (CFP)

Memory size vs software size (CFP)


Case: Italian web software supplier - effort

estimation is more accurate with CFP than with

FP 2) 24

Wo

rk-h

ou

r R

esid

ual

s

25 industrial Web applications

Conclusions: ‘The results of the … study revealed that COSMIC outperformed Function Points as indicator of development effort by providing significantly better estimations’


Case: A Canadian supplier of security and

surveillance software systems: Agile

estimating 3) 25

A customer request for new or changed function is called a ‘task’

The supplier uses the Scrum method; iterations last 3 – 6 weeks

Teams estimate tasks in USP by a ‘planning poker’ process within each iteration, and convert directly to effort in work-hours

The study involved measurements on 24 tasks from nine iterations for which estimated and actual effort were available

Each task was re-measured in CFP


The actual Effort vs USP size graph (24 tasks)

would be poor for effort estimation 26

0

20

40

60

80

100

120

140

160

180

200

0 20 40 60 80 100 120 140 160 180 200

Act

ual E

ffor

t (ho

urs)

Estimated Effort (Hours)

Effort = 0.47 x Story Points + 17.6 hours and R2 = 0.33)

Notice the wide spread and the 17.6 hours ‘overhead’


The final actual Effort vs CFP size graph is

much better for estimating 27

0

20

40

60

80

100

120

140

160

180

200

0 10 20 30 40 50 60 70 80

Act

ual

Eff

or

(Ho

urs

)

Functional Size in CFP

Y = 2.35 x CFP - 0.08hrs and R2 = 0.977)

Resulting from CFP measurement, two tasks were identified with very low effort/CFP. These were found to involve significant software re-use, so were considered separately


Case: An Australian User view of ‘COSMIC for

Agile’ 4) 28

“The reality of achieving predictable project performance has driven me to investigate many methods of prediction. COSMIC is the method that lets me sleep at night.”

Denis Krizanovic, Aon Australia, August 2014


Agenda 29


FSM methods



achieved its goals

Conclusions


Conclusion: COSMIC has achieved all its design

goals: a major advance in FSM 30

The method is stable and ‘future-proof’.

Proven wide scope of applicability CFP sizes correlate very well with effort and code size Used around the world:

Measurement Manual available in 11 languages 50% of known users are software houses > 30,000 downloads of research & conference papers

ISO standard. Endorsed by US Agencies GAO, NIST.


Thank you for your attention

(www.cosmic-sizing.org)

Charles Symons [email protected]

31





mailto:[email protected]

References 32

1. Alexandre Oriou et al, “Manage the automotive embedded software development cost & productivity with the automation of a Functional Size Measurement Method (COSMIC), Alexandre Oriou et al, IWSM/Mensura Conference 2014, www.ieeexplore.org

2. S. Di Martino, F. Ferruci. C. Gravion, F. Sarro, “Web Effort Estimation: Function Point Analysis vs. COSMIC”, Information and Software Technology 72 (2016) 90–109

3. C. Commeyne, A. Abran, R. Djouab. “Effort Estimation with Story Points and COSMIC Function Points - An Industry Case Study”, Software Measurement News, Vol 21, No. 1, 2016 *

4. Comment on Linkedin discussion, September 4th, 2014


http://www.ieeexplore.org/

http://dx.doi.org/10.1016/j.infsof.2015.12.001




Documents

THE VALUE OF COSMIC SOFTWARE SIZE MEASUREMENT …psmo.pl/wp-content/uploads/2017/06/PoSMA2017-COSMIC_Achievements.pdfTHE VALUE OF COSMIC SOFTWARE SIZE MEASUREMENT FOR PROJECT ESTIMATING