Iwsm2014 cosmic approximate sizing using a fuzzy logic approach (alain abran)

1

COSMIC Approximate Sizing using a

Fuzzy Logic Approach -A Case Study with

Industry Participants

Francisco Valdés Souto & Alain AbranÉcole de Technologie Supérieure

[email protected]@etsmtl.ca

© 2014 Valdés-Souto & Abran

2

The Sizing Problem

FSM methods work best when the information to be measured– is fully known.

Early phases: only non detailed information is available.

UC Identification


FP Identification

3

Henderson et al.: investigated the relationshipbetween FP & KLOC

Meli: A. Early Function

Points (EFP), based on IFPUG 4.0,

B. Extended FP (XFP): EFP & 3 correction factors.

Desharnais et al.: analysed 2 techniques: Function Points Simplified (FPS) a& Backfiring

Conte et al.: Early & Quick (E&Q) COSMIC - more tests needed to adjust or to confirm it

Vogelezang et al. : study of 50 projects to define size bands using the quartile approach.

Santillo: Analytic Hierarchy Process, for making choices among alternatives

Related Works onApproximation of Functional Size

1992 1997 2003 2004 2007 2011 2012 2013

Valdés et al. proposed a solution using the fuzzy logic model from [3-5], referred to as the EPCU model

Almakadmeh:A framework to assign scaling factors for identifying the level of granularity of functional requirements specifications.

The state of the art on approximate COSMIC FSM was discussed at IWSM/MENSURA 2013


4

2nd Generation FSM method: COSMIC & General Approach to Approximate Sizing


5

Approximate Sizing Approaches in the COSMIC Measurement Manual

Early sizing: • for use early in the

life cycle of a project:– before the

Functional User Requirements (FUR) are detailed and specified.

Rapid sizing: • for use when there is

not enough time to measure the required software using the standard method


6

Example 1: Average Functional Process approach. Example 2: Fixed Size Classification approach. Example 3: Equal Size Bands approach. (Vogelezang: Refined

Approximate or Quartile approach)

Example 4: Average Use Case approach.

Approximate Sizing Examples in the COSMIC Advanced & Related Topics Manual

Each example based on 2 main assumptions:1. Historical data exist for calculating the scaling factor (average, or

size bands).2. The whole set of requirements is described, or at least there is a

commitment, defined by the requirements, about the scope of the software to be developed.


7

Example 3: Equal Size Bands approach

• Historical data set: – 37 business application development projects, each having a total size

greater than 100 CFP. (Vogelezang , 2007):

• Quartile values of the Functional Process from this dataset: Small = 4.8 CFP, Medium =7.7 CFP, Large = 10.7, and Very Large = 16.4 CFP


8

Fuzzy Logic EstimationModel


9

Roles in the Fuzzy Logic Model

The domain expert:• Selects the types of input & output variables• Selects the ranges of values for each variable• Assigns the inference rules between the inputs

& output variablesThe estimator (junior or expert):• Select among the range of values of the input

variables


10

Roles in the Fuzzy Logic Model

The tool builder :• Selects:– the fuzzy logic maths options• Ex. Trapezoidal shape, triangle shape, etc

– Fuzzification options– De-fuzzication options

• Builds the estimation software tool as a shell for the variables selected by the domain experts


11

Fuzzy Logic Estimation Model

F. Valdés & A. Abran - IWSM-Mensura 2007 (Palma de Mallorca, Spain)

Tool used for this exploratory research:

The fuzzy logic estimation model developed in the PhD thesis of Francisco Valdes in 2011:

EPCU: Estimation of Projects in a Context of Uncertainty

Model initially tested with the estimation of projects duration


12

EPCU Model with Fuzzy Logic

Some characteristics of fuzzy logic estimation models (Valdés et al., 2007, 2010, 2011):

1. Designed to deal with vague information (i.e. usually described by linguistic variables).

2. Generates estimates with less dispersion than the experience-based approach.

3. Enables a systematic replication: whatever the level of skills of the people who assign the values for the input variables.

4. In the early phases (imperfect information environments) may be preferable to the experience-based estimation approach, under similar experimental conditions.

5. The performance of the EPCU estimation process for most of the projects is significantly better than that of the experience-based estimation approach, based on the quality criteria used.


13

COSMIC Approximate Sizing Using the EPCU Model

• Variable 1: Perception of the size of the Use Case (subjective, experience-based).

• Variable 2: The number of Objects of Interest related to the Use Case (subjective, experience-based).

Functional Size Estimated for each Use Case (CFP)

What variables influence the size of a Use Case? What is the possible range for

the output variable?

(Vogelezang , 2007)© 2014 Valdés-Souto & Abran

14

Case Study– Model Designer

A) Defines the Input variables with linguistic values (fuzzy sets):– Input variable 1: Size of Use Case= relative size from small

to very large– Input variable 2: No. of objects of interest = Low, Average,

and High

• Domain of membership function: from 0 to 5 ε R.

B) Defines the Output variable with a:– Min of a Use Case = 2 FP– Max of a Use Case = 16.4 CFP


15

Case Study Participants

• Case Study with 8 practitioners:– not familiar with the COSMIC method, – with no historical data for approximating the FSM using COSMIC, – did not know the EPCU model,– did not participate in the definition of the EPCU context.

The only information available had = – A form with the list of Use Cases – Their own experience with the business process related to the project

• The Case Study= a simulation of the early size estimation step with both approaches (Equal size band & Fuzzy Logic).


16

Experiment Design

ALFA software system/ 14 Use

Case descriptions

EPCU Model

1- Define Input variables & membership functions, 2- Define the Inference rules between the input variables & output variable (Functional Size in CFP)

3-Define Output variable & membership functions

2. Selecting a Measurement Reference

3. Knowing the ALFA Software System

4. Data Collection

5. Data Analysis

1. Define de EPCU Context for Approximate Sizing

Participants provided only with the ALFA list of Use Cases: assign values to the 2 input variables, based on their experience.


17

Participants tasks

A) Equal Size Band Model:- Consider the UC as FP and Classify each of the 14

Use Cases from Small to Very Large (4 linguistic categories)

B) Fuzzy logic model:B1) Assign for each Use Case a relative size between 0

& 5 B2) Assign for each Use Case a relative number of

Objects of Interest between 0 & 5.


18

Case Study Data & Analysis


19

Case Study Data Analysis

• MMRE= 63% and SDMRE = 5%

• Maximum MMRE = 67% & Min MMRE = 54%.

Equal Size Bands approach & real value Fuzzy Logic EPCU model & the real value

• MMRE = 45% & SDMRE= 18%.• Maximum MMRE = 75% & Minimum MMRE =

20%.• Practitioners are not familiar with the COSMIC

method.


20

Exploratory Research Observations

• Participants did not know the COSMIC sizing method

• The Equal Size Band led to less accurate COSMIC sizes:– With 9 participants for the same set of UC.

• The fuzzy logic model led to more accurate of size:– The better results obtained could be associated to the use of use cases

instead of functional process, even though the use cases is at a higher level of granularity than the functional processes.


21

Exploratory Research Observations

• Fuzzy Logic approach: – it does not use bands, but rather a continuous range in ε R, which is

represented by a membership function.– But it is sensitive to min-max values

Large scale experiments needed with:• More case studies• More participants for each case study• Analyze the original data set of Equal Size Bands Approach in

order to define a “cut-off” instead to use the average for the last band.


22

Questions?


Documents

Iwsm2014 cosmic approximate sizing using a fuzzy logic approach (alain abran)