View
2
Download
0
Category
Preview:
Citation preview
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
MTAT.03.243
Software Engineering Management
Lecture 07:
SPI & Measurement –
Part A
Dietmar Pfahl
email: dietmar.pfahl@ut.ee Spring 2013
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Structure of Lecture 7
• Motivation and Definitions (Measure, Measurement)
• Example Measures (Process, Product, Resource)
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Project Planning and Control
All activities related to match the current course of the project with the planned course of the project
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Types of Process Models
Processes
Engineering Processes Non-Engineering Processes
Product-Engineering
Processes Process-Engineering
Processes Business
Processes Social
Processes
Development
Processes
Maintenance
Processes
Project Mgmt
Processes
Quality Mgmt
Processes
Conf Mgmt
Processes Product Line
Processes
Improvement
Processes
Process Modeling
Processes
Measurement
Processes
Process Taxonomy
Software
Knowledge
Process
Models
Product
Models
Quality
Models
Life Cycle
Models
Engineering Process Models
Business Process Models
Social Process Models
Technical Process Models
Managerial Process Models
Process Engineering
Proc. Models
. . .
. . .
PROFES
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
SPI Planning and Control
All activities related to match the current course of the SPI project with the planned course of the SPI project
SPIP Start
SPIP Planning SPIP Control
SPIP Steering
SPIP Enactment SPIP End
SPIP = SPI Project
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Measurement in PROFES
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Definitions:
Measurement and Measure
Measurement:
• Measurement is the process through which values (e.g., numbers) are assigned to attributes of entities of the real world.
Measure:
• A measure is the result of the measurement process, so it is the assignment of a value to an entity with the goal of characterizing a specified attribute.
Source: Sandro Morasca, “Software Measurement”, in “Handbook of Software Engineering and Knowledge Engineering - Volume 1: Fundamentals” (refereed book), pp. 239 - 276, Knowledge Systems Institute, Skokie, IL, USA, 2001, ISBN: 981-02- 4973-X.
4 e *
3 d *
2 c *
1 b *
0 a *
A B
Scale & Unit
LOC (lines of code)
Size Measure
Entity: Program Attribute: Size
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Software Measurement Challenge
• Measuring physical properties (attributes):
entity attribute unit* scale (type) value range*
Human Height cm ratio 178 (1, 300)
Human Temperature C° interval 37 (30, 45)
• Measuring non-physical properties (attributes):
entity attribute unit* scale (type) value range*
Human Intelligence/IQ index ordinal 135 [0, 200]
Program Modifiability ? ? ? ?
• Software properties are usually non-physical:
size, complexity, functionality, reliability, maturity, portability, flexibility, understandability, maintainability, correctness, testability, coupling, coherence, interoperability, …
‘unit’ and ‘range’ are sometimes used synonymously with ‘scale’
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Measurement Scale Types
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Measurement Scale Types – cont’d
The classification of scales has an important impact on their practical use, in particular on the statistical techniques and indices that can be used. Example: Indicator of central tendency of a distribution of values (“Location”).
Mode = most frequent value of distribution Median = the value such that not more than 50% of the values of the distribution are less than the median and not more than 50% of the values of the distribution are greater than the median
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Scale types and meaningful measurement
• Scales are defined through their admissible transformations
• Scales (and their admissible transformations) help us decide
– whether a statement involving measures is meaningful
– what type of statistical analyses we can apply
• Definition of Meaningfulness:
A statement S with measurement values (i.e., measures m1,
…, mn) is meaningful iff its truth of falsity value is invariant
under admissible transformations Tr.
iff: “if and only if”
Tr(S[m1, …, mn]) is true iff S[Tr(m1), …, Tr(mn)] is true
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Meaningfulness of Measurement-Based Statements
Definition:
A statement involving measures is
meaningful, if its truth value remains
unchanged under any admissible
transformation
Example:
“In Madrid, during summer, it’s on average
twice as hot as in Tartu” (measured on the
Celsius scale: e.g., 40 C vs. 20 C)
-> Meaningful?
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Meaningfulness of Measurement-Based Statements
Procedure to check for meaningfulness:
1. Apply the admissible transformation to
measures in a statement S and obtain a
transformed statement S’.
2. If S’ can be shown to be equivalent to S,
then the statement S is meaningful for the
scale associated with the admissible
transformation.
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Meaningfulness – Example 1
• Is statement (1) on the right meaningful, if X is measured on a ratio scale?
• Apply any admissible transformation M’=aM (a>0) for ratio scales:
• By arithmetic manipulation, (2) can always be made equivalent to (1). Therefore, the first statement is meaningful for a ratio scale.
(1)
(2)
mxx
2
21
Ratio Scale
maxaxa
2
21
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Meaningfulness – Example 2
• Is statement (1) on the right meaningful, if X is measured on an interval scale?
• Apply any admissible transformation M’=aM+b (a>0) for interval scales:
• By arithmetic manipulation, (2) can always be made equivalent to (1). Therefore, the first statement is meaningful for an interval scale.
(1)
(2)
mxx
2
21
Interval Scale
bmabxabxa
2
21
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Meaningfulness – Example 3
• Is statement (1) on the right meaningful, if X is measured on an ordinal scale?
• Apply an admissible transformation for ordinal scales, e.g., x’=x3:
• For any pair of measurements x1 and x2, there exists always one admissible transformation such that statement (2) is false when (1) is true. Therefore, statement (1) is not meaningful for an ordinal scale.
(1)
(2)
mxx
2
21
Ordinal Scale
3
213
3
2
3
1
22
xxm
xx
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Meaningfulness – Geometric Mean
• The geometric mean of a data set [a1,
a2, ..., an] is given by
• On which scale type is the geometric
mean meaningful?
Scale Type ?
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Structure of Lecture 7
• Motivation and Definitions (Measure, Measurement)
• Example Measures (Process, Product, Resource)
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Measurable entities in a SW Process Model
• An entity can represent any of the
following:
–Process/Activity: any activity (or
set of activities) related to software
development and/or maintenance
(e.g., requirements analysis,
design, testing) – these can be
defined at different levels of
granularity
–Product/Artifact: any artifact
produced or changed during
software development and/or
maintenance (e.g., source code,
software design documents)
–Resources: people, time, money,
hardware or software needed to
perform the processes
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Examples of Software Product Attributes
• Size
–Length, Complexity,
Functionality
• Modularity
• Cohesion
• Coupling
• Quality
• Value (Price)
• ...
• Quality (-> ISO 9126)
–Functionality
–Reliability
–Usability
–Efficiency
–Maintainability
–Portability
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Product Measure – Ex. 1: ’Code Size’
• Entity
• Attribute
• Unit
• Scale Type
• Range
• Who collects/reports the data?
• When (how often) is the data collected?
• How is the data collected?
• Who is responsible for data validity?
• Code module
• Size (or better: Length)
• Netto Lines of Code (NLOC)
• Ratio
• (0, ∞)
• Developer
• Once, at end of week
• Using tool ’CoMeas’
• Project Manager
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Product Measure – Ex. 2: ’Code Quality 1’
• Entity
• Attribute
• Unit
• Scale Type
• Range
• Who collects/reports the data?
• When (how often) is the data collected?
• How is the data collected?
• Who is responsible for data validity?
• Code module (class file)
• Quality (or better: Correctness)
• Defects (Def)
• Ratio
• [0, ∞)
• Developer
• Continuously during unit testing
• Using defect reporting tool ’TRep’
• Project Manager
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Product Measure – Ex. 3: ’Code Quality 2’
• Entity
• Attribute
• Unit
• Scale Type
• Range
• Who collects/reports the data?
• When (how often) is the data collected?
• How is the data collected?
• Who is responsible for data validity?
• Code module (file)
• Quality (or better: Defect Density)
• Def / NLOC
• Ratio
• [0, ∞)
• Developer
• Continuously during unit testing
• Using tools ’TRep’ and ’CoMeas’
• Project Manager
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Direct vs. Indirect (Derived) Measures
• Direct measure: a measure that directly characterizes an empirical
property and does not require the prior measurement of some other
property
• Indirect measure: uses one or more (direct or indirect) measures of one
or more attributes in order to measure, indirectly, another supposedly
related attribute.
– Requires first the measurement of two or more attributes,
– then it combines them using a mathematical model.
speed = distance / time [km/h] accuracy = ( |actual – estimate| / estimate ) * 100% [Percentage]
Is ’estimate’ a measure?
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Types and Uses of Measures
• Types of Measures
– Direct vs. Indirect
– Subjective vs.
Objective
– Qualitative vs.
Quantitative
• Uses of Measures
– Assessment vs.
Prediction
NB: Measurement for
prediction requires a
prediction model
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Indirect Measures
• Examples:
– Defect Density (DD)
– Reliability (Rel)
– Productivity (Prod)
• Scale type of an indirect measure M will generally be the
weakest of the scale types of the direct measures M1, …,
Mn
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Indirect Measures
• Examples:
– DD = Quality 1 / Size [Unit: #Def/NLOC]
– Reliability = Quality 1 / Time [#Def/hour]
– Productivity 1 = Size / Time [NLOC/hour]
– Productivity 2 = Size / Effort [NLOC/person-hour]
– ...
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Examples of Software Process and Resource
Attributes
• Process Efficiency:
How fast, how much effort, how much quantity/quality per time or
effort unit?
• Process Effectiveness:
Do we get the quantity/quality we want?
• Process Maturity:
CMMI level
• People/Organisation-related:
Skills, knowledge, learning, motivation
• Method/Technique/Tool-related:
Effectiveness, Efficiency, Learnability, Cost
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Process Measure – Ex. 1: ’Acceptance Test Time’
• Entity
• Attribute
• Unit
• Scale Type
• Range
• Who collects/reports the data?
• When (how often) is the data collected?
• How is the data collected?
• Who is responsible for data validity?
• Acceptance Test
• Time (or ’Duration’)
• Calendar Day
• Interval
• [0, ∞)
• Customer XYZ
• At end of every test day
• Using reporting template ’RT’
• Product Owner
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Process Measure – Ex. 2: ’Coding Effort’
• Entity
• Attribute
• Unit
• Scale Type
• Range
• Who collects/reports the data?
• When (how often) is the data collected?
• How is the data collected?
• Who is responsible for data validity?
• Coding
• Effort
• Person-hour
• Ratio
• [0, ∞)
• Developer
• At end of every work day
• Using reporting template ’RE’
• Project Manager
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Effort vs. Time Trade-Off
Person Effort = 4 person-days (pd)
1 1 1 1
Person Effort = 4 pd
1 1 1 1
Person Effort = 4 pd
1
1 1
1
Day
Day
Day
What does it mean when I
say:
• ”This task takes 4 days”
• ”This task needs 4 person-
days”
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Effort vs. Time Trade-Off
Person Effort = 4 person-days (pd)
1 1 1 1
Person Effort = 4 pd
1 1 1 1
?
Person Effort = 4 pd
1
1 1
1
?
COCOMO
Source: B. Boehm: Software Engineering Economics, Prentice-Hall, 1981.
Day
Day
Day ?
COCOMO Intermediate model: Effort Adjustment Factor -> Schedule (SCED)
b d
devE aS EAF T cE
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Agile Measurement:
Sprint Burndown Chart – Example
Sprint
Backlog
(Task List)
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Agile Measurement: Velocity (1)
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Agile Measurement: Velocity (2)
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Agile Measurement: Velocity (3)
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Agile Measurement: Velocity (4)
Solid agile teams have consistent velocity (+/- 20%) Fluctuations? -> Look to stabilize team / environment Velocity trending up/down? -> Look at technical debt handling (rework) and team processes ...
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Agile Measurement: Velocity (5)
Pitfalls to avoid
• Velocity measures team history of delivered software (in terms
of story points or similar units)
Remarks
• Do not use not-done work in velocity calculations
• Velocity between teams is not comparative or additive
• Velocity does not show relative performance or productivity of
teams
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Resource Measure – Ex. 1: ’Programming
Skill’
• Entity
• Attribute
• Unit
• Scale Type
• Range
• Who collects/reports the data?
• When (how often) is the data collected?
• How is the data collected?
• Who is responsible for data validity?
• Team Member
• Programming Skill
• Programming Test Score (PTS)
• Ordinal
• [0, 1, 2, 3, 4, 5] – NB: Each
number needs explanation!
• Skill Test Agency
• Whenever a test is conducted
• Programming Test
• Certification Agency
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Resource Measure – Ex. 2: ’Personality’
• Entity
• Attribute
• Unit
• Scale Type
• Range
• Who collects/reports the data?
• When (how often) is the data collected?
• How is the data collected?
• Who is responsible for data validity?
• Team Member
• Personality
• Myer Briggs Type Indicator (MBTI)
• Nominal
• Set of 16 Types: ISTJ, ISFJ, INFJ,
..., ENTJ
• Test Agency
• Whenever a test is conducted
• MBTI Instrument
• Certification Agency
MTAT.03.243 / Lecture 07 / © Dietmar Pfahl 2013
Next Lecture
• Topic: SPI & Measurement – Part B
• For you to do:
– Have a look at the PROFES Quick Reference and
Manual -> What does it say about measurement?
– Have a look at some of the industry case examples in
the book “The Goal/Question/Metric Method: a practical
guide for quality improvement of software development”
(it’s in the reading materials)
Recommended