Software Measurement Theoretical issues. Measurement Theory – Sources Measurement theory in mathematics Metrology (physical science & Engineering) – Definitions

Software Measurement

Theoretical issues

Measurement Theory– Sources

• Measurement theory in mathematics• Metrology (physical science & Engineering)

– Definitions of : units – quantities – scales – etalon - ….• Representational Theory

– 1st comprehensive framework– S.S. Stevens (1946)– Social Science (Psychology)

– In Wikipedia ".. measurement is the process of obtaining the magnitude of a

quantity, such as length or mass, relative to a unit of measurement, such as a meter or a kilogram.

"The term can also be used to refer to the result obtained after performing the process"

Measurement Theory• Representational Theory

– Measurement = Characterizing a given attribute of a given entity by a numerical value (symbolic)

- Measurement = mapping

Albert

Gabriel

Paula

180

90

172

Attribute : tallness

Measurement TheoryThe intelligence barrier (cfr. Kritz)

reasoning meaningful relations – operations….




• In Software : no consensus about how to characterize (define) software attributes

• Example : the term "complexity"– Size – Coupling– Connectivity (fan-in / fan-out)– Understandability – Lack of Cohesion – ….




• In Software : ambiguity & misuse about the concerned software entity (the product)

• Ex. "maintainability" is an attribute of– code– design – code + design– ….

• Ex. "size" is an attribute of– code– requirement –




• How to measure = F "a precise characterization of what we are measuring"– Ex To measure the mass of an sphere, measure

• the time it takes the object to fall from some height?• its diameter?• its external area?• its volume?• a calculation on basis of those

– Ex To measure the reliability of a light bulb, take• its color ?• the length of the wire?• the number of watts?• the number of times it is switched on? (during ??? )• the average lifetime? (of how many bulbs ??? how to choose them)

Measurement Theory• measurement should capture one’s own intuitive understanding on an

attribute of a set of entities

Albert

Gabriel

Paula

180

90

172

Mapping property 1: Order preservation Albert "taller" Paula (Albert) > (Paula)

Measurement Theory• measurement should capture one’s own intuitive understanding on an

attribute of a set of entities

Albert

Gabriel

Paula

180

90

172

Mapping property 2 : Operations preservation :if Gabriel "on the shoulder of" Albert Albert Gabrielthen (Gabriel "on the shoulder of" Albert ) = (Albert) (Gabriel)

ex (x) (y) is defined as 0,87 ((x) + (y)) ?

Measurement Theory• measurement should capture one’s own intuitive understanding on

an attribute of a set of entities

• Question : which are our understanding ?– Which order ? Which operations ?

• Ex Sound characterization – Timbre:

• same timbre / different timbers (identity)– Loudness:

• a sound louder than another sound (order)– Pitch:

• same pitch / different pitches (identity)• a sound higher than another sound (order)• a “distance” from a reference note (distance) ??

Measurement Theory Mathematically

• A measure involves

– Empirical Relational System = a triple• E : a set of entities• R1, R2, …, Rn: a collection of relations among entities• o1, o2, …, om: a collection of binary operations between entities• R1, R2, …, Rn and o1, o2, …, om determine meaningful (consensual

knowledge on the domain)• !!! (no numbers)

– Numerical Relational System = a triple• V : a set of values• S1, S2, …, Sn: a collection of relations among values,

– such as : > , >= • op1, op2, …, opm: a collection of binary operations between values

– such as : +• !!! no real entities


• Example 1:– program size number of LOC– concatenation +

• Empirical Relational System = a triple– E : a set of program pieces p1, p2,….– R1 : px "is longer than" py (observable on all pairs)– O1 : program concatenation (cfr. technically…)

• Numerical Relational System = a triple– V : non negative real numbers– S1 : > – op1 : +


• Example 1:– program size number of LOC– concatenation +

• What about representational condition ??

– The relations (the order)• For all px, py

– px "is longer than" py size(px) > size(py)

– The operation• For all px, py

– size (px "concatenation" py) = size(px) + size(py)


• Example :– program complexity graph complexity (McCabe)– Composition (via call) +

– Empirical Relational System = a triple• E : a set of procedures pr1, pr2,….• R1 : prx "is more complex than" pry • O1 : procedure composition via "call"

– Numerical Relational System = a triple• V : non negative real numbers• S1 : > • op1 : +


• Example :– program complexity graph complexity (McCabe)– Composition (via call) +

• What about representational condition ??

– The relations (the order)• For all px, py

– prx "is more complex than" py complexity(x) > complexity (py)

– The operation • For all prx, pry

– complexity (prx "call" pry) = complexity(prx) + complexity(pry)

?

Measurement Theory Scale & Validity

• A scale = – Empirical Relational System– Numerical Relational System– Measure (mapping) satisfying the "Representation Condition"

• A scale is not (necessary) unique– distance : KM, M, MM, …– Temperature : degrees celsius , kelvin, fahrenheit …– Electric Resistance : ohm– Electric Current : ampere– Pressure : pascal, bar,

• A family of scales – Scales + admissible transformations – Admissible transformations = preserving "knowledge" , (i.e., statements like "x > y" , "x

op y = z", …) (i.e., "meaningful statements")


• meaningful statements in real world ? – Dupont is taller than Tintin – Yesterday’s temperature was 5 times as high as today’s ?

– Program P1 is three times as long as program P2 ?– Program P1 is three times as complex as program P2 ?– Failure F1 is more critical than failure F2 ? – Failure F1 is five times as critical as failure F2 ? – The average "note" of my students is 12.33 ? – The average "grade" of my students is "distinction" – The UML design D1 (with the pattern P) is better then the design D2– The UML modeling language is more expressive than the ERA

– Question : Which "knowledge" is consensually admitted ?

• weaker view /researcher view – which "knowledge" is potentially a good candidate for a consensus– How to reach such a consensus


• 5 classical "scale types" – nominal

• least informative one / largest set of admissible transformations• no order • classification : ex. (yes/no) (blue/green/red) ...)

– ordinal• order only (no +/-) (no distance) • ex. Students grade : A, B, C, D, E• Statistics : no mean (debate in social science) median

– interval• order with +/- ; with distance (no ratio) • ex. Celsius temperature • Statistics : mean correlation etc

– ratio• order with +/- with ratio with zero, …• ex. Distance in Km/m/cm/….• Statistics : (geometric) mean correlation etc

– absolute• most informative one / smallest set of admissible transformations• Absolute : Ratio + uniqueness• Ex.

Measurement Theory Scales : nominal scale

• Examples– Types of (vehicles /energy sources / …) gender / birthplaces … – SE: pgming languages / methodology /…

• Mainly a classification (rather than an order)

• Meaningful assertions in the empirical world– Entity e belong / do not belong to class C

• Meaningful operations in the numerical world – Mainly a classification (no order)– Any collections of symbols

• (x,y,z) • (yes / no/ no answer) • (C / Java / Fortran)…. • (0,1,2)…. But without operations nor order

• Admissible Transformations from scale to scale– one-to-one

Measurement Theory Scales : nominal scale

• empirical world:– P : set of programs p1, p2,… (or modules)– order : – operation "same lgge as""

• numerical world:– X: {java, C ,C#,Fortran}– order : – operation : =

• Example of operations– p1 "same lgge than" p23– not (p1 "same lgge than" p12)

• Example of statistics– C (30%), FORTRAN(10%), Java (35%), C# (25%)– Frequencies ok ;here p(C) = .3 ; p(FORTRAN) = .1 ; p(Java) = .35 ; p(C#) = .25– Mode (the most likely value(s) ) ok ; here Java

Measurement Theory Scales : ordinal scale

• Examples– Ranking in general : Michelin guide stars, wine, students … – SE: maturity (CMMi), criticality, security levels (some norms) ; (some def) complexity ;

• A simple order

• Meaningful assertions in the empirical world– Entity e1 is "as …. as" e2– Entity e2 is "more …. than" e5

• Meaningful operations in the numerical world – Order : – Any collections of symbols with defined order

• (a,b,c,d,e) • (0,1,2)…. With order but with no distance

• Admissible Transformations from scale to scale– Admissible transformations : any monotonically increasing

t: V→V (m(e1) > m(e2) ⇔ t(m(e1)) > t(m(e2)) )– two entities are at the same level before the transformation if and only if they are at the same level

after the transformation– one entity is at a higher level before the transformation if and only if it is at a higher level after the

transformation– Km to Cm / Celsius to Fahernheit /

Measurement Theory Scales : ordinal scale

• empirical world:– P : set of programs p1,p2,… (or modules)– order : more maintainable – operation

• numerical world:– X: {easy, medium, difficult, too difficult}– order : >– operation :

• Example of operations– p1 "as maintainable as" p23– not (p1 "more maintainable than" p12)

• Example of statistics– median ok – average : problems !!

Measurement Theory Scales : interval scale

• Examples– calendar – Celsius temperature– SE: milestones

• A simple order + relative distance between values

• Meaningful assertions in the empirical world– Entity e1 is "as …. as" e2– Entity e2 is "more …. than" e5– From e1 to e2 is equal/greater/twice….. than from e2 to e3

• Meaningful operations in the numerical world – Order & distance– Ratio on distances– A collections of symbols with order and distance

• integer – real • (0,1,2)…. With order but and with distance (no zero)

• Admissible Transformations from scale to scale– Admissible transformations : linear transformations ( m’ = am + b )– (m(e1) – m(e2)) / (m(e3) – m(e4)) = (m’(e1) – m’(e2)) / (m’(e3) – m’(e4))

= (am(e1) + b – am(e2) - b) / (am(e3) + b – am(e4) - b)– !!!! a subset of the admissible transformations of ordinal scale

Measurement Theory Scales : ratio scale

• Examples– length – weight – SE: size – complexity (some definition)

• A simple order + distance from a zero

• Meaningful assertions in the empirical world– … +– distance from entity e1 to origin is equal/greater/twice…..

than distance from e2 to origin

• Meaningful operations in the numerical world – Order & distance– Ratio on distances from zero – Numbers (Reals…)

• Meaningful statistics: – all those for interval scales, plus (descriptive) geometric mean

• Admissible Transformations from scale to scale– m’ = am (with a > 0)– we can still change the unit of measurement as we like– we cannot change the zero

Measurement Theory Scales : absolute scale

• Examples– probabilities and any direct counting– SE: size in LOC – number of faults -

• Meaningful assertions in the empirical world– // ratio scales

• Meaningful operations in the numerical world – Order & distance & zero – Ratio on distances from zero – Numbers

• Meaningful statistics: – // ratio scales

• Admissible Transformations from scale to scale– m’ = m (no possible transformation)

– .

Documents

Software Measurement Theoretical issues. Measurement Theory – Sources Measurement theory in mathematics Metrology (physical science & Engineering) – Definitions